[DNM] Implement Float16 #21738

stephentyrone · 2019-01-09T13:36:37Z

Implement IEEE 754 binary16 as Float16 in the stdlib conforming to BinaryFloatingPoint and SIMDScalar. This PR does not include availability tags to make it easier for people to experiment, but all of this API will be marked available(9999) in the final PR.

Evolution proposal: swiftlang/swift-evolution#1120
Pitch thread: https://forums.swift.org/t/add-float16/33019/25

stephentyrone · 2019-01-09T14:17:00Z

@swift-ci please test

swift-ci · 2019-01-09T14:47:52Z

Build failed
Swift Test Linux Platform
Git Sha - 5df25f561616d995499dbd3ecac442b634739421

swift-ci · 2019-01-09T14:52:31Z

Build failed
Swift Test OS X Platform
Git Sha - 5df25f561616d995499dbd3ecac442b634739421

Azoy · 2019-01-21T08:37:09Z

@stephentyrone I tried out Float16 locally and I have to say that constructing literals is painful:

func x() {
  var x: Float16 = 1.0
}

results in the following SIL: https://gist.github.com/Azoy/12c131dd50b9d3610ac98e290123f8a8 .
I'm no floating point expert, but I tested going through float32 first then to float16 for literals to see if we could hit the constant folding pass in SIL:

% if bits == 16:
// Trunc to Float then trunc to Float16
self = ${Self}(Float(_builtinFloatLiteral: value))
% elif ...

which thankfully did get picked up by constant folding, but required adding float16 semantics:

// lib/SILOptimizer/Utils/ConstantFolding.cpp Line: 963
case BuiltinFloatType::IEEE16:
  return IEESemantics(16, 5, 10, false);

this resulted in some very pleasing SIL: https://gist.github.com/Azoy/e9f644508454befd21c4b8cb1bfd99f0
which is lowered as a simple store half 0xH3C00, half* %x._value, align 2 in LLVM IR.

We'd have to keep the workaround for the init(_ other: Float80) initializer, but I wanted to ask if it's feasible to make this change for the init(_builtinFloatLiteral value: Builtin.FPIEEE80) initializer so that literals don't have to go through this workaround.

Azoy · 2019-01-21T08:48:35Z

@rjmccall I also wanted to ask if it was at all feasible if floating point types could take advantage of a Builtin.FloatLiteral type like you did for integers. I'm not too sure how we would lower this type specifically in LLVM, but all occurrences of the float literal type should get replaced by one of the Builtin.FPIEEE{bits} types eventually which makes me think that this might be somewhat feasible. Although, at this point during the release cycle is seems very unlikely that such change would go through as it affects some of the stdlib API.

stephentyrone · 2019-01-21T15:40:34Z

I wanted to ask if it's feasible to make this change for the init(_builtinFloatLiteral value: Builtin.FPIEEE80) initializer so that literals don't have to go through this workaround.

No, because that results in a double-rounding and gets some values wrong because of it. This is a temporary fix to get things working correctly, we can easily improve on it in the future.

stephentyrone · 2019-01-21T15:42:34Z

@rjmccall I also wanted to ask if it was at all feasible if floating point types could take advantage of a Builtin.FloatLiteral type like you did for integers.

Yes, we can (and will) do this at some future point. In order to work efficiently and maintain ABI stability, it will probably be a bit more complex than the integer literals are, but it's on the radar.

rjmccall · 2019-01-21T19:44:12Z

The biggest complexity with arbitrary-precision floating-point literals is that we cannot losslessly change the base of the source literal: if the user wrote 0.1, our representation has to be 1 × 10^-1. That's true even if we have no intention of ever supporting non-binary FP types (and I'd to, eventually). We wouldn't necessarily need to support bases other than 2 and 10, but we'd need to support at least those.

The representation would therefore need to include:

a sign
a base (possibly just a 2 vs. 10 flag)
an exponent (a fixed-width Int32 would probably be fine for real use cases, but maybe we'd want this to be arbitrary-precision as well just because)
an arbitrary-precision integer as the "significand"
some way of expressing the infinities and various NaNs (maybe using the significand as a payload?)

And then maybe you'd also throw in a pre-computed Double, because it is not otherwise going to be efficient to turn one of these into an IEEE format.

stephentyrone · 2019-01-21T21:51:53Z

@rjmccall Right. We want to have good support for radix 2 and 10, and nothing else really matters. There are some details to work out, but supporting those radices efficiently isn't too bad (and yes, one option is including a pre-computed Double and Float, though it's not the only option).

…ss` to CSApply

…ness of a member type

…ions inside Problem(s) which caused `ErrorExpr` to be added to AST should be diagnosed already and solver wouldn't be able to produce a viable solution in such case anyway.

… constraints for float literals

The Android CI machines cannot execute the test in the target devices. Marking the test as executable makes the test filtering know that this test needs to be executed in the target device.

…g the data structure by hand.

…-sillyness Runtime: Less eager instantiation of type metadata in the conformance cache

Tests seem to pass without these. Two calls still remain, in ModuleDecl::addFiles() and removeFiles(). It will take a bit more refactoring to eliminate those.

…a8a3deb651d2dd2fc3f8ad6 [ownership] Implement movable guaranteed scopes in ossa.

This fixes a recent source break. We need to perform the normal unqualified lookup before we handle the special case of 'Self', because there might be a type named Self defined in an outer context. Fixes <https://bugs.swift.org/browse/SR-12133> / <rdar://problem/59216636>

…gnostics Use single quotes instead of backticks in diagnostics

…g terminators when fed by load [copy], copy_value. This extends the (copy (guaranteed x)) -> (guaranteed x) optimization to handle transforming terminator arguments. This will begin to enable us to eliminate RC ops around optionals or casts. I still need to add support for eliminating copies that forward through br args and phis. <rdar://problem/56720436> <rdar://problem/56720519>

…osure.

…ng#29813)

@inline

The implementation was done quite a while ago. Now, that we have support in lldb (swiftlang/llvm-project#773), we can enable it by default in the compiler. LLDB now shows the runtime failure reason, for example: * thread swiftlang#1, queue = 'com.apple.main-thread', stop reason = Swift runtime failure: arithmetic overflow frame swiftlang#1: 0x0000000100000f0d a.out`testit(a=127) at trap_message.swift:4 1 2 @inline(never) 3 func testit(_ a: Int8) -> Int8 { -> 4 return a + 1 5 } 6 For details, see swiftlang#25978 rdar://problem/51278690

…af73160c99cfaa4641e6b22

…isable-watchos-x86-simulator

…ndroidModules_RequiresCpuArm64

[docs] Updating doc list for the diagnostics ported to new framework

…-hacks [Constraint solver] Remove performance hacks for pattern type computation

build: bifurcate `_add_swift_executable_single`

[NFC] Minimize uses of 'fileprivate' after SE-0169

This isn't quite right, because it introduces double-rounding, but it will do in the short-term.

…-float16

stephentyrone · 2020-02-28T21:43:26Z

Hmm, that's a merge gone wrong. Let's try that again.

Azoy · 2020-02-28T22:04:02Z

2630 commits to implement Float16? I knew IEEE754 was tough but I didn’t know it was that tough.

stephentyrone · 2020-02-28T22:07:04Z

Look, I just take "keep commits small" really seriously. One character max.

Azoy · 2020-02-28T22:09:28Z

Easier for code review I see 😆

stephentyrone · 2020-02-28T22:20:30Z

Real PR: #30130

beccadax added the swift evolution pending discussion Flag → feature: A feature that has a Swift evolution proposal currently in review label Jan 11, 2019

dwaite mentioned this pull request Dec 12, 2019

Float16 (half-precision floating-point) apple/swift-numerics#8

Closed

stephentyrone force-pushed the add-float16 branch from 7eb5f8a to 29b9207 Compare February 5, 2020 14:31

xedin and others added 18 commits February 17, 2020 16:09

[CSDiag] NFC: Move `diagnoseDeprecatedConditionalConformanceOuterAcce…

8722530

…ss` to CSApply

[ConstraintSystem] Fix unwrap optional base fix to preserve l-value…

15036bc

…ness of a member type

[ConstraintSystem] Don't attempt to solve closures with error express…

3edefea

…ions inside Problem(s) which caused `ErrorExpr` to be added to AST should be diagnosed already and solver wouldn't be able to produce a viable solution in such case anyway.

[CSGen] Validate presence of _MaxBuiltinFloatType before generating…

81c3f62

… constraints for float literals

[android][test] Mark sil_combine_alloc_stack as executable.

0271370

The Android CI machines cannot execute the test in the target devices. Marking the test as executable makes the test filtering know that this test needs to be executed in the target device.

Use single quotes instead of backticks in diagnostics

a348640

[sil] Use FrozenMultiMap in PredictableMemOpts instead of implementin…

b44fbae

…g the data structure by hand.

Merge pull request swiftlang#29887 from slavapestov/conformance-cache…

c4203dd

…-sillyness Runtime: Less eager instantiation of type metadata in the conformance cache

Sema: Remove a couple of calls to clearLookupCache()

0555de5

Tests seem to pass without these. Two calls still remain, in ModuleDecl::addFiles() and removeFiles(). It will take a bit more refactoring to eliminate those.

Merge pull request swiftlang#29875 from gottesmm/pr-a905499d41ad40cb3…

8626cc4

…a8a3deb651d2dd2fc3f8ad6 [ownership] Implement movable guaranteed scopes in ossa.

Merge pull request swiftlang#29895 from mdiep/remove-backticks-in-dia…

d8bc35b

…gnostics Use single quotes instead of backticks in diagnostics

Add guidance about cost of var boxes being captured by an escaping cl…

457b5dd

…osure.

[GSB] Delay mapping an invalid subject type to an error type (swiftla…

c435646

…ng#29813)

Merge pull request swiftlang#29902 from gottesmm/pr-c7fabcf6a57817aae…

7063a79

…af73160c99cfaa4641e6b22

Merge pull request swiftlang#29871 from swiftwasm/swiftwasm-stdlib-tests

bb20a87

swift-ci and others added 18 commits February 22, 2020 22:50

Merge pull request swiftlang#30019 from nate-chandler/result-future-d…

86122f7

…isable-watchos-x86-simulator

Merge pull request swiftlang#30017 from 3405691582/TestSerializationA…

cd4f3b0

…ndroidModules_RequiresCpuArm64

[NFC] Replace two uses of 'fileprivate' with 'private'

ae423c5

[docs] Updating doc list for the diagnostics ported to new framework

86f7c6b

[NFC][Foundation] Remove or replace unnecessary uses of 'fileprivate'

1d7ec4c

Merge pull request swiftlang#29984 from dan-zheng/TF-1168

dddcfb8

build: bifurcate _add_swift_executable_single

2e2e886

Merge pull request swiftlang#30023 from LucianoPAlmeida/patch-1

c7552bf

[docs] Updating doc list for the diagnostics ported to new framework

Merge pull request swiftlang#29851 from DougGregor/remove-performance…

d81161a

…-hacks [Constraint solver] Remove performance hacks for pattern type computation

Merge pull request swiftlang#30026 from compnerd/executable-splitting

af97fd1

build: bifurcate `_add_swift_executable_single`

Merge pull request swiftlang#30021 from xwu/private-not-fileprivate

4d59bc5

[NFC] Minimize uses of 'fileprivate' after SE-0169

Implement Float16

50ab652

Add support for simd vectors of Float16, and Mirrors.

46b651c

Add conversions from Float16 to integer types.

9257cd7

Quick implementation of FloatingPointParsing for Float16

9e9deb3

This isn't quite right, because it introduces double-rounding, but it will do in the short-term.

Add codable conformance for Float16, remove AnyHashableBox.

602085f

Add importer support for mapping __fp16 and _Float16 to Float16.

ff14be9

add _swift_stdlib_strtof16 shim

794ab7a

xwu added swift evolution approved Flag → feature: A feature that was approved through the Swift evolution process and removed swift evolution pending discussion Flag → feature: A feature that has a Swift evolution proposal currently in review labels Feb 27, 2020

stephentyrone added 3 commits February 28, 2020 11:22

More float16 work.

f1c369b

Merge branch 'add-float16' of github.com:stephentyrone/swift into add…

c6e9205

…-float16

Add 9999 availability to all Float16 API.

fd5b305

stephentyrone closed this Feb 28, 2020

stephentyrone deleted the add-float16 branch March 9, 2020 17:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DNM] Implement Float16 #21738

[DNM] Implement Float16 #21738

Uh oh!

stephentyrone commented Jan 9, 2019 •

edited

Loading

Uh oh!

stephentyrone commented Jan 9, 2019

Uh oh!

swift-ci commented Jan 9, 2019

Uh oh!

swift-ci commented Jan 9, 2019

Uh oh!

Azoy commented Jan 21, 2019 •

edited

Loading

Uh oh!

Azoy commented Jan 21, 2019

Uh oh!

stephentyrone commented Jan 21, 2019

Uh oh!

stephentyrone commented Jan 21, 2019

Uh oh!

rjmccall commented Jan 21, 2019

Uh oh!

stephentyrone commented Jan 21, 2019

Uh oh!

stephentyrone commented Feb 28, 2020

Uh oh!

Azoy commented Feb 28, 2020

Uh oh!

stephentyrone commented Feb 28, 2020

Uh oh!

Azoy commented Feb 28, 2020

Uh oh!

stephentyrone commented Feb 28, 2020

Uh oh!

Uh oh!

[DNM] Implement Float16 #21738

[DNM] Implement Float16 #21738

Uh oh!

Conversation

stephentyrone commented Jan 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephentyrone commented Jan 9, 2019

Uh oh!

swift-ci commented Jan 9, 2019

Uh oh!

swift-ci commented Jan 9, 2019

Uh oh!

Azoy commented Jan 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Azoy commented Jan 21, 2019

Uh oh!

stephentyrone commented Jan 21, 2019

Uh oh!

stephentyrone commented Jan 21, 2019

Uh oh!

rjmccall commented Jan 21, 2019

Uh oh!

stephentyrone commented Jan 21, 2019

Uh oh!

stephentyrone commented Feb 28, 2020

Uh oh!

Azoy commented Feb 28, 2020

Uh oh!

stephentyrone commented Feb 28, 2020

Uh oh!

Azoy commented Feb 28, 2020

Uh oh!

stephentyrone commented Feb 28, 2020

Uh oh!

Uh oh!

stephentyrone commented Jan 9, 2019 •

edited

Loading

Azoy commented Jan 21, 2019 •

edited

Loading