[stdlib] Float16/Intel: Add an explicit Sendable conformance to work around a swiftinterface issue #36669

lorentey · 2021-03-31T01:44:58Z

This extension (introduced in #35264) was placed in a file location where it wasn’t correctly guarded against mentioning Float16 on macOS/x86_64, so the generated .swiftinterface file included a reference to an unavailable declaration. (The dummy stand-in Float16 type that we currently use on Intel macOS.)

Moving the declaration out of the “AnyHashable” section and into a file region that’s more suitable for it (i.e., enclosed in #if !((os(macOS) || targetEnvironment(macCatalyst)) && arch(x86_64))) resolves the issue.

Note: The reason why the original extension produces an unusable .swiftinterface file without triggering a build error is currently a mystery to me.

This is a workaround for a compiler bug that omits the full availability information of an unavailable struct from the implicit Sendable conformance declaration emitted into the generated .swiftinterface file. See the discussion below for details.

rdar://76025365

This extension (introduced in swiftlang#35264) was placed in a file location where it wasn’t correctly guarded against mentioning Float16 on macOS/x86_64, so the generated .swiftinterface file included a reference to an unavailable declaration. (The dummy stand-in Float16 type that we currently use on Intel macOS.) Moving the declaration out of the “AnyHashable” section and into a file region that’s more suitable for it (i.e., enclosed in `#if !((os(macOS) || targetEnvironment(macCatalyst)) && arch(x86_64))`) resolves the issue. rdar://76025365

lorentey · 2021-03-31T01:45:36Z

@swift-ci test

swift-ci · 2021-03-31T02:24:19Z

Build failed
Swift Test Linux Platform
Git Sha - ec7258e

lorentey · 2021-03-31T02:32:54Z

Build failed
Swift Test Linux Platform
Git Sha - ec7258e

Looks like the connection got disconnected.

@swift-ci test linux platform

lorentey · 2021-03-31T02:51:01Z

Windows failures are due to currently flaky concurrency support on MSVC; I'm disabling these tests in #36671.

lorentey · 2021-03-31T02:51:21Z

@swift-ci smoke test windows platform

stephentyrone · 2021-03-31T03:52:05Z

Note: The reason why the original extension produces an unusable .swiftinterface file without triggering a build error is currently a mystery to me.

Please open a bug to track this if you haven't already. =)

swift-ci · 2021-03-31T05:39:57Z

Build failed
Swift Test Linux Platform
Git Sha - ec7258e

lorentey · 2021-03-31T05:42:56Z

22:39:36 FAIL: swift-package-tests :: test-snapshot-binaries/linux_load_commands.py (13 of 31)
22:39:36 ******************** TEST 'swift-package-tests :: test-snapshot-binaries/linux_load_commands.py' FAILED ********************
...
22:39:36     lines = list(reversed(subprocess.check_output([args.read_elf, "-program-headers", lib]).split("\n")[:-1]))
22:39:36 TypeError: a bytes-like object is required, not 'str'

lorentey · 2021-03-31T06:16:37Z

The Linux failure is hopefully addressed by swiftlang/swift-integration-tests#85.

lorentey · 2021-03-31T06:16:52Z

@swift-ci smoke test linux platform

lorentey · 2021-04-01T02:53:10Z

So it turns out this is all wrong. The generated x86_64-apple-macos.swiftinterface file still includes the Float16 extension, even though it is definitely 100% for sure guarded by the same #if !((os(macOS) || targetEnvironment(macCatalyst)) && arch(x86_64))) condition as all the other Float16 extensions in the same file.

Given the following (de-gybbed) input:

// .../swift-macosx-x86_64/stdlib/public/core/8/FloatingPointTypes.swift

#if !((os(macOS) || targetEnvironment(macCatalyst)) && arch(x86_64))
...
@available(macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0, *)
@frozen
public struct Float16 {...}
...
@available(macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0, *)
extension Float16: BinaryFloatingPoint {...}
...
@available(macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0, *)
extension Float16 : Sendable { }
#else

/// A half-precision (16b), floating-point value type.
@frozen
@available(macOS 11, iOS 14, tvOS 14, watchOS 7, *)
@available(macOS, unavailable)
@available(macCatalyst, unavailable)
public struct Float16 {...}

#endif

The generated interface includes the struct declaration from the #else branch and the Sendable conformance from the other branch, but not anything else:

// .../swift-macosx-x86_64/lib/swift/macosx/Swift.swiftmodule/x86_64-apple-macos.swiftinterface
...
@available(macOS 11, iOS 14, tvOS 14, watchOS 7, *)
@available(macOS, unavailable)
@available(macCatalyst, unavailable)
@frozen public struct Float16 {
  @_transparent public init() {
    fatalError("Float16 is not available")
  }
}
...
#if compiler(>=5.3) && $MarkerProtocol
@available(iOS 14, tvOS 14, watchOS 7, *)
@available(macOS, unavailable)
@available(macCatalyst, unavailable)
extension Float16 : Swift.Sendable {}
#endif

I'm guessing those magically emitted conditionals might be interfering with the regular #if logic.

lorentey · 2021-04-01T03:44:59Z

Given the above, this won't fix the original issue, but it may be still worth merging this as a purely decorative change.

lorentey · 2021-04-01T07:13:35Z

Ha, it turns out the puzzle piece I was missing was that the compiler is able (and willing!) to implicitly derive Sendable conformance for types for which it can prove on its own are safe to pass across concurrency domains. As it happens, the dummy, empty @frozen public struct Float16 is perfectly concurrency-safe, so it is implicitly Sendable -- this is why its Sendable conformance appears in the generated .swiftinterface.

The real issue is that the generated conformance declaration omits the subtle, but crucial @available(macOS, introduced: 11) part, which makes for a broken interface. The fix is to add the conformance explicitly:

%  if bits == 16:
@available(macOS 11, iOS 14, tvOS 14, watchOS 7, *)
@available(macOS, unavailable)
@available(macCatalyst, unavailable)
extension ${Self}: Sendable { }
%  end

With this explicit extension, the compiler simply copies the availability declaration to the generated interface, and all is well:

@available(macOS 11, iOS 14, tvOS 14, watchOS 7, *)
@available(macOS, unavailable)
@available(macCatalyst, unavailable)
@frozen public struct Float16 {
  @_transparent public init() {
    fatalError("Float16 is not available")
  }
}

#if compiler(>=5.3) && $MarkerProtocol
@available(macOS 11, iOS 14, tvOS 14, watchOS 7, *)
@available(macOS, unavailable)
@available(macCatalyst, unavailable)
extension Float16 : Swift.Sendable {
}
#endif

Tricky!

…iftinterface issue

lorentey · 2021-04-01T07:54:15Z

@swift-ci test

swift-ci · 2021-04-01T09:28:06Z

Build failed
Swift Test Linux Platform
Git Sha - cb67f45

swift-ci · 2021-04-01T09:46:52Z

Build failed
Swift Test OS X Platform
Git Sha - cb67f45

DougGregor · 2021-04-01T15:59:43Z

@swift-ci please smoke test

lorentey · 2021-04-01T20:29:19Z

These will be addressed by swiftlang/llvm-project#2770:

02:46:16 Failed Tests (5):
02:46:16   lldb-api :: lang/swift/array_tuple_resilient/TestSwiftArrayTupleResilient.py
02:46:16   lldb-api :: lang/swift/generic_objcclasswrapper/TestGenericObjcClassWrapper.py
02:46:16   lldb-api :: lang/swift/swiftieformatting/TestSwiftieFormatting.py
02:46:16   lldb-shell :: SwiftREPL/ResilientArray.test
02:46:16   lldb-shell :: SwiftREPL/ResilientDict.test

lorentey · 2021-04-01T23:11:06Z

@swift-ci smoke test macOS platform

DougGregor · 2021-04-02T02:09:26Z

@swift-ci smoke test macOS

lorentey · 2021-04-02T04:45:50Z

@swift-ci test macOS platform

swift-ci · 2021-04-02T09:23:21Z

Build failed
Swift Test OS X Platform
Git Sha - cb67f45

lorentey requested review from stephentyrone and DougGregor March 31, 2021 01:45

DougGregor approved these changes Mar 31, 2021

View reviewed changes

lorentey mentioned this pull request Mar 31, 2021

[test] Disable some concurrency tests on Windows #36671

Merged

stephentyrone approved these changes Mar 31, 2021

View reviewed changes

lorentey changed the title ~~[stdlib] Fix incorrect Float16 extension~~ [stdlib][NFC] Adjust location of Sendable decl for floating point types Apr 1, 2021

[stdlib] Float16: Add explicit Sendable conformance to work around sw…

cb67f45

…iftinterface issue

lorentey changed the title ~~[stdlib][NFC] Adjust location of Sendable decl for floating point types~~ [stdlib] Float16/Intel: Add an explicit Sendable conformance to work around a swiftinterface issue Apr 1, 2021

DougGregor approved these changes Apr 1, 2021

View reviewed changes

stephentyrone approved these changes Apr 1, 2021

View reviewed changes

xwu mentioned this pull request Apr 1, 2021

[stdlib][SR-7556] Re-implement string-to-integer parsing #36623

Merged

lorentey merged commit 63d0b73 into swiftlang:main Apr 2, 2021

lorentey deleted the fix-float16-sendable branch April 2, 2021 08:37

[stdlib] Float16/Intel: Add an explicit Sendable conformance to work around a swiftinterface issue #36669

[stdlib] Float16/Intel: Add an explicit Sendable conformance to work around a swiftinterface issue #36669

Uh oh!

Conversation

lorentey commented Mar 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lorentey commented Mar 31, 2021

Uh oh!

swift-ci commented Mar 31, 2021

Uh oh!

lorentey commented Mar 31, 2021

Uh oh!

lorentey commented Mar 31, 2021

Uh oh!

lorentey commented Mar 31, 2021

Uh oh!

stephentyrone commented Mar 31, 2021

Uh oh!

swift-ci commented Mar 31, 2021

Uh oh!

lorentey commented Mar 31, 2021

Uh oh!

lorentey commented Mar 31, 2021

Uh oh!

lorentey commented Mar 31, 2021

Uh oh!

lorentey commented Apr 1, 2021

Uh oh!

lorentey commented Apr 1, 2021

Uh oh!

lorentey commented Apr 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lorentey commented Apr 1, 2021

Uh oh!

swift-ci commented Apr 1, 2021

Uh oh!

swift-ci commented Apr 1, 2021

Uh oh!

DougGregor commented Apr 1, 2021

Uh oh!

lorentey commented Apr 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lorentey commented Apr 1, 2021

Uh oh!

DougGregor commented Apr 2, 2021

Uh oh!

lorentey commented Apr 2, 2021

Uh oh!

swift-ci commented Apr 2, 2021

Uh oh!

Uh oh!

lorentey commented Mar 31, 2021 •

edited

Loading

lorentey commented Apr 1, 2021 •

edited

Loading

lorentey commented Apr 1, 2021 •

edited

Loading