[AutoDiff upstream] Add stdlib `Differentiable` conformances and derivatives. #30875

dan-zheng · 2020-04-08T10:03:40Z

Add Differentiable conformances and derivatives for stdlib types and functions to the _Differentiation module. Most upstreaming work done by @ematejska!

`Differentiable` conformances

FloatingPoint-conforming types: Float16, Float, Double, Float80
Array where Element: Differentiable
SIMD-conforming types where Scalar: Differentiable & BinaryFloatingPoint, Scalar.TangentVector: BinaryFloatingPoint

Derivatives

Floating-point operations: +, +=, -, -=, *, *=, /, /=
Array operations: Array.init(repeating:count:), Array.subscript.get, Array.+(_:_:)
SIMD operations: +, -, *, /
Most tgmath operations: exp, log, cos, sin, etc.

Differentiable collection higher-order functions:

Array.differentiableMap(_:), Array.differentiableReduce(_:_:)

Add end-to-end derivative tests.

Todos:

Differentiable.zeroTangentVector protocol requirement and implementations.
- TF-1238: to be upstreamed.
SIMD.sum differentiation.
- TF-1103: @_alwaysEmitIntoClient derivative registration.

…ysEmitIntoClient since that's not supported yet.

- Standardize filenames: `XXXDifferentiation.swift`. - Use Pascal or snake case consistently. - Formatting changes.

…m_conformances_derivatives_master

Add `@available` attributes for declarations using `Float16`.

…rmances. Move `Differentiable` conformances out of Differentiable.swift. All `Differentiable` conformances exist in separate files.

Temporarily disable not-yet-supported differentiation tests: - Forward-mode differentiation. - TF-1237: to be upstreamed. - `Differentiable.zeroTangentVector`. - TF-1238: to be upstreamed. - `SIMD.sum` differentiation. - TF-1103: `@_alwaysEmitIntoClient` derivative registration.

…ions. Test `Array.differentiableMap(_:)` and `Array.differentiableReduce(_:_:)`.

dan-zheng · 2020-04-08T10:04:11Z

@swift-ci Please test

swift-ci · 2020-04-08T10:33:02Z

Build failed
Swift Test Linux Platform
Git Sha - b6d7653

swift-ci · 2020-04-08T11:11:20Z

Build failed
Swift Test OS X Platform
Git Sha - b6d7653

dan-zheng · 2020-04-08T12:36:14Z

Two tests fail with -O due to SR-12548: SILCombiner::visitPartialApplyInst SIL verification failure. I'll submit a fix soon.

ematejska · 2020-04-08T16:40:05Z

stdlib/public/Differentiation/FloatingPointDifferentiation.swift.gyb

+//===----------------------------------------------------------------------===//
+
+import Swift
+import SwiftShims


We might not need to import SwiftShims on master. What do you think? Aside from that looks good.

ematejska

Looks like there are a couple of test failures still.

…m-stdlib-differentiation

dan-zheng · 2020-04-08T21:46:44Z

Test failures due to SR-12548 fixed in #30877.
@swift-ci Please test

swift-ci · 2020-04-08T23:32:22Z

Build failed
Swift Test OS X Platform
Git Sha - 01db90d

dan-zheng · 2020-04-08T23:56:44Z

Fixed test failures for 32-bit platforms.
@swift-ci Please test

swift-ci · 2020-04-08T23:59:28Z

Build failed
Swift Test Linux Platform
Git Sha - 01db90d

swift-ci · 2020-04-09T00:00:39Z

Build failed
Swift Test OS X Platform
Git Sha - 01db90d

- Remove unnecessary imports from test/AutoDiff/stdlib/simd.swift. - Use platform-correct `Builtin` integer type in `PullbackEmitter::getArrayAdjointElementBuffer`.

dan-zheng · 2020-04-09T02:17:34Z

@swift-ci Please test

swift-ci · 2020-04-09T02:20:14Z

Build failed
Swift Test OS X Platform
Git Sha - ffbe041d8c4eafa4e463b71849d58251c183c713

swift-ci · 2020-04-09T02:20:23Z

Build failed
Swift Test Linux Platform
Git Sha - ffbe041d8c4eafa4e463b71849d58251c183c713

dan-zheng · 2020-04-09T02:45:08Z

@swift-ci Please test macOS

dan-zheng · 2020-04-09T05:02:33Z

@swift-ci Please test Windows

The added test was not correctly indicated as a running test.

test: repair the Android CI after #30875

New files were added in swiftlang#30875 which did not include os(OpenBSD), so add this. add_swift_target_library in AddSwiftStdlib subsequently required modification. _add_target_variant_link_flags likely needs modification as well, but this is better suited to a separate PR.

@available

* [build-script] Add the installation prefix to the toolchain path In several places, it was assumed to be /usr and hard-coded to that. This makes sure the installation prefix actually passed in is used. * [stdlib] StaticString: extension per conformance * Fix @available(macCatalyst N, iOS N+M) bug Previously, availability checking computed a declaration’s availability as the intersection of all @available attributes with active platforms. This meant that Swift would not allow you to use an API that was available earlier in a child platform than in its parent until it was also available in the parent platform. That was incorrect. This PR corrects availability checking to find the most specific @available attribute with an introduced version and use that instead. Fixes rdar://60892534. * [stdlib] Buildfix differentiation for OpenBSD. New files were added in swiftlang#30875 which did not include os(OpenBSD), so add this. add_swift_target_library in AddSwiftStdlib subsequently required modification. _add_target_variant_link_flags likely needs modification as well, but this is better suited to a separate PR. * Handle multiple @available attributes correctly The previous commit broke a promise made by the implementation it replaced: if there were two or more @available(introduced:) attributes for the same platform, the greatest version wins. This commit restores that property while still completely overriding the parent platform’s availability with a child platform’s. * [Diagnostics] Provide anchors on demand instead of storing in `FailureDiagnostic` This decouples `FailureDiagnostic` from expression. * [Diagnostics] Switch `get{Raw}Anchor` to return `TypedNode` In preparation to anchor `ConstraintLocator` from `TypedNode` let's refactor diagnostics (which is the biggest user of locators) to support `TypedNode` instead of `Expr *`. * [Diagnostics] Switch `getConstraintLocator` variants to accept `TypedNode` * [Diagnostics] Switch `getContextualType*` to use `TypedNode` * [Diagnostics] Switch `getType` to use `TypedNode` * [Diagnostics] Resolve const-ness problems introduced by `TypedNode` Since `TypedNode` elements are all marked as `const` diagnostics need to get some of the APIs adjusted to support passing `const Expr *`. * Add compiler option to *disable* warnings-as-errors `-no-warnings-as-errors` This functionality is required for build systems to be able to overload/disable a given Swift project's preference of treating warnings as errors. Resolves rdar://problem/35699776 * [Diagnostics] Add helper functions to work with anchors - `{castTo, getAs, is}Expr` * [Diagnostics] Adjust `getCallInfo` to accept `TypedNode` instead of expression * Avoid unnecessary performTypeChecking() in REPL * [SIL Diagnostics] Improve diagnostics for capture of inout values within escaping autoclosures. <rdar://problem/60551844> * [NFC] Refactor validateTBDIfNeeded Inline the conditions for the astGuaranteedToCorrespondToSIL parameter and clean up the logic here to be more const-friendly. Also try to centralize and document the reasons we skip TBD validation. * Revert "[Darwin] Further restrict inference of the simulator environment" * [Gardening] Const-qualify CompilerInstance::getInvocation() * [NFC] Formalize isModuleExternallyConsumed Yank this parameter and document the historical scope creep of this value. * More StringRef -> std::string conversion fixes. This is needed for the master-next branch. * Devirtualizer: fix a miscompile due to handling of cast instructions. getInstanceWithExactDynamicType returns a new instance and for this the class decl has to be updated. https://bugs.swift.org/browse/SR-12538 rdar://problem/61911112 * [NFC] Fix typo in SerializedModuleLoader.cpp Co-authored-by: Butta <[email protected]> Co-authored-by: Ben Rimmington <[email protected]> Co-authored-by: Brent Royal-Gordon <[email protected]> Co-authored-by: 3405691582 <[email protected]> Co-authored-by: Pavel Yaskevich <[email protected]> Co-authored-by: Artem Chikin <[email protected]> Co-authored-by: Rintaro Ishizaki <[email protected]> Co-authored-by: Ravi Kandhadai <[email protected]> Co-authored-by: Robert Widmann <[email protected]> Co-authored-by: Saleem Abdulrasool <[email protected]> Co-authored-by: Mishal Shah <[email protected]> Co-authored-by: Michael Forster <[email protected]> Co-authored-by: swift-ci <[email protected]> Co-authored-by: Erik Eckstein <[email protected]> Co-authored-by: Ravi Kandhadai <[email protected]> Co-authored-by: Pavel Yaskevich <[email protected]>

ematejska and others added 9 commits April 2, 2020 19:55

First pass at upstreaming Differentiable conformances and derivatives

f07ae89

Merge branch 'master' of https://github.com/apple/swift

b72aca9

fixing up some tests and turning off SIMD extension which uses @_alwa…

36cf566

…ysEmitIntoClient since that's not supported yet.

[AutoDiff upstream] Gardening.

da36555

- Standardize filenames: `XXXDifferentiation.swift`. - Use Pascal or snake case consistently. - Formatting changes.

Merge branch 'master' of github.com:apple/swift into autodiff_upstrea…

e377436

…m_conformances_derivatives_master

[AutoDiff upstream] Add Float16 derivatives.

ece4fb7

Add `@available` attributes for declarations using `Float16`.

[AutoDiff upstream] Deduplicate floating-point Differentiable confo…

2c11214

…rmances. Move `Differentiable` conformances out of Differentiable.swift. All `Differentiable` conformances exist in separate files.

[AutoDiff upstream] Test differentiable collection higher-order funct…

b6d7653

…ions. Test `Array.differentiableMap(_:)` and `Array.differentiableReduce(_:_:)`.

dan-zheng requested review from ematejska, marcrasi and rxwei April 8, 2020 10:03

dan-zheng mentioned this pull request Apr 8, 2020

[SILOptimizer] Disable SILCombiner::visitPartialApplyInst for method callees. #30877

Merged

ematejska approved these changes Apr 8, 2020

View reviewed changes

ematejska reviewed Apr 8, 2020

View reviewed changes

rxwei approved these changes Apr 8, 2020

View reviewed changes

Merge branch 'master' of github.com:apple/swift into autodiff-upstrea…

01db90d

…m-stdlib-differentiation

Fix tests for 32-bit platforms.

4495f6b

- Remove unnecessary imports from test/AutoDiff/stdlib/simd.swift. - Use platform-correct `Builtin` integer type in `PullbackEmitter::getArrayAdjointElementBuffer`.

dan-zheng force-pushed the autodiff-upstream-stdlib-differentiation branch from ffbe041 to 4495f6b Compare April 9, 2020 02:17

dan-zheng merged commit 75548c3 into master Apr 9, 2020

dan-zheng deleted the autodiff-upstream-stdlib-differentiation branch April 9, 2020 05:31

compnerd added a commit to compnerd/apple-swift that referenced this pull request Apr 9, 2020

test: repair the Android CI after swiftlang#30875

6f28e54

The added test was not correctly indicated as a running test.

compnerd added a commit that referenced this pull request Apr 9, 2020

Merge pull request #30925 from compnerd/30875

1d7b3a5

test: repair the Android CI after #30875

3405691582 mentioned this pull request Apr 15, 2020

[stdlib] Buildfix differentiation for OpenBSD. #31033

Merged

[AutoDiff upstream] Add stdlib Differentiable conformances and derivatives. #30875

[AutoDiff upstream] Add stdlib Differentiable conformances and derivatives. #30875

Uh oh!

Conversation

dan-zheng commented Apr 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Differentiable conformances

Derivatives

Uh oh!

dan-zheng commented Apr 8, 2020

Uh oh!

swift-ci commented Apr 8, 2020

Uh oh!

swift-ci commented Apr 8, 2020

Uh oh!

dan-zheng commented Apr 8, 2020

Uh oh!

ematejska Apr 8, 2020

Choose a reason for hiding this comment

Uh oh!

ematejska left a comment

Choose a reason for hiding this comment

Uh oh!

dan-zheng commented Apr 8, 2020

Uh oh!

swift-ci commented Apr 8, 2020

Uh oh!

dan-zheng commented Apr 8, 2020

Uh oh!

swift-ci commented Apr 8, 2020

Uh oh!

swift-ci commented Apr 9, 2020

Uh oh!

dan-zheng commented Apr 9, 2020

Uh oh!

swift-ci commented Apr 9, 2020

Uh oh!

swift-ci commented Apr 9, 2020

Uh oh!

dan-zheng commented Apr 9, 2020

Uh oh!

dan-zheng commented Apr 9, 2020

Uh oh!

Uh oh!

[AutoDiff upstream] Add stdlib `Differentiable` conformances and derivatives. #30875

[AutoDiff upstream] Add stdlib `Differentiable` conformances and derivatives. #30875

dan-zheng commented Apr 8, 2020 •

edited

Loading

`Differentiable` conformances