Skip to content

[AutoDiff upstream] Add differential operators and some utilities. #30711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Apr 1, 2020
Merged

[AutoDiff upstream] Add differential operators and some utilities. #30711

merged 4 commits into from
Apr 1, 2020

Conversation

rxwei
Copy link
Contributor

@rxwei rxwei commented Mar 29, 2020

  • Add all differential operators.
  • Add withoutDerivative(at:), used for efficiently stopping the derivative propagation at a value and causing the derivative at the value to be zero.
  • Add utility differentiableFunction(from:), used for creating a @differentiable function from an original function and a derivative function.

Mostly work done by @marcrasi and @dan-zheng.
Partially resolves TF-843.

TODO:

  • Add AnyDerivative.
  • Add Array.differentiableMap(_:) and differentiableReduce(_:_:).

@rxwei rxwei requested review from dan-zheng and marcrasi March 29, 2020 09:19
@rxwei
Copy link
Contributor Author

rxwei commented Mar 29, 2020

@swift-ci please smoke test

@rxwei
Copy link
Contributor Author

rxwei commented Mar 29, 2020

Build error:

04:25:08 Failed to reconstruct type for $sq_xIelgnr_D
04:25:08 Original type:
04:25:08 (sil_function_type type=@differentiable(linear) @callee_guaranteed (@in_guaranteed τ_0_1) -> @out τ_0_0
04:25:08   (input=generic_type_param_type depth=0 index=1)
04:25:08   (result=generic_type_param_type depth=0 index=0)
04:25:08   (substitution_map generic_signature=<nullptr>)
04:25:08   (substitution_map generic_signature=<nullptr>))
04:25:08 Stack dump:
04:25:08 0.	Program arguments: /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/buildbot_incremental/swift-macosx-x86_64/bin/swift -frontend -c /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/swift/stdlib/public/Differentiation/Differentiable.swift /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/swift/stdlib/public/Differentiation/DifferentialOperators.swift /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/swift/stdlib/public/Differentiation/DifferentiationUtilities.swift -disable-objc-attr-requires-foundation-module -target x86_64-apple-macosx10.9 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -I /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/buildbot_incremental/swift-macosx-x86_64/./lib/swift/macosx -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/../../../Developer/Library/Frameworks -warn-implicit-overrides -enable-library-evolution -g -module-cache-path /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/buildbot_incremental/swift-macosx-x86_64/./module-cache -module-link-name swift_Differentiation -parse-stdlib -resource-dir /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/buildbot_incremental/swift-macosx-x86_64/./lib/swift -swift-version 5 -O -D INTERNAL_CHECKS_ENABLED -D SWIFT_ENABLE_RUNTIME_FUNCTION_COUNTERS -D SWIFT_ENABLE_EXPERIMENTAL_DIFFERENTIABLE_PROGRAMMING -enable-ownership-stripping-after-serialization -enforce-exclusivity=unchecked -enable-experimental-differentiable-programming -parse-as-library -module-name _Differentiation -o /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/buildbot_incremental/swift-macosx-x86_64/stdlib/public/Differentiation/OSX/x86_64/_Differentiation.o -runtime-compatibility-version none -disable-autolinking-runtime-compatibility-dynamic-replacements 
04:25:08 1.	Apple Swift version 5.3-dev (LLVM 6287796090, Swift 649b83dfe6)
04:25:08 2.	While evaluating request IRGenWholeModuleRequest(IR Generation for module _Differentiation)
04:25:08 3.	While emitting IR SIL function "@$s16_Differentiation9transpose2ofxq_XIq_xXI_tAA14DifferentiableRz13TangentVectorAaDPQzRszAaDR_AeFQy_Rs_r0_lF".
04:25:08  for 'transpose(of:)' (at /Users/buildnode/jenkins/workspace/swift-PR-osx-smoke-test/branch-master/swift/stdlib/public/Differentiation/DifferentialOperators.swift:22:8)
04:25:08 0  swift                    0x000000010828e1d8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
04:25:08 1  swift                    0x000000010828d158 llvm::sys::RunSignalHandlers() + 248
04:25:08 2  swift                    0x000000010828e7cd SignalHandler(int) + 285
04:25:08 3  libsystem_platform.dylib 0x00007fff69c2e5fd _sigtramp + 29
04:25:08 4  libsystem_platform.dylib 000000000000000000 _sigtramp + 18446603338741783072
04:25:08 5  libsystem_c.dylib        0x00007fff69b00808 abort + 120
04:25:08 6  swift                    0x00000001083c8526 (anonymous namespace)::IRGenDebugInfoImpl::getOrCreateType(swift::irgen::DebugTypeInfo) (.cold.2) + 102
04:25:08 7  swift                    0x000000010429421a (anonymous namespace)::IRGenDebugInfoImpl::getOrCreateType(swift::irgen::DebugTypeInfo) + 3626

I haven't been able to reproduce it on macOS...

@marcrasi
Copy link

marcrasi commented Mar 30, 2020 •
edited
Loading

I think that may be https://github.com/apple/swift/blob/9c4212a49b483f455bcd1caf4de6f2ef12a5ad54/include/swift/AST/IRGenOptions.h#L333. I don't know how hard that is to fix for real. We obviously can't check DisableRoundTripDebugTypes(true) into master. Perhaps we could make DisableRoundTripDebugTypes default to true when differentiable programming is enabled.

@dan-zheng
Copy link
Contributor 

04:25:08 Failed to reconstruct type for $sq_xIelgnr_D

I fixed this issue (TF-1225) and added a test in 391a586.
It was a type demangling deficiency for differentiable function types.

This PR should be unblocked now, @marcrasi will take over!

rxwei and others added 3 commits March 30, 2020 14:15
@rxwei
[AutoDiff upstream] Add differential operators and some utilities.
57d228b 
* Add all [differential operators](https://github.com/apple/swift/blob/master/docs/DifferentiableProgramming.md#list-of-differential-operators).
* Add `withoutDerivative(at:)`, used for efficiently stopping the derivative propagation at a value and causing the derivative at the value to be zero.
* Add utility `differentiableFunction(from:)`, used for creating a `@differentiable` function from an original function and a derivative function.

Mostly work done by @marcrasi and @dan-zheng.
Partially resolves TF-843.

TODO:
* Add `AnyDerivative`.
* Add `Array.differentiableMap(_:)` and `differentiableReduce(_:_:)`.
@dan-zheng
Fix differentiable function type demangling.
0218844 
Handle differentiability kind (`@differentiable` and `@differentiable(linear)`)
in `ASTBuilder::createImplFunctionType`.

Resolves TF-1225.
improve differential operator tests
64d0e76
@marcrasi marcrasi force-pushed the differential-operators branch from 391a586 to 64d0e76 Compare March 30, 2020 21:58
@marcrasi
Copy link

I have updated the included test to test all the reverse mode differential operators. (Forward mode ones are less important so I don't think it's worth it to spend more time writing tests for them now.) This is ready now, so I'll do a test then merge it.

@marcrasi
Copy link

@swift-ci please test

@marcrasi
Copy link

@swift-ci please smoke test

2 similar comments
@marcrasi
Copy link

@swift-ci please smoke test

@marcrasi
Copy link

@swift-ci please smoke test

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 672d180da957dd2e77db54f1ca46c3c4a6530823

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 672d180da957dd2e77db54f1ca46c3c4a6530823

@rxwei
Copy link
Contributor Author

rxwei commented Mar 30, 2020

Thanks for taking over the work!

@marcrasi
Copy link

@swift-ci please smoke test

1 similar comment
@marcrasi
Copy link

@swift-ci please smoke test

@marcrasi
Copy link

@swift-ci please smoke test macos

@rxwei
Copy link
Contributor Author

rxwei commented Mar 31, 2020

@swift-ci please smoke test macOS

@marcrasi
Copy link

Hmm, Python/python_lint.swift seems to keep flaking. I guess I'll keep running it until it passes.

@marcrasi
Copy link

@swift-ci please smoke test macOS

2 similar comments
@marcrasi
Copy link

@swift-ci please smoke test macOS

@rxwei
Copy link
Contributor Author

rxwei commented Mar 31, 2020

@swift-ci please smoke test macOS

@marcrasi marcrasi force-pushed the differential-operators branch from b26f252 to 013a66b Compare March 31, 2020 23:25
@marcrasi
Copy link

Aha, it was a python lint error, not actually a flake. Now it should be fixed.

@marcrasi
Copy link

@swift-ci please smoke test

@marcrasi
Copy link

@swift-ci please smoke test

@marcrasi marcrasi merged commit 77fd034 into swiftlang:master Apr 1, 2020
@marcrasi
Copy link

marcrasi commented Apr 1, 2020

We broke the long-test: https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04-long-test/5804/ .

I have a temporary fix that should make the CI green again: #30756

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dan-zheng dan-zheng Awaiting requested review from dan-zheng

@marcrasi marcrasi Awaiting requested review from marcrasi

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@rxwei @marcrasi @dan-zheng @swift-ci