Skip to content

[AutoDiff] Replace SIMD use of @differentiable(jvp:vjp:) with @derivative(of:) #28930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 4, 2020

Conversation

bartchr808
Copy link
Contributor

@bartchr808 bartchr808 commented Dec 22, 2019

With the change of using @derivative(of:) as done in this PR, I started off with some parts of TF-1085 by updating SIMD.

@bartchr808 bartchr808 added the tensorflow This is for "tensorflow" branch PRs. label Dec 22, 2019
@bartchr808
Copy link
Contributor Author

@swift-ci Please test tensorflow

@bartchr808 bartchr808 changed the title [AutoDiff] WIP: Replace SIMD use of @differentiable(jvp:vjp:) with @derivative(of:) [AutoDiff] Replace SIMD use of @differentiable(jvp:vjp:) with @derivative(of:) Jan 4, 2020
@dan-zheng
Copy link
Contributor

ParseableInterface/verify_stdlib.swift fails due to TF-1094:

/Users/swiftci/jenkins/workspace/swift-PR-TensorFlow-macOS/Ninja-ReleaseAssert+stdlib-Release/swift-macosx-x86_64/lib/swift/macosx/Swift.swiftmodule/x86_64.swiftinterface:23857:4: error: function is not differentiable
  @differentiable(wrt: value where Self : Swift.Differentiable, Self == Self.TangentVector, Self.Scalar : Swift.BinaryFloatingPoint, Self.Scalar : Swift.Differentiable, Self.Scalar == Self.Scalar.TangentVector)
  ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/swiftci/jenkins/workspace/swift-PR-TensorFlow-macOS/Ninja-ReleaseAssert+stdlib-Release/swift-macosx-x86_64/lib/swift/macosx/Swift.swiftmodule/x86_64.swiftinterface:23858:24: note: when differentiating this function definition
  @_transparent public init(repeating value: Self.Scalar) {
                       ^
/Users/swiftci/jenkins/workspace/swift-PR-TensorFlow-macOS/Ninja-ReleaseAssert+stdlib-Release/swift-macosx-x86_64/lib/swift/macosx/Swift.swiftmodule/x86_64.swiftinterface:23860:14: note: cannot differentiate through a non-differentiable result; do you want to use 'withoutDerivative(at:)'?
    for i in indices { self[i] = value }
             ^
<unknown>:0: error: failed to load module 'Swift'

The reason is SIMD.init(repeating:) is serialized but SIMD._vjpInit(repeating:) is not. This causes SIMD._vjpInit(repeating:) to not be registered as a derivative during .swiftinterface compilation.

A workaround is to mark SIMD._vjpInit as @inlinable to make it serialized. We can visit a robust solution for TF-1094 later.

Mark `SIMD._vjpInit(repeating:)` as `@inlinable`.
@dan-zheng
Copy link
Contributor

@swift-ci Please test tensorflow

@dan-zheng
Copy link
Contributor

@swift-ci Please test tensorflow macOS

@dan-zheng dan-zheng merged commit 743b6f5 into swiftlang:tensorflow Jan 4, 2020
@bartchr808 bartchr808 deleted the simd-derivative-attr branch January 4, 2020 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tensorflow This is for "tensorflow" branch PRs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants