[AutoDiff] Move stdlib sources to stdlib/public/core/Differentiation.

[dan-zheng]

Move differentiable-programming-related stdlib sources to stdlib/public/core/Differentiation.

Related master branch PR: #27511.
master branch will have a _Differentiation support library containing the Differentiable protocol and related APIs.

On tensorflow branch, the Swift source files in stdlib/public/core/Differentiation will be directly built as part of swiftCore for simplicity. Swift for TensorFlow users can continue to use the Differentiable protocol and related APIs without adding import _Differentiation.

Rename protocol _Differentiable back to protocol Differentiable.

[dan-zheng]

@swift-ci Please test tensorflow

[dan-zheng]

@swift-ci Please test tensorflow

[dan-zheng]

Currently, #27511 adds a test subdirectory called test/Differentiation and a lit configuration "available feature" called differentiable_programming.

---

Two notes:

• It may make sense to merge the subdirectories test/AutoDiff (on tensorflow branch) and test/Differentiation (introduced on master branch) into one.

  • Personally, I prefer the name test/AutoDiff, but test/Differentiation is consistent with stdlib/public/Differentiation, but test/AutoDiff is shorter.

• Since tensorflow branch does not have a _Differentiation module, upstreamed tests need to use a guarded import:

  // RUN: ...
  // REQUIRES: differentiable_programming
  
  // Note: guarded import because `_Differentiation` module does not exist on
  // `tensorflow` branch.
  #if canImport(_Differentiation)
  import _Differentiation
  #endif
  
  // References to `Differentiable` protocol, etc.

  • I'm not sure how to avoid this except to add the _Differentiation module to tensorflow branch, same as master branch. This is a breaking change for Swift for TensorFlow users, but will be a necessary step when differentiable programming is upstreamed.

---

For now, I'll rename test/Differentiation to test/AutoDiff and add the guarded import _Differentiation to test/AutoDiff/differentiable_protocol.swift. Other suggestions are welcome.

[marcrasi]

Could we have an empty _Differentiation module on the tensorflow branch so that we do not have to remember to do canImport(_Differentiation)?

[rxwei]

Have you considered making the _Differentiation module be imported implicitly in this branch?

[dan-zheng]

Have you considered making the _Differentiation module be imported implicitly in this branch?

We considered this - the team made the initial decision to not investigate auto-import logic to avoid the engineering effort and complexity. (A -parse-differentiation driver flag may be needed, similar to -parse-stdlib.)

I think we can revisit this decision though if there are strong reasons for adding an auto-import.

[dan-zheng]

If there are no strong objections, let's proceed by adding an empty _Differentiation module to tensorflow branch for now. We can revisit later!

[dan-zheng]

@swift-ci Please test tensorflow

[dan-zheng]

One failing test on macOS:

Failing Tests (1):
    Swift(macosx-x86_64) :: ParseableInterface/verify_stdlib.swift

Command Output (stdout):
--
/Users/swiftci/jenkins/workspace/swift-PR-TensorFlow-macOS/Ninja-ReleaseAssert+stdlib-Release/swift-macosx-x86_64/lib/swift/macosx/Swift.swiftmodule/x86_64.swiftinterface

--
Command Output (stderr):
--
/Users/swiftci/jenkins/workspace/swift-PR-TensorFlow-macOS/Ninja-ReleaseAssert+stdlib-Release/swift-macosx-x86_64/lib/swift/macosx/Swift.swiftmodule/x86_64.swiftinterface:33522:1: error: conditional conformance of type 'SIMD2<Scalar>' to protocol 'EuclideanDifferentiable' does not imply conformance to inherited protocol 'Differentiable'
extension SIMD2 : Swift.EuclideanDifferentiable where Scalar : Swift.BinaryFloatingPoint, Scalar : Swift.EuclideanDifferentiable, Scalar.TangentVector : Swift.BinaryFloatingPoint {
^
/Users/swiftci/jenkins/workspace/swift-PR-TensorFlow-macOS/Ninja-ReleaseAssert+stdlib-Release/swift-macosx-x86_64/lib/swift/macosx/Swift.swiftmodule/x86_64.swiftinterface:33522:1: note: did you mean to explicitly state the conformance like 'extension SIMD2: Differentiable where ...'?
extension SIMD2 : Swift.EuclideanDifferentiable where Scalar : Swift.BinaryFloatingPoint, Scalar : Swift.EuclideanDifferentiable, Scalar.TangentVector : Swift.BinaryFloatingPoint {
^
...

For some reason, with this patch, x86_64.swiftinterface drops the SIMD2 : Differentiable conformance:

// x86_64.swiftinterface
// Before this patch:
extension SIMD2 : Swift.Differentiable & Swift.EuclideanDifferentiable where Scalar : Swift.BinaryFloatingPoint, Scalar : Swift.EuclideanDifferentiable, Scalar.TangentVector : Swift.BinaryFloatingPoint {
// With this patch:
extension SIMD2 : Swift.EuclideanDifferentiable where Scalar : Swift.BinaryFloatingPoint, Scalar : Swift.EuclideanDifferentiable, Scalar.TangentVector : Swift.BinaryFloatingPoint {

I reproduced this issue upstream and filed a minimal reproducer: SR-11723.

I'll work around the issue for now by breaking up SIMD${n}: Differentiable and SIMD${n}: EuclideanDifferentiable conformances into separate extensions.

[dan-zheng]

@swift-ci Please test tensorflow