[AutoDiff] switch array and floating point derivatives to @derivative(of:)

[marcrasi]

Also I added a test for the FloatingPoint.swift derivatives because there isn't an existing test for them.

Once this and #28933 are merged, there will be no more vjp:s in the stdlib.

[rxwei]

Nice!

Minor comment: The title initially came across as "removing VJPs from stdlib", and I only realized it's removing @differentiable(vjp: ...)s when I looked at the diff. How about rewording the PR title to something like "[AutoDiff] Switch to '@Derivative(of:)' for array and floating point derivative registrations"?

[marcrasi]

@swift-ci please test tensorflow

[dan-zheng]

Is it (or will it eventually be) necessary to keep @differentiable attributes on the public original functions so that the derivatives are registered with public access?

---

We discussed this as part of derivative visibility. Currently, derivative visibility is equal to the access level of the @derivative declaration, but we could potentially decouple it (e.g. by adding syntax like @derivative(of: foo, access: public)).

[rxwei]

We discussed this and preferred making the derivative registration have the same visibility as the derivative function. The only thing that's preventing this is the fact that we do not yet support anonymous function declarations (with func _) and do not want to export named functions as public symbols.

The current code pattern (as it is in the library) will likely not compile in the final design, because when @differentiable is put on a public declaration, it cannot be satisfied with an internal @derivative(of:) registration.

[marcrasi]

@swift-ci please test tensorflow

[marcrasi]

@swift-ci please test tensorflow

[marcrasi]

@swift-ci please test tensorflow

[marcrasi]

@swift-ci please test tensorflow

[marcrasi]

@swift-ci please test tensorflow