[AutoDiff] Fix incorrect derivative

[philipturner]

The VJP derivative of FloatingPoint.addingProduct ignores the value of the pullback, assuming it is 1. This slipped by tests, which call gradient(at:of:) - a function that always passes 1 into the pullback. If the value passed into the pullback is 2, the output gradient will be 1/2 of what it should be.

The test for FloatingPoint.addingProduct has been modified to have two function calls - one which passes 1 into the pullback, and one which passes 2. Due to the magnitude of the test file, I am putting a hold on overhauling the entire thing with this new validation technique. After further discussion, such a change may be added to the pull request.

[BradLarson]

@swift-ci Please test.

[BradLarson]

@dan-zheng - Do you think it would make sense to extend the use of non-1 values in pullback(at:of:) to other tests that just use gradient(at:of:) to make sure we're properly exercising the pullback?

[dan-zheng]

Thanks for the catch and fix!

[dan-zheng]

@dan-zheng - Do you think it would make sense to extend the use of non-1 values in pullback(at:of:) to other tests that just use gradient(at:of:) to make sure we're properly exercising the pullback?

Thanks for the question!

I think it is a case-by-case decision per operation f, whether to test pullback(at: x, of: f) with explicit non-1 seed v values in addition to gradient(at: x, of: f) tests (where the seed is always 1). No need to add these tests proactively, unless you want to check correctness of pullbacks for specific operations.

The purpose would not be to test whether gradient(at:of:) is implemented correctly in terms of pullback(at:of:), but to test whether custom VJP/pullback implementations are correct for non-1 seed values – so the correctness depends per custom VJP/pullback implementation, like for _vjpAddingProduct here.