[AutoDiff] Create JVPEmitter #25954

bartchr808 · 2019-07-03T01:11:49Z

Creates a very barebones JVPEmitter which copies the body of the original function returning (original, undef), and creates the differential function with a body that returns an undef.

Apologies for the weird commit history, there was something wrong with my tensorflow branch on the fork of this repo that caused this. However, I fixed it now so future PRs won't be this ugly.

…orflow

) (swiftlang#25778) Xcode puts the bin/ and lib/ directories in a subdirectory based on configuration; account for this when copying over compiler-rt for non-host Apple platforms using one of the existing helpers in build-script-impl.

`PointwiseMultiplicative` represents a ring over multiplication with identity element "one" and a multiplicative inverse (reciprocal). Conform `Differentiable` synthesized struct types to `PointwiseMultiplicative` protocol if possible. Enables revamping optimizers, which requires division (i.e. multiplication with multiplicative inverse).

…swiftlang#25803) Partial fix for `@differentiable` function thunk with output opaque abstraction pattern. Avoid `AbstractionPattern` methods that crash for opaque patterns. Small step towards fixing TF-123. Reproducers no longer crash, SIL verification fails instead. Robust fix requires more work - see TF-123 for more details.

tensorflow/swift-apis@60f63e1 `Tensor` now conforms to `PointwiseMultiplicative`.

…orflow

…wiftlang#25967) In VJPEmitter, if the original function call has substitutions, we `partial_apply` it with no arguments to specialize it. This `partial_apply` is not being released. JVP and VJP are being specialized the same way, but they are not being released either. To fix this, we release the `@differentiable` function returned by `autodiff_function`, which will release the original and the associated functions tht are to be filled in later altogether. If the original function does not have substitutions, we retain the original function to balance out the release of the `@differentiable` function that comes later. As a result, `ADContext::promoteToDifferentiableFunction` no longer needs to retain the associated functions. Example where the original function has substitutions: ``` f' = partial_apply f<...>() f_diff = autodiff_function f' release_value f_diff ``` Example where the original function does not have substitutions: ``` retain_value f f_diff = autodiff_function f release_value f_diff ``` Note: This makes the `autodiff_function` folding optimization no longer able to detect the pattern, but it is necessary. We can rewrite the optimization later. This should fix [TF-621](https://bugs.swift.org/browse/TF-621).

bartchr808 · 2019-07-09T15:15:12Z

lib/SILOptimizer/Mandatory/Differentiation.cpp

@@ -5875,108 +6123,104 @@ ADContext::declareExternalAssociatedFunction(
  return assocFn;
 }

-static SILFunction* createJVP(
+static SILFunction *createEmptyVJP(


The git diff here is acting funny in the next several lines, may be hard to follow but I more so made a copy of createEmptyVJP and named it createEmptyJVP and removed the previous createJVP to more closely resemble how VJPEmitter looks like and JVPEmitter will become.

bartchr808 · 2019-07-09T15:48:31Z