Merge branch 'tensorflow' into tensorflow-merge #27745
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Inverse trigonometric functions. * Exponents and logarithms. * Hyperbolic functions. * Error functions ( erf, erfc ). * Free generic functions: sqrt, fma. Partially resolves [TF-812](https://bugs.swift.org/browse/TF-812).
Previously, `LinearMapInfo::addLinearMapToStruct` did not remap `apply` callee type in derivative context. Now, remapping is done. Remapping is significant when the derivative has a more constrained generic signature. Resolves TF-817.
Rename the `Differentiable` protocol to `_Differentiable`. This matches upstreaming PR #27511, which adds `protocol _Differentiable` in a new `Differentiable.swift` file. `tensorflow` branch defines `typealias Differentiable = _Differentiable` for usability. There should be no user-facing changes, except that some error messages now show '_Differentiable' instead of 'Differentiable'.
…#27579) The differentiation order field in `differentiable_function` and `differentiable_function_extract` instructions is unsupported and will not be used by the current design. Quite a lot of dead code exists to try to handle `order`, but it is mostly incomplete and untested. This PR removes the differentiation order from the code base to simplify what we upstream to the 'master' branch. Changes include: * Remove `differentiationOrder` from `DifferentiableFunctionInst` and `DifferentiableFunctionExtractInst`. * Make `DifferentiableFunctionInst::DifferentiableFunctionInst` take an optional pair of JVP and VJP instead of a variable-size array. * Rename "associated functions" to "derivative functions" in `DifferentiableFunctionInst` to align better with [the design](https://forums.swift.org/t/differentiable-programming-mega-proposal/28547). Filed task [TF-882](https://bugs.swift.org/browse/TF-882) to track the renaming of all other occurrences of "associated functions". Resolves [TF-880](https://bugs.swift.org/browse/TF-880).
…rentiation.cpp. (#27597)
…7603) `assocFn` -> `derivativeFn` `AssocFn` -> `DerivativeFn` `assocFunc` -> `derivativeFunc` `AssocFunc` -> `DerivativeFunc` `associatedFunction` -> `derivativeFunction` `AssociatedFunction` -> `DerivativeFunction` `autodiff associated function` -> `derivative function` `autodiff-associated function` -> `derivative function` `AD associated function` -> `derivative function` `associated differentiation function` -> `derivative function` This is a follow-up to #27597. Resolves [TF-882](https://bugs.swift.org/browse/TF-882).
Per @DougGregor's review comment (#27555 (comment)), `AutoDiffIndexSubset` data structure could have a more general name. We rename it to `IndexSubset` and move it to its own file.
…#27613) Type-check `@differentiable` attributes during `TypeChecker::validateDecl` for all relevant declaration kinds (initializers, subscripts, variables), not just function declarations. Resolves TF-888. TF-789 tracks proper request-based type-checking for `@differentiable` attribute. Exposes TF-892: `ElementaryFunctions` linker error on Linux.
…e. (#27627) - Move some 'IndexSubset' method implemenetations from AutoDiff.cpp to a new file called IndexSubset.cpp. - Fix some indentation errors caused by the earlier renaming of `AutoDiffIndexSubset`.
…27638) `ValueOwnershipKindClassifier` and `OperandOwnershipKindClassifier` should have the same classification for `autodiff_function_extract`. `ValueOwnershipKindClassifier`'s classification was fixed by #27199, which gave the correct ownership verification results. Now we fix it in `OperandOwnershipKindClassifier`. `DifferentiableFunctionExtractOriginalExpr`'s SILGen is now corrected to borrowing the argument and emitting a copy for the extracted original function.
) Handle the case where `@differentiable(linear)` is followed by a type alias or another type attribute. ```swift typealias linear = (Float) -> Float let property: @differentiable(linear) linear // okay let property: @differentiable(linear) @convention(c) linear // okay ``` Resolves [TF-576](https://bugs.swift.org/browse/TF-576).
Previously, `@differentiable` attribute SILGen lowering was skipped if the attribute did not have resolved parameter indices. This led to unexpected and difficult-to-debug bugs like TF-888. Re-add the assertion for robustness. This is possible after #27613, which fixed the underlying issue with parameter indices resolution.
SIL differentiability witnesses are a new top-level SIL construct mapping "original" SIL functions to derivative SIL functions. They will replace SIL function `[differentiable]` attributes, additionally enabling cross-module retroactive derivative registration. SIL differentiability witnesses have the following components: - Original `SILFunction`. - Linkage. - Parameter indices (`IndexSubset`). - Result indices (`IndexSubset`). - Derivative generic signature (optional). - JVP `SILFunction` (optional). - VJP `SILFunction` (optional). - "Is serialized?" bit. This patch adds the `SILDifferentiabilityWitness` data structure, along with parsing, printing, verification, and serialization (including lookup by key). The TF-866 master issue tracks follow-up, including SILGen and differentiation transform changes.
…eRepr'. (#27656) ASTGen did not know how to handle `@differentiable(linear)`, which caused parsed `@differentiable(linear)` function types to lose the "linear" bit when they are an argument or a result in a SIL function's type signature. This patch teaches ASTGen to check for `linear` in `TypeAttributes` when generating AST. Resolves [TF-901](https://bugs.swift.org/browse/TF-901).
… type serialization. (#27659) Before `DifferentiabilityKind` was introduced, a function was either `@differentiable` or not `@differentiable`. `DifferentiabilityKind` added a third option: `@differentiability(linear)`. SIL function type serialization did not care about the specific `DifferentiabilityKind` but just whether the function is differentiable -- it is now fixed in this patch. Resolves [TF-903](https://bugs.swift.org/browse/TF-903).
…27661) This patch adds support for lowering `@differentiable(linear)` function types to LLVM IR. * Add `SILFunctionType` transpose type calculation utility: `SILFunctionType:: getAutoDiffTransposeFunctionType`. * Refactor `TypeClassifierBase` methods used for computing recursive properties of `@differentiable` function types and eliminated the need for repeatedly checking differentiability kind. * Add `LinearDifferentiableSILFunctionTypeLowering`. Rename the original `DifferentiableSILFunctionTypeLowering` to `NormalDifferentiableSILFunctionTypeLowering` to make clear it's not for linear functions. * Add `TypeConverter::convertLinearDifferentiableFunctionType`. Rename the original `TypeConverter::convertDifferentiableFunctionType` to `TypeConverter::convertNormalDifferentiableFunctionType` to make clear it's not for linear functions. Resolves [TF-902](https://bugs.swift.org/browse/TF-902).
…instructions. (#27637) Introduce `linear_function` and `linear_function_extract` instructions, which are used for creating and destructing `@differentiable(linear)` functions. ### `linear_function` instruction Bundles a function with its transpose function into a `@differentiable(linear)` function. ``` sil-instruction ::= 'linear_function' sil-linear-function-parameter-indices? sil-value ':' sil-type sil-linear-function-transpose-function-clause? sil-linear-function-parameter-indices ::= '[' 'parameters' [0-9]+ (' ' [0-9]+)* ']' sil-linear-transpose-function-clause ::= with_transpose sil-value ':' sil-type ``` ### `linear_function_extract` instruction Extracts the original function or a transpose function from the given ``@differentiable(linear)`` function. It must be provided with an extractee: ``[original]`` or ``[transpose]``. ``` sil-instruction ::= 'linear_function_extract' sil-linear-function-extractee sil-value ':' sil-type sil-linear-function-extractee ::= '[' sil-linear-function-extractee ']' sil-linear-function-extractee-name ::= 'original' | 'transpose' linear_function_extract [original] %0 : $@differentiable(linear) (T) -> T linear_function_extract [transpose] %0 : $@differentiable(linear) (T) -> T ``` Resolves [TF-907](https://bugs.swift.org/browse/TF-907).
Diagnose unsupported forward-mode control flow instead of crashing.
…7687) - Support the following implicit conversions: - Non-differentiable to `@differentiable(linear)` - Sema: Emit `LinearFunctionExpr`. - SILGen: Lower `LinearFunctionExpr` to `linear_function`. - `@differentiable(linear)` to non-differentiable - Sema: Emit `LinearFunctionExtractOriginalExpr`. - SILGen: Lower `LinearFunctionExtractOriginalExpr` to `linear_function_extract [original]`. - Reject the following implicit conversions: - `@differentiable` to `@differentiable(linear)` - This conversion is not rejected because a `@differentiable` function can never come directly from a closure expression or from a declaration/member reference. - `@differentiable(linear)` to `@differentiable` ([TF-908](https://bugs.swift.org/browse/TF-908)) - This is supported by design, but is not yet implemented due to its complexity. This requires thunking `@differentiable(linear)` to a derivative (JVP) function where the derivative function returns the original result and the same linear map with `@nondiff` parameters partial-applied away. - Properly handle `linear_function`, `differentiable_function_extract`, and `linear_function_extract` in `swift::isGuaranteedForwardingValueKind` and `swift::isOwnershipForwardingValueKind`. This fixed a previously uncaught assertion, which was because `differentiable_function_extract` was in `swift::isOwnershipForwardingValueKind` while it should really be in `swift::isGuaranteedForwardingValueKind`. For complete conversion rules among `@differentiable` functions, see [Differentiable Programming Mega-Proposal - Type conversion](https://github.com/dan-zheng/swift/blob/differentiable-programming/docs/DifferentiableProgramming.md#type-conversion). Resolves [TF-900](https://bugs.swift.org/browse/TF-900).
…el type. (#27688) Many places in the compiler that are completely unrelated to `DifferentiableFunctionExtractInst` are using `DifferentiableFunctionExtractInst::Extractee`, including `@differentiable` type lowering (IRGen/GenDiffFunc.cpp). This patch refactors it and renames it to `NormalDifferentiableFunctionTypeComponent` so that it is no longer part of `DifferentiableFunctionInst`. Resolves [TF-904](https://bugs.swift.org/browse/TF-904).
In `differentiable_function` instruction's syntax, use `parameters` instead of `wrt` and 'with_derivative' instead of `with`. This is to align with `linear_function` instruction and the future direction that both parameter indices and result indices will be included in this instruction. Note: `with_derivative` is not named `with_derivatives` because VJPs will be dropped from this instruction when we complete JVP + linear map transposition. Resolves [TF-909](https://bugs.swift.org/browse/TF-909).
…lation. (#27711) When we differentiate a function (example below) with respect to a proper subset of its indirect parameters and when the function only has a derivative with respect to a proper superset of those indirect parameters, the pullback returns more indirect results than we need. However, unneeded indirect results are not destroyed, which causes a memory lifetime verification failure. This patch fixes this bug by releasing all pullback indirect results instead of just releasing the ones needed for calculating the derivative. ```swift @differentiable(wrt: x) func foo<T: Differentiable>(_ x: T, _ y: T, apply: @differentiable (T, T) -> T) -> T { return apply(x, y) } ``` This patch also uncomments a test in test/AutoDiff/superset_adjoint.swift which is now passing. This fixed a FIXME. Resolves [TF-914](https://bugs.swift.org/browse/TF-914).
Generate SIL differentiability witnesses from AST `@differentiable` attributes, using lowered parameter indices, result indices (currently with capacity 1 and set index 0), and derivative generic signature. Resolves TF-869. The TF-866 master issue tracks all retroactive derivative registration tasks.
|
I had to resolve merge conflicts in: include/swift/AST/Attr.h Also, I made the |
|
@swift-ci please clean test tensorflow |
rxwei
approved these changes
Oct 17, 2019
dan-zheng
approved these changes
Oct 17, 2019
|
I don't believe CI is strictly necessary on merge PRs other than the final |
|
Oh I thought it's to the tensorflow branch. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Bring the new changes in the tensorflow branch.