Skip to content

[Docs] [AutoDiff] Rename '@nondiff' to '@noDerivative'. #28309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 17, 2019
Merged

[Docs] [AutoDiff] Rename '@nondiff' to '@noDerivative'. #28309

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fa6c07d
[Docs] [AutoDiff] Rename '@nondiff' to '@noDerivative'.
rxwei Nov 17, 2019
01b53db
Remove a redundant paragraph.
rxwei Nov 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 12 additions & 15 deletions docs/DifferentiableProgramming.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1827,15 +1827,12 @@ The `@differentiable` attribute requires the function type it is attached to
have differentiable parameters and results. Each parameter and result must
conform to the `Differentiable` protocol (or `Differentiable &
AdditiveArithmetic` when the attribute is `@differentiable(linear)`) unless it
is marked with `@nondiff`.
is marked with `@noDerivative`.

<p align="center">
<img src="assets/DifferentiableProgramming/differentiable-function-subtyping.png">
</p>

_Note: `@nondiff` stands for "not being differentiated", and will likely be
unified with `@noDerivative`._

#### Type conversion

The subtyping relation among `@differentiable(linear)`, `@differentiable`, and
Expand Down Expand Up @@ -1899,13 +1896,13 @@ let f2: (Float) -> Float = f1
```

A `@differentiable` function can also be converted to a function which is
identical except that more of its parameters are marked with `@nondiff`.
identical except that more of its parameters are marked with `@noDerivative`.

```swift
func addOne(_ x: Float) -> Float { x + 1 }
let f0: @differentiable (Float, Float, Float) -> Float = addOne
let f1: @differentiable (@nondiff Float, Float, Float) -> Float = f0
let f2: @differentiable (@nondiff Float, Float, @nondiff Float) -> Float = f1
let f1: @differentiable (@noDerivative Float, Float, Float) -> Float = f0
let f2: @differentiable (@noDerivative Float, Float, @noDerivative Float) -> Float = f1
```

#### Implied generic constraints
Expand Down Expand Up @@ -1975,33 +1972,33 @@ Neural network trainer objects that store loss functions, e.g.
Like function declarations with a `@differentiable` attribute, differentiable
function values can also be differentiable with respect to a subset of
parameters. This is expressed as part of type information, in `@differentiable`
and `@differentiable(linear)` function types, using a `@nondiff` attribute at
and `@differentiable(linear)` function types, using a `@noDerivative` attribute at
each parameter that is not being differentiated with respect to.

By default, all parameters are being differentiated with respect to. When a
`@nondiff` attribute is specified for a parameter in a `@differentiable`
`@noDerivative` attribute is specified for a parameter in a `@differentiable`
function type, values of this function type are not differentiable (or linear)
with respect to the parameter.

```swift
let f0: @differentiable (Float, Float) -> Float = { $0 * $1 }
let f1: @differentiable(linear) (Float, Float) -> Float = { $0 + $1 }
let f2: @differentiable(linear) (Float, @nondiff Float) -> Float = { $0 * $1 }
let f3: @differentiable (@nondiff Int, Float, @nondiff Int) -> Float = {
let f2: @differentiable(linear) (Float, @noDerivative Float) -> Float = { $0 * $1 }
let f3: @differentiable (@noDerivative Int, Float, @noDerivative Int) -> Float = {
$0 ? Float($1) + $2 : 0
}
```

Differentiability of parameters in a function type is important for type
conversions and is part of the subtyping rule: Any `@differentiable` or
`@differentiable(linear)` function type is a subtype of the same function type
with more `@nondiff` parameters than there originally are.
with more `@noDerivative` parameters than there originally are.

```swift
let f0: @differentiable (Float, Float) -> Float = { $0 * $1 }
_ = f0 as @differentiable (Float, @nondiff Float) -> Float
_ = f0 as @differentiable (@nondiff Float, Float) -> Float
_ = f0 as @differentiable (@nondiff Float, @nondiff Float) -> Float
_ = f0 as @differentiable (Float, @noDerivative Float) -> Float
_ = f0 as @differentiable (@noDerivative Float, Float) -> Float
_ = f0 as @differentiable (@noDerivative Float, @noDerivative Float) -> Float
```

### Differentiable operators
Expand Down