[AutoDiff] Support ref_element_addr differentiation.

[dan-zheng]

Support differentiation of ref_element_addr: class stored property references.

Activity analysis:

• Propagate activity for ref_element_addr like struct_element_addr.
• Do not propagate activity for ref_element_addr to @noDerivative members.

Pullback generation rules:

  Original: y = ref_element_addr x, <n>
   Adjoint: adj[x] += struct (0, ..., #field': adj[y], ..., 0)
                                      ^~~~~~~
                    field in tangent space corresponding to #field

• Add ref_element_addr case to PullbackEmitter::getAdjointProjection.
  • The adjoint projection of a ref_element_addr is a local allocation
    initialized with the corresponding field value from the class's base
    adjoint value.

Exposes TF-1149: cannot differentiate active value with loadable type but
address-only TangentVector type. Diagnose for now.

Resolves SR-12152.

Todos:

• SR-12152: support class initializer differentiation.
• TF-1149: robust fix for original/tangent value category mismatch.
• TF-970: replace invalid TangentVector struct assertions with diagnostics.

---

Example:

class C: Differentiable {
  @differentiable
  var float: Float

  init(_ float: Float) {
    self.float = float
  }

  @differentiable
  func method(_ x: Float) -> Float {
    x * float
  }
}
print(valueWithGradient(at: C(10), 3, in: { c, x in c.method(x) }))

Before:

$ swift tf-645.swift
tf-645.swift:2:4: error: function is not differentiable
  @differentiable
  ~^~~~~~~~~~~~~~
tf-645.swift:3:7: note: when differentiating this function definition
  var float: Float
      ^
tf-645.swift:3:7: note: differentiating class properties is not yet supported
  var float: Float
      ^

After:

$ swift tf-645.swift
value: 30.0, gradient: (C.TangentVector(float: 3.0), 10.0))

[dan-zheng]

@swift-ci Please test tensorflow

[rxwei]

Very nice!

[dan-zheng]

@swift-ci Please test tensorflow

[dan-zheng]

5012f1b fixes incorrect ref_element_addr zero derivatives:

Add `ref_element_addr` case to `PullbackEmitter::getAdjointProjection`.

The adjoint projection of a `ref_element_addr` is a local allocation initialized
with the corresponding field value from the class's base adjoint value.

[dan-zheng]

@swift-ci Please test tensorflow

[dan-zheng]

@swift-ci Please test tensorflow

[rxwei]

We should get rid of this header soon...

[dan-zheng]

I can do it soon :)