[AutoDiff upstream] Add TangentSpace.

[dan-zheng]

TangentSpace represents the tangent space of a type.

• For Differentiable-conforming types:
  • The tangent space is the TangentVector associated type.
• For tuple types:
  • The tangent space is a tuple of the elements' tangent space types, for the
    elements that have a tangent space.
• For function types whose innermost result type has a tangent space:
  • The tangent space is the same function type, replacing the innermost
    result type with its tangent space type.
• Other types have no tangent space.

TangentSpace is used to:

• Compute the derivative function type of a given original function type.
• Compute the type of tangent/adjoint values during automatic differentiation.

---

Progress towards TF-828: upstream @differentiable attribute type-checking.
@differentiable attribute type-checking mega-patch: #29091

---

Tests exist on tensorflow branch and will be upstreamed later.
Currently, not enough code has been upstreamed for meaningful testing.

[rxwei]

For function types whose innermost result type has a tangent space:

• The tangent space is the same function type, replacing the innermost
  result type with its tangent space type.

Function is a dead case that should be eliminated before upstreaming. Function's tangent space is AnyDerivative and will be implemented later.

[dan-zheng]

Function is a dead case that should be eliminated before upstreaming. Function's tangent space is AnyDerivative and will be implemented later.

Thanks for the info! I've removed the Function case, could you please re-review?

[dan-zheng]

@swift-ci Please test

[swift-ci]

Build failed
Swift Test Linux Platform
Git Sha - 60290a4214632e121a8d7d975eded0332982aeb8

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - 60290a4214632e121a8d7d975eded0332982aeb8

[dan-zheng]

macOS CI seems to have flaked (error when cloning sources). Retriggering.
@swift-ci Please smoke test macOS platform

[dan-zheng]

@swift-ci Please smoke test macOS platform