[AutoDiff] Remove 'differentiableFunction(from:)' and 'linearFunction(from:)'

[rxwei]

Remove unused APIs differentiableFunction(from:) and linearFunction(from:). They were never official APIs, are not included in the initial proposal, and are unused by existing supported client libraries (SwiftFusion and S4TF). Most importantly, they block crucial optimizations on linear map closures (#34935) and would need nontrivial work in SILGen to support.

[rxwei]

@swift-ci please test

[dan-zheng]

LGTM! I'm not familiar with existing use cases or reasons not to remove - my memory is that you added them as a cute idea.

[swift-ci]

Build failed
Swift Test Linux Platform
Git Sha - ee216181050e4d9c9b5ca7056a581edd95e0ee65

[swift-ci]

Build failed
Swift Test OS X Platform
Git Sha - ee216181050e4d9c9b5ca7056a581edd95e0ee65

[rxwei]

@swift-ci please test

[porterchild]

@rxwei @dan-zheng I was making use of differentiableFunction for checkpointing, is there a way to achieve gradient checkpointing without differentiableFunction? Like here

[porterchild]

Would something like this do it?

@differentiable
func makeRecomputedInGradient<T: Differentiable, U: Differentiable>(
    _ original: @escaping @differentiable(T) -> U
) -> @differentiable(T) -> U {
    
    var gradientHolder: U.TangentVector? = nil
    
    @differentiable
    func newFunc(_ t: T) -> U {
        var t = t
        t = t.withDerivative { $0 = pullback(at: t, in: original)(gradientHolder!) }
        var result = withoutDerivative(at: original(t))
        result = result.withDerivative{ gradientHolder = $0 }
        return result
    }
    
    return newFunc
}

Huh. That crashes the compiler...

[rxwei]

func _original<T, R>(_ x: T, _ f: @differentiable (T) -> R) -> R {
  f(x)
}

@derivative(of: _original, wrt: x)
func _vjp<T, R>(
  _ x: T, _ f: @differentiable (T) -> R
) -> (value: R, pullback: (R.TangentVector) -> T.TangentVector) {
  (value: f(x), pullback: pullback(at: x, in: f))
}

func makeRecomputed<T, R>(
  _ f: @escaping @differentiable (T) -> R
) -> @differentiable (T) -> R {
  { x in _original(x, f) }
}

[porterchild]

Thanks Richard!