[AutoDiff] Introduce 'linear_function' and 'linear_function_extract' instructions.

[rxwei]

Introduce linear_function and linear_function_extract instructions, which are used for creating and destructing @differentiable(linear) functions.

linear_function instruction

Bundles a function with its transpose function into a @differentiable(linear) function.

sil-instruction ::= 'linear_function'
                    sil-linear-function-parameter-indices?
                    sil-value ':' sil-type
                    sil-linear-function-transpose-function-clause?

sil-linear-function-parameter-indices ::=
    '[' 'parameters' [0-9]+ (' ' [0-9]+)* ']'
sil-linear-transpose-function-clause ::=
    with_transpose sil-value ':' sil-type

linear_function_extract instruction

Extracts the original function or a transpose function from the given @differentiable(linear) function. It must be provided with an extractee: [original] or [transpose].

sil-instruction ::= 'linear_function_extract'
                    sil-linear-function-extractee
                    sil-value ':' sil-type

sil-linear-function-extractee ::=
    '[' sil-linear-function-extractee ']'
sil-linear-function-extractee-name ::= 'original' | 'transpose'

linear_function_extract [original] %0 : $@differentiable(linear) (T) -> T
linear_function_extract [transpose] %0 : $@differentiable(linear) (T) -> T

Resolves TF-907.

[dan-zheng]

LGTM!

Please add round-trip tests (parsing, (de)serialization, printing) à la differentiable_function_inst.sil.

[dan-zheng]

Minor: Oxford comma please (used below for linear_function) 🙂

Suggested change

	/// `differentiable_function` - given a function, differentiation indices and
	/// `differentiable_function` - given a function, differentiation indices, and

[rxwei]

As mentioned in the PR description, this PR is work in progress.

[rxwei]

There turns out to be a large portion of @differentiable(linear) type lowering that's not handled at all. I'll deal with that first.

[rxwei]

Ok, everything is working now after various fixes in parsing and lowering. I will split this patch into multiple.

[rxwei]

@swift-ci please test tensorflow

[rxwei]

@swift-ci please test tensorflow

[rxwei]

@swift-ci please test tensorflow

[rxwei]

@swift-ci please test tensorflow