[Tests] Port tensor control flow AD tests

[jon-tow]

#408

[rxwei]

Was it tested locally?

[jon-tow]

@rxwei Yup. It surely worked on the previous nightly toolchain:
(Swift version 5.1-dev (LLVM 200186e28b, Swift b2720a3460)
I now get the error after updating to the latest nightly.

I'm not sure why testConcatenationPlusPlus() is failing because the return expression in the gradient(at:) call works fine when checked in the REPL with the same tensors. I'll try to see what gradient(at:) is doing with the arguments when I get the chance.

[rxwei]

It should be a compiler bug that’s introduced recently. The fix is swiftlang/swift#26448 and we need to wait for that to get in.

[rxwei]

The nightly toolchain used for CI is not up to date because of toolchain build failures. The team is looking into this.

[jon-tow]

@dan-zheng Would you like me to port over the new tests in swift/test/AutoDiff/control_flow.swift?

[dan-zheng]

@dan-zheng Would you like me to port over the new tests in swift/test/AutoDiff/control_flow.swift?

The goal is to move all Tensor-specific AD tests to tensorflow/swift-apis. The Tensor control flow AD tests are the last of these.

AutoDiff tests from apple/swift (including swift/test/AutoDiff/control_flow.swift) are intentionally Tensor-independent and should not be ported to tensorflow/swift-apis.

[marcrasi]

@jon-tow It looks like tests on this are failing because it's based off an older commit of swift-apis. If you merge master into this, we can probably rerun tests and merge this. Could you try that when you get a chance?

[jon-tow]

Hey @marcrasi. I merged master into the branch here but I'm getting compiler errors on conditionals that compare Tensor<Float>s with scalar literals. E.g. if x > 0 where x is a Tensor<Float>. This worked just fine a few months ago and any form of casting seems to not resolve the issue.
Any suggestions would be appreciated. Thanks!

[marcrasi]

It looks like the comparisons with scalars were deprecated and then removed (#516). The suggestion in the deprecation warning is to use (lhs .> rhs).all(). Does that fix things?

[jon-tow]

It sure does! Thanks for pointing that out. Invoking all() resolved the issue. :)

[marcrasi]

Oh no, a very recent new commit on master made a backwards-incompatible change so CI is still failing. I think if you merge from master again we should be able to get CI passing.

[jon-tow]

Sure thing! Not a problem. Let me know if you need me to address anything.