[TF] Moved most Tensor APIs to 'tensorflow/swift-apis'.

[eaplatanios]

This PR is moving stuff over from stdlib to swift-apis and switching to using eager mode raw op bindings.

I have also incorporated the following updates to tensorflow/swift-apis#109:

• PR [TF] Refactor gradients computation with expandingShape #24249

Friend PRs: tensorflow/swift-apis#109 and tensorflow/swift-bindings#26 .

[rxwei]

Since we are preparing v0.3, we'll get to this next week!

[eaplatanios]

Sounds good! I'll keep making changes to this PR and the swift-apis one until then, if that's ok.

[rxwei]

Sure thing.

[rxwei]

Could you resolve the conflicts?

[eaplatanios]

@rxwei The conflicts should be resolved now.

[rxwei]

@swift-ci please clean test tensorflow Linux

[rxwei]

@swift-ci please clean test tensorflow Linux

[eaplatanios]

I can try to look into later today but it may take a while because I need to recompile S4TF on one of our GPU servers. Also, I noticed from the tests that they only fail when we assign to a tensor, not when we're simply indexing. I'll try to figure out why.

[eaplatanios]

@rxwei also, I'm not sure if you're aware of this, but when I try to use #tfop from outside of stdlib, I get linkage errors for symbols such as _TFE_DeleteOp, _TFE_NewOp, _TFE_OpSetAttrInt, etc.

[rxwei]

@rxwei also, I'm not sure if you're aware of this, but when I try to use #tfop from outside of stdlib, I get linkage errors for symbols such as _TFE_DeleteOp, _TFE_NewOp, _TFE_OpSetAttrInt, etc.

Yeah, this is a known bug.

[eaplatanios]

Why can't we just export these symbols when compiling CTensorFlow?

[rxwei]

It's a linker flag issue. Symbols should already be exported module-wise.

[eaplatanios]

Where would be a good place to look into for fixing that?

[rxwei]

Where would be a good place to look into for fixing that?

I think that @dan-zheng knows this best!

[dan-zheng]

@rxwei also, I'm not sure if you're aware of this, but when I try to use #tfop from outside of stdlib, I get linkage errors for symbols such as _TFE_DeleteOp, _TFE_NewOp, _TFE_OpSetAttrInt, etc.

Could you please share a reproducer?

---

Here's some quick experimentation:

// tfop.swift
import TensorFlow

let x = Tensor<Float>(1)
let tensor: Tensor<Float> =
  #tfop("Identity", x, T$dtype: Float.tensorFlowDataType,
        __shapes: [x.shape])
print(tensor)

$ swift tfop.swift
1.0

$ swiftc tfop.swift
Undefined symbols for architecture x86_64:
  "_TFE_DeleteOp", referenced from:
      _$s10TensorFlow0A0V4tfopE10toHostTest5shapeACyxGAA0A5ShapeV_tF in tfop-5805d4.o
  "_TFE_NewOp", referenced from:
      _$s10TensorFlow0A0V4tfopE10toHostTest5shapeACyxGAA0A5ShapeV_tF in tfop-5805d4.o
  "_TFE_OpSetAttrType", referenced from:
      _$s10TensorFlow0A0V4tfopE10toHostTest5shapeACyxGAA0A5ShapeV_tF in tfop-5805d4.o
  "_TF_DeleteStatus", referenced from:
      _$s10TensorFlow0A0V4tfopE10toHostTest5shapeACyxGAA0A5ShapeV_tF in tfop-5805d4.o
  "_TF_NewStatus", referenced from:
      _$s10TensorFlow0A0V4tfopE10toHostTest5shapeACyxGAA0A5ShapeV_tF in tfop-5805d4.o
ld: symbol(s) not found for architecture x86_64
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

$ swiftc -ltensorflow tfop.swift # succeeds

$ ./tfop
1.0

[eaplatanios]

Thanks @dan-zheng ! I actually wasn't using -ltensorflow because I assumed it would be included by default in S4TF. It's a bit surprising that it is needed for this but not for using other functions provided by the TensorFlow library more generally. I also noticed that, irrespective of #tfop, if I have a SwiftPM package that compiles fine using swift build, linking fails when I use swift build -c release. This is resolved by using swift build -c release -Xlinker -ltensorflow instead. However, I find this behavior a bit surprising.

[eaplatanios]

I'm closing this in favor of a new PR I'm about to open.