Skip to content

[AutoDiff] Fix autodiff_function instruction worklist. #24941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 21, 2019

Conversation

dan-zheng
Copy link
Contributor

Fix autodiff_function instruction worklist registration/invalidation.

  • In VJPEmitter, add cloned autodiff_function instructions in VJP
    to worklist.
  • If autodiff_function instructions are deleted in
    foldAutoDiffFunctionExtraction, also remove their worklist occurrences.

Resolves TF-515.


Writing a reproducer test is difficult because the nature of the bug is ad-hoc.
Verified that swift test works for tensorflow/swift-models@f9687bf.

Fix `autodiff_function` instruction worklist registration/invalidation.

- In `VJPEmitter`, add cloned `autodiff_function` instructions in VJP
  to worklist.
- If `autodiff_function` instructions are deleted in
  `foldAutoDiffFunctionExtraction`, also remove their worklist occurrences.

Resolves TF-515.
@dan-zheng dan-zheng added the tensorflow This is for "tensorflow" branch PRs. label May 21, 2019
@dan-zheng dan-zheng requested review from rxwei and saeta May 21, 2019 01:45
@rxwei
Copy link
Contributor

rxwei commented May 21, 2019

In Differentiation::run(), all autodiff_function instructions in a module are pushed to the work list. Could you please explain why it is necessary to push more in VJPEmitter::visitAutoDiffFunctionInst?

@dan-zheng
Copy link
Contributor Author

dan-zheng commented May 21, 2019

In Differentiation::run(), all autodiff_function instructions in a module are pushed to the work list. Could you please explain why it is necessary to push more in VJPEmitter::visitAutoDiffFunctionInst?

If an original function has an autodiff_function instruction, it will be found and pushed to the worklist.
However, when VJPEmitter clones the original function into a VJP function, the newly cloned autodiff_function instruction needs to be manually added to the worklist.

Otherwise, the cloned autodiff_function instruction in the VJP won't be processed, leading to verification errors:

Assertion `!getAssociatedFunctions().empty() && "No associated functions. Maybe " "the differentiation pass has not run?"'

@dan-zheng
Copy link
Contributor Author

@swift-ci Please test tensorflow

@dan-zheng
Copy link
Contributor Author

@swift-ci Please clean test tensorflow

@rxwei
Copy link
Contributor

rxwei commented May 21, 2019

It's a little unfortunate that while the cloned autodiff_function is an exact copy of the original, it has to go through emitAssociatiedFunctionReference twice.

@dan-zheng
Copy link
Contributor Author

It's a little unfortunate that while the cloned autodiff_function is an exact copy of the original, it has to go through emitAssociatiedFunctionReference twice.

Yeah, that is true.

@dan-zheng dan-zheng merged commit a2ba998 into swiftlang:tensorflow May 21, 2019
@dan-zheng dan-zheng deleted the TF-515 branch May 21, 2019 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tensorflow This is for "tensorflow" branch PRs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants