Skip to content

fix bug where AD is not always seeing custom [differentiable] attrs #21565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

marcrasi
Copy link

When SILGen generates SIL for an expression like 3 * x it creates a SILFunciton for Float.* but does not populate that SILFunction with data like the list of [differentiable] attrs. The AD pass does not load this data until after running lookUpMinimalDifferentiationTask, so lookUpMinimalDifferentiationTask does not find the custom adjoint. So AD differentiates Float.* wrt the second parameter instead of using the custom adjoint.

The struct_extract differentiation patch that I am working on exposes this, so I split this fix out into its own PR.

@marcrasi marcrasi requested review from rxwei and dan-zheng December 29, 2018 23:10
@marcrasi
Copy link
Author

@swift-ci please test tensorflow

@marcrasi marcrasi merged commit cb2eacc into swiftlang:tensorflow Dec 30, 2018
@marcrasi marcrasi deleted the load-differentiable-attributes branch December 30, 2018 01:45
@rxwei
Copy link
Contributor

rxwei commented Jan 13, 2019

Do you know why SILGen did not populate the SILFunction with differentiable attrs? It does add attributes like specialize to the function declaration. See https://forums.swift.org/t/help-needed-with-differentiable-attribute-serialization/19454 for context.

loadFunction deserializes the function body, which we should not do in the long term.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants