Skip to content

[AutoDiff] improve symbol linkage #28582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Dec 5, 2019
Merged

Conversation

marcrasi
Copy link

@marcrasi marcrasi commented Dec 5, 2019

Implements most of the symbol linkage design, enough to fix two linker issues: TF-690 & TF-1025. (There are a few other linker issues in JIRA. I'll check later if this fixes any of those).

Specific changes:

  • Changed the linkage and serialization of witnesses, derivative functions, and closures to match the design.
  • Made deserialized witnesses from other modules not get emitted into the current module IR, by setting their linkage to public_external, and teaching IRGen not to emit public_external witnesses. (Fixes TF-1025).
  • Added diagnostic for calling private derivatives from serialized functions (Fixes TF-690).

This PR does not include the following pieces of the design:

  • Fixing linkage of thunks. I'll do this in a later PR.
  • The solution to allow closures in differentiable function default arguments (TF-1030).

This seems "dangerous" so I'll run swift-apis and swift-models tests on it. (e.g. without tensorflow/swift-apis#573, swift-apis would fail to compile after this PR).

@marcrasi marcrasi requested review from rxwei and dan-zheng December 5, 2019 03:37
Copy link
Contributor

@dan-zheng dan-zheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the great documentation!

@marcrasi
Copy link
Author

marcrasi commented Dec 5, 2019

@swift-ci please test tensorflow

@@ -823,7 +823,8 @@ void SILGenModule::emitDifferentiabilityWitness(
auto *diffWitness = SILDifferentiabilityWitness::createDefinition(
M, originalFunction->getLinkage(), originalFunction, loweredParamIndices,
config.resultIndices, config.derivativeGenericSignature,
/*jvp*/ nullptr, /*vjp*/ nullptr, originalFunction->isSerialized(),
/*jvp*/ nullptr, /*vjp*/ nullptr,
/*isSerialized*/ hasPublicVisibility(originalFunction->getLinkage()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably shouldn't always serialize witnesses for any public declarations, because things we emit should be opaque by default. Maybe you should add logic to the cross module optimizer that was recently added (#28407) to make differentiability witnesses serialized.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do this later because this PR is now very tested and almost ready to go. https://bugs.swift.org/browse/TF-1035

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thanks.

@marcrasi
Copy link
Author

marcrasi commented Dec 5, 2019

@swift-ci please test tensorflow

2 similar comments
@marcrasi
Copy link
Author

marcrasi commented Dec 5, 2019

@swift-ci please test tensorflow

@marcrasi
Copy link
Author

marcrasi commented Dec 5, 2019

@swift-ci please test tensorflow

@marcrasi
Copy link
Author

marcrasi commented Dec 5, 2019

@swift-ci please test tensorflow macos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants