Skip to content

[Explicit Module Builds] Support explicit module builds with external dependencies. #181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 4, 2020

Conversation

artemcm
Copy link
Contributor

@artemcm artemcm commented Jul 23, 2020

This change introduces support for external dependencies in Explicit Module Build compilation mode.

Building a Swift module in Explicit Module Build mode requires passing all of its module dependencies as explicit arguments to the build command.

When the driver's clients (build systems, e.g. SwiftPM) are planning a build that involves multiple Swift modules, planning for each individual module may take place before its dependencies have been built. This means that the dependency scanning action will not be able to discover such modules. In such cases, the clients must provide the driver with information about such external dependencies, including the path to where their compiled .swiftmodule will be located, once built, and a full inter-module dependency graph for each such dependence.

The driver will pass down the information about such external dependencies to the scanning action, which will generate placeholder swift modules for them in the resulting dependency graph. The driver will then use the complete dependency graph provided by the client for each external dependency and use it to "resolve" the dependency's "placeholder" module.

This PR:

  • Adds API for the clients to pass in external module dependency artifacts (module file path and dependency graph).
  • Updates the Inter Module Dependency Graph to represent swiftPlaceholder dependencies.
  • Updates the scanning action invocation to pass in information about external dependencies.
  • Introduces the placeholder dependency resolution logic that updates the dependency graph by substituting in dependency graphs of external modules in place of placeholder modules.

This change works in tandem with the corresponding Dependency Scanner change in:
swiftlang/swift#33032
And a corresponding SwiftPM change:
swiftlang/swift-package-manager#2827

@artemcm artemcm requested review from DougGregor and nkcsgexi July 23, 2020 16:07
@nkcsgexi
Copy link
Contributor

Have we had a consensus about how to resolve cross-import overlays yet?

@artemcm
Copy link
Contributor Author

artemcm commented Jul 23, 2020

Have we had a consensus about how to resolve cross-import overlays yet?

No, not yet, that needs further discussion.

@nkcsgexi
Copy link
Contributor

@DougGregor @brentdax What do you think of this plan:

  • In the ExternalDependencyMap passed down to the -scan-dependency action, swift-driver also specifies transitively imported Swift module names.
  • Dependency scanner reads those transitive imports to resolve additional cross-import overlays

artemcm added 5 commits July 29, 2020 11:30
Resolves `placeholder` dependencies in the dependency graph computed by the scanning action using external dependencies passed in from the client.
Copy link
Member

@DougGregor DougGregor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to do a line-by-line review (sorry), but the structure of this looks exactly like I'd expected and it LGTM.

@artemcm
Copy link
Contributor Author

artemcm commented Jul 29, 2020

The toolchain snapshot build with the corresponding Dependency Scanner changes passed so the next created toolchain will allow this to be submitted. I expect it to become available tonight.

@artemcm artemcm force-pushed the ExplicitPackageBuilds branch from ba0e516 to 30149cb Compare July 30, 2020 21:20
@DougGregor
Copy link
Member

@swift-ci please test

@DougGregor
Copy link
Member

@swift-ci test

@artemcm artemcm merged commit ffe24e5 into swiftlang:master Aug 4, 2020
@artemcm artemcm deleted the ExplicitPackageBuilds branch January 20, 2021 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants