Skip to content

[Dependency Scanning] Perform cross-import overlay resolution per source-file #79550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 26, 2025

Conversation

artemcm
Copy link
Contributor

@artemcm artemcm commented Feb 22, 2025

Previous implementation took the entire transitive dependency set and cross-referenced all of its members to determine which ones introduce requried cross-import overlays. That implementation differed from the cross-import overlay loading logic during source compilation, where a corrsponding cross-import overlay module is only requested if the two constituent modules are reachable via direct 'import's from the same source file. Meaning the dependency scanner before this change would report cross-import overlay dependencies which never got loaded by the corresponding client source compile.

This change implements a new implementation of cross-import overlay discovery which first computes sub-graphs of module dependencies directly reachable by 'import's for each source file of the module under scan and then performs pairwise cross-import overlay query per each such sub-graph.

Resolves rdar://145157171

@artemcm
Copy link
Contributor Author

artemcm commented Feb 22, 2025

@swift-ci test

@artemcm artemcm force-pushed the EBM_CrossImportsPerSourceFile branch 2 times, most recently from 0442ccd to 5afb81c Compare February 24, 2025 18:25
@artemcm
Copy link
Contributor Author

artemcm commented Feb 24, 2025

@swift-ci test

@artemcm artemcm marked this pull request as ready for review February 24, 2025 18:25
Copy link
Contributor

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general. Some coding style comments in line.

Also can cross-import introduce new dependencies?

@artemcm artemcm force-pushed the EBM_CrossImportsPerSourceFile branch from 5afb81c to 9d26fba Compare February 24, 2025 19:26
@artemcm
Copy link
Contributor Author

artemcm commented Feb 24, 2025

Also can cross-import introduce new dependencies?

They sure can, so we have resolveCrossImportOverlayDependencies call back into resolveImportedModuleDependencies on the set of discovered/imported cross-import overlay modules. We make a crucial assumption that cross-import overlays cannot introduce a need for other new cross-import overlays.

@artemcm
Copy link
Contributor Author

artemcm commented Feb 24, 2025

@swift-ci test

@cachemeifyoucan
Copy link
Contributor

We make a crucial assumption that cross-import overlays cannot introduce a need for other new cross-import overlays.

That is actually my question and thanks for the answer.

@artemcm artemcm force-pushed the EBM_CrossImportsPerSourceFile branch from 9d26fba to 6b2acd5 Compare February 24, 2025 21:51
@artemcm
Copy link
Contributor Author

artemcm commented Feb 24, 2025

@swift-ci test

@artemcm artemcm force-pushed the EBM_CrossImportsPerSourceFile branch from 6b2acd5 to 37c9ea1 Compare February 24, 2025 23:33
@artemcm
Copy link
Contributor Author

artemcm commented Feb 24, 2025

@swift-ci test

@artemcm artemcm enabled auto-merge February 24, 2025 23:33
@artemcm artemcm force-pushed the EBM_CrossImportsPerSourceFile branch from 37c9ea1 to 037b7af Compare February 25, 2025 18:04
@artemcm
Copy link
Contributor Author

artemcm commented Feb 25, 2025

@swift-ci test

…rce-file

Previous implementation took the entire transitive dependency set and cross-referenced all of its members to determine which ones introduce requried cross-import overlays. That implementation differed from the cross-import overlay loading logic during source compilation, where a corrsponding cross-import overlay module is only requested if the two constituent modules are reachable via direct 'import's from *the same source file*. Meaning the dependency scanner before this change would report cross-import overlay dependencies which never got loaded by the corresponding client source compile.

This change implements a new implementation of cross-import overlay discovery which first computes sub-graphs of module dependencies directly reachable by 'import's for each source file of the module under scan and then performs pairwise cross-import overlay query per each such sub-graph.

Resolves rdar://145157171
@artemcm artemcm force-pushed the EBM_CrossImportsPerSourceFile branch from 037b7af to 214fe59 Compare February 25, 2025 21:40
@artemcm
Copy link
Contributor Author

artemcm commented Feb 25, 2025

@swift-ci test

@xymus
Copy link
Contributor

xymus commented Feb 25, 2025

Does it mean that we end up brining in more cross-import overlays when rebuilding from one swiftinterface than when building the same module from many source files?

@artemcm
Copy link
Contributor Author

artemcm commented Feb 25, 2025

Does it mean that we end up brining in more cross-import overlays when rebuilding from one swiftinterface than when building the same module from many source files?

That's a great question. I may have over-complicated my understanding of the actual algorithm used during source compilation for overlay discovery, based on what I see in compiler sources. I now think that we do not need to follow any dependency edges at all and only do detection of cross-import overlays amongst directly-imported modules.
@beccadax does that sound right to you?

@artemcm artemcm merged commit 14dd14d into swiftlang:main Feb 26, 2025
4 of 5 checks passed
@artemcm artemcm deleted the EBM_CrossImportsPerSourceFile branch February 26, 2025 18:07
@artemcm
Copy link
Contributor Author

artemcm commented Feb 26, 2025

I had forgotten enabling the auto-merger for this PR even though I wanted to address the question above...
I will follow-up shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants