Skip to content

[ModuleTrace] Emit import graph to ease debugging import issues. #33980

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

varungandhi-apple
Copy link
Contributor

@varungandhi-apple varungandhi-apple commented Sep 17, 2020

Adds functionality to see an import graph describing the imports of different modules.

Here is an example for a module only importing Combine, which has a small dependency footprint.

combine

In practice, you'll likely have a much larger dependency footprint. For example, here's what you're dragging in if you import MapKit and SwiftUI.

cross-import

There are still some non-determinism issues in the graphviz output, need to look at the printing code again more carefully.

Marking as a draft because there are some unresolved questions:

  • How do we test this? Ideally, I'd like to reuse the existing import graphs instead of having its own separate module graphs. Should I just add extra lines to our tests for the module trace and for cross-import overlays?
  • Should we add functionality for pruning particular nodes? In an ideal world, the graph manipulation would be handled by a separate tool (the equivalent of jq but for GraphViz graphs), but unfortunately I don't think such a tool exists. It seems like it would be easy functionality to add though, maybe even within the scope of a StarterBug.
  • Can we make the output less janky? There are too many wobbly edges for my liking. 😬

Current TODOs:

  • Rebasing - Worked on this a little bit, but there's more to be done. Potentially going to run into problems in case [NFC] Clarify import filtering logic and naming. #33986 needs to be reverted.
  • WidgetKit crash - Trying to print an import graph for WidgetKit caused a crash in Clang, need to look at that.

Answering potential questions:

Q: Why have redundancy in the colors and the "(S)"/"(C)"?
A: The colors make it easier to spot things for people who can distinguish orange/grey. If you cannot visually distinguish the colors, the additional "(S)" and "(C)" are helpful.

Q: Can you change the edges labels to use a different scheme/convention instead?
A: Yes, I'm happy to change those, so long as the new scheme shows both default + SPI when both imports are used along the same edge (leading to a compiler warning). Simply showing the attribute (@_exported/@_implementationOnly/@_spi), and hence not showing "Def" at all, doesn't have that property.

It used to be a method because it used a cache in
ABIDependencyEvaluator; however that cache has been removed.
When looking at a module being shadowed by a separate (cross-import) overlay:

before:
  filter (potentially) = {ShadowedBySeparateOverlay}
  requiredFilter = {Private, ShadowedBySeparateOverlay}
  filter.contains(requiredFilter) = false // incorrect

after:
  filter (potentially) = {ShadowedBySeparateOverlay}
  requiredFilter = {ShadowedBySeparateOverlay}
  filter.contains(requiredFilter) = true
This should make it easier to spot cycles in import graphs, as well as
funky behavior involving both default/private imports + SPI imports.
@varungandhi-apple
Copy link
Contributor Author

@swift-ci please test

@varungandhi-apple varungandhi-apple marked this pull request as draft September 17, 2020 07:50
@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - d0ac0d4

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - d0ac0d4

Comment on lines +1194 to -1199
else if (!separatelyImportedOverlays.lookup(desc.module.importedModule).empty())
requiredFilter |= ModuleDecl::ImportFilterKind::ShadowedBySeparateOverlay;
else
requiredFilter |= ModuleDecl::ImportFilterKind::Private;

if (!separatelyImportedOverlays.lookup(desc.module.importedModule).empty())
requiredFilter |= ModuleDecl::ImportFilterKind::ShadowedBySeparateOverlay;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "fix" is incorrect. The ShadowedBySeparateOverlay filter is essentially a different axis altogether.

             | @_exported | private | @_implementationOnly | @_spi
-------------+------------+---------+----------------------+------
Not shadowed |     EN     |   PN    |        IN            |  SN
Shadowed     |     ES     |   PS    |        IS            |  SS

Ideally, we'd be passing in two OptionSets, not one IMO. If you pass in newFilter = {{@_exported}, {not shadowed}}, you get EN. If you pass in newFilter = {{@_exported | private}, {shadowed}}, you'd get ES and PS.

Today, if you pass in filter = {@_exported}, you get EN whereas if you pass in filter = {shadowed}, you get nothing. That's unnecessarily confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, @_spi is a different axis, so the table in ^ is wrong.

@shahmishal
Copy link
Member

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

  • How to change the base branch: (Link)
  • More detail about the branch update: (Link)

@varungandhi-apple varungandhi-apple changed the base branch from master to main October 1, 2020 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants