Skip to content

Bring back [Explicit Module Builds] Serialize and re-use the inter-module dependency graph as incremental build state #1236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 5, 2022

Conversation

artemcm
Copy link
Contributor

@artemcm artemcm commented Dec 2, 2022

Second attempt at landing #1230
Which previously caused compiler lit test regressions in main CI.

Building on top of #621, on an incremental build, load a prior dependency graph, check if it is still valid:

  • import set of the current module has not changed
  • input files of every module in the prior graph are older than the output and if it is, re-use it, skipping invoking the dependency scanner again.

More concretely, computation of the initial incremental state in computeIncrementalStateForPlanning will attempt to read out and validate a prior serialized inter-module dependency graph. If validated, such graph will become a part of the initial incremental state itself.

During build planning, at the stage of computing inter-module dependencies, the dependency scanning action will be skipped if the initial incremental state contains a still-valid prior inter-module dependency graph.

In both cases, if a prior graph is being re-used, or if a new graph was computed via a dependency scanning action, the graph will become a component of the incremental build state to be serialized after the build is complete.

This change does not yet result in skipping the actual module build jobs in the planning output, which can be done with this, but this is left to a followup PR.

There is also some notable overlap between this and what is already done for detecting changes in input files to the current compilation:

  • Suppose a Swift module interface of a module dependency has changed. This means that, according to the above mechanism, a dependency scanning action must be performed on an incremental build. This will result in a new dependency graph and a subsequent re-build of the dependency. But, moreover, there is an existing mechanism in place to ensure that source-files also get re-built on changes to previously-recorded inputs to compilation so the source-files will also be re-built when a dependency changes.

Once explicit builds become the default, we can consolidate the two approaches into one.

Partially resolves rdar://66801475

…as incremental build state

On an incremental build, load a prior dependency graph, check if it is still valid:
1. import set of the current module has not changed
2. input files of every module in the prior graph are older than the output
and if it is, re-use it, skipping invoking the dependency scanner again.

Partially resolves rdar://66801475
'guard let X else {}' works with latest Swift compilers (such as one used in swift-driver CI) but isn't supported in some older compilres used in CI and other bots.
@artemcm artemcm force-pushed the IncrementalBabyOneMoreTime branch from 6d13641 to 816c738 Compare December 2, 2022 22:46
@artemcm artemcm requested review from nkcsgexi and tshortli December 2, 2022 22:48
@artemcm
Copy link
Contributor Author

artemcm commented Dec 2, 2022

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Dec 2, 2022

The original PR removed:

if parsedOptions.hasArgument(.v) {
      try printVersion(outputStream: &stderrStream)
}

from Driver.run. This change wasn't actually related to the PR and was left in it by accident. But this output missing was enough to trip up some lit-tests in the compiler repo...

@artemcm artemcm merged commit caa81e5 into swiftlang:main Dec 5, 2022
@artemcm artemcm deleted the IncrementalBabyOneMoreTime branch December 5, 2022 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants