Skip to content

[3.0] [Driver] Make sure to rebuild dependents when a dirty file fails. #3957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

jrose-apple
Copy link
Contributor

@jrose-apple jrose-apple commented Aug 3, 2016

  • Explanation: Certain changes and subsequent fixes to a file A.swift can result in a downstream file B.swift not getting rebuilt, essentially resulting in undefined behavior at the interface between those two files. (That is, this is effectively a miscompile.)
  • Scope: The conditions under which the issue occurs aren't that unlikely (see commit message). The fix affects all incremental builds. A clean build (including the Archive action) will continue to avoid any such issues.
  • Issue: rdar://problem/25405605
  • Reviewed by: (not yet reviewed) @modocache
  • Risk: Medium.
  • Testing: Added compiler regression tests for individual cases, as well as the originally-reported test case.

Before merging this pull request to apple/swift repository:

  • Test pull request on Swift continuous integration.

Triggering Swift CI

The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:

Smoke Testing

Platform Comment
All supported platforms @swift-ci Please smoke test
All supported platforms @swift-ci Please smoke test and merge
OS X platform @swift-ci Please smoke test OS X platform
Linux platform @swift-ci Please smoke test Linux platform

A smoke test on macOS does the following:

  1. Builds the compiler incrementally.
  2. Builds the standard library only for macOS. Simulator standard libraries and
    device standard libraries are not built.
  3. lldb is not built.
  4. The test and validation-test targets are run only for macOS. The optimized
    version of these tests are not run.

A smoke test on Linux does the following:

  1. Builds the compiler incrementally.
  2. Builds the standard library incrementally.
  3. lldb is built incrementally.
  4. The swift test and validation-test targets are run. The optimized version of these
    tests are not run.
  5. lldb is tested.

Validation Testing

Platform Comment
All supported platforms @swift-ci Please test
All supported platforms @swift-ci Please test and merge
OS X platform @swift-ci Please test OS X platform
OS X platform @swift-ci Please benchmark
Linux platform @swift-ci Please test Linux platform

Lint Testing

Language Comment
Python @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

…iftlang#3935)

Otherwise we get into a situation like this:

1. Change made to the interface of file A.swift that also causes an
   error in A.swift.
2. Fixing the error in A.swift does not affect A.swift's interface.
3. File B.swift that depends on A.swift is not rebuilt, since the most
   recent change to A.swift did not change its interface.

To fix this, mark downstream files as needing to be rebuilt even when
a compilation job fails with errors. Additionally, attempt to be extra
conservative when there's a crash.

rdar://problem/25405605
(cherry picked from commit 848b3eb)
@jrose-apple
Copy link
Contributor Author

No one else works on this area of the compiler, but I'd still like someone else's eyes on the change. @bitjammer, @modocache, and/or @gribozavr, would you mind taking a look? (Feel free to ask questions.)

@jrose-apple
Copy link
Contributor Author

@swift-ci Please test

@jrose-apple jrose-apple changed the title [Driver] Make sure to rebuild dependents when a dirty file fails. [3.0] [Driver] Make sure to rebuild dependents when a dirty file fails. Aug 3, 2016
@modocache
Copy link
Contributor

My pleasure! I appreciate the mention, I definitely want to get more familiar with this code.

/cc @Samjohn, who expressed interested in incremental compilation.

if os.path.basename(primaryFile) == 'bad.swift':
sys.exit(1)
else:
sys.exit(129)

dir = os.path.dirname(os.path.abspath(__file__))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit-pick: dir is actually a built-in Python function. In a future commit, it might be nice to prevent the shadowing that function here.

@shahmishal shahmishal added this to the Swift 3.0 milestone Aug 3, 2016
@jrose-apple
Copy link
Contributor Author

sigh @Samjohn, I still owe you some more details and better tools, if not actual improvements. (Re-link to DependencyAnalysis.rst for all involved.)

Thanks, Dmitri.

(cherry picked from commit b8cc267)
StringRef DependenciesFile =
Output.getAdditionalOutputForType(types::TY_SwiftDeps);
if (!DependenciesFile.empty() &&
(ReturnCode == EXIT_SUCCESS || ReturnCode == EXIT_FAILURE)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I interpret this reference to mean that ReturnCode must always be either EXIT_SUCCESS or EXIT_FAILURE. Is there a case in which it is neither? In other words, is this check necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's a check for crashes. It's not the most direct way to check for abnormal exit, but it ought to work.

@jrose-apple
Copy link
Contributor Author

@modocache, since all your suggestions are cosmetic, does it look okay to land otherwise?

@swift-ci Please test

@modocache
Copy link
Contributor

@jrose-apple Yup! Thanks again for the chance to look at it.

@tkremenek tkremenek merged commit 04b9f0d into swiftlang:swift-3.0-branch Aug 4, 2016
@jrose-apple jrose-apple deleted the swift-3.0-we-can-rebuild-them branch August 4, 2016 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants