Skip to content

[Dependency Scanning] Add functionality to validate contents of a loaded scanner cache state for incremental scans #78962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 3, 2025

Conversation

artemcm
Copy link
Contributor

@artemcm artemcm commented Jan 27, 2025

Checking each module dependency info if it is up-to-date with respect to when the cache contents were serialized in a prior scan.

// Initial Scan Client module
// RUN: %target-swift-frontend -scan-dependencies -scanner-module-validation -module-load-mode prefer-interface -Rdependency-scan-cache -serialize-dependency-scan-cache -dependency-scan-cache-path %t/cache.moddepcache -module-cache-path %t/module-cache %t/Client.swift -o %t/deps_initial.json -I %t/Modules -I %S/../Inputs/CHeaders -module-name Client

// Clean re-scan
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional tests added/modified to reflect functionality introduced in this PR in swiftlang/swift-driver#1786.

@artemcm
Copy link
Contributor Author

artemcm commented Jan 28, 2025

@swift-ci test

@artemcm artemcm marked this pull request as ready for review January 28, 2025 21:50
@artemcm artemcm force-pushed the IncrementalScanValidate branch from a5d11e2 to 8b0d74e Compare January 28, 2025 23:21
@artemcm
Copy link
Contributor Author

artemcm commented Jan 28, 2025

@swift-ci test

@artemcm artemcm force-pushed the IncrementalScanValidate branch from 8b0d74e to 59caf34 Compare January 29, 2025 16:23
@artemcm
Copy link
Contributor Author

artemcm commented Jan 29, 2025

@swift-ci test

@artemcm artemcm force-pushed the IncrementalScanValidate branch from 59caf34 to 5dd7bec Compare January 30, 2025 16:28
@artemcm
Copy link
Contributor Author

artemcm commented Jan 30, 2025

@swift-ci test

@artemcm artemcm force-pushed the IncrementalScanValidate branch 2 times, most recently from bd3697f to efc9d0b Compare January 30, 2025 20:35
@artemcm
Copy link
Contributor Author

artemcm commented Jan 30, 2025

@swift-ci test

Copy link
Contributor

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than some timing windows, the overall approach looks good to me.

@artemcm artemcm force-pushed the IncrementalScanValidate branch from efc9d0b to 8976c80 Compare February 3, 2025 18:01
@artemcm
Copy link
Contributor Author

artemcm commented Feb 3, 2025

@swift-ci smoke test

Copy link
Contributor

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

…ded scanner cache state

Checking each module dependency info if it is up-to-date with respect to when the cache contents were serialized in a prior scan.

- Add a timestamp field to the serialization format for the dependency scanner cache
- Add a flag "-validate-prior-dependency-scan-cache" which, when combined with "-load-dependency-scan-cache" will have the scanner prune dependencies from the deserialized cache which have inputs that are newer than the prior scan itself

With the above in-place, the scan otherwise proceeds as-is, getting cache hits for entries still valid since the prior scan.
@artemcm artemcm force-pushed the IncrementalScanValidate branch from 8976c80 to acb4e84 Compare February 3, 2025 19:05
@artemcm
Copy link
Contributor Author

artemcm commented Feb 3, 2025

@swift-ci smoke test

@artemcm artemcm enabled auto-merge February 3, 2025 19:06
@artemcm artemcm merged commit 06637ae into swiftlang:main Feb 3, 2025
3 checks passed
@artemcm artemcm deleted the IncrementalScanValidate branch February 7, 2025 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants