Skip to content

make package.resolved more deterministic in edge cases #3967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jan 6, 2022

Conversation

tomerd
Copy link
Contributor

@tomerd tomerd commented Dec 24, 2021

motivation: in some cases like two URLs with casing differences, package.resolved could be non-deterministic

changes:

  • base managed dependencies lookup on canonical location (case insensitive, ignore .git suffix)
  • refactor how package.resolved is saved to test for non material differences
  • adjust call sites that manipulate the pin store to be more efficient about when to save the resolved file
  • add warning when expected dependency that should be stored in the resolved file is not found
  • adjust call sites
  • add and adjust tests

@tomerd tomerd added the WIP Work in progress label Dec 24, 2021
motivation: in some cases like two URLs with casing differences, package.resolved could be non-deterministic

changes:
* make the maanged dependencies location lookup case insensitive
* refactor how package.resolved is saved to test for non material differences
* adjust call sites that manipulate the pin store to be more efficient about when to save the resolved file
* add warning when expected dependency that should be stored in the resolved file is not found
* adjust call sites

TODO: add tests
… it to compare locations where required

* rename LegacyPackageIdentity as PackageIdentityParser and simplify how identity parsing is wired (removed unusued complexity)
@tomerd tomerd force-pushed the fix/manage-deps-comparison branch from 1b7138c to e37c93b Compare December 24, 2021 08:04
@tomerd
Copy link
Contributor Author

tomerd commented Dec 24, 2021

@swift-ci smoke test

@tomerd
Copy link
Contributor Author

tomerd commented Dec 24, 2021

@swift-ci please test package compatibility

// load the pin store from disk so we can compare for any changes
// this is needed as we want to avoid re-writing the resolved files
// unless absolutely required
guard let storedPinStore = observabilityScope.trap({ try self.pinsStore.load() }) else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if this fails, such as because of an I/O error, would it be better to try to write the new file anyway? I guess I'm questioning whether returning here is the right thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you suggesting we try rewrite if we cant read? I can go with that too

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly (I should have been more clear). Since the only reason we're reading it is to avoid unnecessary changes, it seems reasonable to write it out if the read fails. Of course, if it's a permission issue, the write may still fail, but that's a separate issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated e237516

@tomerd tomerd force-pushed the fix/manage-deps-comparison branch from 7488194 to fbbda78 Compare January 5, 2022 01:41
@tomerd tomerd changed the title [wip] make package.resolved deterministic in edge cases make package.resolved deterministic in edge cases Jan 5, 2022
@tomerd tomerd added ready Author believes the PR is ready to be merged & any feedback has been addressed and removed WIP Work in progress labels Jan 5, 2022
@tomerd
Copy link
Contributor Author

tomerd commented Jan 5, 2022

@swift-ci please smoke test

@tomerd tomerd changed the title make package.resolved deterministic in edge cases make package.resolved more deterministic in edge cases Jan 5, 2022
@tomerd
Copy link
Contributor Author

tomerd commented Jan 5, 2022

@swift-ci please test package compatibility

}

observabilityScope.trap{
// exist early is there is nothing to do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// exist early is there is nothing to do
// exit early if there is nothing to do

@tomerd tomerd merged commit 5a1ac27 into swiftlang:main Jan 6, 2022
tomerd added a commit to tomerd/swift-package-manager that referenced this pull request Jan 6, 2022
motivation: in some cases like two URLs with casing differences, package.resolved could be non-deterministic

changes:
* base managed dependencies lookup on canonical location (case insensitive, ignore .git suffix)
* refactor how package.resolved is saved to test for non material differences
* adjust call sites that manipulate the pin store to be more efficient about when to save the resolved file
* add warning when expected dependency that should be stored in the resolved file is not found
* rename CanonicalPackageIdentity as CanonicalPackageLocation and use it to compare locations where required
* rename LegacyPackageIdentity as PackageIdentityParser and simplify how identity parsing is wired (removed unusued complexity)
* adjust call sites
* add and adjust test
tomerd added a commit that referenced this pull request Jan 6, 2022
motivation: in some cases like two URLs with casing differences, package.resolved could be non-deterministic

changes:
* base managed dependencies lookup on canonical location (case insensitive, ignore .git suffix)
* refactor how package.resolved is saved to test for non material differences
* adjust call sites that manipulate the pin store to be more efficient about when to save the resolved file
* add warning when expected dependency that should be stored in the resolved file is not found
* rename CanonicalPackageIdentity as CanonicalPackageLocation and use it to compare locations where required
* rename LegacyPackageIdentity as PackageIdentityParser and simplify how identity parsing is wired (removed unusued complexity)
* adjust call sites
* add and adjust test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready Author believes the PR is ready to be merged & any feedback has been addressed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants