Disable fingerprint checking when storage is not available #3928
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yim-lee
requested review from
abertelrud,
elsh,
neonichu and
tomerd
as code owners
|
@swift-ci please smoke test |
neonichu
approved these changes
Dec 10, 2021
Motivation: Source compat test failure: https://ci.swift.org/job/swift-PR-source-compat-suite/5701/artifact/swift-source-compat-suite/ ``` error: Failed to get source control fingerprint for swift-log remoteSourceControl https://github.com/apple/swift-log.git version 1.4.2 from storage: Error Domain=NSCocoaErrorDomain Code=513 "You don't have permission to save the file "fingerprints" in the folder "security"." UserInfo={NSFilePath=/Users/buildnode/.swiftpm/security/fingerprints, NSUnderlyingError=0x7feaae439370 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}} error: Error Domain=NSCocoaErrorDomain Code=513 "You don't have permission to save the file "fingerprints" in the folder "security"." UserInfo={NSFilePath=/Users/buildnode/.swiftpm/security/fingerprints, NSUnderlyingError=0x7feaae439370 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}} ``` Modifications: - Make `PackageFingerprintStorage` optional in `RegistryClient` and `SourceControlPackageContainer`, which would turn off fingerprint read/write and essentially disable the TOFU feature. - `SwiftTool` will try to create the shared security directory (under which fingerprints are stored), and if it fails (e.g., permission errors) set `PackageFingerprintStorage` to none. - Don't perform integrity check on fingerprint write. The validation failure will happen on read.
yim-lee
force-pushed
the
fingerprint-storage-err
branch
from
877640a to
63d0de2
Compare
|
@swift-ci please smoke test |
yim-lee
changed the title
neonichu
approved these changes
Dec 10, 2021
tomerd
reviewed
Dec 10, 2021
| completion(.success(checksum)) | ||
| case .none: | ||
| case .failure(let error): | ||
| // Don't throw write errors |
There was a problem hiding this comment.
out of curiosity, why do we no longer throw in this case?
There was a problem hiding this comment.
We were doing two types of throw before:
- checksum has changed
- actual write failures (e.g., file system, serialization, etc.)
1 is sort of integrity check, and I think it's a bit strange to do it on a write operation when we are already doing it on read.
I suppose we can continue throwing 2 if that's the preference.
There was a problem hiding this comment.
I'm seeing a failure in Swift CI due to the removed none case at https://ci.swift.org/job/swift-PR-Linux-smoke-test/31790/consoleFull#6525590103122a513-f36a-4c87-8ed7-cbc36a1ec144:
/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-main/swiftpm/Tests/WorkspaceTests/RegistryPackageContainerTests.swift:421:39: error: type 'FingerprintCheckingMode' has no member 'none'
01:01:48 fingerprintCheckingMode: .none,
01:01:48 ~^~~~
01:01:48 error: fatalError
tomerd
approved these changes
Dec 10, 2021
I think it would be good to throw in that case, follow up PR is fine, this is not urgent to fix |
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 11, 2021
Per review feedback, reverting part of the changes introduced in swiftlang#3928.
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 11, 2021
Per review feedback, reverting part of the changes introduced in swiftlang#3928.
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 11, 2021
Motivation: Source compat test continues to fail even with swiftlang#3928: ``` "You don’t have permission to save the file “fingerprints” in the folder “security” ``` https://ci.swift.org/job/swift-PR-source-compat-suite/5709/artifact/ Not sure how/why, but it seems like `security` directory is created just fine but not the `fingerprints` sub directory (`fingerprints` is a directory not a file, and the code add a file to the `fingerprints` dir for each downloaded package). Modifications: Test creating `fingerprints` directory instead of `security`, and if that fails, disable TOFU feature.
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 12, 2021
Motivation: Source compat test continues to fail even with swiftlang#3928: ``` "You don’t have permission to save the file “fingerprints” in the folder “security” ``` https://ci.swift.org/job/swift-PR-source-compat-suite/5709/artifact/ Not sure how/why, but it seems like `security` directory is created just fine but not the `fingerprints` sub directory (`fingerprints` is a directory not a file, and the code add a file to the `fingerprints` dir for each downloaded package). Modifications: Test creating `fingerprints` directory instead of `security`, and if that fails, disable TOFU feature.
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 13, 2021
Motivation: Source compat test continues to fail even with swiftlang#3928: ``` "You don’t have permission to save the file “fingerprints” in the folder “security” ``` https://ci.swift.org/job/swift-PR-source-compat-suite/5709/artifact/ The tests we did with swiftlang#3938 shows that we can create directories but not write files. Modifications: Test creating `security` directory and writing file in it, and disable TOFU feature if the test fails.
yim-lee
added a commit
that referenced
this pull request
Dec 14, 2021
Motivation: Source compat test continues to fail even with #3928: ``` "You don’t have permission to save the file “fingerprints” in the folder “security” ``` https://ci.swift.org/job/swift-PR-source-compat-suite/5709/artifact/ The tests we did with #3938 shows that we can create directories but not write files. Modifications: Test creating `security` directory and writing file in it, and disable TOFU feature if the test fails.
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 14, 2021
Per review feedback, reverting part of the changes introduced in swiftlang#3928.
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 14, 2021
…ang#3941) Motivation: Source compat test continues to fail even with swiftlang#3928: ``` "You don’t have permission to save the file “fingerprints” in the folder “security” ``` https://ci.swift.org/job/swift-PR-source-compat-suite/5709/artifact/ The tests we did with swiftlang#3938 shows that we can create directories but not write files. Modifications: Test creating `security` directory and writing file in it, and disable TOFU feature if the test fails.
yim-lee
added a commit
to yim-lee/swift-package-manager
that referenced
this pull request
Dec 14, 2021
Per review feedback, reverting part of the changes introduced in swiftlang#3928.
yim-lee
added a commit
that referenced
this pull request
Dec 14, 2021
Per review feedback, reverting part of the changes introduced in #3928.
yim-lee
added a commit
that referenced
this pull request
Dec 15, 2021
* Disable fingerprint checking when storage is not available Motivation: Source compat test failure: https://ci.swift.org/job/swift-PR-source-compat-suite/5701/artifact/swift-source-compat-suite/ ``` error: Failed to get source control fingerprint for swift-log remoteSourceControl https://github.com/apple/swift-log.git version 1.4.2 from storage: Error Domain=NSCocoaErrorDomain Code=513 "You don't have permission to save the file "fingerprints" in the folder "security"." UserInfo={NSFilePath=/Users/buildnode/.swiftpm/security/fingerprints, NSUnderlyingError=0x7feaae439370 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}} error: Error Domain=NSCocoaErrorDomain Code=513 "You don't have permission to save the file "fingerprints" in the folder "security"." UserInfo={NSFilePath=/Users/buildnode/.swiftpm/security/fingerprints, NSUnderlyingError=0x7feaae439370 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}} ``` Modifications: - Make `PackageFingerprintStorage` optional in `RegistryClient` and `SourceControlPackageContainer`, which would turn off fingerprint read/write and essentially disable the TOFU feature. - `SwiftTool` will try to create the shared security directory (under which fingerprints are stored), and if it fails (e.g., permission errors) set `PackageFingerprintStorage` to none. - Don't perform integrity check on fingerprint write. The validation failure will happen on read. * Throw fingerprint write errors Per review feedback, reverting part of the changes introduced in #3928. * Create 'security' directory test should try writing files too (#3941) Motivation: Source compat test continues to fail even with #3928: ``` "You don’t have permission to save the file “fingerprints” in the folder “security” ``` https://ci.swift.org/job/swift-PR-source-compat-suite/5709/artifact/ The tests we did with #3938 shows that we can create directories but not write files. Modifications: Test creating `security` directory and writing file in it, and disable TOFU feature if the test fails. * Fix test that fails to compile (#3936) Co-authored-by: Doug Gregor <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation:
Source compat test failure: https://ci.swift.org/job/swift-PR-source-compat-suite/5701/artifact/swift-source-compat-suite/
Modifications:
PackageFingerprintStorageoptional inRegistryClientandSourceControlPackageContainer, which would turn off fingerprint read/write and essentially disable the TOFU feature.SwiftToolwill try to create the shared security directory (under which fingerprints are stored), and if it fails (e.g., permission errors) setPackageFingerprintStorageto none.