Add PackageRegistryManager

[mattt]

A partial tree of #3640 containing additive changes that don't affect how dependencies are downloaded or resolved. This PR itself shouldn't affect the behavior of Swift Package Manager, and is intended to serve as a backstop should we need to rollback any changes related to package resolution.

Motivation:

Improve velocity of registry implementation and reduce risk of merging future changes.

Modifications:

• Add RegistryManager and associated tests
• Add SourceArchiver used by RegistryManager
• Extract Registry from RegistryConfiguration
• Add scopeAndName property to PackageIdentity
• Raise error when dependency is declared using invalid package identifier
• Rename identity to id in PackageDescription to match SE-0292

Result:

No changes.

[mattt]

@swift-ci Please smoke test

[tomerd]

iirc one can configure the request with acceptable response codes to avoid such check, tho you do make other checks here as well

[mattt]

Yeah, if it were just the status check or a range of status codes, I would've gone with that. But it's less code overall to do it this way here.

[tomerd]

move to Basics?

[tomerd]

I think there is some code in workspace that also does similar things (for binary artifacts). maybe we can reuse / converge on one implementation?

[mattt]

There's ZipArchiver in TSCUtility, which I based this type on: https://github.com/apple/swift-tools-support-core/blob/f9bbd6b80d67408021576adf6247e17c2e957d92/Sources/TSCUtility/Archiver.swift#L34

(Hence some of the callouts from copy-pasted code, like dispatching to the global queue)

I discuss the rationale for this type in the doc comments, but the main takeaway is that this is probably only useful for the registry. I'd also be interested in getting some more 👀 on this, especially from security folks.

As far as converging on a single implementation, there may be an opportunity to make the delegated system call and directory prefix stripping a configurable property. But at that point, I think you may as well have two separate implementations of the Archiver protocol.

[tomerd]

I discuss the rationale for this type in the doc comments, but the main takeaway is that this is probably only useful for the registry. I'd also be interested in getting some more 👀 on this, especially from security folks.

cc @andyp-apple and @robertlacroix

As far as converging on a single implementation, there may be an opportunity to make the delegated system call and directory prefix stripping a configurable property. But at that point, I think you may as well have two separate implementations of the Archiver protocol.

makes sense, we can revisit this later too

[tomerd]

try? intentional? should it throw instead?

[mattt]

If we throw here, package resolution would fail for anyone who doesn't have a .swiftpm/configuration/registries.json file, even if they only use URL-based dependencies. The check for valid registry configuration is pushed back to be done only when necessary.

[tomerd]

in similar situation in SwiftTool we return nil when it okay for it to be missing. so how about changing self.getRegistriesConfig to return optional when configuration/registries.json does not exist, and throw if it exists but cant be loaded?

[mattt]

Sure thing! Fixed by dd323fb.

And I was mistaken — doing try here shouldn't throw an error if the configuration file is missing. Instead, it should return an empty configuration object.

[tomerd]

lgtm, couple of small things

[mattt]

@swift-ci Please smoke test

[tomerd]

@mattt seems like this is ready to go? do you want to squash and provide the commit message, or should I do best-effort while merging?

[mattt]

@tomerd Yeah, this is good to go. Thanks for your help on getting this ready. Feel free to squash and merge whenever you see the CI go ✅ .