Skip to content

[bugfix] Write back lockfile after devbox update #1291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions internal/devpkg/package.go
Original file line number Diff line number Diff line change
Expand Up @@ -518,8 +518,8 @@ func (p *Package) ContentAddressedPath() (string, error) {

ux.Fwarning(
os.Stderr,
"calculating ca_store_path. This may be slow. "+
"Run `devbox update` to speed this up for next time.\n",
fmt.Sprintf("calculating ca_store_path for %s. This may be slow. "+
"Run `devbox update` to speed this up for next time.\n", sysInfo.StorePath),
)
localPath, err := nix.ContentAddressedStorePath(sysInfo.StorePath)
if err != nil {
Expand Down
30 changes: 11 additions & 19 deletions internal/impl/update.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ func (d *Devbox) Update(ctx context.Context, pkgs ...string) error {
}
}

if err := d.lockfile.Save(); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One downside to this approach is that if anything breaks while installing packages (which is fairly common due to stuff like insecure packages) then we leave the lockfile in a bad state that doesn't represent reality. This would have happened while updating nodejs@16

An alternative approach is to have local.IsUpToDate take a lock file instead of reading from disk. This would guarantee not running into this issue again. Similarly, localLock.Update() would take a lock file.

Not for this PR, but I think we could improve the local lock file API. Specifically I would make it private and just have the lock struct interact with it. e.g.

d.lockfile.LocalIsUpToDate()

And privatize local.Update (just have it be called by lock.Save() since they are both called together anyway.

This would make all the local stuff private.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative approach is to have local.IsUpToDate take a lock file instead of reading from disk.

Yeah.. I have an implementation of this, but I prefer to get this in and later think about how to refactor the local lock stuff, since it may have larger implications.

return err
}

if err := d.ensurePackagesAreInstalled(ctx, ensure); err != nil {
return err
}
Expand Down Expand Up @@ -107,34 +111,22 @@ func (d *Devbox) updateDevboxPackage(
return nil
}

// Check if the package's system info is missing, or not complete.
// Add any missing system infos for packages whose versions did not change.
if featureflag.RemoveNixpkgs.Enabled() {
userSystem, err := nix.System()
if err != nil {
return err
}

// If the newEntry has a system info for the user's system,
// then check if we need to update system info
if newEntry.Systems[userSystem] != nil {

// Check if the system info is missing for the user's system.
sysInfo := d.lockfile.Packages[pkg.Raw].Systems[userSystem]
if sysInfo == nil {
// If the newEntry has a system info for the user's system, then add/overwrite it. We don't overwrite
// other system infos because we don't want to clobber system-dependent CAStorePaths.
if newSysInfo, ok := newEntry.Systems[userSystem]; ok {
if !newSysInfo.Equals(existing.Systems[userSystem]) {
// We only guard this so that the ux messaging is accurate. We could overwrite every time.
if d.lockfile.Packages[pkg.Raw].Systems == nil {
d.lockfile.Packages[pkg.Raw].Systems = map[string]*lock.SystemInfo{}
}
d.lockfile.Packages[pkg.Raw].Systems[userSystem] = newEntry.Systems[userSystem]
ux.Finfo(d.writer, "Updated system information for %s\n", pkg)
return nil
}

// Check if the CAStorePath is missing for the user's system.
// Since any one user cannot add this field for all systems,
// we'll need to progressively add it to a project's lockfile.
if sysInfo.CAStorePath == "" {
// Update the CAStorePath for the user's system
d.lockfile.Packages[pkg.Raw].Systems[userSystem].CAStorePath = newEntry.Systems[userSystem].CAStorePath
d.lockfile.Packages[pkg.Raw].Systems[userSystem] = newSysInfo
ux.Finfo(d.writer, "Updated system information for %s\n", pkg)
return nil
}
Expand Down
11 changes: 9 additions & 2 deletions internal/lock/package.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ type Package struct {
type SystemInfo struct {
// StorePath is the input-addressed path for the nix package in /nix/store
// It is the cache key in the Binary Cache Store (cache.nixos.org)
// It is of the form <hash>-<name>-<version>
// It is of the form /nix/store/<hash>-<name>-<version>
// <name> may be different from the canonicalName so we store the full store path.
StorePath string `json:"store_path,omitempty"`
// CAStorePath is the content-addressed path for the nix package in /nix/store
// It is of the form <hash>-<name>-<version>
// It is of the form /nix/store/<hash>-<name>-<version>
CAStorePath string `json:"ca_store_path,omitempty"`
}

Expand All @@ -43,3 +43,10 @@ func (p *Package) IsAllowInsecure() bool {
}
return p.AllowInsecure
}

func (i *SystemInfo) Equals(other *SystemInfo) bool {
if i == nil || other == nil {
return i == other
}
return *i == *other
}