[update] Add sync flag which syncs all dependencies to same resolved version under current working dir #1501
Conversation
…version under current working dir
internal/lock/sync.go
Outdated
| return err | ||
| } | ||
|
|
||
| if !fi.IsDir() && filepath.Base(path) == "devbox.lock" { |
There was a problem hiding this comment.
Is this going to walk all of my node_modules files, or are we excluding directories according to .gitignore somewhere
There was a problem hiding this comment.
That's a good call. Yeah I think using .gitignore is good.
There was a problem hiding this comment.
Want to add that the monorepo might have a top level .gitignore that handles all the sub directories, and individual .gitignore in each subdirectory to add more stuff to ignore
internal/lock/sync.go
Outdated
| return err | ||
| } | ||
|
|
||
| if !fi.IsDir() && filepath.Base(path) == "devbox.lock" { |
internal/lock/sync.go
Outdated
| } | ||
| latestTime, err := time.Parse(time.RFC3339, latestPkg.LastModified) | ||
| if err != nil { | ||
| return err |
There was a problem hiding this comment.
Does this error if one of the lockfile is super old and does not have LastModified in there? (or is that scenario non existent)
There was a problem hiding this comment.
I'm not sure that ever happens, but we should handle that. I think a missing last modified should be considered old and replaced if possible.
internal/lock/sync.go
Outdated
| func latestPackages() (map[string]*Package, error) { | ||
| latestPackages := make(map[string]*Package) | ||
|
|
||
| err := filepath.Walk(".", func(path string, fi os.FileInfo, err error) error { |
There was a problem hiding this comment.
Is it possible to have a separate function that simply returns all the devbox.json file paths, so that we only walk it once? That way if a .gitignore doesn't catch a giant directory such as node_modules, at least the delay only happens once 🤷♀️
|
@LucilleH took me a bit to figure out how to handle the multi gitignores but I think I figured it out. This runs on devbox in 1s (which has 45 devbox.lock and similar number of gitignores) and runs in 300-600ms in our front-end repo with all dependencies installed.\ PTAL |
internal/lock/sync.go
Outdated
|
|
||
| var filterFunc = func(path string) bool { | ||
| // Start at the top of the stack which has most specific gitignore. | ||
| for i := len(filterFuncStack) - 1; i >= 0; i-- { |
There was a problem hiding this comment.
I'm trusting that this works as intended 😅
|
@LucilleH it works, but the performance improvement doesn't really work. Turns out that testing (almost) every single file and directory against gitignore slows it down more than the savings of not going into every directory. Given the complexity I lean towards removing it. |
|
There's a chance that limiting it to only directories could yield gains, but given how fast it is right now I'm tempted to remove the gitignore code and ship it. |
|
@mikeland73 sounds good. Just one more thing: can you try against the frontend repo with |
Looks like there's caching of some sort at the OS level. First run is 900ms and subsequent ones are ~350ms. |
Summary
Adds new
devbox update --sync-lockcommand. It syncs all lockfiles to the same version.I'm also thinking of adding a new
--sync-from [lockfile]flag for @LucilleH's use case. Specifically that would allow a certain lockfile to take priority.How was it tested?
devbox update --sync-lockon this repo (see lockfile changes)