[remove-nixpkgs] Disable remove-nixpkgs to fix non-search fallback #1463
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ipince
approved these changes
Sep 8, 2023
savil
added a commit
that referenced
this pull request
Sep 12, 2023
…1468) ## Summary In this PR, we loosen the conditions under which `devpkg.Package.IsInBinaryCache` can be called. Internally, it checks an in-memory map for the status of the package in the binary cache. Previously, it was _required_ that callers pre-compute this status by invoking `FillNarInfoInCache`. The motivation then was to ensure that we avoid a synchronous http request if `IsInBinaryCache` is called within a loop. However, this condition and the fact that golang lacks async-await support resulted in a loosely coupled load-and-check-later design whereby calls to `FillNarInfoInCache` would be sprinkled in various parts of the codebase and then `IsInBinaryCache` checked in other parts. This loose-coupling made it hard to reason about _why_ any particular FillNarInfoInCache was present. The straw that broke the proverbial camel's back was the issue that #1463 was seeking to fix. In this PR, we add a fallback http request inside `IsInBinaryCache` and remove all but one of the `FillNarInfoInCache` calls. Reason: 1. The remaining callsite is in the hot codepath of calculating `shellenv`. 2. The other callsites were in the codepaths of adding or installing packages. This is necessarily a slow operation, and so a small bit of synchrony is acceptable. NOTE: this PR also re-enables Remove Nixpkgs feature flag. ## How was it tested? - did some adhoc testing of opening a shell in an example project - added testscript unit-test for adding packages that are not in the Devbox Search Index. re: #1463's scenario - prior testscript unit-tests and project-tests should pass.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Restores functionality implemented by #1264 and refactored in #1279
which was accidentally broken by #1437
tldr: the
FillNarInfoCachefunction breaks if packages are not in search, but we support non-search packages via fallback (.e.g. stdenv.cc.cc.lib)I'm keeping surface area as small as possible, but as a follow up I think we can remove the
devpkg.FillNarInfoCachecalls fromimpl.Addif we simply ignore theIsInBinaryCacheerrors in ValidateExists or removeIsInBinaryCachefrom that function completely (I'm not convinced it is needed, @savil @ipince correct me if I'm wrong)How was it tested?
devbox add stdenv.cc.cc.lib