Skip to content

Reduce cabalProject IFD deps using dummy GHCs #607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 15, 2020
Merged

Conversation

hamishmack
Copy link
Collaborator

The cabalProject function works by running cabal v2-configure followed by plan-to-nix. However for v2-configure to correctly calculate the plan (including flag settings and buildable components) it needs to know what GHC we want to use.

This causes significant problems:

  • For hydra to assemble a list of jobs from components.tests it must first have GHC that will be used. If a patch has been applied to the GHC to be used it must be rebuilt before the list of jobs can be assembled. If a lot of different GHCs are being tests that can be a lot of work all happening in the eval stage where little feedback is available.

  • Once the jobs are running the compilation of the GHC needed (the eval stage already must have done it, but the outputs there are apparently not added to the cache) happens inside the IFD part of cabalProject. This causes a very large amount of work to be done in the IFD and our understanding is that this can cause problems on nix and/or hydra.

  • When using cabalProject we cannot examine the properties of the project without building or downloading the GHC (less of an issue as we would normally need it soon anyway).

The solution here is to:

  • Create a dummy ghc and ghc-pkg from the real ones by capturing the ghc and ghc-pkg outputs that cabal v2-configure requests.

  • The dummy ghc and ghc-pkg is used instead of the real one in cabalProject when running cabal v2-configure.

  • The captured outputs are materialized into the haskell.nix repo so that the real GHC is only needed when checkMaterialization is set.

@hamishmack hamishmack requested a review from angerman May 15, 2020 02:32
@angerman angerman merged commit 341341c into master May 15, 2020
@angerman angerman deleted the hkm/dummy-ghc branch May 15, 2020 06:00
@rvl
Copy link
Contributor

rvl commented May 15, 2020

This is quite impressive @hamishmack 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants