Add evalSystem and evalPackages project args

[hamishmack]

This adds a way to specify the evalSystem or evalPackages explicitly when calling the project functions.

Currently if we want to make a flake that supports multiple systems we have few options:

• Require builders for all the supported systems (even just for nix flake show).

• Pass --impure so that haskell.nix can see builtins.currentSystem to set up pkgs.evalPackages to use that. Unfortunately this prevents nix from caching some of the work it does and often results in it recalculating for each supported system when it would otherwise be cached and take no time at all.

• Add an overlay to replace evalPackages. This works, but it is not straight forward.

• Materialize the nix files for the project.

This change allows evalSystem = "x86_64-linux"; to be passed telling haskell.nix to run cabal and nix-tools on that system. The user will have to have a builder for that system, but does not need to have builders for the others (unless building outputs for them).

[michaelpj]

I wonder if it would make sense to ask the user to provide evalPackages instead? The only way we use evalSystem is to get evalPackages, and it means we have to do the awful dance with re-importing nixpkgs by getting the original path and everything, whereas the user should just be able to re-import their nixpkgs straightforwardly, so it makes things a little less evil. WDYT?

[hamishmack]

I have removed evalPackages from pkgs and replaced it with an evalPackages arg. I had to refactor hackage-project (and the functions that use it) to be nix modules based (so that the nix-tools argument could be passed correctly to match evalPackages). The hackage-quirks are now a list of project level nix modules used by hackage-package.

I removed caller arg as it was not actually used and I think name is more useful to have in error messages anyway.

I left evalSystem in, but it is just used in the default for evalPackages now.

[michaelpj]

Maybe this was a bad idea, I don't know. It seems like we now don't have evalPackages in the places where we need them?

[michaelpj]

Are these changes actually okay? Won't this again mean that you can't eval without a build machine of the right system, because it will try and clean the source on a build machine?

[michaelpj]

Same for the other places where this switch happens.

[hamishmack]

I am not sure about cleanGit. It should be ok on CI where .git does not exist, but it may fail for local evals. I'll do some tests.

[michaelpj]

Yeah, I guess it should be fairly obvious if you try building a darwin derivation when you don't have a darwin machine or something. If it works then I'm happy! It just seems suspicious - presumably it means that the usage of evalPackages was actually unncessary in each of these places to begin with?

[hamishmack]

I think there is a situation when not using flakes (so cleanGit would have work to do) and cross compiling where this would have been useful so I have included a note in the change log.

[michaelpj]

could also throw in this case and require the user to specify?

[hamishmack]

I've changed the default to use buildPackages. I also removed the default to make sure nothing in haskell.nix itself (including the CI) depends on the default. The import code here should now only get used if evalSystem was explicitly passed in (for instance if the user passed evalSystem = builtins.currentSystem;).

[michaelpj]

Can we not do this? it just makes it hard to know what types things have

[hamishmack]

I have changed hackage-tool so it only takes a single module. If there is a way to compose nix modules we could change the others too (hackage-package, hackage-project, cabalProject, etc.). Alternatively we could also split out variants that take a list and have wrappers for those that take a single module. Let me know if you think that would be nicer.

[michaelpj]

I think we suffer a lot from not having types, so I guess the heuristic I end up going for a lot is that an argument named X should always have the same type. So I would just use a list of modules everywhere just so it's super consistent, even if that's a bit more annoying in places where you probably only want one?

[hamishmack]

I think it might make sense to move cabalProject and stackProject code from overlays/haskell.nix to modules/cabal-project.nix and overlays/stack-project.nix. I think then we might be able to simplify the way we eval the project modules. However I started trying that and it seems like a much bigger change. So for now I have changed hackage-tool to accept only a list.

[hamishmack]

bors try

[iohk-bors]

try

Build failed:

• buildkite/haskell-dot-nix

[hamishmack]

bors try

[iohk-bors]

try

Build succeeded:

• buildkite/haskell-dot-nix
• ci/hydra:Cardano:haskell-nix:required

[dhess]

I believe this commit broke the evaluation of our haskell.nix project with the following error:

error: value is the partially applied built-in function 'mapAttrs' while a set was expected
(use '--show-trace' to show detailed location information)

I've tracked it down to this commit by pinning our haskell.nix Flake input to 7f02c05; after that commit, this error occurs during evaluation.

edit: Here's a trace:

error: value is the partially applied built-in function 'mapAttrs' while a set was expected

       at /nix/store/sg4hv9sjzlp9xp4qr7vdcmh7s1cblyha-source/flake.nix:303:19:

          302|                   #ghc8107Tools.cabal-edit
          303|                   ghc8107Tools.cabal-fmt
             |                   ^
          304|

       … while evaluating 'isDerivation'

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/lib/attrsets.nix:427:18:

          426|   */
          427|   isDerivation = x: x.type or null == "derivation";
             |                  ^
          428|

       … from call site

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/pkgs/stdenv/generic/make-derivation.nix:193:8:

          192|   checkDependencyList' = positions: name: deps: lib.flip lib.imap1 deps (index: dep:
          193|     if lib.isDerivation dep || isNull dep || builtins.typeOf dep == "string" || builtins.typeOf dep == "path" then dep
             |        ^
          194|     else if lib.isList dep then checkDependencyList' ([index] ++ positions) name dep

       … while evaluating anonymous lambda

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/pkgs/stdenv/generic/make-derivation.nix:192:81:

          191|   checkDependencyList = checkDependencyList' [];
          192|   checkDependencyList' = positions: name: deps: lib.flip lib.imap1 deps (index: dep:
             |                                                                                 ^
          193|     if lib.isDerivation dep || isNull dep || builtins.typeOf dep == "string" || builtins.typeOf dep == "path" then dep

       … from call site

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/lib/lists.nix:117:32:

          116|   */
          117|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                                ^
          118|

       … while evaluating anonymous lambda

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/lib/lists.nix:117:29:

          116|   */
          117|   imap1 = f: list: genList (n: f (n + 1) (elemAt list n)) (length list);
             |                             ^
          118|

       … from call site

       … while evaluating anonymous lambda

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/pkgs/stdenv/generic/make-derivation.nix:221:13:

          220|       (map (drv: drv.__spliced.hostHost or drv) (checkDependencyList "depsHostHost" depsHostHost))
          221|       (map (drv: drv.crossDrv or drv) (checkDependencyList "buildInputs" buildInputs))
             |             ^
          222|     ]

       … from call site

       … while evaluating 'getOutput'

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/lib/attrsets.nix:598:23:

          597|   */
          598|   getOutput = output: pkg:
             |                       ^
          599|     if ! pkg ? outputSpecified || ! pkg.outputSpecified

       … from call site

       … while evaluating the attribute 'buildInputs' of the derivation 'ghc-shell-for-packages'

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/pkgs/stdenv/generic/make-derivation.nix:270:7:

          269|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          270|       name =
             |       ^
          271|         let

       … while evaluating the attribute 'drvPath'

       at /nix/store/l4ysisamwr0jdf6w2za4zygj9ajlf0wb-source/lib/customisation.nix:210:7:

          209|     in commonAttrs // {
          210|       drvPath = assert condition; drv.drvPath;
             |       ^
          211|       outPath = assert condition; drv.outPath;

where ghc8107Tools is defined in our overlay as:

ghc8107Tools = final.haskell-nix.tools "ghc8107" {
    cabal-fmt = "latest";
    cabal-edit = "latest";
};

[hamishmack]

Should be fixed by #1570

[dhess]

Should be fixed by #1570

Thanks! I can confirm that it's fixed now.