Adds documentation

docs/architecture.md

@@ -0,0 +1,48 @@
+# Architecture
+
+There are multiple components that play a part in the haskell.nix
+infrastructure.  These are `nix-tools`, `haskell.nix`, `hackage.nix`,
+`stackage.nix` and optionally `iohk-nix`.
+
+## [nix-tools](https://github.com/input-output-hk/nix-tools)
+
+nix-tools is a haskell package that provides the following tools:
+
+- `cabal-to-nix`: a `.cabal` to `.nix` transformer that retains
+  conditional expressions.
+
+- `stack-to-nix`: a `stack.yaml` to `.nix` transformer that will read
+  in a `stack.yaml` expression an generate a `pkgs.nix` file suited for
+  use with `haskell.nix`.
+
+- `plan-to-nix`: a `plan.json` to `.nix` transformer that will read in
+  a `plan.json` file and generate a `pkgs.nix` file suited for use
+  with `haskell.nix`.
+
+as well as a few other tools used to generate `hackage.nix` and `stackage.nix`.
+
+## [haskell.nix](https://github.com/input-output-hk/haskell.nix)
+
+haskell.nix is the runtime system for this haskell infrastructure.  It
+contains the component builder, as well as the system package and
+license mapping.  Without haskell.nix the expressions generated by
+either of the `nix-tools` tools make little sense on their own.
+
+## [hackage.nix](https://github.com/input-output-hk/hackage.nix)
+
+hackage.nix provides all cabal expressions from hackage as nix
+expressions.  It is periodically updated to keep in sync with the set
+of packages available on hackage.
+
+## [stackage.nix](https://github.com/input-output-hk/stackage.nix)
+
+stackage.nix is similar to hackage.nix but provides all stackage
+snapshots (lts, and nightly) as nix expressions.  It naturally depends
+on hackage.nix to resolve package names, versions and revisions to the
+repsective packages from hackage.nix.
+
+## [iohk-nix](https://github.com/input-output-hk/iohk-nix)
+
+iohk-nix is IOHK's shared nix library.  It provides some templates to
+make working with haskell.nix trivial but is non-essential to use
+haskell.nix infrastructure.

docs/index.md

@@ -0,0 +1,89 @@
+# haskell.nix
+
+haskell.nix is an alternative haskell infrastructure for nix.
+
+## Motivation
+
+Why do we need another haskell infrastructure for nix?  Doesn't nixpkgs
+provide a sufficiently good haskell infrastructure already?  These are
+good questions.  And it boils down to the following reasons for us to
+embark on a new infrastructure:
+
+- first class support for cross compilation
+- first calss support for package sets
+- component level control when building packages
+- reduction of `dontCheck` for cyclic dependencies
+- reducing build times by building libraries and tests in parallel
+- moving more logic into nix
+- decoupeling of haskell and nixpkgs
+
+### cross compilation
+
+`nixpkgs` has quite good support for cross compilation, however the
+haskell infrastructure suffers from the fact that it heavily relies on
+the `cabal2nix` tool.  `cabal2nix` (as well as tools that depend on it
+like `stack2nix`) flattens the `.cabal` file at conversion time to a
+given os/arch/flags configuration.  Thus to make cross compilation
+work with `cabal2nix` you will have to generate a separate `nix`
+expression for each configuration.  This becomes a major maintaince
+burden over time.  Therefore the tooling that translates cabal files
+into nix-expressions for use with haskell.nix retains the full
+contidional tree from the cabal file and exposes it to `nix`.  In
+addition it will also expose the `build-type` value, which allows us
+to cache the `Setup.hs` for build-type simple and not have to rebuild
+it every time.
+
+### package sets
+
+We often rely on either package sets as provided by stackage or
+computed by cabal.  `nixpkgs` provides it's own curated package set
+which might or might not work for the projects we work on.
+`stack2nix` tries to solve this issue, here we go one step further and
+provide the infrastructure to allow any form of package set.
+
+### component level control
+
+The haskell builder in `nixpkgs` provides control over executables and
+libraries, to build a specific executable only however is rather
+tricky to do.  This also leads to the cyclic dependencies issue.
+
+### cyclic dependencies
+
+Because the haskell builder in `nixpkgs` exposes packages at the
+package level, if packages mutually depend on each other through tests
+and libraries, lead to cyclic dependencies that nix can't resolve.  By
+exposing the components to nix as separate derivations this will only
+occure if you have mutally depdendent components.
+
+### build times
+
+The haskell builder in nixpkgs build package sequentially, first the
+library than the executables and finally the tests.  It then executes
+the tests before the package is considered done.  The upshot of this
+is that packages are only considered done if the test-suites
+passed.  The downside is that if you have to compile multiple packages
+the likelyhood of them failing is low, you have unnecessarily
+serialized you build.  In a more aggressive setting libraries could
+start building as early as their dependent libraries are built.  Of
+course they will have to be invalidated later should the test-suites
+of their dependencies fail, but this way we can make use of parallel
+building.  In an ideal scenario this will reduce build times close to
+the optimum.
+
+### more logic in nix
+
+The `cabal2nix` tool has a resolver that resolved system dependencies
+and licenses to values in `nixpkgs`.  This logic end up being a simple
+dictionary lookup and can be a simple nix expression.  This also
+offloads some of the work the cabal to nix translation tool needs to
+to into nix, and as such if changes are necessary (or needed to be
+performed ad-hoc) there is no need to rebuild the conversion tool and
+subsequently mark every derived expression as out of date.
+
+### decoupleing
+
+Finally by treating haskell.nix and nixpkgs as separate entities we
+can decouple the haskell packages and infrastructure from the nixpkgs
+package set, and rely on it to provide us with system packages while
+staying up to date with haskell packages from hackage while retaining
+a stable (or known to be good) nixpkgs revision.

docs/user-guide-cabal.md

@@ -0,0 +1,65 @@
+# User Guide (cabal project)
+
+Here we will look into how to generate the `pkgs.nix` file for a
+`cabal.project` project.  For the full integration please see the [User
+Guide](./user-guide)
+
+## Using `plan-to-nix`
+
+*We currently don't have a `project-to-nix` tool yet, as such creating
+ the relevant `pkgs.nix` file for a `cabal.project` is slightly more
+ involved than for a corresponding stack project*.
+
+With [nix-tools](https://github.com/input-output-hk/nix-tools) in
+`PATH`, we can simply run the following command on a stack project:
+
+```bash
+# make sure the cabal project is configured (the plan.json file is generated)
+cabal new-configure
+# convert the plan.json file into a pkgs.nix file
+plan-to-nix dist-newstyle/cache/plan.json > nix/plan.nix
+```
+
+This will produce a `nix/plan.nix` file that looks like the following:
+```nix
+hackage:
+  {
+    packages = {
+      "o-clock" = hackage.o-clock."0.1.1".revisions.default;
+      ...
+    };
+    compiler = { ... };
+  }
+```
+
+it specifically does not include any of our local packages yet. We
+will need to run
+
+```bash
+cabal-to-nix $path > nix/$pkg.nix
+```
+or
+```bash
+cabal-to-nix $url $rev > nix/$pkg.nix
+```
+for each local (or source) package.
+
+With this in place we can then proceed to build the `nix/pkgs.nix`
+file as follows:
+
+```nix
+let plan = import ./plan.nix; in
+{ ... }:
+{ pkg-def = plan;
+  overlay =
+    { local-package-a = ./local-package-a.nix;
+      local-package-b = ./local-package-b.nix;
+      source-import-a = ./source-import-a.nix;
+      source-import-b = ./source-import-b.nix;
+      ...
+    };
+}
+```
+
+*If you came here from the [User Guide](./user-guide), go back and
+ complete the setup.*

docs/user-guide-stack.md

@@ -0,0 +1,49 @@
+# User Guide (stack project)
+
+Here we will look into how to generate the `pkgs.nix` file for a
+`stack.yaml` project.  For the full integration please see the [User
+Guide](./user-guide)
+
+## Using `stack-to-nix`
+
+With [nix-tools](https://github.com/input-output-hk/nix-tools) in
+`PATH`, we can simply run the following command on a stack project:
+
+```bash
+stack-to-nix -o nix stack.yaml > nix/.stack-pkgs.nix
+```
+
+This will produce a `nix/.stack-pkgs.nix` file that looks like the following:
+```nix
+{
+  resolver = "lts-12.17";
+  overlay = hackage:
+    {
+      packages = {
+        "o-clock" = hackage.o-clock."0.1.1".revisions.default;
+        ...
+      } // {
+        my-package = ./.stack.nix/my-package.nix;
+        ...
+      };
+    };
+}
+```
+
+This file contains the stackage resolver, as well as an overlay of
+packages.  The overlay specifies which extra-deps (here: clock-0.1.1)
+we wanted to overlay over the stackage snapshot, and what local
+packages we want (here: my-package).
+
+We will then create the following `nix/pkgs.nix` file:
+
+```nix
+let stack-pkgs = import ./.stack-pkgs.nix; in
+{ stackage, ... }:
+{ pkg-def = stackage.${stack-pkgs.resolver};
+  inherit (stack-pkgs) overlay;
+}
+```
+
+*If you came here from the [User Guide](./user-guide), go back and
+ complete the setup.*

docs/user-guide.md

@@ -0,0 +1,113 @@
+# User Guide
+
+So you want to use `haskell.nix` with your stack or cabal project. The
+general appraoch will be to pick the right tool from `nix-tools` and
+produce a `pkgs.nix` expressions.  Getting a copy of the haskell.nix,
+hackage.nix (and potentially stackage.nix) source will then equip us
+to produce derivations that we can `nix build`.
+
+## Setup
+
+The general structure will be the same for haskell.nix, independent of
+the use of stack or cabal.  Let us assume for now that we have
+generated a `pkgs.nix` expression in `nix`.
+
+## default.nix
+
+We will start with defining a helper function in our `default.nix`
+that will allow us to override the source imports with `-I
+haskell=/path/to/haskell.nix` in case we need it.
+
+```nix
+{ pkgs ? import <nixpkgs> {}
+}:
+let
+  overrideWith = override: default:
+   let
+     try = builtins.tryEval (builtins.findFile builtins.nixPath override);
+   in if try.success then
+     builtins.trace "using search host <${override}>" try.value
+   else
+     default;
+in
+```
+
+Next we will use this to import `haskell.nix`, `hackage.nix` and
+`stackage.nix` (if we use a stack project).
+
+**NOTE**: update the `rev` and `sha256` values to the recent ones as
+  found on github.  Especially `hackage.nix` and `stackage.nix` will
+  evolve with package release on hackage and stackage releases
+  respectively.
+
+```nix
+let
+  # all packages from hackage as nix expressions
+  hackage = import (overrideWith "hackage"
+                    (pkgs.fetchFromGitHub { owner  = "input-output-hk";
+                                            repo   = "hackage.nix";
+                                            rev    = "1bee48237cb87d9ad6432a9ddb7777f858674400";
+                                            sha256 = "0sllxmffrcvcq63wlvz64s0i0ls2249vwix645899k6xh159z5pj";
+                                            name   = "hackage-exprs-source"; }))
+                   ;
+  # a different haskell infrastructure
+  haskell = import (overrideWith "haskell"
+                    (pkgs.fetchFromGitHub { owner  = "input-output-hk";
+                                            repo   = "haskell.nix";
+                                            rev    = "dae8025469d94e739aa52f23685bcd87840c72e3";
+                                            sha256 = "0bj577aj2v7yl6d1l6rinp0bbn4k9m8k4jd46m4hspqaisfd58hg";
+                                            name   = "haskell-lib-source"; }))
+                   hackage;
+
+  # the set of all stackage snapshots
+  stackage = import (overrideWith "stackage"
+                     (pkgs.fetchFromGitHub { owner  = "input-output-hk";
+                                             repo   = "stackage.nix";
+                                             rev    = "5ccfc7662469843768a5c4924d91faafbe5824e1";
+                                             sha256 = "1zwasyscqn4751i10165imwj4715hh5arwmccqkpvpn9bnb6c5ck";
+                                             name   = "stackage-snapshot-source"; }))
+                   ;
+in
+```
+
+Finally we string this together and produce a package set:
+
+```nix
+let
+  my-pkgs = import ./nix/pkgs.nix { inherit stackage; };
+
+  pkgSet = haskell.mkPkgSet {
+    inherit pkgs;
+    pkg-def = my-pkgs.pkg-def;
+    pkg-def-overlays = [
+      # this overlay will provide additional packages
+      # ontop of the package set.  E.g. extra-deps
+      # for stack packages. or local packages for
+      # cabal.projects
+      my-pkgs.overlay
+    ];
+    modules = [
+      # specific package overrides would go here
+      # example:
+      #  packages.cbors.patches = [ ./one.patch ];
+      #  packages.cbors.flags.optimize-gmp = false;
+    ];
+  };
+
+in pkgSet.config.hsPkgs // { _config = pkgSet.config; }
+```
+
+With this setup you can then start building the components of
+interest:
+
+```bash
+nix build -f default.nix $pkg.components.library
+```
+
+to build the library for `$pkg` or
+
+```bash
+nix build -f default.nix $pkg.components.exes.$exe
+```
+
+to build a specific executable. The same holds for test suites and benchmarks.

mkdocs.yml

@@ -0,0 +1,25 @@
+# You can use mkdocs from nix, by running the following:
+#
+#   nix-shell -p '[ mkdocs ]'
+#
+# that should pop you into a shell, where you can then
+# run
+#
+#   mkdocs serve
+#
+# and point your browser to localhost:8000
+#
+site_name: haskell.nix - alt .\ infra
+theme: readthedocs
+site_url: null
+repo_url: https://github.com/input-output-hk/haskell.nix
+
+# this is called `nav` in more recent mkdocs versions. Mine
+# right now is 0.17.5
+pages:
+  - 'Introduction': index.md
+  - 'Architecture': architecture.md
+  - 'User Guide': user-guide.md
+  - 'User Guide (Stack)': user-guide-stack.md
+  - 'User Guide (Cabal)': user-guide-cabal.md
+  - 'Removing withPackage wrapper': removing-with-package-wrapper.md