Reorganize tutorials #702

domenkozar · 2020-06-17T13:14:45Z

I've spend some time on ZuriHac helping and observing others and used that knowledge to restructure things.

renamed User Guide to Tutorials as it's more generally accepted term
moved Getting Started into the first Tutorial to ease confusion where to start
moved README to Getting Started as everyone went directly into documentation skipping the most useful bits in README
manually generating Nix files is now it's own tutorial, but by defafult IFD is what getting started offers
merged Cabal and Stack getting started snippet for easier copy pasting and less confusing flow
moved a bunch of things from getting started into their own tutorials
renamed tutorials to better suit what the user might want to do

domenkozar · 2020-06-17T14:03:08Z

michaelpj · 2020-06-17T15:09:44Z

mkdocs.yml

-    - 'How to cross-compile your project': user-guide/cross-compilation.md
+  - 'Tutorials':
+    - 'Getting Started': tutorials/getting-started.md
+    - 'Mapping non-Haskell dependencies to Nixpkgs': dev/pkg-map.md


I'm not sure this should be in here, or not so prominent. Most people shouldn't have to do this, it's more advanced/troubleshooting.

I think it's quite common to hit this (someone is having trouble just now on #haskell.nix, but the tutorial should be expanded), currently it's 4th item in the tutorial list.

Also cover #674

michaelpj · 2020-06-17T15:13:06Z

docs/tutorials/getting-started.md

+
+Assuming you have [Nix installed](https://nixos.org/download.html), you can start setting up your project.
+
+## Setting up the Cachix binary cache


I think this should be in the main README too: we basically can't say this enough. This is the #1 thing where if you don't do it you'll have a really bad time.

The basic idea is that everyone should go through getting started. I confirmed that's what people were doing.

The reason noone did this was that setting up caches was always a step after installing nix-tools or building, which is way too late.

There is a slightly related issue in that our primary cache is still hydra :-/

@angerman not sure what exactly to do about that, I've copied what you had before, but let me know if we should change to something else.

I'm quite biased here, but Cachix is easier for people to setup.

Certainly. Our Hercules instance feeds the cachix cache. Due to some inability to upgrade Hercules for a while it’s been falling behind. Hydra is the primary ci and feeds exclusively into the iohk hydra cache.

michaelpj · 2020-06-17T15:13:59Z

docs/tutorials/getting-started.md

+## Scaffolding
+
+The following configuration will use `stack.yaml` if it exists,
+otherwise fallback to `cabal.project`.


I'm not sure about this: nobody actually wants this (they have one or the other or they'll pick one), and it actually makes the example more complicated since we have this file existence testing stuff.

I think it's fine to just have two examples.

I think that it's better to give one example to copy and people can edit out their own workflow.

Most important feature is that scaffold gives a good default.

Just a few days ago I fixed 2793871 because the two templates weren't in sync.

Also having to split the flow during a getting started is considered bad practice in documentation.

We could alternatively offer two snippets that user can comment/uncomment, but that means copy/pasting just won't work out of the box.

There are tradeoffs here, but I think saying prefer Stack over Cabal is a good default.

Stack over Cabal

You might have come to the wrong repo 😈; especially with Stack's pantry stuff, which we simply can't support this is a dangerous direction to go as people will expect Stack to be the preferred tool of choice. For the vast majority of iohk that's simply not the case, and thus the likelihood of Cabal fixes will be much higher than for Stack. Simply because we hardly use Stack at all, and thus won't see much of Stack breakage outside of a selected few repositories. We sadly rely on the community to fix Stack to a large degree 😢

michaelpj · 2020-06-17T15:14:30Z

docs/tutorials/getting-started.md

+  }
+```
+
+!!! note "git dependencies"


Markdown has admonitions now?

As an extension :)

michaelpj · 2020-06-17T15:14:59Z

docs/tutorials/getting-started.md

+nix-build -A your-package-name.components.library
+```
+
+To build an executable:


Maybe a good place to explain the general structure of the attributes?

michaelpj · 2020-06-17T15:16:05Z

docs/tutorials/hackage-stackage.md

+  })]; })
+, haskellNixArgs ? (import (builtins.fetchTarball https://github.com/input-output-hk/haskell.nix/archive/master.tar.gz) {}).nixpkgsArgs
+}:
+  pkgs


This is different to the way described in the existing docs - any reason for that? (I prefer the current way)

This was moved from getting started. I'm a bit confused as there are two ways to specify, via hackageSourceJSON and sourcesOverride. We should probably pick one and use it consistently.

Ah, I mised that :) We're gradually moving towards niv for everything, so hackageSourceJSON will hopefully vanish in favour of sourceOverrides.

Maybe we can just point to the other doc about pinning hackage/stackage?

What other docs do you have in mind?

I agree about sourceOverrides, it's a more general interface.

hackage-stackage.md

Wait, this is the same file, it's just a few paragraphs up!

michaelpj · 2020-06-17T15:18:25Z

mkdocs.yml

+    - 'Mapping non-Haskell dependencies to Nixpkgs': tutorials/pkg-map.md
+    - 'Bumping Hackage and Stackage snapshots': tutorials/hackage-stackage.md
+    - 'Cross-compiling your project': tutorials/cross-compilation.md
+    - 'Installing nix-tools': tutorials/installing-nix-tools.md


I also don't think users should usually do this. You can do materialization and everything without ever doing this. Another advanced/troubleshooting topic IMO.

That's why I moved it to a separate tutorial. I wouldn't split tutorials into basic/advanced, but rather explain in the tutorial itself when something is needed and what's the goal.

So you're saying that manually generating expressions is never really recommended anymore and materialized is the new way to go avoiding IFD?

Not sure I can answer that fully, but at least you can get fully materialized Nix files for your project without ever installing nix-tools, just by running specific scripts to generate the files (which of course use nix-tools under the hood).

@angerman what do you think? It would be great if we can communicate that in the documentation.

Either way I think having docs to install nix-tools is useful, but maybe they can be moved to ceveloper section.

Can we change the "Manually generating Nix expressions" tutorial so that it shows how to use the materialization scriptt? Then the info about installing and using nix-tools and can be put in a dev note.

The whole materialisation mechanism is there to supersede the need for manual generation of expression with nix tools and doing so in a principled way that reduces potential pitfalls.

Conceptually pre-generating nix expressions is a form of caching. Caching is hard. Materialisation is a form of making sure haskell.nix generates the right nix expression you want in the first place. And then telling you how to copy that "cached" value out, and use it instead of recomputing.

Generally it is not a bad idea to understand how this all works the very basic level; this will provide intuition for this, and understanding how things fit together. My general approach when teaching concepts is to start at the very basic labour intensive setup. Then do the same thing again but use one level of abstraction; and finally do it all over again but this time use yet another layer of abstraction. This should help the person learning it to understand how the abstractions try to group together functionality and produce shorthands, that the student would have hopefully arrived at on his own as well. Now finding the right abstractions is really hard. Have we found them with haskell.nix? I'm not sure... only time will tell.

Thus, yes the recommended way is: IFD > materialisation > custom regeneration script > some custom setup that sits on top of what nix-tools generates.

The educational route is precisely the opposite way.

🤷‍♂️

Most people won't want to understand it, but get things done with it.

I think that's a much bigger audience than those wanting to fiddle, so those docs should exist.

I would propose to address materialization vs manual expression in a separate PR after this one is merged.

rvl

Nice!

rvl · 2020-06-17T23:42:24Z

mkdocs.yml

+    - 'Mapping non-Haskell dependencies to Nixpkgs': tutorials/pkg-map.md
+    - 'Bumping Hackage and Stackage snapshots': tutorials/hackage-stackage.md
+    - 'Cross-compiling your project': tutorials/cross-compilation.md
+    - 'Installing nix-tools': tutorials/installing-nix-tools.md


Can we change the "Manually generating Nix expressions" tutorial so that it shows how to use the materialization scriptt? Then the info about installing and using nix-tools and can be put in a dev note.

rvl · 2020-06-17T23:44:21Z

docs/tutorials/manually-generating-nix-expressions.md

+We believe that imports from derivations (IFDs) provide tremendous
+value in nix and the aversion towards them stems mostly from 
+poor tooling and ci support for them.  We do not believe
+that poor tooling or ci support should cripple nix capability
+of abstraction.  Hence haskell.nix makes excessive use of
+IFDs.


https://youtu.be/RFzZ-xpyfoo

docs/tutorials/development.md

Co-authored-by: Rodney Lorrimar <[email protected]>

domenkozar · 2020-06-18T14:58:46Z

@angerman I've added an implemention of getting started as discused on #haskell.nix and removed duplicated pinning content. Happy to improve this further in more focused approach :)

hamishmack · 2020-06-18T15:58:06Z

Would this help #703 ?

domenkozar · 2020-06-18T16:42:48Z

Yes! That's great! I think we ought to remove so many layers, but let's first get the public interface right. One step at the time.

* fix hackage and stackage to work * user-guides -> tutorials * Restructure tutorials * getting-started: clarify on packages and components * Update docs/tutorials/development.md Co-authored-by: Rodney Lorrimar <[email protected]> * getting-started: bail out if cabal.project and stack.yaml are present * hackage-stackage: delete duplicated content Co-authored-by: Rodney Lorrimar <[email protected]>

domenkozar added 3 commits June 16, 2020 10:58

fix hackage and stackage to work

b382dd0

user-guides -> tutorials

0963b03

Restructure tutorials

f951513

domenkozar mentioned this pull request Jun 17, 2020

Improve stack workflow/UX with haskell.nix #689

Closed

2 tasks

michaelpj reviewed Jun 17, 2020

View reviewed changes

getting-started: clarify on packages and components

7e17064

rvl approved these changes Jun 17, 2020

View reviewed changes

domenkozar and others added 3 commits June 18, 2020 11:15

Update docs/tutorials/development.md

f7ab391

Co-authored-by: Rodney Lorrimar <[email protected]>

getting-started: bail out if cabal.project and stack.yaml are present

306621c

hackage-stackage: delete duplicated content

4c76a6b

michaelpj approved these changes Jun 18, 2020

View reviewed changes

hamishmack merged commit 2852a95 into input-output-hk:master Jun 19, 2020


		Assuming you have [Nix installed](https://nixos.org/download.html), you can start setting up your project.

		## Setting up the Cachix binary cache

Reorganize tutorials #702

Reorganize tutorials #702

Uh oh!

Conversation

domenkozar commented Jun 17, 2020

Uh oh!

domenkozar commented Jun 17, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rvl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

domenkozar commented Jun 18, 2020

Uh oh!

hamishmack commented Jun 18, 2020

Uh oh!

domenkozar commented Jun 18, 2020

Uh oh!