Copy draft-3 user guide.

Author: Peter Amstutz

v1.0/UserGuide.yml

@@ -3,6 +3,9 @@
   doc:
     - $include: userguide-intro.md
 
+    - |
+      # Wrapping Command Line Tools
+
     - |
       ## First example
 
@@ -11,9 +14,9 @@
       output.  CWL documents are written in [JSON](http://json.org) or
       [YAML](http://yaml.org), or a mix of the two.
 
-      *1st.cwl*
+      *1st-tool.cwl*
       ```
-    - $include: examples/1st.cwl
+    - $include: examples/1st-tool.cwl
     - |
       ```
 
@@ -29,7 +32,7 @@
       command line:
 
       ```
-      $ cwl-runner 1st.cwl echo-job.yml
+      $ cwl-runner 1st-tool.cwl echo-job.yml
       [job 140199012414352] $ echo 'Hello world!'
       Hello world!
       Final process status is success
@@ -311,7 +314,7 @@
       command line:
       ```
       $ rm hello.tar || true && touch goodbye.txt && tar -cvf hello.tar
-      $ cwl-runner example-tar-param.cwl example-tar-param-job.yml
+      $ cwl-runner tar-param.cwl tar-param-job.yml
       [job 139868145165200] $ tar xf /home/example/hello.tar goodbye.txt
       Final process status is success
       {
@@ -480,8 +483,7 @@
 
       ```
       $ cwl-runner array-inputs.cwl array-inputs-job.yml
-      /home/peter/work/cwl-venv/bin/cwltool 1.0.20160325210917
-      [job 140334923640912] /home/peter/work/common-workflow-language/draft-3/examples$ echo -A one two three -B=four -B=five -B=six -C=seven,eight,nine
+      [job 140334923640912] /home/example$ echo -A one two three -B=four -B=five -B=six -C=seven,eight,nine
       -A one two three -B=four -B=five -B=six -C=seven,eight,nine
       Final process status is success
       {}
@@ -497,6 +499,48 @@
       You can specify arrays of arrays, arrays of records, and other complex
       types.
 
+    - |
+      ## Array outputs
+
+      You can also capture multiple output files into an array of files using `glob`.
+
+      *array-outputs.cwl*
+      ```
+    - $include: examples/array-outputs.cwl
+    - |
+      ```
+
+      *array-outpust-job.yml*
+      ```
+    - $include: examples/array-outputs-job.yml
+    - |
+      ```
+
+      Now invoke `cwl-runner` providing the tool wrapper and the input object
+      on the command line:
+
+      ```
+      $ cwl-runner array-outputs.cwl array-outputs-job.yml
+      [job 140190876078160] /home/example$ touch foo.txt bar.dat baz.txt
+      Final process status is success
+      {
+        "output": [
+          {
+            "size": 0,
+            "path": "/home/peter/work/common-workflow-language/draft-3/examples/foo.txt",
+            "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
+            "class": "File"
+          },
+          {
+            "size": 0,
+            "path": "/home/peter/work/common-workflow-language/draft-3/examples/baz.txt",
+            "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
+            "class": "File"
+          }
+        ]
+      }
+      ```
+
     - |
       ## Record inputs, dependent and mutually exclusive parameters
 
@@ -677,14 +721,143 @@
 
       ```
       $ cwl-runner linkfile.cwl arguments-job.yml
-      [job 139928309171664] /home/example$ docker run -i --volume=/home/peter/work/common-workflow-language/draft-3/examples/Hello.java:/var/lib/cwl/job557617295_examples/Hello.java:ro --volume=/home/peter/work/common-workflow-language/draft-3/examples:/var/spool/cwl:rw --volume=/tmp/tmpmNbApw:/tmp:rw --workdir=/var/spool/cwl --read-only=true --net=none --user=1001 --rm --env=TMPDIR=/tmp java:7 javac Hello.java
+      [job 139928309171664] /home/example$ docker run -i --volume=/home/example/Hello.java:/var/lib/cwl/job557617295_examples/Hello.java:ro --volume=/home/example:/var/spool/cwl:rw --volume=/tmp/tmpmNbApw:/tmp:rw --workdir=/var/spool/cwl --read-only=true --net=none --user=1001 --rm --env=TMPDIR=/tmp java:7 javac Hello.java
       Final process status is success
       {
       "classfile": {
         "size": 416,
-        "path": "/home/peter/work/common-workflow-language/draft-3/examples/Hello.class",
+        "path": "/home/example/Hello.class",
         "checksum": "sha1$2f7ac33c1f3aac3f1fec7b936b6562422c85b38a",
         "class": "File"
         }
       }
       ```
+
+    - |
+      # Writing Workflows
+
+      ## First workflow
+
+      This workflow extracts a java source file from a tar file and then
+      compiles it.
+
+      *1st-workflow.cwl*
+      ```
+    - $include: examples/1st-workflow.cwl
+    - |
+      ```
+
+      Use a JSON object in a separate file to describe the input of a run:
+
+      *1st-workflow-job.yml*
+      ```
+    - $include: examples/1st-workflow-job.yml
+    - |
+      ```
+
+      Now invoke `cwl-runner` with the tool wrapper and the input object on the
+      command line:
+
+      ```
+      $ cwl-runner 1st-workflow.cwl 1st-workflow-job.yml
+      [job untar] /tmp/tmp94qFiM$ tar xf /home/example/hello.tar Hello.java
+      [step untar] completion status is success
+      [job compile] /tmp/tmpu1iaKL$ docker run -i --volume=/tmp/tmp94qFiM/Hello.java:/var/lib/cwl/job301600808_tmp94qFiM/Hello.java:ro --volume=/tmp/tmpu1iaKL:/var/spool/cwl:rw --volume=/tmp/tmpfZnNdR:/tmp:rw --workdir=/var/spool/cwl --read-only=true --net=none --user=1001 --rm --env=TMPDIR=/tmp java:7 javac -d /var/spool/cwl /var/lib/cwl/job301600808_tmp94qFiM/Hello.java
+      [step compile] completion status is success
+      [workflow 1st-workflow.cwl] outdir is /home/example
+      Final process status is success
+      {
+        "classout": {
+          "path": "/home/example/Hello.class",
+          "checksum": "sha1$2f7ac33c1f3aac3f1fec7b936b6562422c85b38a",
+          "class": "File",
+          "size": 416
+        }
+      }
+      ```
+
+      What's going on here?  Let's break it down:
+
+      ```
+      cwlVersion: cwl:draft-3
+      class: Workflow
+      ```
+
+      The 'cwlVersion` field indicates the version of the CWL spec used by the
+      document.  The `class` field indicates this document describes a workflow.
+
+
+      ```
+      inputs:
+        - id: inp
+          type: File
+        - id: ex
+          type: string
+      ```
+
+      The `inputs` section describes the inputs of the workflow.  This is a
+      list of input parameters where each parameter consists of an identifier
+      and a data type.  These parameters can be used as sources for input to
+      specific workflows steps.
+
+      ```
+      outputs:
+        - id: classout
+          type: File
+          source: "#compile/classfile"
+      ```
+
+      The `outputs` section describes the outputs of the workflow.  This is a
+      list of output parameters where each parameter consists of an identifier
+      and a data type.  The `source` connects the output parameter `classfile`
+      of the `compile` step to the workflow output parameter `classout`.
+
+      ```
+      steps:
+        - id: untar
+          run: tar-param.cwl
+          inputs:
+            - id: tarfile
+              source: "#inp"
+            - id: extractfile
+              source: "#ex"
+          outputs:
+            - id: example_out
+      ```
+
+      The `steps` section describes the actual steps of the workflow.  In this
+      example, the first step extracts a file from a tar file, and the second
+      step compiles the file from the first step using the java compiler.
+      Workflow steps are not necessarily run in the order they are listed,
+      instead the order is determined by the dependencies between steps (using
+      `source`).  In addition, workflow steps which do not depend on one
+      another may run in parallel.
+
+      The first step, `untar` runs `tar-param.cwl` (described previously in
+      [Parameter references](#Parameter_references)).  This tool has two input
+      parameters, `tarfile` and `extractfile` and one output parameter
+      `example_out`.
+
+      The `inputs` section of the workflow step connects these two input
+      parameters to the inputs of the workflow, `#inp` and `#ex` using
+      `source`.  This means that when the workflow step is executed, the values
+      assigned to `#inp` and `#ex` will be used for the parameters `tarfile`
+      and `extractfile` in order to run the tool.
+
+      The `outputs` section of the workflow step lists the output parameters
+      that are expected from the tool.
+
+      ```
+        - id: compile
+          run: arguments.cwl
+          inputs:
+            - id: src
+              source: "#untar/example_out"
+          outputs:
+            - id: classfile
+      ```
+
+      The second step `compile` depends on the results from the first step by
+      connecting the input parameter `src` to the output parameter of `untar`
+      using `#untar/example_out`.  The output of this step `classfile` is
+      connected to the `outputs` section for the Workflow, described above.

v1.0/examples/1st.cwl

@@ -1,5 +1,6 @@
 cwlVersion: cwl:draft-3
 class: CommandLineTool
+label: Example trivial wrapper for Java 7 compiler
 baseCommand: javac
 hints:
   - class: DockerRequirement

v1.0/examples/arguments.cwl

@@ -4,7 +4,9 @@ Hello!
 
 This guide will introduce you to writing tool wrappers and workflows using the
 Common Workflow Language (CWL).  This guide describes the current stable
-specification, draft 3.
+specification, version 1.0.
+
+Note: This document is a work in progress.  Not all features are covered, yet.
 
 <!--ToC-->
 
@@ -24,5 +26,3 @@ technologies such as Docker, be used with CWL implementations from different
 vendors, and is well suited for describing large-scale workflows in cluster,
 cloud and high performance computing environments where tasks are scheduled in
 parallel across many nodes.
-
-# Wrapping Command Line Tools