Squashed 'cwltool/schemas/' changes from 40086a12..f96bca69

Peter Amstutz · Peter Amstutz · commit 041bc492533f · 2018-03-30T09:41:04.000-04:00
f96bca69 Merge pull request #633 from wtsi-hgi/parameter-like-reference-parsing e1308521 Merge pull request #624 from common-workflow-language/new-test/trailing-newline 46b40226 Merge pull request #660 from common-workflow-language/v1.1.0-fix-wf-bindings 9f979e91 break out Changlogs into seperate sections and cross-link them. e7b87b5e descriptions for {in,out}putBinding c30b27a1 diverse CLT changelog updates 566a735c Changelog: the `doc` field may be an array of strings. ea9faa89 Fix userguide. Fix schema table of contents ordering (a bit). 0bc210a9 sync some fixes 28218d9f Fix doc for CommandInputParameter.inputBinding c373b9fc Be more specific that loadContents in inputBinding (and inputBinding on WorkflowInputParameter) will go away in v2.0 1646a398 Merge pull request #659 from common-workflow-language/cern e1e3cc1a Add Tibor Simko; CERN dfdb1467 Merge pull request #656 from common-workflow-language/mr-c-patch-1 9cb8f9c8 Revert accidental change 57e2a0ec Remove reference to parameter-reference.cwl 3128b673 Move parameter-reference test to inline-js 602e077f Add prefix to parameter-reference 36644bc7 Add output to parameter-reference.cwl conformance test 98f62660 change parameter-reference type from string to boolean ad3c1640 Conformance test for parameter-reference-like javascript 4e2a75fb Add test for trailing whitespace in file entries of InitialWorkDir git-subtree-dir: cwltool/schemas git-subtree-split: f96bca6911b6688ff614c02dbefe819bed260a13
diff --git a/README.md b/README.md
@@ -191,6 +191,7 @@ Conduct](https://github.com/common-workflow-language/common-workflow-language/bl
 * [EMBL Australia Bioinformatics Resource](https://www.embl-abr.org.au/)
 * [IBM Spectrum Computing](https://www.ibm.com/spectrum-computing)
 * [DNAnexus](https://www.dnanexus.com/)
+* [CERN](https://home.cern/)
 
 ## Individual Contributors
 
@@ -244,6 +245,7 @@ Conduct](https://github.com/common-workflow-language/common-workflow-language/bl
 * Tom Morris, Curoverse; https://orcid.org/0000-0003-0435-7851
 * Anton Nekrutenko, The Pennsylvania State University / Galaxy Project; https://orcid.org/0000-0002-5987-8032
 * Brian O'Connor, University of California Santa Cruz; https://orcid.org/0000-0002-7681-6415
+* Tibor Simko, CERN, https://orcid.org/0000-0001-7202-5803
 * Nihar Sheth, DNAnexus; https://orcid.org/0000-0003-4128-4364
 * Stian Soiland-Reyes, University of Manchester; https://orcid.org/0000-0001-9842-9718
 * James Taylor, Johns Hopkins University / Galaxy Project; https://orcid.org/0000-0001-5079-840X
diff --git a/v1.0/conformance_test_v1.0.yaml b/v1.0/conformance_test_v1.0.yaml
@@ -533,6 +533,17 @@
     Test InitialWorkDirRequirement with expression in filename.
   tags: [ initial_work_dir ]
 
+- job: v1.0/string-job.json
+  output:
+    out:
+        class: File
+        checksum: sha1$6a47aa22b2a9d13a66a24b3ee5eaed95ce4753cf
+        location: example.conf
+        size: 16
+
+  tool: v1.0/iwdr-entry.cwl
+  doc: Test if trailing newline is present in file entry in InitialWorkDir
+
 - job: v1.0/wc-job.json
   output:
     output: 16
@@ -1216,7 +1227,7 @@
   tags: [ required ]
 
 - output:
-    args: [-A,'2',-B,baz,-C,'10','9','8','7','6','5','4','3','2','1']
+    args: [-A,'2',-B,baz,-C,'10','9','8','7','6','5','4','3','2','1',-D]
   job: v1.0/empty.json
   tool: v1.0/inline-js.cwl
   doc: Test InlineJavascriptRequirement with multiple expressions in the same tool
diff --git a/v1.0/contrib.md b/v1.0/contrib.md
@@ -1,19 +1,19 @@
 Authors:
 
-* Peter Amstutz <peter.amstutz@curoverse.com>, Arvados Project, Curoverse
-* Michael R. Crusoe <michael.crusoe@gmail.com>, Common Workflow Language
+* Peter Amstutz <pamstutz@veritasgenetics.com>, Arvados Project, Veritas Genetics
+* Michael R. Crusoe <mrc@commonwl.org>, Common Workflow Language
   project
 * Nebojša Tijanić <nebojsa.tijanic@sbgenomics.com>, Seven Bridges Genomics
 
 Contributors:
 
 * Brad Chapman <bchapman@hsph.harvard.edu>, Harvard Chan School of Public Health
 * John Chilton <jmchilton@gmail.com>, Galaxy Project, Pennsylvania State University
-* Michael Heuer <heuermh@berkeley.edu,>,UC Berkeley AMPLab
+* Michael Heuer <heuermh@berkeley.edu>, UC Berkeley AMPLab
 * Andrey Kartashov <Andrey.Kartashov@cchmc.org>, Cincinnati Children's Hospital
 * Dan Leehr <dan.leehr@duke.edu>, Duke University
 * Hervé Ménager <herve.menager@gmail.com>, Institut Pasteur
 * Maya Nedeljkovich <maja.nedeljkovic@sbgenomics.com>, Seven Bridges Genomics
 * Matt Scales <mscales@icr.ac.uk>, Institute of Cancer Research, London
-* Stian Soiland-Reyes [soiland-reyes@cs.manchester.ac.uk](mailto:soiland-reyes@cs.manchester.ac.uk), University of Manchester
+* Stian Soiland-Reyes <soiland-reyes@cs.manchester.ac.uk>, University of Manchester
 * Luka Stojanovic <luka.stojanovic@sbgenomics.com>, Seven Bridges Genomics
diff --git a/v1.0/v1.0/echo-tool.cwl b/v1.0/v1.0/echo-tool.cwl
@@ -3,7 +3,7 @@
 class: CommandLineTool
 cwlVersion: v1.0
 inputs:
-  text:
+  in:
     type: Any
     inputBinding: {}
 outputs:
diff --git a/v1.0/v1.0/inline-js.cwl b/v1.0/v1.0/inline-js.cwl
@@ -38,3 +38,8 @@ arguments:
         }
         return r;
       }
+  # Test errors similar to https://github.com/common-workflow-language/cwltool/issues/648 are fixed
+  - prefix: -D
+    valueFrom: $(true)
+  - prefix: -E
+    valueFrom: $(false)
diff --git a/v1.0/v1.0/iwdr-entry.cwl b/v1.0/v1.0/iwdr-entry.cwl
@@ -0,0 +1,20 @@
+#!/usr/bin/env cwl-runner
+
+class: CommandLineTool
+cwlVersion: v1.0
+baseCommand: ["cat", "example.conf"]
+
+requirements:
+  InitialWorkDirRequirement:
+    listing:
+      - entryname: example.conf
+        entry: |
+          CONFIGVAR=$(inputs.message)
+
+inputs:
+  message: string
+outputs:
+  out:
+    type: File
+    outputBinding:
+      glob: example.conf
diff --git a/v1.0/v1.0/string-job.json b/v1.0/v1.0/string-job.json
@@ -0,0 +1,3 @@
+{
+  "message": "hello"
+}
diff --git a/v1.1.0-dev1/CommandLineTool.yml b/v1.1.0-dev1/CommandLineTool.yml
@@ -48,9 +48,19 @@ $graph:
       underspecified behavior may have slightly different behavior in
       v1.1.0-dev1.
 
+      ## Changelog
+
         * Default values for some fields are now expressed in the schema.
-        * When defining record types, fields of type `File` may now include `secondaryFiles` and `format`.
-        * InputSchema now permits a `doc` field
+        * When defining record types with `CommandInputRecordSchema`, fields of
+          type `File` may now include `format`, `loadContents`,
+          `secondaryFiles` and `streamable`.
+        * `CommandInputRecordSchema`, `CommandInputEnumSchema`, and
+          `CommandInputArraySchema` now have an optional `doc` field.
+        * `inputBinding` has been added as an optional field for
+          `CommandInputRecordSchema` (was previously in CWL `draft-3` but
+          disappeared in `v1.0`)
+        * Any `doc` field may be an array of strings in addition to the
+          previously allowed single string.
         * Addition of `stdin` type shortcut for `CommandInputParameter`.
         * Clarify that the designated output directory should be empty
           except for files or directories specified using
@@ -59,6 +69,8 @@ $graph:
         * Expressions are now allowed to evaluate to `null` or `Dirent` in
           [InitialWorkDirRequirement.listing](#InitialWorkDirRequirement).
 
+      See also the [CWL Workflow Description, v1.1.0-dev1 changelog](Workflow.html#Changelog).
+
       ## Purpose
 
       Standalone programs are a flexible and interoperable form of code reuse.
@@ -111,6 +123,7 @@ $graph:
 - type: record
   name: CommandLineBinding
   extends: InputBinding
+  docParent: "#CommandInputParameter"
   doc: |
 
     When listed under `inputBinding` in the input schema, the term
@@ -245,6 +258,7 @@ $graph:
     inputBinding:
       type: CommandLineBinding?
       jsonldPredicate: "cwl:inputBinding"
+      doc: Describes how to turn this object into command line arguments.
 
 - name: CommandInputRecordField
   type: record
@@ -306,6 +320,9 @@ $graph:
     - name: outputBinding
       type: CommandOutputBinding?
       jsonldPredicate: "cwl:outputBinding"
+      doc: |
+        Describes how to generate this output object based on the files
+        produced by a CommandLineTool
 
 
 - name: CommandOutputRecordSchema
@@ -370,6 +387,9 @@ $graph:
         Specify valid types of data that may be assigned to this parameter.
     - name: inputBinding
       type: CommandLineBinding?
+      doc: |
+        Describes how to turns the input parameters of a process into
+        command line arguments.
       jsonldPredicate: "cwl:inputBinding"
 
 - type: record
@@ -404,6 +424,8 @@ $graph:
     - name: outputBinding
       type: CommandOutputBinding?
       jsonldPredicate: "cwl:outputBinding"
+      doc: Describes how to generate this output object based on the files
+        produced by a CommandLineTool
 
 - name: stdin
   type: enum
diff --git a/v1.1.0-dev1/Process.yml b/v1.1.0-dev1/Process.yml
@@ -580,8 +580,8 @@ $graph:
       jsonldPredicate: "cwl:loadContents"
       doc: |
         Use of `loadContents` in `InputBinding` is deprecated.
-        Preserved for v1.0 backwards compatability.  May be removed in
-        a future spec.  Use `InputParameter.loadContents` instead.
+        Preserved for v1.0 backwards compatability.  Will be removed in
+        CWL v2.0.  Use `InputParameter.loadContents` instead.
 
         Only valid when `type: File` or is an array of `items: File`.
 
diff --git a/v1.1.0-dev1/README.md b/v1.1.0-dev1/README.md
@@ -2,8 +2,8 @@
 
 The CWL specifications are divided up into several documents.
 
-The [User Guide](UserGuide.html) provides a gentle introduction to writing CWL
-command line tools and workflows.
+The [User Guide](http://www.commonwl.org/user_guide/) provides a gentle
+introduction to writing CWL command line tools and workflows.
 
 The [Command Line Tool Description Specification](CommandLineTool.html)
 specifies the document schema and execution semantics for wrapping and
diff --git a/v1.1.0-dev1/Workflow.yml b/v1.1.0-dev1/Workflow.yml
@@ -46,12 +46,28 @@ $graph:
       underspecified behavior may have slightly different behavior in
       v1.1.0-dev1.
 
+      ## Changelog
+
         * Default values for some fields are now expressed in the schema
-        * When defining record types, fields of type `File` may now include `secondaryFiles` and `format`.
-        * InputSchema now permits a `doc` field
+        * When defining record types, fields of type `File` may now include
+          `format`, `secondaryFiles`, and `streamable`.
+        * The Input schemas (Record, Array, Enum),  now permits a `doc` field
+        * Any `doc` field may be an array of strings in addition to the
+          previously allowed single string.
         * The `run` field now introduces a subscope so that embedded
           definitions do not have identifier conflicts with the
           outer workflow.
+        * `Workflow.inputs` and `ExpressionTool.inputs` now refer to the type
+          `WorkflowInputParameter` instead of `InputParameter`; the only syntax
+          difference from v1.0 is that `inputBinding` is now deprecated (as
+          there is no command line to bind input values to) and `inputBinding`
+          in a `Workflow` context will be removed in v2.0. Use the new
+          `WorkflowInputParameter.loadContents` instead.
+        * The presence of `OutputRecordField.outputBinding` in v1.0 was due to
+          a technical error and as it makes no semantic sense it was removed.
+
+
+      See also the [CWL Command Line Tool Description, v1.1.0-dev1 changelog](CommandLineTool.html#Changelog).
 
       ## Purpose
 
@@ -95,6 +111,7 @@ $graph:
 - name: WorkflowInputParameter
   type: record
   extends: InputParameter
+  docParent: "#Workflow"
   fields:
     - name: type
       type:
@@ -120,7 +137,9 @@ $graph:
         Specify valid types of data that may be assigned to this parameter.
     - name: inputBinding
       type: InputBinding?
-      doc: Deprecated.  Preserved for v1.0 backwards compatability.  Use `InputParameter.loadContents` instead.
+      doc: |
+        Deprecated.  Preserved for v1.0 backwards compatability.  Will be removed in
+        CWL v2.0.  Use `WorkflowInputParameter.loadContents` instead.
       jsonldPredicate: "cwl:inputBinding"
 
 
diff --git a/v1.1.0-dev1/concepts.md b/v1.1.0-dev1/concepts.md
@@ -6,7 +6,7 @@
 
 **YAML**: http://yaml.org
 
-**Avro**: https://avro.apache.org/docs/current/spec.html
+**Avro**: https://avro.apache.org/docs/1.8.1/spec.html
 
 **Uniform Resource Identifier (URI) Generic Syntax**: https://tools.ietf.org/html/rfc3986)
 
@@ -388,5 +388,5 @@ To discover CWL documents look in the following locations:
 
 `$XDG_DATA_HOME/commonwl/` (usually `$HOME/.local/share/commonwl`)
 
-`$XDF_DATA_HOME` is from the [XDG Base Directory
+`$XDG_DATA_HOME` is from the [XDG Base Directory
 Specification](http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html)
diff --git a/v1.1.0-dev1/contrib.md b/v1.1.0-dev1/contrib.md
@@ -1,7 +1,7 @@
 Authors:
 
 * Peter Amstutz <pamstutz@veritasgenetics.com>, Arvados Project, Veritas Genetics
-* Michael R. Crusoe <michael.crusoe@gmail.com>, Common Workflow Language
+* Michael R. Crusoe <mrc@commonwl.org>, Common Workflow Language
   project
 * Nebojša Tijanić <nebojsa.tijanic@sbgenomics.com>, Seven Bridges Genomics
 
@@ -15,5 +15,5 @@ Contributors:
 * Hervé Ménager <herve.menager@gmail.com>, Institut Pasteur
 * Maya Nedeljkovich <maja.nedeljkovic@sbgenomics.com>, Seven Bridges Genomics
 * Matt Scales <mscales@icr.ac.uk>, Institute of Cancer Research, London
-* Stian Soiland-Reyes <[soiland-reyes@cs.manchester.ac.uk](mailto:soiland-reyes@cs.manchester.ac.uk)>, University of Manchester
+* Stian Soiland-Reyes <soiland-reyes@cs.manchester.ac.uk>, University of Manchester
 * Luka Stojanovic <luka.stojanovic@sbgenomics.com>, Seven Bridges Genomics

-Original file line number
+Diff line change
@@ @@ -1,19 +1,19 @@ @@
 Authors:
 -* Peter Amstutz <peter.amstutz@curoverse.com>, Arvados Project, Curoverse
 -* Michael R. Crusoe <[email protected]>, Common Workflow Language
 +* Peter Amstutz <pamstutz@veritasgenetics.com>, Arvados Project, Veritas Genetics
 +* Michael R. Crusoe <[email protected]>, Common Workflow Language
   project
 * Nebojša Tijanić <[email protected]>, Seven Bridges Genomics
 Contributors:
 * Brad Chapman <[email protected]>, Harvard Chan School of Public Health
 * John Chilton <[email protected]>, Galaxy Project, Pennsylvania State University
 -* Michael Heuer <[email protected],>,UC Berkeley AMPLab
 +* Michael Heuer <[email protected]>, UC Berkeley AMPLab
 * Andrey Kartashov <[email protected]>, Cincinnati Children's Hospital
 * Dan Leehr <[email protected]>, Duke University
 * Hervé Ménager <[email protected]>, Institut Pasteur
 * Maya Nedeljkovich <[email protected]>, Seven Bridges Genomics
 * Matt Scales <[email protected]>, Institute of Cancer Research, London
 -* Stian Soiland-Reyes [[email protected]](mailto:[email protected]), University of Manchester
 +* Stian Soiland-Reyes <[email protected]>, University of Manchester
 * Luka Stojanovic <[email protected]>, Seven Bridges Genomics
Original file line number	Diff line number	Diff line change
`@@ -38,3 +38,8 @@ arguments:`
`38`	`38`	`}`
`39`	`39`	`return r;`
`40`	`40`	`}`
	`41`	`+ # Test errors similar to https://github.com/common-workflow-language/cwltool/issues/648 are fixed`
	`42`	`+ - prefix: -D`
	`43`	`+ valueFrom: $(true)`
	`44`	`+ - prefix: -E`
	`45`	`+ valueFrom: $(false)`