Skip to content

Commit 79ef5c3

Browse files
author
Peter Amstutz
committed
Squashed 'cwltool/schemas/' changes from 46ce0c3..2cf0896
2cf0896 Merge pull request #289 from denis-yuen/patch-1 68f5f90 Merge pull request #291 from bmeg/master a82cf0d Merge https://github.com/bmeg/common-workflow-language e7325dc Adding docker hint for conformance test 70501d5 Merge pull request #290 from alaindomissy/patch-1 58a9354 Update concepts.md 1b052b5 Fix a typo 4a02bc7 Test that expressionLib requirement of individual tool step overrides expressionLib of workflow. 80e2501 add path to record-job3.yaml c9df19c Add checksum to cwl.output.json results. e604399 Add tests that $HOME and $TMPDIR are set correctly according to spec. a479bbd Add checksums and sizes to secondaryFiles in draft-3 tests. dfcbfeb Merge pull request #286 from common-workflow-language/test-nested-array 91bb7b4 Test case for command line generation of array-of-arrays. 5baf924 Merge pull request #285 from common-workflow-language/checksum-secondary 1c02570 Merge pull request #282 from common-workflow-language/test-inputbinding-dir 1835540 Add checksums to secondaryFiles and files in Directory objects. 04d9dac Add test for command line inputBinding of Directory input parameter. 8053c4b Merge pull request #279 from simonovic86/patch-1 168d1a2 Update README.md e727a46 Merge pull request #277 from common-workflow-language/add-biostars-link bfddadc fix typo 9ec9642 add biostars link b67a4cf Merge pull request #272 from StarvingMarvin/master 570ae5f Merge pull request #266 from common-workflow-language/remove-empty-baseCommand 721ffdc Merge remote-tracking branch 'origin/master' into remove-empty-baseCommand 6931eeb minor userguide fixes a296c5f Remove "baseCommand: []" from tests. git-subtree-dir: cwltool/schemas git-subtree-split: 2cf0896
1 parent 362cca3 commit 79ef5c3

21 files changed

+203
-31
lines changed

‎README.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ portability of data analysis workflows. Our goal is to create specifications
77
that enable data scientists to describe analysis tools and workflows that are
88
powerful, easy to use, portable, and support reproducibility.
99

10-
CWL builds on technologies such as [JSON-LD](http://json-ld.org) and
11-
[Avro](https://avro.apache.org/) for data modeling and
12-
[Docker](http://docker.com) for portable runtime environments.
10+
CWL builds on technologies such as [JSON-LD](http://json-ld.org)
11+
for data modeling and [Docker](http://docker.com) for portable runtime\
12+
environments.
1313

1414
CWL is designed to express workflows for data-intensive science, such as
1515
Bioinformatics, Medical Imaging, Chemistry, Physics, and Astronomy.
@@ -64,6 +64,15 @@ https://ci.commonwl.org
6464

6565
[Github repository of example tools and workflows.](https://github.com/common-workflow-language/workflows)
6666

67+
## Support
68+
69+
The best place to ask a question about all things CWL is on
70+
[Biostars](https://www.biostars.org/t/cwl/).
71+
72+
[![Biostars
73+
CWL](https://www.biostars.org/static/biostar2.logo.png)](https://www.biostars.org/t/cwl/)
74+
</a>
75+
6776
## Development and testing
6877

6978
[The CWL project is on Github.](https://github.com/common-workflow-language/common-workflow-language)
@@ -91,7 +100,7 @@ and send a pull request!
91100
[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/common-workflow-language/common-workflow-language?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
92101

93102
Your CWL Community Engineer, Michael R. Crusoe, publishes a blog about his work
94-
with weekly updates at http://mrc.commonwl.org.
103+
with updates at http://mrc.commonwl.org.
95104

96105
### Code of Conduct
97106

@@ -142,6 +151,7 @@ Conduct](https://github.com/common-workflow-language/common-workflow-language/bl
142151
* Maxim Mikheev <[email protected]>
143152
* Tim Pierce <[email protected]>
144153
* Josh Randall <[email protected]>
154+
* Janko Simonović <[email protected]>
145155
* Stian Soiland-Reyes <[email protected]>
146156
* Luka Stojanovic <[email protected]>
147157
* Nebojša Tijanić <[email protected]>

‎draft-3/conformance_test_draft-3.yaml

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -276,23 +276,33 @@
276276
"secondaryFiles": [
277277
{
278278
"path": "input.txt.idx1",
279-
"class": "File"
279+
"class": "File",
280+
"checksum": "sha1$553f3a09003a9f69623f03bec13c0b078d706023",
281+
"size": 1500
280282
},
281283
{
282284
"path": "input.idx2",
283-
"class": "File"
285+
"class": "File",
286+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
287+
"size": 0
284288
},
285289
{
286290
"path": "input.txt.idx3",
287-
"class": "File"
291+
"class": "File",
292+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
293+
"size": 0
288294
},
289295
{
290296
"path": "input.txt.idx4",
291-
"class": "File"
297+
"class": "File",
298+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
299+
"size": 0
292300
},
293301
{
294302
"path": "input.txt.idx5",
295-
"class": "File"
303+
"class": "File",
304+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
305+
"size": 0
296306
}
297307
],
298308
"size": 1111
@@ -583,7 +593,9 @@
583593
output: {
584594
"foo": {
585595
"path": "foo",
586-
"class": "File"
596+
"class": "File",
597+
"checksum": "sha1$f1d2d2f924e986ac86fdf7b36c94bcdf32beec15",
598+
"size": 4
587599
}
588600
}
589601
tool: draft-3/test-cwl-out.cwl

‎site/userguide-v1.0-input.json

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,11 @@
4545
"class": "File",
4646
"location": "../v1.0/examples/record-job2.yml"
4747
},
48+
{
49+
"basename": "record-job3.yml",
50+
"class": "File",
51+
"location": "../v1.0/examples/record-job3.yml"
52+
},
4853
{
4954
"basename": "echo-job.yml",
5055
"class": "File",
@@ -145,4 +150,4 @@
145150
],
146151
"class": "File",
147152
"location": "../v1.0/UserGuide.yml"
148-
}
153+
}

‎v1.0/UserGuide.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@
8585
8686
Available primitive types are *string*, *int*, *long*, *float*, *double*,
8787
and *null*; complex types are *array* and *record*; in addition there are
88-
special types *File* and *Any*.
88+
special types *File*, *Directory* and *Any*.
8989
9090
The following example demonstrates some input parameters with different
9191
types and appearing on the command line in different ways:
@@ -601,7 +601,7 @@
601601
602602
*record-job3.yml*
603603
```
604-
- $include: examples/record-job2.yml
604+
- $include: examples/record-job3.yml
605605
- |
606606
```
607607
@@ -681,7 +681,7 @@
681681
Sometimes you need to create a file on the fly from input parameters,
682682
such as tools which expect to read their input configuration from a file
683683
rather than the command line parameters. To do this, use
684-
`CreateFileRequirement`.
684+
`InitialWorkDirRequirement`.
685685
686686
*createfile.cwl*
687687
```
@@ -712,7 +712,7 @@
712712
Normally, input files are located in a read-only directory separate from
713713
the output directory. This causes problems if the underlying tool
714714
expects to write its output files alongside the input file in the same
715-
directory. You use `CreateFileRequirement` to stage input files into the
715+
directory. You use `InitialWorkDirRequirement` to stage input files into the
716716
output directory. In this example, we use a Javascript expression to
717717
extract the base name of the input file from its leading directory path.
718718

‎v1.0/Workflow.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ $graph:
6262
type: int
6363
doc: Second input parameter
6464
```
65-
* The common field 'description` has been renamed to `doc`.
65+
* The common field `description` has been renamed to `doc`.
6666
6767
## Purpose
6868

‎v1.0/concepts.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ https://github.com/common-workflow-language/common-workflow-language/tree/master
9595

9696
If an object contains an `id` field, that is used to uniquely identify the
9797
object in that document. The value of the `id` field must be unique over the
98-
entire document. Identifiers may be resolved relative to other the document
98+
entire document. Identifiers may be resolved relative to either the document
9999
base and/or other identifiers following the rules are described in the
100100
[Schema Salad specification](SchemaSalad.html#Identifier_resolution).
101101

‎v1.0/conformance_test_v1.0.yaml

Lines changed: 65 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@
4545
}
4646
job: v1.0/cat-job.json
4747
tool: v1.0/template-tool.cwl
48-
doc: Test CreateFileRequirement ExpressionEngineRequirement.engineConfig feature
48+
doc: Test InitialWorkDirRequirement ExpressionEngineRequirement.engineConfig feature
4949

5050
- job: v1.0/cat-job.json
5151
output:
@@ -327,29 +327,39 @@
327327
{
328328
"location": "input.txt.idx1",
329329
"class": "File",
330+
"checksum": "sha1$553f3a09003a9f69623f03bec13c0b078d706023",
331+
"size": 1500
330332
},
331333
{
332334
"location": "input.idx2",
333335
"class": "File",
336+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
337+
"size": 0
334338
},
335339
{
336340
"location": "input.txt.idx3",
337341
"class": "File",
342+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
343+
"size": 0
338344
},
339345
{
340346
"location": "input.txt.idx4",
341347
"class": "File",
348+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
349+
"size": 0
342350
},
343351
{
344352
"location": "input.txt.idx5",
345353
"class": "File",
354+
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
355+
"size": 0
346356
}
347357
],
348358
"size": 1111
349359
}
350360
tool: "v1.0/search.cwl#main"
351361
doc: |
352-
Test CreateFileRequirement linking input files and capturing secondaryFiles
362+
Test InitialWorkDirRequirement linking input files and capturing secondaryFiles
353363
on input and output.
354364
355365
- job: v1.0/rename-job.json
@@ -361,7 +371,7 @@
361371
size: 1111
362372
tool: v1.0/rename.cwl
363373
doc: |
364-
Test CreateFileRequirement with expression in filename.
374+
Test InitialWorkDirRequirement with expression in filename.
365375
366376
- job: v1.0/wc-job.json
367377
output:
@@ -633,7 +643,9 @@
633643
output: {
634644
"foo": {
635645
"location": "foo",
636-
"class": "File"
646+
"class": "File",
647+
"checksum": "sha1$f1d2d2f924e986ac86fdf7b36c94bcdf32beec15",
648+
"size": 4
637649
}
638650
}
639651
tool: v1.0/test-cwl-out.cwl
@@ -713,7 +725,7 @@
713725
"class": "File"
714726
}
715727
tool: v1.0/dir.cwl
716-
doc: Test directory input
728+
doc: Test directory input with parameter reference
717729

718730
- job: v1.0/dir-job.yml
719731
output:
@@ -733,11 +745,15 @@
733745
"listing": [
734746
{
735747
"class": "File",
736-
"location": "hello.txt"
748+
"location": "hello.txt",
749+
"checksum": "sha1$47a013e660d408619d894b20806b1d5086aab03b",
750+
"size": 13
737751
},
738752
{
739753
"class": "File",
740-
"location": "goodbye.txt"
754+
"location": "goodbye.txt",
755+
"checksum": "sha1$dd0a4c4c49ba43004d6611771972b6cf969c1c01",
756+
"size": 24
741757
}
742758
],
743759
}
@@ -809,3 +825,45 @@
809825
size: 21
810826
tool: v1.0/nameroot.cwl
811827
doc: Test nameroot/nameext expression in arguments, stdout
828+
829+
- job: v1.0/dir-job.yml
830+
output:
831+
"outlist": {
832+
"size": 20,
833+
"location": "output.txt",
834+
"checksum": "sha1$13cda8661796ae241da3a18668fb552161a72592",
835+
"class": "File"
836+
}
837+
tool: v1.0/dir6.cwl
838+
doc: Test directory input with inputBinding
839+
840+
- job: v1.0/nested-array-job.yml
841+
output:
842+
echo:
843+
checksum: sha1$3f786850e387550fdab836ed7e6dc881de23001b
844+
location: echo.txt
845+
class: File
846+
size: 2
847+
tool: v1.0/nested-array.cwl
848+
doc: Test command line generation of array-of-arrays
849+
850+
- job: v1.0/empty.json
851+
output: {}
852+
tool: v1.0/envvar.cwl
853+
doc: Test $HOME and $TMPDIR are set correctly
854+
855+
- job: v1.0/empty.json
856+
output: {}
857+
tool: v1.0/envvar2.cwl
858+
doc: Test $HOME and $TMPDIR are set correctly in Docker
859+
860+
- job: v1.0/empty.json
861+
output:
862+
"out": {
863+
"checksum": "sha1$7448d8798a4380162d4b56f9b452e2f6f9e24e7a",
864+
"location": "whatever.txt",
865+
"class": "File",
866+
"size": 2
867+
}
868+
tool: "v1.0/js-expr-req-wf.cwl#wf"
869+
doc: Test that expressionLib requirement of individual tool step overrides expressionLib of workflow.

‎v1.0/v1.0/bwa-mem-tool.cwl

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ class: CommandLineTool
77
hints:
88
- class: ResourceRequirement
99
coresMin: 4
10+
- class: DockerRequirement
11+
dockerPull: python
1012

1113
inputs:
1214
- id: reference

‎v1.0/v1.0/dir.cwl

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ outputs:
99
type: File
1010
outputBinding:
1111
glob: output.txt
12-
baseCommand: []
1312
arguments: ["cd", "$(inputs.indir.path)",
1413
{shellQuote: false, valueFrom: "&&"},
1514
"find", ".",

‎v1.0/v1.0/dir2.cwl

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ outputs:
1111
type: File
1212
outputBinding:
1313
glob: output.txt
14-
baseCommand: []
1514
arguments: ["cd", "$(inputs.indir.path)",
1615
{shellQuote: false, valueFrom: "&&"},
1716
"find", ".",

‎v1.0/v1.0/dir4.cwl

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ outputs:
99
type: File
1010
outputBinding:
1111
glob: output.txt
12-
baseCommand: []
1312
arguments: ["cd", "$(inputs.inf.dirname)",
1413
{shellQuote: false, valueFrom: "&&"},
1514
"find", ".",

‎v1.0/v1.0/dir5.cwl

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ outputs:
1111
type: File
1212
outputBinding:
1313
glob: output.txt
14-
baseCommand: []
1514
arguments: ["find", ".",
1615
{shellQuote: false, valueFrom: "|"},
1716
"sort"]

‎v1.0/v1.0/dir6.cwl

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
class: CommandLineTool
2+
cwlVersion: v1.0
3+
requirements:
4+
- class: ShellCommandRequirement
5+
inputs:
6+
indir:
7+
type: Directory
8+
inputBinding:
9+
prefix: cd
10+
position: -1
11+
outputs:
12+
outlist:
13+
type: File
14+
outputBinding:
15+
glob: output.txt
16+
arguments: [
17+
{shellQuote: false, valueFrom: "&&"},
18+
"find", ".",
19+
{shellQuote: false, valueFrom: "|"},
20+
"sort"]
21+
stdout: output.txt

‎v1.0/v1.0/envvar.cwl

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
class: CommandLineTool
2+
cwlVersion: v1.0
3+
inputs: []
4+
outputs: []
5+
requirements:
6+
ShellCommandRequirement: {}
7+
arguments: [
8+
echo, {valueFrom: '"HOME=$HOME"', shellQuote: false}, {valueFrom: '"TMPDIR=$TMPDIR"', shellQuote: false},
9+
{valueFrom: '&&', shellQuote: false},
10+
test, {valueFrom: '"$HOME"', shellQuote: false}, "=", $(runtime.outdir),
11+
"-a", {valueFrom: '"$TMPDIR"', shellQuote: false}, "=", $(runtime.tmpdir)]

0 commit comments

Comments
 (0)