Skip to content

[wip] simple input object generation #462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 13, 2017
Merged

[wip] simple input object generation #462

merged 2 commits into from
Jul 13, 2017

Conversation

mr-c
Copy link
Member

@mr-c mr-c commented Jul 12, 2017

Inspired by Dockstore's equivalent feature

Will close #384

@ctb
Copy link
Contributor

ctb commented Jul 13, 2017

Note that

cwltool --make-template https://github.com/CancerCollaboratory/dockstore-tool-bamstats/raw/develop/Dockstore.cwl > params.yaml

succeeds but shows the following error message that made me think it did not succeed:

/home/tx160085/env/bin/cwltool 1.0
https://github.com/CancerCollaboratory/dockstore-tool-bamstats/raw/develop/Dockstore.cwl:3:1: unrecognized extension field `http://purl.org/dc/terms/creator`.  Did you include a $schemas section?

Any way to change this to a warning, or be more verbose about the fact that params.yml is legit?

@ctb
Copy link
Contributor

ctb commented Jul 13, 2017

Also - the dockstore output includes a section for the output file, e.g. bamstats_report in the example that ngs-docs/2017-cloud-workflows-misc#1 is modifying. This doesn't seem to be the case with the --make-template command. Is that what you mean in this comment by uses the standardized --outdir option? thx!

@mr-c
Copy link
Member Author

mr-c commented Jul 13, 2017

@ctb thank you for the verification. This is incomplete but I'll release it for later improvement :-)

Re: output file name in the input object -- yes, that is a dockstore specific extension (and one that I never personally understood). Use the standardized --outdir if you'd like to put your outputs anywhere besides the current working directory. If you'd like to discover the filename of a particular output you can parse the output JSON to find out (some workflows the file names vary).

@mr-c
Copy link
Member Author

mr-c commented Jul 13, 2017

re: the warning, this is a known regression #455

@mr-c mr-c merged commit 1075312 into master Jul 13, 2017
@mr-c mr-c deleted the generate_input_object branch July 13, 2017 14:21
@denis-yuen
Copy link
Member

denis-yuen commented Jul 13, 2017

@ctb @mr-c FYI, the output file name is for specifying an output location https://dockstore.org/docs/advanced-features#file-provisioning

Note that putting it in /tmp works but is probably a less interesting example than putting the output into S3, Synapse, etc. (if you have the plugins installed)

@mr-c
Copy link
Member Author

mr-c commented Jul 13, 2017

@denis-yuen Ah, that makes more sense. It would be more portable if you added these as a dockstore specific directives, otherwise it looks like another input which is confusing to CWL runners and humans alike:

Example:

bam_input:
    class: File
    path: /tmp/NA12878.chrom20.ILLUMINA.bwa.CEU.low_coverage.20121211.bam
dockstore:outputs:
    bamstats_report:
        class: File
        path: /tmp/bamstats_report.zip

This should be proposed as a standardized optional feature, then dockstore:outputs could become cwl:outputs

@denis-yuen
Copy link
Member

@mr-c I think that would be a reasonable improvement. It would simplify our code too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convert cwl to json input
3 participants