I wrote a pipeline, a set of apps in Python. Every app outputs thousands of files. Every app will create a main work directory (aka $app_workdir, path provided in command line) and put all the final output data in $app_workdir/out/, all the logs in $app_workdir/log/, all temporaries in $app_workdir/tmp/ and so on.
Now I am trying to integrate this pipeline into Galaxy as a set of custom tools. I would prefer not to modify apps too much but rather create Galaxy custom tool XML files so that they can be chained with each other. The problem I can not solve is how to 'pack' a directory with thousands of files in it as *one* output and then how to provide this as one input to the next app.
I am trying the approaches mentioned here: https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files but can not make them work.
I guess the most appropriate approach for me should be the "Pass the application a specific output directory" but it does not seem to work for me. I made the first app work and create output, but the second app can not read the input. For the second app, Galaxy complains:
NotFound: cannot find 'files_path' while searching for 'input.files_path'
What am I doing wrong?
Here are the excerpts from my tool description XMLs:
# app #1 <command> fasttree.py --input "$input" --workdir "$output.files_path" </command> <inputs> <param name="input" type="data" format="hg_mft" label="Alignment files manifest"/> </inputs> <outputs> <data name="output" format="html" label="Tree files"/> </outputs> # app # <command> analyze_trees.py --input "$input.files_path/out/trees" --workdir "$output.files_path" </command> <inputs> <param name="input" type="data" format="html" label="Tree files"/> </inputs> <outputs> <data name="output" format="html" label="Tree analysis"/> </outputs>