Question: Custom galaxy tool: multiple output files
1
gravatar for rleach
2.3 years ago by
rleach10
rleach10 wrote:

This is my first attempt to install a custom tool on galaxy and I'm having some trouble getting all of the output files to display in galaxy in the output collection. My tool is a barcode splitter, so I used fastx_barcode_splitter's xml file as a template.

The tool works and it generates all the files, but the resultant collection linked in the history is missing 26 of the 68 output files. I found the files hidden in the main history instead of residing inside the collection and they each had the message: "This dataset has been hidden". When I unhide them, the files are displayed in the main history instead of in the collection with the rest of the output files. In fact, among the files that are hidden are the 8 files of the 68 total that are the only ones that actually contain output (the unmatched ones). The files that should be linked-to and downloadable in collection are named:

RNAseq-read-1.fastq
RNAseq-read-2.fastq
RNAseq-read-3.fastq
RNAseq-read-4.fastq
i5-1-read-1.fastq
i5-1-read-2.fastq
i5-1-read-3.fastq
i5-1-read-4.fastq
i5-10-read-1.fastq
i5-10-read-2.fastq
i5-10-read-3.fastq
i5-10-read-4.fastq
i5-11-read-1.fastq
i5-11-read-2.fastq
i5-11-read-3.fastq
i5-11-read-4.fastq
i5-12-read-1.fastq
i5-12-read-2.fastq
i5-12-read-3.fastq
i5-12-read-4.fastq
i5-2-read-1.fastq
i5-2-read-2.fastq
i5-2-read-3.fastq
i5-2-read-4.fastq
i5-3-read-1.fastq
i5-3-read-2.fastq
i5-3-read-3.fastq
i5-3-read-4.fastq
i5-4-read-1.fastq
i5-4-read-2.fastq
i5-4-read-3.fastq
i5-4-read-4.fastq
i5-5-read-1.fastq
i5-5-read-2.fastq
i5-5-read-3.fastq
i5-5-read-4.fastq
i5-6-read-1.fastq
i5-6-read-2.fastq
i5-6-read-3.fastq
i5-6-read-4.fastq
i5-7-read-1.fastq
i5-7-read-2.fastq
i5-7-read-3.fastq
i5-7-read-4.fastq
i5-8-read-1.fastq
i5-8-read-2.fastq
i5-8-read-3.fastq
i5-8-read-4.fastq
i5-9-read-1.fastq
i5-9-read-2.fastq
i5-9-read-3.fastq
i5-9-read-4.fastq
multimatched-read-1.fastq
multimatched-read-2.fastq
multimatched-read-3.fastq
multimatched-read-4.fastq
oldi5-1-read-1.fastq
oldi5-1-read-2.fastq
oldi5-1-read-3.fastq
oldi5-1-read-4.fastq
oldi5-2-read-1.fastq
oldi5-2-read-2.fastq
oldi5-2-read-3.fastq
oldi5-2-read-4.fastq
unmatched-read-1.fastq
unmatched-read-2.fastq
unmatched-read-3.fastq
unmatched-read-4.fastq

Everything after "i5-7-read-2.fastq" is hidden in the main history (instead of in the collection).

I copied the command for the run in the info view and ran it on the command line from my home directory:

bash /Users/rleach/PROJECT/GALAXY/hepcat72-galaxy/galaxy/tools/robs_tools/barcode_splitter_galaxy_wrapper.sh split fastq --bcfile /Users/rleach/PROJECT/GALAXY/hepcat72-galaxy/galaxy/database/files/000/dataset_28.dat --mismatches 1 --galaxy /Users/rleach/PROJECT/GALAXY/hepcat72-galaxy/galaxy/database/files/000/dataset_32.dat /Users/rleach/PROJECT/GALAXY/hepcat72-galaxy/galaxy/database/files/000/dataset_31.dat /Users/rleach/PROJECT/GALAXY/hepcat72-galaxy/galaxy/database/files/000/dataset_30.dat /Users/rleach/PROJECT/GALAXY/hepcat72-galaxy/galaxy/database/files/000/dataset_29.dat --idxread 1 2 > /Users/rleach/PROJECT/GALAXY/hepcat72-galaxy/galaxy/database/files/000/dataset_326.dat

It creates the 'split' directory holding all the files.

Here is the portion of the tool's xml file that specifies the output files:

<outputs>
  <data format="tabular" name="summary" label="${tool.name} on ${on_string}: Summary" />
  <collection name="split_output" type="list" format_source="input" label="${tool.name} on ${on_string}">
    <discover_datasets pattern="__designation_and_ext__" directory="split" visible="false" label="${designation}"/>
  </collection>
</outputs>

How do I make it so that all the output files get into the collection?

galaxy • 1.1k views
ADD COMMENTlink modified 2.3 years ago • written 2.3 years ago by rleach10
0
gravatar for rleach
2.3 years ago by
rleach10
rleach10 wrote:

When you run a tool that generates a lot of files, those files get copied into the collection folder by a background process. By stopping and restarting the galaxy server, you interrupt that copying. All files start out in the main history. They get removed from the main history and put into the collection file by file. Any that had not yet moved when the restart occurred, will remain hidden in the main history.

ADD COMMENTlink written 2.3 years ago by rleach10
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 181 users visited in the last hour