Galaxy failed to unzip gz files after uploading

Question: Galaxy failed to unzip gz files after uploading

22 months ago by

Canada

jamwest101 • 20 wrote:

Hi All: Compressed files used to be decompressed in galaxy after uploading. Recently, the compressed files (gz files) are failed to be decompressed after uploaded into Galaxy. Is it a common problem or just happened to me? Do I need to unzip files before uploading or manually decompress files using some galaxy tools? Thanks!

unzip galaxy gz chip-seq • 1.6k views

ADD COMMENT • link •

modified 5 months ago by maisarahabsamad • 0 • written 22 months ago by jamwest101 • 20

It happened to me just now.

ADD REPLY • link written 22 months ago by atcglj • 10

This has been reported a few times. The Main server at http://usegalaxy.org is currently undergoing updates. More feedback soon. Jen, Galaxy team

ADD REPLY • link written 22 months ago by Jennifer Hillman Jackson ♦ 25k

Thank you! Problem solved.

ADD REPLY • link written 22 months ago by jamwest101 • 20

Admin change: marked answer as accepted. Next time you can do this yourself to help others when an answer resolves a problem. Thanks! Jen, Galaxy team

ADD REPLY • link written 22 months ago by Jennifer Hillman Jackson ♦ 25k

22 months ago by

Martin Čech ♦♦ 4.9k

United States

Martin Čech ♦♦ 4.9k wrote:

This is now fixed on usegalaxy.org.

They way it works now is that .gz files do not uncompress on upload but rather on tool run. You can now select the fastq.gz files as inputs for tools that expect fastq and Galaxy will uncompress them before sending to the tool. See example below.

enter image description here

ADD COMMENT • link written 22 months ago by Martin Čech ♦♦ 4.9k

I would love to be able to do this on a local instance. Is this feature only available on the main public server or will it be in the next release as well?

ADD REPLY • link written 22 months ago by jdv • 20

We will release Galaxy 17.01 (which contains this feature) in the upcoming weeks. If you don't want to wait you can clone it already from our repo (branch release_17.01) but beware that there might be more bugs than usual since we are still testing it.

ADD REPLY • link written 22 months ago by Martin Čech ♦♦ 4.9k

I can wait, but this is great - it is a long-desired feature to be able to handle compressed read data transparently. Is this a fundamental datatype (i.e. can it be set as an output format for tools that can natively write compressed fastq or which we can pipe through gzip when history size is more important than speed of execution)?

ADD REPLY • link written 22 months ago by jdv • 20

Correct, these are new datatypes (e.g. fastq.gz, fastqsanger.gz). It works in a backward compatible manner and allows for future tools to be adapted to work with .gz archives efficiently.

ADD REPLY • link written 22 months ago by Martin Čech ♦♦ 4.9k

Hi,

I was hoping that what you described was that case. But when I uploaded my fastq.gz files and run Bowtie with them as input (this is a part of a workflow), Bowtie failed to run and this is the error message:

Error aligning sequence. Error: reads file does not look like a FASTQ file terminate called after throwing an instance of 'int' /bin/sh: line 1: 20345 Aborted (core dumped) bowtie -q -p 6 -S -n 3 -e 70 -l 28 --maxbts 800 -y -k 1 -m 1 --best --strata --phred33-quals /galaxy/data/galGal4/bowtie_index/galGal4 /galaxy-repl/main/files/018/641/dataset_18641959.dat > /galaxy-repl/main/files/018/642/dataset_18642032.dat

ADD REPLY • link written 22 months ago by Bao Ho • 80

5 months ago by

maisarahabsamad • 0

maisarahabsamad • 0 wrote:

Hi, I am second to Bao Ho's case.

I uploaded the fastq.gz and the data format become fastqsanger. It means that the gz file is already unzip. However, I cannot use any tool FastQC, Fastq Groomer, Bowtie2 etc for the input.

I receive this error message: Fatal error: Exit code 1 () Error: reads file does not look like a FASTQ file terminate called after throwing an instance of 'int' (ERR): bowtie2-align died with signal 6 (ABRT) (core dumped)

Please help. Thanks.

ADD COMMENT • link written 5 months ago by maisarahabsamad • 0

Hi Maisarah,

If I'm not mistaken, since galaxy version 17.01, when uploaded the fastq.gz file, it should not uncompress. Galaxy will uncompress the file when it is on tool run (I re-tested it by uploaded fastq.gz files to see if any error occurred or it changed as fastqsanger but I did not get any). This way, the user's quota space can be saved when dealing with files with large size.

Did you set the type of uploaded file as fastq.gz or fastqsanger or left it to auto-detect? If you uploaded a fastq.gz file, you must set it as a fastq.gz type (or the header will change and resulted with an error when running tool). If you correctly set the file as fastq.gz, you must first run Fastq Groomer to change the format to fastqsanger.gz. You can then used it with tools in galaxy.

ADD REPLY • link written 5 months ago by hadiaziz • 20

hadiaziz is correct in both cases.

If you do not select your format as compressed Galaxy will unzip it.
Your second problem is that your fastq file is not in fastqsanger format. Use the groomer hadiaziz recommended.

ADD REPLY • link written 4 months ago by Martin Čech ♦♦ 4.9k

Please log in to add an answer.

Similar posts • Search »