Question: Data import from EBI SRA to Galaxy does not work
1
gravatar for scholtz
20 months ago by
scholtz60
Hungary
scholtz60 wrote:

Dear Galaxy Team,

There seems to be a persistent problem with importing fastq files from EBI SRA - I've been trying to upload data since about Christmas, and keep getting the same error message as reported by others. Although the alternative "Extract reads in FASTQ/A format from NCBI SRA" works, the fastq splitter is taking forever (memory problems?). I would appreciate if you could look into the problem - or should we try to contact the EBI staff? There is no announcement at their website about maintenance or anything...

Best regards,

Beata

ebi sra data import galaxy • 1.2k views
ADD COMMENTlink modified 19 months ago by Jennifer Hillman Jackson25k • written 20 months ago by scholtz60
1
gravatar for Mo Heydarian
20 months ago by
Mo Heydarian820
United States
Mo Heydarian820 wrote:

Hello Beata,

I am also seeing an extended run time for FASTQ splitter.

An alternative to importing data with NCBI SRA Tools -> FASTQ splitter is to run NCBI SRA Tools with a list of SRR IDs, which will import paired end data as separate files corresponding to the forward and reverse reads. This way you won't have to run FASTQ splitter and your reads will be ready for analysis directly after import.

Note that when you use a list of SRR IDs to import your data with NCBI SRA Tools, every dataset provided will be imported in one list (collection). If you don't want to use collections, you should provide a list of individual SRR IDs to NCBI SRA Tools.

Thanks to Youri Hoogstrate for pointing out this usage of NCBI SRA Tools.

Hope this helps!

Cheers,

Mo Heydarian

ADD COMMENTlink written 20 months ago by Mo Heydarian820

Related thread https://biostar.usegalaxy.org/p/21042

ADD REPLYlink written 20 months ago by Jennifer Hillman Jackson25k

Dear All,

Some details to the workaround option suggested above (also, it seems to be faster than uploading from EBI): 1. Upload the SRR ID as a "list", (tabular file) using the NCBI SRA Tools - Extract reads in FASTQ/A format. Input type: List of SRA accession, one per line 2. Once uploaded, click on the "hide hidden" in the History panel 3. Unhide the forward/reverse data files

Proceed with the analysis as usual.

Hope this helps, and thanks a lot to the Galaxy team for pointing out this (lifesaving) option.

Beata

ADD REPLYlink written 19 months ago by scholtz60
1
gravatar for scholtz
20 months ago by
scholtz60
Hungary
scholtz60 wrote:

Hello Mo,

Thank you very much, I will try that for now. Also, this is the reponse I received from the EBI folks to my inquiry - the problem will hopefully be resolved by next week :)):

"On 2017.01.06., at 13:50, Marc Rossello mrosello@ebi.ac.uk wrote:

Dear colleagues,

The problems with the FTP servers that were limiting data downloads (or transferring files to Galaxy) have now largely been resolved and connections can be made and data downloaded. We are expecting the service to be stable by EOB today. It is still in an "at-risk" condition and the performance could be slower than usual until the beginning of next week.

Sorry for the inconvenience.

Regards, Marc"

ADD COMMENTlink written 20 months ago by scholtz60
0
gravatar for Jennifer Hillman Jackson
20 months ago by
United States
Jennifer Hillman Jackson25k wrote:

UPDATE 2016-01-13

The technical team at EBI SRA sent an email this morning that states that the service is back up and running normally as of a few days ago. Please try your queries again and let us know if you have problems.

Thanks for your patience, Jen & the Galaxy team

ADD COMMENTlink written 20 months ago by Jennifer Hillman Jackson25k
0
gravatar for Jennifer Hillman Jackson
19 months ago by
United States
Jennifer Hillman Jackson25k wrote:

UPDATE 2016-02-13

Many issues have been reported again with this tool. We are in communication with the data provider and an update will posted back here from our team when more is known.

Workaround options:

  1. Use the NCBI data retrieval tools when the data exists in both places.
  2. Download the data locally, directly from the ENA using their methods (if possible, may take a few attempts). Then upload to Galaxy using FTP.

Thanks again to everyone for the patience! Jen, Galaxy team

ADD COMMENTlink modified 19 months ago • written 19 months ago by Jennifer Hillman Jackson25k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 105 users visited in the last hour