Latest
Open
RNA-Seq
ChIP-Seq
SNP
Assembly
Forum
Planet
All »
Home
Log In
Question: How to download a batch consisting of 50 bam files, using CURL or wget
1
gravatar for ppurkayastha2010
8 months ago by
ppurkayastha2010 • 30
ppurkayastha2010 • 30 wrote:

How could I download a batch consists of 50 bam files, using CURL or wget.

I tried the following solution,

https://biostar.usegalaxy.org/p/26527/

but still is not working :(

wget -O --no-check-certificate 'https://usegalaxy.org/api/dataset_collections/d103ecff2e1c7b62/download'
--2018-04-04 01:59:36--  https://usegalaxy.org/api/dataset_collections/d103ecff2e1c7b62/download
Resolving usegalaxy.org usegalaxy.org)... 129.114.60.56, 129.114.60.60
Connecting to usegalaxy.org usegalaxy.org)|129.114.60.56|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2018-04-04 01:59:37 ERROR 403: Forbidden.

I tried doing all the listed options in the following link for big data

https://galaxyproject.org/support/download-data/#download-tip-big-data

but still it is giving me the same error.

curl linux galaxy wget • 442 views
ADD COMMENT • link •
modified 7 months ago by Martin Čech ♦♦ 4.9k • written 8 months ago by ppurkayastha2010 • 30
2
gravatar for Martin Čech
7 months ago by
Martin Čech ♦♦ 4.9k
United States
Martin Čech ♦♦ 4.9k wrote:

For dataset collections and datasets within collections you have to supply your API key with the request. You can add it to the end of the collection download url, the command would look something like this:

$ wget https://usegalaxy.org/api/dataset_collections/d20ad3e1ccd4595de/download?key=MYSECRETAPIKEY (note the ?key= before the API key itself)

I updated our docs at https://galaxyproject.org/support/download-data to reflect this requirement.

ADD COMMENT • link written 7 months ago by Martin Čech ♦♦ 4.9k

Thanks a lot :-) I could download the files now..

ADD REPLY • link written 7 months ago by ppurkayastha2010 • 30

but this is not allowing to download more than a GB data.

Showing an error

Read error at byte 1079804844 (Success).Retrying.

ADD REPLY • link modified 7 months ago • written 7 months ago by ppurkayastha2010 • 30

Related post closed out as a duplicate with more details: https://biostar.usegalaxy.org/p/27392/

Quote:

I am trying to download a data-set consisting of 50 bam files, I use the following command to download

$ wget https://usegalaxy.org/api/dataset_collections/d20ad3e1ccd4595de/download?key=MYSECRETAPIKEY

but I am unable to download the data beyond 1 GB.

I tried to download using Curl, but I still get the following error

curl -O 'https://usegalaxy.org/api/dataset_collections/e3f40e4987/download?key=MYSECRETKEY'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1029M    0 1029M    0     0   256k      0 --:--:--  1:08:26 --:--:--  259k
curl: (18) transfer closed with outstanding read data remaining

Please help

ADD REPLY • link written 7 months ago by Jennifer Hillman Jackson ♦ 25k
Please log in to add an answer.

Similar posts • Search »

Content
Help
Access
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 175 users visited in the last hour