Hello.
I tried to export datasets from usegalaxy.org to our local server. I followed the following command:
curl -o outputfile --insecure '<link>'
I used to get smooth download in the past, but now, i get a message that says "Couldn't resolve host 'usegalaxy.org." What does this mean, and how can this be fixed?
Thank you!
Hello,
This should work. I ran a test on a small 50 line tabular file and it ran fine. Try this to see if it works:
$ curl -o outfile --insecure 'https://usegalaxy.org/datasets/bbd44e69cb8906b50cf794ca01349017/display?to_ext=sam'
If it does, then something is wrong with the particular dataset you are downloading.
As a first pass solution, please try resetting your dataset's metadata by clicking on the pencil icon and using the "autodetect" button on the first tab of the Edit Attributes forms. Then try the curl command again.
Should that fail, let us know. We'll want to look at your dataset to find out what might be going on.
https://galaxyproject.org/support/download-data/
Thanks! Jen, Galaxy team
Thank you, Jen. I will have to try this out when I get to my office computer. My datasets are trimmed fastq files in a dataset collection of paired reads. I want to download the collection as a whole (with the different subfolders for each sample and pairs), but since Galaxy doesn't have this feature yet, apparently, I am downloading each single file (i.e. forward and reverse separately) manually by right-clicking on the diskette icon and downloading using the command line, and the above-mentioned issue was all I got.
I also tried downloading the file directly to my computer by clicking on the diskette icon since each dataset in the collection is just less than 500Mb. However, this method also doesn't show the download window. I tried this in two different computers using Google chrome and internet explorer. Is there anything wrong with my connection?
Hello Jen. I tried the command and link you made:
$ curl -o outfile --insecure 'https://usegalaxy.org/datasets/bbd44e69cb8906b50cf794ca01349017/display?to_ext=sam'
This also shows the same message: ![enter image description here][1]
Hello Jen, I tried all your suggestions but I still get the same message. My files are in fastqsanger format. I also tried downloading the file directly to my computer by clicking on the diskette icon since each dataset in the collection is just less than 500Mb. However, this method also doesn't show the download window. I tried this in two different computers using Google chrome and internet explorer. Is this because of my system?
You may check my datasets, please.
Hi! I took a look at your account and found that the data are nested into a collection (I either missed this before or we didn't chat about it yet). Data in nested collections will not download directly using the disc icon and/or the associated link with curl.
I didn't get the same error that you did with curl, instead, an html file downloaded with some metadata, without the actual data content. The disc icon did not work at all. Both are expected for data in nested collections, at least for now in the current release (there are some new ideas for collection handling to enhance download options in the future). For now, the collection must be expanded (aka flattened) first. Changing the format in this way does not use up additional account quota space.
Now, this may not address your connection error - but you could try running this tool on the collection: Collection Operations > Flatten Collection into a flat list of datasets to obtain datasets that can be downloaded using either of those two methods (browser download or curl) to see if that works the same as it did for me. My guess is that it won't given your particular error from curl - yet perhaps try anyway? Just to be certain? The browser download might be ok.
I tested with Chrome/MAC OSX for the browser "disc" icon download and with Terminal/MAC OSX for the curl command. The tested collection/datasets were from the history: Apiales > 110: Trimmomatic on collection 76: paired. Both methods accessed the "flattened" version of the data.
The connection problems with usegalaxy.org could very well be a firewall issue from the settings on your computer or on the network you are connecting from. Is there a local admin that you could check in with? If you are not working from an institution/business with admin support, checking with your ISP provider about potential firewalls is another area to look into. I am not sure what we could do on our side to help resolve this but will bring in our own admin for advice. Given the time of day EST, we wouldn't be able to get back to you until tomorrow about this, but we will definitely follow up.
Take care and very sorry that you are still having problems with this!! Jen, Galaxy team
Dear Jen, Thank you so much for your kind reply. Sorry, the time zones are not in our favor, but I really appreciate your help. I asked our local network admin, and he confirmed that there might be some problem with our server configuration these days, so he is looking into it.
And I tried your suggestion to flatten and the disk icon now works, although a lot slower than when I used curl in the past. I am okay with this for now as long as I can have my data while our admin is trying to resolve the issue. Thank you once again! I really appreciate it a lot!
Take care, Nomar
