Transfer from one Galaxy server to another?

Question: Transfer from one Galaxy server to another?

4.3 years ago by

United States

Hi!

I am learning the basic of Galaxy by going through Blankenberg and Hillman-Jackson MethodsMolBiol2014.

The mapping step is taking a while on the public server http://usegalaxy.org so I want to perform the learning procedure on a local galaxy server which is already install at my university. What are my options for this step?

Do I must download the fastq file to my computer and than upload it to the desired galaxy server or there is a way to share the history or files from server to server?

Thanks and have a nice day

galaxy • 3.3k views

ADD COMMENT • link •

modified 2.7 years ago by Guy Reeves • 1.0k • written 4.3 years ago by jean-philippe.brosseau • 0

4.3 years ago by

Martin Čech ♦♦ 4.9k

United States

Martin Čech ♦♦ 4.9k wrote:

You can tell Galaxy to download accessible datasets from any source including other Galaxies.

In order to do that:

find the URL of the dataset you want to transfer (if the dataset is in other Galaxy click on it in the history and right-click-save the URL of the dataset accessible through the floppy disk icon)
in your Galaxy click the upload data button (top right corner of the picture)
Select Paste/Fetch data
Enter the URL and hit start
The dataset will be transferred from server to server, you don't even need to have Galaxy opened.

the picture: https://www.dropbox.com/s/u5f2mpeat3rn9fz/Screenshot%202014-08-08%2015.05.08.png

ADD COMMENT • link modified 4.3 years ago • written 4.3 years ago by Martin Čech ♦♦ 4.9k

Agreed - dataset transfer is a great alternative! Jen

ADD REPLY • link written 4.3 years ago by Jennifer Hillman Jackson ♦ 25k

I didnt get it for #2 Where should I paste this URL?

thanks!

ADD REPLY • link written 4.3 years ago by jean-philippe.brosseau • 0

see the attached picture, click the button in the top left and the modal window, where you can paste the url, will open (on the picture the URL is already pasted there, just need to hit 'Start')

ADD REPLY • link written 4.3 years ago by Martin Čech ♦♦ 4.9k

I can now see this icon that you are talking about but only on the usegalaxy site.

I cannot see that icon on the galaxy site from my university, which the one that I want to upload some files. Is it normal?

ADD REPLY • link written 4.3 years ago by jean-philippe.brosseau • 0

Probably old version of Galaxy. You should be able to find the old upload form under 'Get Data'/'Upload File' in the toolpanel (left). Just paste the URL in the box and hit Execute.

ADD REPLY • link written 4.3 years ago by Martin Čech ♦♦ 4.9k

When I try to do so, i get an empty file and the following message (although it goes to completion)

Unable to fetch https://usegalaxy.org/datasets/bbd44e69cb8906b5be2ea87a3d394827/display?to_ext=fastqsanger [Errno socket error] [Errno 110] Connection timed out

ADD REPLY • link written 4.3 years ago by jean-philippe.brosseau • 0

This is a good time to contact the local administrator of your instance. There may be transfer limits or external import blocks in place. Or, as Martin mentions, an older version of Galaxy or one that has been customized could be the root cause of the issue - it almost certainty does not have the most recent fixes or could have (local to your system) undetected conflicts with dataset import based on configuration.

Upgrading the instance would be the first step toward resolving any issue - in particular because of the recent security fix: http://wiki.galaxyproject.org/DevNewsBriefs/2014_08_11#Security

For immediate analysis, you call always fall back to direct data upload.

Jen, Galaxy team

ADD REPLY • link modified 4.3 years ago • written 4.3 years ago by Jennifer Hillman Jackson ♦ 25k

The file has ~12GB so direct upload is not an option. Although you can use FTP to upload it to your galaxy if it is configured in a way to support it.

ADD REPLY • link written 4.3 years ago by Martin Čech ♦♦ 4.9k

All correct, poor choice of wording on my part. By direct, I meant using upload in that Galaxy instance (by a configured option: FTP, Data library), instead of the Galaxy server-to-server transfer. Hope this works out for you! Jen, Galaxy team

ADD REPLY • link modified 4.3 years ago • written 4.3 years ago by Jennifer Hillman Jackson ♦ 25k

2.7 years ago by

Guy Reeves • 1.0k

Germany

Guy Reeves • 1.0k wrote:

Hi This is a really great feature to move individual datasets (Martins message). The only down side is that the name of the dataset is the URL address and not the dataset name in the original galaxy instance. If you try to do it with .bam files you need to click on the download button and then right click the 'download dataset' popup.

I tried to import the history (Jens message) in the hope that names would be kept (two datasets in history). Using the 'export tor file' AND 'import to file' I could get it to work.

Download entire histories by selecting "Export to File" from the History menu, and clicking on the link generated.

2.Transfer entire histories by selecting "Export to File" from the History menu, generating the link, coping the link in the "from" Galaxy instance, then in the "to" Galaxy instance select "Import from File" from the History menu, and paste in the link into the new form.

https://wiki.galaxyproject.org/Support

While you get a page saying ' This history will be visible when the import is complete' you actually have to click on the 'view all histories' button to see the new history. It will have the original name with the prefix 'imported from archive:' and all the datasets will have the original names which is great.

But you do need to be patient as the "export to file" command might take a while to complete for large histories and the "import from file" command will not work until it has. I find it useful to copy the export URL to a text file in case I overwrite the clipboard while I am waiting .

Thanks Guy

ADD COMMENT • link modified 2.7 years ago • written 2.7 years ago by Guy Reeves • 1.0k

4.3 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

Uploading the initial datasets (fastq, etc) is the simplest and surest route for a successful analysis.

That said, you do have the option of copying a history between Galaxy servers. Under the History menu are two options "Export to File" and "Import from File". The first time used with any history, a URL will be presented at export, which can be used for the import, eliminating the need to actually perform the download. This functionality is still undergoing full development, so if you choose to try this and it fails or presents problems, back up and start with the direct dataset upload.

A ticket (that also links to other tickets) listing out future import/export development plans is here:
http://trello.com/c/qCfAWeYU

Best, Jen, Galaxy team

ADD COMMENT • link modified 4.3 years ago • written 4.3 years ago by Jennifer Hillman Jackson ♦ 25k

Please log in to add an answer.

Similar posts • Search »