Question: How to move data from a server to galaxy running on another server
0
gravatar for zfrenchee
2.5 years ago by
zfrenchee0
zfrenchee0 wrote:

I'm running Bjoern's Docker Galaxy and have been wanting to move data from a local server (but not my local computer) to galaxy. I initially looked into FXP but that didn't really go anywhere. I also have a script which uses the bioblend galaxy api to upload groups of files to data libraries. But I was wondering whether I was overlooking something, some easy way to move large files to galaxy, short of this script.

Thanks!

software error • 669 views
ADD COMMENTlink modified 2.5 years ago by Hotz, Hans-Rudolf1.8k • written 2.5 years ago by zfrenchee0
1

Hi

Using the bioblend galaxy api in a script to generate datalibraries sounds like the best way to me.

Maybe you can elaborate a bit, how the 'external' data looks like: - is it static/dynamic - all files have the same type, i.e. it is just a bunch of fastg files - the date is stored in a complicated/random directory - etc.

Regards, Hans-Rudolf

ADD REPLYlink written 2.5 years ago by Hotz, Hans-Rudolf1.8k

Let's assume it's a bunch of big fastq files on a server, called B. You're at your local computer A, and galaxy is on a cloud-service provider VM called C. You want to get your fastq files from B to C without passing through A. You can ssh into both B and C. What's the best way to move say 10GB, 100BG, 1TB, from B to C?

ADD REPLYlink written 2.5 years ago by zfrenchee0
0
gravatar for Hotz, Hans-Rudolf
2.5 years ago by
Switzerland
Hotz, Hans-Rudolf1.8k wrote:

Hi

I still consider "using the bioblend galaxy api in a script to generate datalibraries" as the best way to do it. As an alternative, I would suggest this 'quick and dirty' option: write your own copy tool

  • generate a tool xml which lists all the fastq files (this can be hard coded or dynamic, this can be one long list or a tree)
  • the user selects the files she/he wants
  • the tool copies the selected files from "B" into Galaxy as history items on "C"

it might not be 'quick'.... ;)

Hans-Rudolf

ADD COMMENTlink written 2.5 years ago by Hotz, Hans-Rudolf1.8k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 176 users visited in the last hour