Question: Upload To Galaxy Via Ftp
0
gravatar for Nate Coraor
8.0 years ago by
Nate Coraor3.2k
United States
Nate Coraor3.2k wrote:

Hello, With the continued advancement of sequencing technology, we've seen the size of files uploaded to Galaxy grow quite large. Although our current file upload methods have worked fine for years, they are not well suited for the extremely large files in common use today. Uploading directly from the browser can be unreliable and browsers don't provide feedback on upload progress and state like they do for downloads. Because of this, we have implemented file uploads to Galaxy via FTP. FTP will allow you to monitor as mentioned above, as well as resume interrupted transfers. To get started using FTP, you'll need to have registered a regular Galaxy account on our public server at: http://usegalaxy.org/ Once registered, you can initiate an FTP connection in your preferred FTP client to the same host usegalaxy.org) using your registered email address and password for the login details. Files uploaded to this server won't automatically be imported to Galaxy - rather, you will be presented with a list of the contents of your FTP directory on the standard "Upload File" tool interface. Files not imported within 3 days will be cleaned up from the FTP site. We hope that this service solves a crucial problem in data transfer speed and reliability. Please report any problems or suggestions with the service to us at: galaxy-bugs@bx.psu.edu Please note that it may not always be practical to use the public Galaxy servers. If you're routinely working with very large data and having to wait for it to upload, a local Galaxy server could be a more practical solution. Instructions on installing your own server can be found at: http://getgalaxy.org/ http://usegalaxy.org/production In addition, with the recent announcement of Amazon's parallel upload capability for the Simple Storage Service (S3), we are investigating incorporating easy access to S3 buckets for Galaxy instances on the Amazon Elastic Compute Cloud (EC2). But you don't need to wait for the pretty interface, you can already access contents of S3 buckets by pasting links to their contents in the "URL/Text:" field of the "Upload File" tool. For an example of how to do this, see the "Watch how the complete analysis can be performed on the Amazon Cloud" screencast at: http://usegalaxy.org/heteroplasmy Thanks, --nate Galaxy Team

ftp • 3.2k views
ADD COMMENTlink modified 4.1 years ago by Martin Čech ♦♦ 4.9k • written 8.0 years ago by Nate Coraor3.2k
0
gravatar for Peter
8.0 years ago by
Peter260
Peter260 wrote:
Hi Nate, Just to clarify - the new functionality is to allow FTP from the user's computer (running an FTP client) to a Galaxy server (running an FTP server), right? I'm already using the "Upload File from your computer" tool to import a file from a remote server into a Galaxy server. Note this functionality would be more discoverable if the tool name or description was changed... how about: "Import File from your computer or another server"? Peter
ADD COMMENTlink written 8.0 years ago by Peter260
That is correct. So many things already refer to "Upload File" that it might be a bit difficult to change. The upside is that this long establishment means that it will hopefully be the first place people look for such functionality. Thanks, --nate
ADD REPLYlink written 8.0 years ago by Nate Coraor3.2k
0
gravatar for SHAUN WEBB
8.0 years ago by
SHAUN WEBB70
SHAUN WEBB70 wrote:
Great, works well. Will there be documentation on how to implement this on a local install? Shaun Quoting Nate Coraor <nate@bx.psu.edu>: -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
ADD COMMENTlink written 8.0 years ago by SHAUN WEBB70
Yes! The short version is to set the two new 'ftp_*' options in the config. I'll add a sample FTP server configuration to the Wiki as well. --nate
ADD REPLYlink written 8.0 years ago by Nate Coraor3.2k
Added here: http://bitbucket.org/galaxy/galaxy-central/wiki/Config/UploadViaFTP And linked from the production server page. --nate
ADD REPLYlink written 8.0 years ago by Nate Coraor3.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 167 users visited in the last hour