Uploading Large Archive Histories

Question: Uploading Large Archive Histories

5.8 years ago by

Galaxy Team, I exported a large history that reduced to an 8GB compressed file. I would like to re-import this history but the "Import from file" options gives me an error (see attached image). I looked around the wiki and there appears to be an ftp upload for large files (http://wiki.galaxyproject.org/FTPUpload). Is there an equivalent FTP upload for exported large histories? I would like to export some of my large analysis histories and save them in an external hard drives to free some space in my account. However, I am not able to re-import large histories. I want to make sure I can re-import these large histories before I delete them from my Galaxy account. Getiria -- Getiria Onsongo, Ph.D. Informatics Analyst, Research Informatics Support System Minnesota Supercomputing Institute for Advanced Computational Research University of Minnesota Minneapolis, MN 55455 Phone: 612-624-0532

galaxy • 898 views

ADD COMMENT • link •

modified 5.8 years ago by Jennifer Hillman Jackson ♦ 25k • written 5.8 years ago by Getiria Onsongo • 40

5.8 years ago by

Jesse Erdmann • 10

Jesse Erdmann • 10 wrote:

After discussing this with Getiria, I think what is needed here is a way to use FTP to upload compressed histories from the import screen as is available from the upload data tool. The problem we're currently experiencing is that when we try to upload a compressed history where the compressed file is greater than 2GB, the process fails. This is preventing us from being able to effectively have users archive data from Galaxy to reduce our disk usage. This is a very pressing issue for us with our production Galaxy server at the Minnesota Supercomputing Institute. Additionally, if someone could provide a better understanding of how the exported histories are treated on the Galaxy server that would be helpful. e.g. how long do the compressed histories stick around on disk, etc? We would like to have users archive their data on their own systems and permanently delete their Galaxy history when an analysis is complete, but if the compressed history persists on the Galaxy server the utility of this is somewhat reduced. Thanks! -- Jesse Erdmann jesse@jesseerdmann.com Twitter: http://twitter.com/jesseerdmann

ADD COMMENT • link written 5.8 years ago by Jesse Erdmann • 10

5.8 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hi Getiria, Jeremy gave advice for the local, but I wanted to follow up about the trouble you were having uploading to the public Main Galaxy instance at https://main.g2.bx.psu.edu/ usegalaxy.org). Would you please try a larger history upload again today, if you have not already done so, and let us know if it is still problematic? Our apologies for the problems later last week. We believe these are cleared now, but want to make sure this function is working from your end as expected. Best, Jen Galaxy team -- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org

ADD COMMENT • link written 5.8 years ago by Jennifer Hillman Jackson ♦ 25k

Please log in to add an answer.

Similar posts • Search »