Upload File Size And User Authorization

Question: Upload File Size And User Authorization

7.7 years ago by

Two questions, either of which may belong more on the developers list. But lets try here first: 1. As we migrate services to Galaxy, a stakeholder expressed concern at uploading large datafiles of "several gigabytes". Given that we can require users to upload from an ftp site or something similar, does anyone have experience with how well this works with gigabyte- size files? I figure I can always write a tool to upload masses of data via a slow but robust method, but it would be good to know if this is required. 2. Similarly, someone else wanted to know if Galaxy authorization could be plugged into another system, such as Active Directory. Any pointers on this, or will I have to cut some code? thanks Paul Agapow (paul-michael.agapow@hpa.org.uk) Bioinformatics, Centre for Infections, Health Protection Agency ********************************************************************** **** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk ********************************************************************** ****

galaxy • 866 views

ADD COMMENT • link •

modified 7.7 years ago by Pieter Neerincx • 360 • written 7.7 years ago by Paul-Michael Agapow • 120

7.7 years ago by

Pieter Neerincx • 360

Pieter Neerincx • 360 wrote:

Hi Paul, Large files are no problem and you may use an FTP site. Depending how much bandwidth you have it may take some time though to move large files around. Please see: https://bitbucket.org/galaxy/galaxy- central/wiki/Config/ProductionServer for details on how to optimize uploads. Downloads are are different thing though as right now you can only get them as an HTTP download. Yes, using "external authentication" via an Apache proxy as described here: https://bitbucket.org/galaxy/galaxy- central/wiki/Config/ApacheProxy I'm using this to link my Galaxy to our University's LDAP and that works like a charm :). Cheers, Pi mobile: +31 6 143 66 783 e-mail: pieter.neerincx@gmail.com skype: pieter.online

ADD COMMENT • link written 7.7 years ago by Pieter Neerincx • 360

Peter, Damn - looked through the documentation and missed that authentication bit. Good to know that the large files are no problem - I'm sure the people around here are going to push it to the limit! Thanks again p Paul Agapow (paul-michael.agapow@hpa.org.uk) Bioinformatics, Centre for Infections, Health Protection Agency To: Paul-Michael Agapow Cc: galaxy-user@lists.bx.psu.edu Subject: Re: [galaxy-user] Upload file size and user authorization Hi Paul, But lets try here first: concern at uploading large datafiles of "several gigabytes". Given that we can require users to upload from an ftp site or something similar, does anyone have experience with how well this works with gigabyte- size files? I figure I can always write a tool to upload masses of data via a slow but robust method, but it would be good to know if this is required. Large files are no problem and you may use an FTP site. Depending how much bandwidth you have it may take some time though to move large files around. Please see: https://bitbucket.org/galaxy/galaxy- central/wiki/Config/ProductionServer for details on how to optimize uploads. Downloads are are different thing though as right now you can only get them as an HTTP download. authorization could be plugged into another system, such as Active Directory. Any pointers on this, or will I have to cut some code? Yes, using "external authentication" via an Apache proxy as described here: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy I'm using this to link my Galaxy to our University's LDAP and that works like a charm :). Cheers, Pi ********************************************************************** ** ** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving.HTTP://www.HPA.org.uk ********************************************************************** ** ** mobile: +31 6 143 66 783 e-mail: pieter.neerincx@gmail.com skype: pieter.online ********************************************************************** **** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk ********************************************************************** ****

ADD REPLY • link written 7.7 years ago by Paul-Michael Agapow • 120

Hi Paul, In your defense: I couldn't find it back myself at first; it's not always easy to find what you are looking for on the wiki... Well, the upload shouldn't be a problem at all, but making sure your server(s) don't run out of disk space in no time can be quite problematic with NGS data :o... Galaxy stores all data - inputs, intermediate files and final results - by default. Files will only vanish if users actively delete them (and the admin runs the clean up scripts): This is very convenient for the end-users :), but it may be a headache for the sys admin... Cheers, Pi mobile: +31 6 143 66 783 e-mail: pieter.neerincx@gmail.com skype: pieter.online

ADD REPLY • link written 7.7 years ago by Pieter Neerincx • 360

Similar posts • Search »