Question: Speed Up Uploading Into Local Galaxy, Terribly Slow!!
0
gravatar for Alejandra Rougon
5.8 years ago by
Alejandra Rougon50 wrote:
Hello, I tried to search in the forums and although this question has appeared many times I still don't have a solution. I cannot manage to upload big files into the local galaxy, it just takes ages. Can I not just copy and paste into a local directory? why do I have to upload the files if it is already installed locally? I do not have a server webpage in order to use the url address option If I do it through ftp (locally) what ftp address shall I put?  ftp localhost:8080?? Is there any other option to speed up uploading, is so slow that is no longer worth using it, please help me!
galaxy • 2.3k views
ADD COMMENTlink modified 5.8 years ago by Greg Von Kuster810 • written 5.8 years ago by Alejandra Rougon50
0
gravatar for Greg Von Kuster
5.8 years ago by
Penn State University
Greg Von Kuster810 wrote:
You can upload your large file to Galaxy data libraries using a combination of "Upload files from filesystem paths" and "Do not copy data into Galaxy's default file store". See this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%2 0Files For all of the information on Galaxy data libraries, see this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries Greg Von Kuster
ADD COMMENTlink written 5.8 years ago by Greg Von Kuster810
Thank you Greg, I saw the wiki but I do not see the admin link in my galaxy interface. How do you open galaxy as an administrator? t ________________________________ To: Alejandra Rougon <alerougon@yahoo.com> Cc: "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu> Subject: Re: [galaxy-user] Speed up uploading into local Galaxy, terribly slow!! You can upload your large file to Galaxy data libraries using a combination of "Upload files from filesystem paths" and "Do not copy data into Galaxy's default file store".   See this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%2 0Files For all of the information on Galaxy data libraries, see this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries Greg Von Kuster Hello, I tried to search in the forums and although this question has appeared many times I still don't have a solution.
ADD REPLYlink written 5.8 years ago by Alejandra Rougon50
Hi, You will want to assign yourself as an administrator in the universe_wsgi.ini file. # Administrative users - set this to a comma-separated list of valid Galaxy # users (email addresses). These users will have access to the Admin section # of the server, and will have access to create users, groups, roles, # libraries, and more. For more information, see: # http://wiki.g2.bx.psu.edu/Admin/Interface #admin_users = None ^^ uncomment and add your login email here Other wikis that may be helpful, especially if user login is not yet set up: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Production%20Server http://wiki.g2.bx.psu.edu/Admin/Config Hopefully this helps, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support
ADD REPLYlink written 5.8 years ago by Jennifer Hillman Jackson23k
Dear Galaxy for genome wide metagenomics, I wonder if there is a way to purge conserved sequences. For example, if I have 99% of 10 million reads have perfect alignment to more then 20 reference sequences, I want them to go away. Inotherwords I only want unique sequences that align to only a few reference sequences. As for 16sDNA, a 100bp fragment taken from between V5 and V6 will align to everything!!! I want to purge that away Sincerely Scott Scott Tighe Advanced Genome Technology Lab Vermont Cancer Center at the University of Vermont 149 Beaumont Avenue Health Science Research Bd RM 305 Burlington Vermont USA 05405 lab 802-656-AGTC (2482) cell 802-999-6666
ADD REPLYlink written 5.8 years ago by Scott Tighe200
Hi Scott, There isn't a pre-made tool to do this, but creating a workflow to do some counts is possible. Using a combination of tools in "Text Manipulation" ('Cut'), "Join, Subtract and Group" ('Group'), and/or maybe "Statistics" ('Count'), should allow you to work with a simplified tabular file of hits to identify the query sequences with low specificity, then filter the associated alignments out of the primary data ('Join' or 'Compare two Datasets'). There are probably several ways to do this, but the general idea would be to cut out the query and hit IDs, perform some counts, then filter to only keep the query IDs that meet your criteria. Then, use that list to pull out the associated alignments from the original set (SAM or Interval or other text based format). If your data are very large, you might need to split this up into a few working files (being sure to keep all of the alignments for a particular query in the same analysis group). Hopefully this helps, Best, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support
ADD REPLYlink written 5.8 years ago by Jennifer Hillman Jackson23k
Dear Glaxy users and admin I ran my sequence data on FASTQC tool, output says it is EncodingSanger / Illumina 1.9 now i want to groom my file, but groomer does not have option for 1.9 in "Input FASTQ quality scores type" any idea which option i should select to grroom my file, later i want to run Bowtie or Tophat, Thanks
ADD REPLYlink written 5.8 years ago by Ateequr Rehman150
Hello, The input quality score type should be set as "Sanger" for your data. Thanks! Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support
ADD REPLYlink written 5.8 years ago by Jennifer Hillman Jackson23k
I have a question about the groomer.  Do all Illumina runs need to be groomed, or are there situations where it can be skipped??  (My data says illumina 1.5, so ive been picking input type as illumina 1.3-1.5.) rich ________________________________ To: Ateequr Rehman <ateeqrr@yahoo.com> Cc: "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu> Subject: Re: [galaxy-user] ILLumina 1.9 Hiseq Hello, The input quality score type should be set as "Sanger" for your data. Thanks! Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org.  Please keep all replies on the list by using "reply all" in your mail client.  For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:   http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at:   http://lists.bx.psu.edu/
ADD REPLYlink written 5.8 years ago by Richard Mark White240
Hi Rich, Tools will allow grooming to be skipped if the data is already in fastqsanger or fastqcssanger format (as appropriate). This can be set at upload or on the "Edit Attributes" form (pencil icon). For Illumina, these would be data resulting from the CASAVA 1.8+ pipeline. For Illumina 1.5, select "Illumina 1.3-1.7" on the FASTQ Groomer tool form to properly format the data. Thanks for bringing up a good topic! Not having to groom can save both time and disk space. Best, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support
ADD REPLYlink written 5.8 years ago by Jennifer Hillman Jackson23k
Hello Admin and users i wanted to visualize my data, i ran Tophat and converted sam to BAM and then cufflink, but totally unable to see any output data, any suggestion, how i could see my results For administrators, on my account run number 76 to 79 are the run i want to visualize Thanks Ateeq
ADD REPLYlink written 5.8 years ago by Ateequr Rehman150
Hi Ateeq, Please share a link to your history so that we can provide feedback. Use "Options -> Share or Publish", generate the share link (first button), copy the link into a reply email, and send that back to me directly. Also, your last few questions have been sent as replies to other questions on the mailing list with a new subject line. This causes them to thread/track incorrectly (and potentially be missed). When sending a new question, please start with a brand new message, address the "to" as "galaxy-user@bx.psu.edu" and this will reach us correctly. Thank you and I will watch for your reply, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support
ADD REPLYlink written 5.8 years ago by Jennifer Hillman Jackson23k
Hi Jen, I have a related question. If Illumina 1.9 is already in Sanger format, is it still necessary to groom the FASTQ files for TopHat? Would it be enough to directly change the data type to Sanger without grooming? Thanks, Carlos
ADD REPLYlink written 5.8 years ago by Carlos Borroto390
Thanks Jennifer, I already uncomment and add my email address but still the admin link does not appear. I then registered my email in users and logged in with that email address and I still don't see the link. when I type /admin at  the of the url it says that I have to be logged in as an administrator. sorry I am a newbe on this admin stuff.   ________________________________ To: Alejandra Rougon <alerougon@yahoo.com> Cc: "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu>; "closeticket@galaxyproject.org" <closeticket@galaxyproject.org> Subject: Re: [galaxy-user] Speed up uploading into local Galaxy, terribly slow!! Hi, You will want to assign yourself as an administrator in the universe_wsgi.ini file. # Administrative users - set this to a comma-separated list of valid Galaxy # users (email addresses).  These users will have access to the Admin section # of the server, and will have access to create users, groups, roles, # libraries, and more.  For more information, see: # http://wiki.g2.bx.psu.edu/Admin/Interface #admin_users = None ^^ uncomment and add your login email here Other wikis that may be helpful, especially if user login is not yet set up: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Production%20Server http://wiki.g2.bx.psu.edu/Admin/Config Hopefully this helps, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support
ADD REPLYlink written 5.8 years ago by Alejandra Rougon50
Do you restart your Galaxy server when you make changes to universe_wsgi.ini (like adding / uncommenting email addresses for the admin_users setting)?
ADD REPLYlink written 5.8 years ago by Greg Von Kuster810
I did close the web browser and opened it again was that enough? ________________________________ To: Alejandra Rougon <alerougon@yahoo.com> Cc: Jennifer Jackson <jen@bx.psu.edu>; "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu>; "closeticket@galaxyproject.org" <closeticket@galaxyproject.org> Subject: Re: [galaxy-user] Speed up uploading into local Galaxy, terribly slow!! Do you restart your Galaxy server when you make changes to universe_wsgi.ini (like adding / uncommenting email addresses for the admin_users setting)? Thanks Jennifer, I already uncomment and add my email address but still the admin link does not appear. I then registered my email in users and logged in with that email address and I still don't see the link. through ftp (locally) what ftp address shall I put? ftp manage your subscriptions to this and other Galaxy lists, at:
ADD REPLYlink written 5.8 years ago by Alejandra Rougon50
No, you'll need to stop and restart your Galaxy server.
ADD REPLYlink written 5.8 years ago by Greg Von Kuster810
Finally I managed to do it, thank you very much. Great improvement!
ADD REPLYlink written 5.8 years ago by Alejandra Rougon50
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 118 users visited in the last hour