Speed Up Uploading Into Local Galaxy, Terribly Slow!!

6.8 years ago by

Penn State University

You can upload your large file to Galaxy data libraries using a combination of "Upload files from filesystem paths" and "Do not copy data into Galaxy's default file store". See this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%2 0Files For all of the information on Galaxy data libraries, see this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries Greg Von Kuster

ADD COMMENT • link written 6.8 years ago by Greg Von Kuster ♦ 810

Thank you Greg, I saw the wiki but I do not see the admin link in my galaxy interface. How do you open galaxy as an administrator? t ________________________________ To: Alejandra Rougon <alerougon@yahoo.com> Cc: "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu> Subject: Re: [galaxy-user] Speed up uploading into local Galaxy, terribly slow!! You can upload your large file to Galaxy data libraries using a combination of "Upload files from filesystem paths" and "Do not copy data into Galaxy's default file store". See this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%2 0Files For all of the information on Galaxy data libraries, see this wiki: http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Libraries Greg Von Kuster Hello, I tried to search in the forums and although this question has appeared many times I still don't have a solution.

ADD REPLY • link written 6.8 years ago by Alejandra Rougon • 50

Hi, You will want to assign yourself as an administrator in the universe_wsgi.ini file. # Administrative users - set this to a comma-separated list of valid Galaxy # users (email addresses). These users will have access to the Admin section # of the server, and will have access to create users, groups, roles, # libraries, and more. For more information, see: # http://wiki.g2.bx.psu.edu/Admin/Interface #admin_users = None ^^ uncomment and add your login email here Other wikis that may be helpful, especially if user login is not yet set up: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Production%20Server http://wiki.g2.bx.psu.edu/Admin/Config Hopefully this helps, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

ADD REPLY • link written 6.8 years ago by Jennifer Hillman Jackson ♦ 25k

Dear Galaxy for genome wide metagenomics, I wonder if there is a way to purge conserved sequences. For example, if I have 99% of 10 million reads have perfect alignment to more then 20 reference sequences, I want them to go away. Inotherwords I only want unique sequences that align to only a few reference sequences. As for 16sDNA, a 100bp fragment taken from between V5 and V6 will align to everything!!! I want to purge that away Sincerely Scott Scott Tighe Advanced Genome Technology Lab Vermont Cancer Center at the University of Vermont 149 Beaumont Avenue Health Science Research Bd RM 305 Burlington Vermont USA 05405 lab 802-656-AGTC (2482) cell 802-999-6666

ADD REPLY • link written 6.8 years ago by Scott Tighe • 200

Hi Scott, There isn't a pre-made tool to do this, but creating a workflow to do some counts is possible. Using a combination of tools in "Text Manipulation" ('Cut'), "Join, Subtract and Group" ('Group'), and/or maybe "Statistics" ('Count'), should allow you to work with a simplified tabular file of hits to identify the query sequences with low specificity, then filter the associated alignments out of the primary data ('Join' or 'Compare two Datasets'). There are probably several ways to do this, but the general idea would be to cut out the query and hit IDs, perform some counts, then filter to only keep the query IDs that meet your criteria. Then, use that list to pull out the associated alignments from the original set (SAM or Interval or other text based format). If your data are very large, you might need to split this up into a few working files (being sure to keep all of the alignments for a particular query in the same analysis group). Hopefully this helps, Best, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

ADD REPLY • link written 6.8 years ago by Jennifer Hillman Jackson ♦ 25k

Dear Glaxy users and admin I ran my sequence data on FASTQC tool, output says it is EncodingSanger / Illumina 1.9 now i want to groom my file, but groomer does not have option for 1.9 in "Input FASTQ quality scores type" any idea which option i should select to grroom my file, later i want to run Bowtie or Tophat, Thanks

ADD REPLY • link written 6.8 years ago by Ateequr Rehman • 150

Hello, The input quality score type should be set as "Sanger" for your data. Thanks! Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

ADD REPLY • link written 6.8 years ago by Jennifer Hillman Jackson ♦ 25k

I have a question about the groomer. Do all Illumina runs need to be groomed, or are there situations where it can be skipped?? (My data says illumina 1.5, so ive been picking input type as illumina 1.3-1.5.) rich ________________________________ To: Ateequr Rehman <ateeqrr@yahoo.com> Cc: "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu> Subject: Re: [galaxy-user] ILLumina 1.9 Hiseq Hello, The input quality score type should be set as "Sanger" for your data. Thanks! Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

ADD REPLY • link written 6.8 years ago by Richard Mark White • 240

Hi Rich, Tools will allow grooming to be skipped if the data is already in fastqsanger or fastqcssanger format (as appropriate). This can be set at upload or on the "Edit Attributes" form (pencil icon). For Illumina, these would be data resulting from the CASAVA 1.8+ pipeline. For Illumina 1.5, select "Illumina 1.3-1.7" on the FASTQ Groomer tool form to properly format the data. Thanks for bringing up a good topic! Not having to groom can save both time and disk space. Best, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

ADD REPLY • link written 6.8 years ago by Jennifer Hillman Jackson ♦ 25k

Hello Admin and users i wanted to visualize my data, i ran Tophat and converted sam to BAM and then cufflink, but totally unable to see any output data, any suggestion, how i could see my results For administrators, on my account run number 76 to 79 are the run i want to visualize Thanks Ateeq

ADD REPLY • link written 6.8 years ago by Ateequr Rehman • 150

Hi Ateeq, Please share a link to your history so that we can provide feedback. Use "Options -> Share or Publish", generate the share link (first button), copy the link into a reply email, and send that back to me directly. Also, your last few questions have been sent as replies to other questions on the mailing list with a new subject line. This causes them to thread/track incorrectly (and potentially be missed). When sending a new question, please start with a brand new message, address the "to" as "galaxy-user@bx.psu.edu" and this will reach us correctly. Thank you and I will watch for your reply, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

ADD REPLY • link written 6.8 years ago by Jennifer Hillman Jackson ♦ 25k

Hi Jen, I have a related question. If Illumina 1.9 is already in Sanger format, is it still necessary to groom the FASTQ files for TopHat? Would it be enough to directly change the data type to Sanger without grooming? Thanks, Carlos

ADD REPLY • link written 6.8 years ago by Carlos Borroto • 390

Thanks Jennifer, I already uncomment and add my email address but still the admin link does not appear. I then registered my email in users and logged in with that email address and I still don't see the link. when I type /admin at the of the url it says that I have to be logged in as an administrator. sorry I am a newbe on this admin stuff. ________________________________ To: Alejandra Rougon <alerougon@yahoo.com> Cc: "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu>; "closeticket@galaxyproject.org" <closeticket@galaxyproject.org> Subject: Re: [galaxy-user] Speed up uploading into local Galaxy, terribly slow!! Hi, You will want to assign yourself as an administrator in the universe_wsgi.ini file. # Administrative users - set this to a comma-separated list of valid Galaxy # users (email addresses). These users will have access to the Admin section # of the server, and will have access to create users, groups, roles, # libraries, and more. For more information, see: # http://wiki.g2.bx.psu.edu/Admin/Interface #admin_users = None ^^ uncomment and add your login email here Other wikis that may be helpful, especially if user login is not yet set up: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Production%20Server http://wiki.g2.bx.psu.edu/Admin/Config Hopefully this helps, Jen Galaxy team -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

ADD REPLY • link written 6.8 years ago by Alejandra Rougon • 50

Do you restart your Galaxy server when you make changes to universe_wsgi.ini (like adding / uncommenting email addresses for the admin_users setting)?

ADD REPLY • link written 6.8 years ago by Greg Von Kuster ♦ 810

I did close the web browser and opened it again was that enough? ________________________________ To: Alejandra Rougon <alerougon@yahoo.com> Cc: Jennifer Jackson <jen@bx.psu.edu>; "galaxy-user@lists.bx.psu.edu" <galaxy-user@lists.bx.psu.edu>; "closeticket@galaxyproject.org" <closeticket@galaxyproject.org> Subject: Re: [galaxy-user] Speed up uploading into local Galaxy, terribly slow!! Do you restart your Galaxy server when you make changes to universe_wsgi.ini (like adding / uncommenting email addresses for the admin_users setting)? Thanks Jennifer, I already uncomment and add my email address but still the admin link does not appear. I then registered my email in users and logged in with that email address and I still don't see the link. through ftp (locally) what ftp address shall I put? ftp manage your subscriptions to this and other Galaxy lists, at:

Similar posts • Search »