Large Data To Upload Problem

Question: Large Data To Upload Problem

4.8 years ago by

do kadya • 130

do kadya • 130 wrote:

Hi, Thank You for answers, I am able to learn everyday. I am having 500 GB hard disk, I am trying to integrate 'x' tool to galaxy for local lab purpose. 'x' tool require bam files which are of 400 GB file size. When I used Admin section to upload file my computer hang because of no space. Is it way that I can tell Galaxy that use these files directly or may be I can convert files to Galaxy required format so that galaxy will not create files by itself. As you can see in my example, I am having 500 GB HDD, and ~400 GB BAM files, replication of data by Galaxy its not possible for me. Some option like tell Galaxy about extension and properties of file, so that it should not replicate. By command line 'x' is working fine. Need some really good solution. Waiting for positive reply. Thank You. -- Reagrds, Chetan dokadya@gmail.com

galaxy • 1.0k views

ADD COMMENT • link •

modified 4.8 years ago by Bjoern Gruening ♦ 5.1k • written 4.8 years ago by do kadya • 130

4.8 years ago by

Bjoern Gruening ♦ 5.1k

Germany

Bjoern Gruening ♦ 5.1k wrote:

Hi Do, the link I send you yesterday, contains a section about linking file. Have a close look at the upload form in the admin section. There should be an option to link the file into Galaxy. As a side note, I really think you will need an additional harddrive, 100GB free space would be not enough, imho. Cheers, Bjoern

ADD COMMENT • link written 4.8 years ago by Bjoern Gruening ♦ 5.1k

Hi, Thank You for reply. I am still not clear about this website. https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP There are total 4 main points highlighted in this website. Do I have to follow all 4 points ? I have tried just on second point which is Configuring Galaxy As I just need same folder for galaxy where I can put my content directly. Now after sometime I will all of the points mentioned. Next Question is WHY GALAXY IS NOT ACCEPTING .bai files? .bai is indexed form of .bam files. I tried uploading using admin section upload, which not worked. next normal file upload as .bai file is just 6 MB., but not worked. next I write xml file to incorporate samtools index into galaxy. Program run successfully but output file is 0 KB. in all of my try .bai file is 0 KB. I need to upload because my 'x' program is not accepting normal BAM format hence it needs Indexed BAM. Waiting for positive reply. Regards, Chetan

ADD REPLY • link written 4.8 years ago by do kadya • 130

Hello Chetan, Björn sent you a different link yesterday. Please read through it - especially this part: *Upload files from filesystem paths*. https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFil es M.

ADD REPLY • link written 4.8 years ago by Martin Čech ♦♦ 4.9k

Hi, Thank you very much for link. I was able to solve the issue (whatever images shown are out dated may be. In my system its dropdown menu insted of checkbox to show "link to files without copy"). So can i do same for result ? Another Question: My 'x' software takes tab delimited list of files, as we all know, Galaxy follows its own file name. My 'x' software unable to look for those file. I have tried to upload all fresh so that my have little more impact but same error. I don't know why its unable to look for that. The folder from where I am uploading got all the files in that. Now when I am running my 'x' tool through galaxy, is it possible that its looking for default location of galaxy instead of ( path / folder) which i am using ? Waiting for positive reply. So in short can I have something like, Galaxy, you dont use your location for uploading, searching and writing results of command. Just use my folder and run my commands in that folder only, so that looking for files will be easy, also i will get to know the result at same place.

ADD REPLY • link written 4.8 years ago by do kadya • 130

Hi, Yes, can be but the idea is the same? ??? Please have a look at the basic Galaxy Tutorials. For example here: http://vimeo.com/79356949 You will find many more videos. In short you need to change the filetype that is associated with your dataset, probably from txt to tabular. ??? No, unless you are using the Galaxy API. With that you can archive partially what you want. Please read the wiki carefully, most of your questions are covered there. Cheers, Bjoern

ADD REPLY • link written 4.8 years ago by Bjoern Gruening ♦ 5.1k

Similar posts • Search »