Question: bowtie2 out of memory
1
gravatar for leontp587
3.3 years ago by
leontp58750
United States
leontp58750 wrote:

I'm trying to run bowtie2 in galaxy on my own centos installation to align paired end data to the hg19 reference. I get an out of memory error:

bowtie2 -p ${GALAXY_SLOTS:-4} -x /home/leon/ref_data/bowtie2/hg19/hg19 -1 /home/leon/genome_data/sra/ERR010982/ERR010982_1.fastq -2 /home/leon/genome_data/sra/ERR010982/ERR010982_2.fastq -I 0 -X 250 | samtools view -Su - | samtools sort -o - - > /home/leon/galaxy-dist/database/files/000/dataset_29.dat

Out of memory allocating the ebwt[] array for the Bowtie index. Please try again on a computer with more memory. Error: Encountered internal Bowtie 2 exception (#1) Command: /home/leon/galaxy-dist/dependency_dir/bowtie2/2.1.0/devteam/package_bowtie2_2_1

There is about 2 GB of free memory in the system, and virtual memory is disabled. Why is this happening? It seems like plenty of memory is available.

memory local scaling galaxy bowtie2 • 3.5k views
ADD COMMENTlink modified 3.3 years ago • written 3.3 years ago by leontp58750
1

Thanks Jen! I took it up to 13GB of memory and it worked very well.

ADD REPLYlink written 3.3 years ago by leontp58750

Please accept the answer so others can find it easily.

ADD REPLYlink written 3.3 years ago by Martin Čech ♦♦ 4.2k
3
gravatar for Jennifer Hillman Jackson
3.3 years ago by
United States
Jennifer Hillman Jackson23k wrote:

Hello,

Free disk for data memory (storage) and processing memory (RAM) are different. The general recommendation is 8 GB processing memory, but that is the very lower limit, and for this tool 16 GB might be best (as far I know, when using a human reference genome and large inputs). 2G storage is almost certainly not enough under any config.

The underlying 3rd party tool requirements must still be met when executing the tools in Galaxy. And certain parameter settings can greatly increase processing memory requirements, as can the size of the reference genome (and indexes) - hg19 is considered large. Review the google group posts for what config others are using for successful processing plus review parameters for those that increase processing memory needs:
http://ccb.jhu.edu/software/tophat/manual.shtml#prer

And are some statistics from the Galaxy community about stand-alone servers, for reference:
http://wiki.galaxyproject.org/Community/GalaxyAdmins/SurveyResults#Standalone_Server

Scaling up your local is one option. More details for local Galaxy implementations can be found here:
http://wiki.galaxyproject.org/Admin

If that is not possible, another Galaxy choice could be the solution:
http://wiki.galaxyproject.org/Support#About_Galaxy
http://wiki.galaxyproject.org/BigPicture/Choices

CloudMan Galaxy is a common pick. Why? AWS offer grants for research purposes (which may apply for project) and scaling it is on demand. That said, there are other cloud Galaxy options described here (Globus, etc.):
http://wiki.galaxyproject.org/Cloud

Hopefully one of these work out for you, Jen, Galaxy team



 

ADD COMMENTlink written 3.3 years ago by Jennifer Hillman Jackson23k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 91 users visited in the last hour