bowtie2 out of memory

Question: bowtie2 out of memory

4.3 years ago by

United States

leontp587 • 50 wrote:

I'm trying to run bowtie2 in galaxy on my own centos installation to align paired end data to the hg19 reference. I get an out of memory error:

bowtie2 -p ${GALAXY_SLOTS:-4} -x /home/leon/ref_data/bowtie2/hg19/hg19 -1 /home/leon/genome_data/sra/ERR010982/ERR010982_1.fastq -2 /home/leon/genome_data/sra/ERR010982/ERR010982_2.fastq -I 0 -X 250 | samtools view -Su - | samtools sort -o - - > /home/leon/galaxy-dist/database/files/000/dataset_29.dat

Out of memory allocating the ebwt[] array for the Bowtie index. Please try again on a computer with more memory. Error: Encountered internal Bowtie 2 exception (#1) Command: /home/leon/galaxy-dist/dependency_dir/bowtie2/2.1.0/devteam/package_bowtie2_2_1

There is about 2 GB of free memory in the system, and virtual memory is disabled. Why is this happening? It seems like plenty of memory is available.

memory local scaling galaxy bowtie2 • 5.1k views

ADD COMMENT • link •

modified 4.3 years ago • written 4.3 years ago by leontp587 • 50

Thanks Jen! I took it up to 13GB of memory and it worked very well.

ADD REPLY • link written 4.3 years ago by leontp587 • 50

Please accept the answer so others can find it easily.

ADD REPLY • link written 4.3 years ago by Martin Čech ♦♦ 4.9k

4.3 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

Free disk for data memory (storage) and processing memory (RAM) are different. The general recommendation is 8 GB processing memory, but that is the very lower limit, and for this tool 16 GB might be best (as far I know, when using a human reference genome and large inputs). 2G storage is almost certainly not enough under any config.

The underlying 3rd party tool requirements must still be met when executing the tools in Galaxy. And certain parameter settings can greatly increase processing memory requirements, as can the size of the reference genome (and indexes) - hg19 is considered large. Review the google group posts for what config others are using for successful processing plus review parameters for those that increase processing memory needs:
http://ccb.jhu.edu/software/tophat/manual.shtml#prer

And are some statistics from the Galaxy community about stand-alone servers, for reference:
http://wiki.galaxyproject.org/Community/GalaxyAdmins/SurveyResults#Standalone_Server

Scaling up your local is one option. More details for local Galaxy implementations can be found here:
http://wiki.galaxyproject.org/Admin

If that is not possible, another Galaxy choice could be the solution:
http://wiki.galaxyproject.org/Support#About_Galaxy
http://wiki.galaxyproject.org/BigPicture/Choices

CloudMan Galaxy is a common pick. Why? AWS offer grants for research purposes (which may apply for project) and scaling it is on demand. That said, there are other cloud Galaxy options described here (Globus, etc.):
http://wiki.galaxyproject.org/Cloud

Hopefully one of these work out for you, Jen, Galaxy team

ADD COMMENT • link written 4.3 years ago by Jennifer Hillman Jackson ♦ 25k

Similar posts • Search »