This job was terminated because it used more memory than it was allocated

Question: This job was terminated because it used more memory than it was allocated

3.9 years ago by

alt82 • 0

Germany

alt82 • 0 wrote:

Hello,

I have always encountered such problem "This job was terminated because it used more memory than it was allocated." since last month， no matter how the input(fastq file) is large.

Here is the parameters I set for TopHat2:

Is this library mate-paired?	single
RNA-Seq FASTQ file	13: Filter by quality on data 7 (22924873 reads)
Use a built in reference genome or own from your history	indexed
Select a reference genome	hg19
TopHat settings to use	full
Max realign edit distance	1000
Max edit distance	3
Library Type	FR Unstranded
Final read mismatches	2
Use bowtie -n mode	No
Anchor length (at least 3)	8
Maximum number of mismatches that can appear in the anchor region of spliced alignment	0
The minimum intron length	70
The maximum intron length	100000
Allow indel search	Yes
Max insertion length.	3
Max deletion length.	3
Maximum number of alignments to be allowed	20
Minimum intron length that may be found during split-segment (default) search	50
Maximum intron length that may be found during split-segment (default) search	100000
Number of mismatches allowed in each segment alignment for reads mapped independently	2
Minimum length of read segments	25
Use Own Junctions	Yes
Use Gene Annotation Model	Yes
Gene Model Annotations	19: UCSC Main on Human: ensGene (genome)
Use Raw Junctions	No
Only look for supplied junctions	No
Use Coverage Search	No
Use Microexon Search	No
Do Fusion Search	Yes
Anchor Length	13
Minimum Distance	100000
Read Mismatches	2
Multireads	2
Multipairs	2
Ignore some chromosomes such as chrM when detecting fusion break points	chrM
Set Bowtie2 settings	No
Specify read group?	no
Job Resource Parameters	no

Does anyone know how to solve this problem? Thanks a lot!

rna-seq tophat preferences accounts registration • 2.5k views

ADD COMMENT • link •

modified 3.9 years ago by Jennifer Hillman Jackson ♦ 25k • written 3.9 years ago by alt82 • 0

3.9 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

For others reading this post, the short solution involves these factors:

An error like this always indicates that a job exceeded the "processing" memory available on the Galaxy server you are working on
This is distinct from the user disk quota for your account
One re-run is recommended to eliminate any transient cluster issues. The re-run will often times be more successful if not started immediately - waiting 30 minutes is sufficient. (This is due to technical reasons that are beyond the scope of this reply).
Often input/parameter adjustments can help to achieve a successful run. Format or content issues in inputs are common, as are choosing certain 'compute intensive" options.
Each resolution can be different in the details. Review the 3rd party tool usage via the link in the help to better understand the tool and best practises for usage.
If the inputs are OK, and the jobs still remains to large, moving to a cloud Galaxy that is set up with more processing memory can be a great solution. AWS has a wonderful grant program than can help with costs. The set-up is more straightforward that ever, even for non-technical users. We can help if you get stuck at a step.

Take care! Jen, Galaxy team

ADD COMMENT • link written 3.9 years ago by Jennifer Hillman Jackson ♦ 25k

Thanks very much for your reply.

Now the same errors happen more frequently. I assume there may be some problems in the galaxy server.

Two months ago, TopHat2 just needed to run for less than 20 hours even when the input contains more than 100 million pair-end reads, under the same parameter settings as above I posted. However, now，even if the input contained 5-10 million single-end reads, the mapping will fail in most time. Does anyone else encounter the similar problem?

ADD REPLY • link written 3.9 years ago by alt82 • 0

Hello,

Thank you for the feedback. For the disparate job timing - my only guess is that the content of the inputs in the current set differ than the data used before (if all else is unchanged).

It was a bit difficult to understand if you were asking about a memory or timing issue. For timing and this tool, consider using the Stampede cluster option. For memory, the cloud is the option.

More help along these topics is now in our wiki:
https://wiki.galaxyproject.org/Support#Error_from_tools

If there is more from our sys admin/devteam, they will comment with more details ;)

Thanks! Jen, Galaxy team

ADD REPLY • link written 3.9 years ago by Jennifer Hillman Jackson ♦ 25k

3.9 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

We have been trying to reach you! Whenever you send in a bug report, the email provided results in a "bounced message". This has been true for all bug reports send in, not just this one - so we are most definitely not ignoring you, just couldn't reply to you with incredibly complex and insightful help! Joke ... :)

The reply I sent earlier this week for for this specific issue. I'd like to forward that to you via email, and reference the the bug report, if possible. All is confidential using this method, which is sometimes better - it depends.

It would be best to use a valid email address for your account (for bug reports and critical - but rare - communications from us). Using your actually galaxy valid account email is generally best for most users when submitting bug reports, too. Older accounts were not required to go through this account validation step, leading to this type of problem, but this is easily remedied. To update your account email at any time (whenever one changes their primary email, jobs, schools, etc), this can be done in the UI under "User -> Preferences -> Manage your information". Please be aware that you will need to confirm the address via an email we will send you.

This all said, some incoming mail can sometimes bounce due to the way the mail server is set up on your end. But since you are using a public server (I won't reveal which here publically), a bounce generally indicates that the email is doesn't not exist or has been retired. (Unlike an institutional email address - those can have other underlying causes that lead to bounces).

If you want to send an email again to galaxy-bugs, after correcting your email, please include a reference URL to this post so I can match all the communications together and send the best info.

Bit long, but this may help others, so seemed a good opportunity to share this info. I'll adjust the tags to reflect that this post has more info beyond the issue stated. Best, Jen, Galaxy team

ADD COMMENT • link modified 3.9 years ago • written 3.9 years ago by Jennifer Hillman Jackson ♦ 25k

Please log in to add an answer.

Similar posts • Search »