Question: This job was terminated because it used more memory than it was allocated
0
gravatar for alt82
3.9 years ago by
alt820
Germany
alt820 wrote:

Hello,

I have always encountered such problem "This job was terminated because it used more memory than it was allocated." since last month, no matter how the input(fastq file) is large.

Here is the parameters I set for TopHat2:

Is this library mate-paired? single  
RNA-Seq FASTQ file 13: Filter by quality on data 7 (22924873 reads)  
Use a built in reference genome or own from your history indexed  
Select a reference genome hg19  
TopHat settings to use full  
Max realign edit distance 1000  
Max edit distance 3  
Library Type FR Unstranded  
Final read mismatches 2  
Use bowtie -n mode No  
Anchor length (at least 3) 8  
Maximum number of mismatches that can appear in the anchor region of spliced alignment 0  
The minimum intron length 70  
The maximum intron length 100000  
Allow indel search Yes  
Max insertion length. 3  
Max deletion length. 3  
Maximum number of alignments to be allowed 20  
Minimum intron length that may be found during split-segment (default) search 50  
Maximum intron length that may be found during split-segment (default) search 100000  
Number of mismatches allowed in each segment alignment for reads mapped independently 2  
Minimum length of read segments 25  
Use Own Junctions Yes  
Use Gene Annotation Model Yes  
Gene Model Annotations 19: UCSC Main on Human: ensGene (genome)  
Use Raw Junctions No  
Only look for supplied junctions No  
Use Coverage Search No  
Use Microexon Search No  
Do Fusion Search Yes  
Anchor Length 13  
Minimum Distance 100000  
Read Mismatches 2  
Multireads 2  
Multipairs 2  
Ignore some chromosomes such as chrM when detecting fusion break points chrM  
Set Bowtie2 settings No  
Specify read group? no  
Job Resource Parameters no

Does anyone know how to solve this problem? Thanks a lot! 

ADD COMMENTlink modified 3.9 years ago by Jennifer Hillman Jackson25k • written 3.9 years ago by alt820
1
gravatar for Jennifer Hillman Jackson
3.9 years ago by
United States
Jennifer Hillman Jackson25k wrote:

For others reading this post, the short solution involves these factors:

  1. An error like this always indicates that a job exceeded the "processing" memory available on the Galaxy server you are working on
  2. This is distinct from the user disk quota for your account
  3. One re-run is recommended to eliminate any transient cluster issues. The re-run will often times be more successful if not started immediately - waiting 30 minutes is sufficient. (This is due to technical reasons that are beyond the scope of this reply).
  4. Often input/parameter adjustments can help to achieve a successful run. Format or content issues in inputs are common, as are choosing certain 'compute intensive" options. 
  5. Each resolution can be different in the details. Review the 3rd party tool usage via the link in the help to better understand the tool and best practises for usage.
  6. If the inputs are OK, and the jobs still remains to large, moving to a cloud Galaxy that is set up with more processing memory can be a great solution. AWS has a wonderful grant program than can help with costs. The set-up is more straightforward that ever, even for non-technical users. We can help if you get stuck at a step.

Take care! Jen, Galaxy team

ADD COMMENTlink written 3.9 years ago by Jennifer Hillman Jackson25k

Thanks very much for your reply.

Now the same errors happen more frequently. I assume there may be some problems in the galaxy server.

Two months ago, TopHat2 just needed to run for less than 20 hours even when the input contains more than 100 million pair-end reads, under the same parameter settings as above I posted. However, now,even if the input contained 5-10 million single-end reads, the mapping will fail in most time. Does anyone else encounter the similar problem?     

ADD REPLYlink written 3.9 years ago by alt820

Hello,

Thank you for the feedback. For the disparate job timing - my only guess is that the content of the inputs in the current set differ than the data used before (if all else is unchanged). 

It was a bit difficult to understand if you were asking about a memory or timing issue. For timing and this tool, consider using the Stampede cluster option. For memory, the cloud is the option.

More help along these topics is now in our wiki:
https://wiki.galaxyproject.org/Support#Error_from_tools

If there is more from our sys admin/devteam, they will comment with more details ;)

Thanks! Jen, Galaxy team

ADD REPLYlink written 3.9 years ago by Jennifer Hillman Jackson25k
0
gravatar for Jennifer Hillman Jackson
3.9 years ago by
United States
Jennifer Hillman Jackson25k wrote:

Hello,

We have been trying to reach you! Whenever you send in a bug report, the email provided results in a "bounced message". This has been true for all bug reports send in, not just this one - so we are most definitely not ignoring you, just couldn't reply to you with incredibly complex and insightful help! Joke ... :) 

The reply I sent earlier this week for for this specific issue. I'd like to forward that to you via email, and reference the the bug report, if possible. All is confidential using this method, which is sometimes better - it depends.

It would be best to use a valid email address for your account (for bug reports and critical - but rare - communications from us). Using your actually galaxy valid account email is generally best for most users when submitting bug reports, too. Older accounts were not required to go through this account validation step, leading to this type of problem, but this is easily remedied. To update your account email at any time (whenever one changes their primary email, jobs, schools, etc), this can be done in the UI under "User -> Preferences -> Manage your information". Please be aware that you will need to confirm the address via an email we will send you.

This all said, some incoming mail can sometimes bounce due to the way the mail server is set up on your end. But since you are using a public server (I won't reveal which here publically), a bounce generally indicates that the email is doesn't not exist or has been retired. (Unlike an institutional email address - those can have other underlying causes that lead to bounces).

If you want to send an email again to galaxy-bugs, after correcting your email, please include a reference URL to this post so I can match all the communications together and send the best info.

Bit long, but this may help others, so seemed a good opportunity to share this info. I'll adjust the tags to reflect that this post has more info beyond the issue stated. Best, Jen, Galaxy team

ADD COMMENTlink modified 3.9 years ago • written 3.9 years ago by Jennifer Hillman Jackson25k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 104 users visited in the last hour