Question: computeMatrix skipping bug is causing a problem with large data sets
0
Dietz.Matthew • 0 wrote:
When running computeMatrix with a large set of data I keep receiving this error multiple times each run:
Skipping (gene), due to being absent in the computeMatrix output.
This is a problem as I am using the resulting matrix to make heatmaps outside of galaxy, but it's impossible to line the genes up with their values because several hundred-thousand genes are being left out. Is anyone else getting this error or know how to fix it? Even just figuring out how to get the full list of genes left out would be fine, as I could line up the ones that remain, but currently galaxy is only giving me four at a time in the info box. Thanks for all the help in advance,
-Matt
ADD COMMENT
• link
•
modified 4 months ago
by
Jennifer Hillman Jackson ♦ 25k
•
written
4 months ago by
Dietz.Matthew • 0
Hi Matt, Could you explain in more detail what steps are leading up to this problem? I'm not sure what "genes being left out" means or how you are preparing the data for this tool.
It would be helpful to include exact tool names (including version) and the copied contents of the Job Details tool run settings (don't post back the job API links, or your data will not remain private). Or you can post back a shared history link (would be public) or send in a bug report from the error dataset (private). If you choose to send in the bug report, be sure to leave all input/output datasets undeleted and include a link to this post so we can link the two.
How-to and troubleshooting FAQs: https://galaxyproject.org/support/
Thanks, Jen, Galaxy team
I'm using the computeMatrix tool in NGS: DeepTools, version 2.5.0.0 and by "genes being left out", I mean that the tool is deciding, for one reason or another, to skip some of the genes and leave them out of the computeMatrix output. The settings are as shown:
computeMatrix has two main output options reference-point
The reference point for the plotting beginning of region (e.g. TSS)
Discard any values after the region end False
Distance upstream of the start site of the regions defined in the region file 1000
Distance downstream of the end site of the given regions 1000
Show advanced output settings yes
Save the matrix of values underlying the heatmap True
Save the regions after skipping zeros or min/max threshold values False
Show advanced options yes
Length, in bases, of non-overlapping bins used for averaging the score over the regions length 50
Sort regions maintain the same ordering as the input files
Method used for sorting mean
Define the type of statistic that should be displayed. mean
Convert missing values to 0? True
Skip zeros False
Minimum threshold Not available.
Maximum threshold Not available.
Scaling factor Not available.
Use a metagene model False
trascript designator transcript
exon designator exon
transcriptID key designator transcript_id
Blacklisted regions in BED/GTF format
Job Resource Parameters no
The full message in the info box is as shown:
Skipping uc009skg.1, due to being absent in the computeMatrix output. Skipping uc009skh.1, due to being absent in the computeMatrix output. Skipping uc029xhh.1, due to being absent in the computeMatrix output. Skipping uc029xhi.1, due to being absent in
Either fixing the skipping issue or even just getting a complete list of the skipped ones would effectively fix the problem I'm having. I'm fairly certain the problem isn't with how the data is prepared, as I have run this through with different sets and with smaller sets and it was fine before. Thanks for looking into this, and feel free to ask if you need any more clarification. When I get time I'll see if I can't figure it out myself.
My initial guess is that the skipped transcripts (genes) map to places not represented in the bigwig score data. The bigWig was created from a BAM that was mapped to the mm10 primary autosomes + chrX, chrY, and chrM (an uploaded BAM, but the BAM headers are a match for mm10, so I don't think there is a genome mismatch problem).
The mm10 UCSC genes track includes transcripts that map to haplotypes and unmapped (the full genome). These are probably what is being skipped.
I'll be checking for that (there are not that many skipped so this makes sense) -- but also please continue to check/troubleshoot your way & considering this info.