Error / Nebula And Repeatmasker

Question: Error / Nebula And Repeatmasker

5.8 years ago by

Sarah Maman • 200 wrote:

Hello, I've installed NEBULA tools and "RepeatMasker" tool on our local Galaxy instance. For some tools (others work), I get the following errors: * - *Here is the error I get when running the tool *FilterControl * *** glibc detected *** java: double free or corruption (!prev): 0x00007fe56800ecd0 *** *** glibc detected *** java: double free or corruption (!prev): 0x00007fe56800ecd0 *** *** glibc detected *** java: double free or corruption (!prev): 0x00007fe56800ecd0 *** *** glibc detected *** java: double free or corruption (!prev): 0x00007fe56800ecd0 *** /usr/local/bioinfo/src/galaxy/galaxy- dist/tools/Nebula/FilterControlPeaks/FilterControlPeaks.sh: line 80: 61310 Abandon java -classpath $LOCAL_PATH/ -Xmx6g FilterPeaks -f $inputfile -c $controlfile -t $minHeight -v $minRatio -o $output > /dev/null *** glibc detected *** java: double free or corruption (!prev): 0x00007f1f5800ecd0 *** /usr/local/bioinfo/src/galaxy/galaxy- dist/tools/Nebula/FilterControlPeaks/FilterControlPeaks.sh: line 81: 61351 Abandon - Here is the error I get when running the tool *IntersectBed *//work/galaxy/database/pbs/galaxy_4129.sh: line 13: Erreur de syntaxe prčs du symbole inattendu Ť ;; ť /work/galaxy/database/pbs/galaxy_4129.sh: line 13: `bedtools intersect -f 0.05/ - Here is the error I get when running the tool *ChIPmunk (thanks Alban for you help (*ChIPMunk v2 is used and all librairies are found) *but here is a new error, maybe a problem of "dirname" ?) */mv: impossible d'évaluer Ť /work/galaxy/database/job_working_directory/004/4132/galaxy_dataset _5992.dat_0.xml.png ť: Aucun fichier ou dossier de ce type mv: impossible d'évaluer Ť /work/gala /- Here is the error I get when running the tool *PeaksToBed * readline() on closed filehandle FILE at /usr/local/bioinfo/src/galaxy/galaxy- dist/tools/Nebula/Peak2Bed/peak2bed.pl line 124 (#1) (W closed) The filehandle you're reading from got itself closed sometime before now. Check your control flow. Use of uninitialized value $fields[1] in concatenation (.) or string at /usr/local/bioinfo/src/galaxy/galaxy- dist/tools/Nebula/Peak2Bed/peak2bed.pl line 154, <file> line 1 (#2) (W uninitialized) An undefined value was used as if it were already defined. It was interpreted as a "" or a 0, but maybe it was a mistake. To suppress this warning assign a defined value to your variables. To help you figure out what was undefined, perl will try to tell you the name of the variable (if any) that was undefined. In some cases it cannot do this, so it also tells you what operation you used the undefined value in. Note, however, that perl optimizes your program and the operation displayed in the warning may not necessarily appear literally in your program. For example, "that $foo" is usually optimized into "that " . $foo, and the warning will refer to the concatenation (.) operator, even though there is no . in your program. Use of uninitialized value in concatenation (.) or string at /usr/local/bioinfo/src/galaxy/galaxy- dist/tools/Nebula/Peak2Bed/peak2bed.pl line 154, <file> line 1 (#2) Use of uninitialized value in numeric ge (>=) at /usr/local/bioinfo/src/galaxy/galaxy- dist/tools/Nebula/Peak2Bed/peak2bed.pl line 156, <file> line 1 (#2) ...... /usr/local/bioinfo/src/galaxy/galaxy- dist/tools/Nebula/Peak2Bed/peak2bed.pl line 156, <file> line 63 (#2) Use of uninitialized value $fields[1] in conc ** I dowloaded tool "RepeatMasker" from Galaxy Tool Shed,and RepeatMasker is installed on our cluster. Here is the error I get when running the tool RepeatMasker: An error occurred running this job: /Epilog : job finished at jeu. janv. 31 14:39:16 CET 2013 /work/galaxy/database/pbs/galaxy_4134.sh: line 13: Erreur de syntaxe prčs du symbole inattendu Ť ;; ť /work/galaxy/database/pbs/galaxy_4134.sh: line 13: `RepeatMasker -parallel 8 -speci/ Do you have any ideas ? Thank you in advance for your help, Sarah Maman -- --*-- Sarah Maman INRA - LGC - SIGENAE http://www.sigenae.org/ Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 --*--

bedtools • 1.4k views

ADD COMMENT • link •

modified 5.8 years ago by Björn Grüning • 350 • written 5.8 years ago by Sarah Maman • 200

5.8 years ago by

Björn Grüning • 350

Björn Grüning • 350 wrote:

Hi Sarah, i looks like something is wrong with the PBS system. The errors for bedtools and repeatmasker are the same. I can't understand french but i guess that ";" are not supported by PBS or in your configuration? Repeatmasker and bedtools creating more than one shell command and concatenating them with ';'. Hope that helps, Bjoern -- Björn Grüning Albert-Ludwigs-Universität Freiburg Institute of Pharmaceutical Sciences Pharmaceutical Bioinformatics Hermann-Herder-Strasse 9 D-79104 Freiburg i. Br. Tel.: +49 761 203-4872 Fax.: +49 761 203-97769 E-Mail: bjoern.gruening@pharmazie.uni-freiburg.de Web: http://www.pharmaceutical-bioinformatics.org/

ADD COMMENT • link written 5.8 years ago by Björn Grüning • 350

Thanks Bjoern, In your xml file, I 've specified the interpreter : replace by but the problem is not solved (";" is supported by our cluster) because the command is runned with an error ";;" (double ;) instead of ";" (simple ;) : `bash /my/path/to/RepeatMasker -parallel 8 -species fungi -dir /work/tmp/tmprRrJh2 -gff /my/path/to/galaxy/database/files/005/dataset_5979.dat > /dev/null 2> /dev/null; cp /my/path/to/work/tmp/tmprRrJh2/dataset_5979.dat.tbl /my/path/to/gal axy/database/job_working_directory/004/4135/galaxy_dataset_5999.dat; cp /work/tmp/tmprRrJh2/dataset_5979.dat.out.gff None; cp /work/tmp/tmprRrJh2/dataset_5979.dat.masked None; tail -n +4 /work/tmp/tmprRrJh2/dataset_5979.dat.out | tr -s ' ' '\t' > /my/path/to/galaxy/database/job_working_directory/004/4135/galaxy_data set_5998.dat; *rm /work/tmp/tmprRrJh2 -r;; * cd /my/path/to/galaxy/galaxy-dist; /my/path/to/galaxy/galaxy-dist/set_metadata.sh /my/path/to/galaxy/database/files /my/path/to/galaxy/database/job_working_directory/004/4135 . /my/path/to/galaxy/galaxy-dist/universe_wsgi.ini /work/tmp/tmpi0u8TZ /my/path/to/galaxy/database/job_working_directory/004/4135/galaxy.json /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_in _HistoryDatasetAssociation_6156_Lp_2X3, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_kw ds_HistoryDatasetAssociation_6156_emJ62i, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_ou t_HistoryDatasetAssociation_6156_Dgw3la, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_re sults_HistoryDatasetAssociation_6156_aTCjOU, /my/path/to/galaxy/database/job_working_directory/004/4135/galaxy_data set_5999.dat, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_ov erride_HistoryDatasetAssociation_6156_I6enyB /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_in _HistoryDatasetAssociation_6155_LRm2gT, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_kw ds_HistoryDatasetAssociation_6155_4xAztH, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_ou t_HistoryDatasetAssociation_6155_14AYvT, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_re sults_HistoryDatasetAssociation_6155_l_1gJ2, /my/path/to/galaxy/database/job_working_directory/004/4135/galaxy_data set_5998.dat, /my/path/to/galaxy/database/job_working_directory/004/4135/metadata_ov erride_HistoryDatasetAssociation_6155_NufEns' So the error is ";;" rm /work/tmp/tmprRrJh2 -r;; instead of ";" rm /work/tmp/tmprRrJh2 *-r;* I've tried to replace ";" by nothing but this generate others errors (files not generated) : cp: impossible d'évaluer Ť /work/tmp/tmpga_QOo/dataset_5979.dat.tbl ť: Aucun fichier ou dossier de ce type cp: impossible d'évaluer Ť /work/tmp/tmpga_QOo/dataset_5979.dat.out.gff ť: Aucun fichier ou dossier de ce type cp: impossible d'évaluer Ť /work/tmp/tmpga_QOo/dataset_5979.dat.masked ť: Aucun fichier ou dossier de ce type tail: impossible d'ouvrir Ť /work/tmp/tmpga_QOo/dataset_5979.dat.out ť en lecture: Aucun fichier ou dossier de ce type Do you have any idea ? Thanks in advance, Sarah Björn Grüning a écrit : -- --*-- Sarah Maman INRA - LGC - SIGENAE http://www.sigenae.org/ Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 --*--

ADD REPLY • link written 5.8 years ago by Sarah Maman • 200

Hi Sarah, I'll try to debug, point per point: I think here you have a misconfiguration of the memory of your java install (according to the need of the tool) If you look at the FilterControlPeaks.sh file, the java is called with the option -Xmx6g. So your java install have to be allowed to use 6G as memory (by default it's 1024M) I don't understand this one, The intersectBed tool is only composed of a xml which simply call bedtools.. Check the command by typing 'bedtools intersect' in a terminal, if it sends you back the option list, the Galaxy tool should work.. If not, you have to check your bedtools install Here, I think that the error is due to the ChIPMunk execution which doesn't work. You can try the ChIPMunk execution this way: Find the line containing: ruby $CHIPMUNK_BIN_DIR/run_chiphorde.rb $LOG $motifs $MODE yes 1.0 s:$inputfile 100 10 1 4 >> /dev/null 2>> /dev/null Add a line with: echo "ruby $CHIPMUNK_BIN_DIR/run_chiphorde.rb $LOG $motifs $MODE yes 1.0 s:$inputfile 100 10 1 4 >> /dev/null 2>> /dev/null" > ~/ChIPMunk.log Then try to launch the command in ~/ChIPMunk.log in a terminal and look at the output. If ChIPMunk has run, the error comes from next step (i.e. the parseChipmunkOutput.pl script). Else, try to debug ChIPMunk, and then the tool will work. For this one, edit the PeaksToBed.pl script this way: The 3 lines: use strict; use warnings; use diagnostics; becomes: #use strict; #use warnings; #use diagnostics; Galaxy doesn't really love to have the return of a script (even if it's not an error). Here the perl scripts gives you warnings that are understood as errors by Galaxy. Hope It will help you, don't hesitate if you need more.. ++, Alban -- Alban Lermine Unité 900: INSERM - Mines ParisTech - Institut Curie " Bioinformatics and Computational Systems Biology of Cancer" 11-13 rue Pierre et Marie Curie (1er étage) - 75005 Paris - France Tel: +33 (0) 1 56 24 69 84 Le 1 févr. 2013 ŕ 14:41, Sarah Maman <sarah.maman@toulouse.inra.fr> a écrit :

ADD REPLY • link written 5.8 years ago by alermine • 40

Hi Alban, Marie-Stephane and Bjoern, * _*For IntersectBed tool,*_ running is OK. I've just deleted ";" in xml file and your tool runs. My cluster add one ";" so a double ";;" gives an error. I thing that this new eroor is due to files tested : Differing number of VCF fields encountered at line: 38. Exiting... *_*For RepeatMasker tool*_, thanks to Marie-Stephane (CONGRATULATIONS MARIE-STEPHANE !), we need to specify that bash commands are included iwith Python. So xml file have been modify in this way : - Specify "perl" before RepeatMasker command. - Specify RepeatMasker path - Include all bash comman in a Python script : #os.system("cp $gff_file $output_gff;") instead of cp $gff_file $output_gff; So the code is : ## The command is a Cheetah template which allows some Python based syntax. ## Lines starting hash hash are comments. Galaxy will turn newlines into spaces ## create temp directory #import tempfile, os #set $dirname = os.path.abspath(tempfile.mkdtemp()) #set $input_filename = os.path.split(str($query))[-1] #set $output_basename = os.path.join($dirname, $input_filename) //*perl /usr/local/bioinfo/bin/RepeatMasker*// -parallel 8 $nolow $noint $norna #if str($species)!="all": $species #end if -dir $dirname #if $adv_opts.adv_opts_selector=="advanced": #if str($adv_opts.gc)!="0": -gc $adv_opts.gc #end if $adv_opts.gccalc #set $output_files_list = str($adv_opts.output_files).split(',') #if "gff" in $output_files_list: -gff #end if #if "html" in $output_files_list: -html #end if $adv_opts.slow_search $adv_opts.quick_search $adv_opts.rush_search $adv_opts.only_alus $adv_opts.is_only #else: ## Set defaults -gff ## End of advanced options: #end if $query ///dev/null 2> /dev/null; / /## Copy the output files to galaxy #if $adv_opts.adv_opts_selector=="advanced": #if "summary" in $output_files_list: ## Write out the summary file (default) #set $summary_file = $output_basename + '.tbl' #os.system("cp $summary_file $output_summary;") #end if #if "gff" in $output_files_list: ## Write out the gff file (default) #set $gff_file = $output_basename + '.out.gff' #os.system("cp $gff_file $output_gff"); #end if #if "html" in $output_files_list: ## Write out the html file #set $html_file = $output_basename + '.out.html' #os.system("cp $html_file $output_html;") #end if #else: ## Write out the summary file (default) #set $summary_file = $output_basename + '.tbl' *#os.system("cp $summary_file $output_summary;") * ## Write out the gff file (default) #set $gff_file = $output_basename + '.out.gff' #os.system("cp $gff_file $output_gff;") ## End of advanced options: #end if ## Write out mask sequence file #set $mask_sequence_file = $output_basename + '.masked' #os.system("cp $mask_sequence_file $output_mask;") ## Write out standard file (default) ## The default '.out' file from RepeatMasker has a 3-line header and spaces rather ## than tabs. Remove the header and replace the whitespaces with tab #set $standard_file = $output_basename + '.out' #os.system("tail -n +4 $standard_file | tr -s ' ' '\t' > $output_std;") ## Delete all temporary files #os.system("rm $dirname -r;" / * _*For the tool FilterControl*_, our cluster is configured to kill jobs that use more than 4Go of memory. I don't achieve to modify qsub options in my Galaxy instance, so I'he changed this option -Xmx6g in "-Xmx4g". Maybe some treatment won't run by lake of memory .. If you have any idea on how to add option on Galaxy qsub, could you please help me ? I would like to add these options to qsub : qsub -l mem=6G -l h_vmem=8G * _*ChIPMunk*_ :Sorry, but all ChIPMunk files (xml, pl; sh) didn't have execution rights... So I just do "chmod a+x" on these files and ChIPMunk tool is OK in my Galaxy instance.... Thanks a lot for all your explanations, Alban, Marie-Stephane and Bjoern . Thanks in advance for qsub and IntersectBed tool (test file), Sarah alermine a écrit : -- --*-- Sarah Maman INRA - LGC - SIGENAE http://www.sigenae.org/ Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 --*--

ADD REPLY • link written 5.8 years ago by Sarah Maman • 200

Hi all, The memory that you specify with -Xmx6g is the memory used by Java and NOT the memory used for the job launched by Galaxy on the cluster (see http://www.auditmypc.com/java-memory-xmx512.asp). So you have to specify in your java configuration the min and max memory usable. Try by specifying in universe_wsgi.ini runner:///queue/-l nodes=1:ppn=1,mem=6gb,h_vmem=8gb/ If it's not understood by Galaxy,To add new qsub options, you have to make some modifications in the source code: 1. Identify your scheduler (pbs, drmaa, sge) 2. Edit the python script which creates jobs for your scheduler: GALAXY_INSTALL_DIR/galaxy-dist/lib/galaxy/jobs/runners/pbs.py GALAXY_INSTALL_DIR/galaxy- dist/lib/galaxy/jobs/runners/drmaa.py GALAXY_INSTALL_DIR/galaxy-dist/lib/galaxy/jobs/runners/sge.py 3. Search in the script the function which parse your scheduler options (ex: for pbs.py, the function is named "def determine_pbs_options( self, url ):") 4. Modify the parsing step to make this function understand the h_vmem option Hope it's help you, ++, Alban -- Alban Lermine Unité 900: INSERM - Mines ParisTech - Institut Curie " Bioinformatics and Computational Systems Biology of Cancer" 11-13 rue Pierre et Marie Curie (1er étage) - 75005 Paris - France Tel: +33 (0) 1 56 24 69 84 Le 5 févr. 2013 ŕ 14:53, Sarah Maman <sarah.maman@toulouse.inra.fr> a écrit :

ADD REPLY • link written 5.8 years ago by alermine • 40

Thanks a lot Alban for these explanations , Everything is OK. ++, Sarah alermine a écrit : -- --*-- Sarah Maman INRA - LGC - SIGENAE http://www.sigenae.org/ Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 --*--

ADD REPLY • link written 5.8 years ago by Sarah Maman • 200

Hi Alban, After a few weeks on other work, I try again the integration of your suite Nebula in my local instance of Galaxy. If possible, could you please help me to understand why the script "parseChipmunkOutput.pl " doesn't found results in my Galaxy instance. Here are the steps performed: _**1* Command launched with sucess in a terminal :*_ ruby /usr/local/bioinfo/src/galaxy/galaxy-dist/tool- data/shared/jars/ChIPMunk/run_chiphorde.rb toto.log 3:10,2:10 mask yes 1.0 s:masked_sequence s_for_CompleteMotifs.fasta 100 10 1 4 Ouput fles : -rw-r--r-- 1 galaxy wbioinfo 256K 27 févr. 09:59 masked_sequences_for_CompleteMotifs.fasta -rw-r--r-- 1 galaxy wbioinfo 61K 27 févr. 10:03 toto.log_chiphorde.log -rw-r--r-- 1 galaxy wbioinfo 3,5K 27 févr. 10:03 toto.log_0.xml -rw-r--r-- 1 galaxy wbioinfo 82K 27 févr. 10:03 toto.log_0.xml.png -rw-r--r-- 1 galaxy wbioinfo 59K 27 févr. 10:03 toto.log_0_occurrences.txt So ChIPMunk work. _* *2*Command launched via Galaxy :*_ /Ť /work/galaxy/database/job_working_directory/004/4397/galaxy_datase t_6397.dat_0.xml.png ť: Aucun fichier ou dossier de ce type/ So the error comes from next step (i.e. the parseChipmunkOutput.pl script). Do you have any idea ? Thanks in advance, Sarah Here is your last message :

ADD REPLY • link written 5.7 years ago by Sarah Maman • 200

Similar posts • Search »