Question: Hello Newbie Questions About Galaxy And Clustering
0
gravatar for leconte
5.7 years ago by
leconte20
leconte20 wrote:
hello I have installed an galaxy instance for purpose testing on a 8 CPU server without problem all run perfectly. I have made some galaxy xml wrapper to understant how galaxy calls program and other little things. Now I want to install a Galaxy server with a cluster configuration. Before I have tested HTCondor but I have no Idea how I can mix Galaxy and HTCondor. The perfect thing was somebody have already tested this type of condor. But I can satisfy myself with any other cluster type. What I don't understant in fact ( maybe my worst problem ) where lie all parts of this puzzle. I explain. I know that I need Galaxy. I know too I need a cluster ( in my case HTCondor ) I understand I need something between these 2 parts ... I think something like PBS or SGE but it's not totaly clear for me I'm search on galaxy mailling list archive ... but I just found 2 mails in 2008 :( has anybody made this conf or have a configuration ? greetings PS : sorry for my poor english :)
galaxy • 997 views
ADD COMMENTlink modified 5.6 years ago • written 5.7 years ago by leconte20
0
gravatar for leconte
5.6 years ago by
leconte20
leconte20 wrote:
hello, My subject is at the border between Galaxy and Condor I'found method to use Galaxy with condor cluster solution. In fact I have wrote a script when Galaxy call clustalw2 , it calls in fact a script called clustalw2. All run exactly as I want excepting ... but Galaxy get in error and said there was a problem I turn my script to debug mode and it puts followings output + prog=/usr/bin/clustalw2 + err=0 + infile=-infile=/home/galaxy/galaxy- dist/database/files/000/dataset_11.dat + outfile=-outfile=/home/galaxy/galaxy- dist/database/job_working_directory/000/46/galaxy_dataset_80.dat + outorder=-OUTORDER=ALIGNED + seqnos=-SEQNOS=OFF + typeseq=-TYPE=DNA + var6= + var7= + var8= + var9= + echo ++ echo -infile=/home/galaxy/galaxy- dist/database/files/000/dataset_11.dat ++ sed -e s/-infile=// + infile2=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat ++ sed -e 's/galaxy_dataset_\(.*\)\.dat//' ++ sed -e s/-outfile=// ++ echo -outfile=/home/galaxy/galaxy- dist/database/job_working_directory/000/46/galaxy_dataset_80.dat + WorkingDir=/home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + mkdir -p /home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + mkdir -p /home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + mkdir -p /home/galaxy/galaxy- dist/database/job_working_directory/000/46/ + [[ -n /usr/bin/clustalw2==clustalw2 ]] + echo 'universe = vanilla' + echo 'executable = /usr/bin/mpirun ' + echo 'arguments = "-np 10 -hostfile /home/galaxy/hostfile /usr/bin/clustalw-mpi -infile=/home/galaxy /galaxy-dist/database/files/000/dataset_11.dat -outfile=/home/galaxy /galaxy- dist/database/job_working_directory/000/46/galaxy_dataset_80.dat -OUTORDER=ALIGNED -SEQNOS=OFF -TYPE=DNA "' + echo 'Getenv = True machine_count = 1 transfer_input_files = /home/galaxy/hostfile,/usr/bin/clustalw-mpi, /home/galaxy/galaxy-dist/database/files/000/dataset_11.dat should_transfer_files = yes Log = /home/galaxy/logs/normal.$(cluster).$(process).log Output = /home/galaxy/logs/normal.$(cluster).$(process).out Error = /home/galaxy/logs/normal.$(cluster).$(process).error notification = never queue 1' + /usr/bin/condor_submit /home/galaxy/submit Submitting job(s). 1 job(s) submitted to cluster 179. I suspect the method I use, doesn't return properly information to galaxy ... but I don't know what I can do ... Any Idea ...
ADD COMMENTlink written 5.6 years ago by leconte20
Hello, I've moved this over to the galaxy-dev list since it concerns a local installation. You'll want to use the 'condor' job running plugin, which is provided with the Galaxy distribution. Have a look at the job_conf.xml.sample_advanced file to see how this would be configured. A very basic job_conf.xml that should submit all jobs to HTCondor: <job_conf> <plugins workers="4"> <plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner"/> </plugins> <destinations default="condor"> <destination id="condor" runner="condor"/> </destinations> </job_conf> --nate
ADD REPLYlink written 5.6 years ago by Nate Coraor3.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 182 users visited in the last hour