Question: Making Galaxy Tools run on multiple cores/treads?
gravatar for vebaev
14 months ago by
vebaev130 wrote:

Dear all,

I have been trying for some days to make Galaxy run tools on more than 1 core (e.g. Bowtie to use 8 or more cores on my machine with 48 cores) by modifying and activating the file jpb_conf.xml but without any luck.

I will appreciate if anyone already done it to point me to a tutorial of this method or to the other method on installing a job manager which is also I think out of mine knowledge?

cores cpu galaxy local • 473 views
ADD COMMENTlink modified 14 months ago • written 14 months ago by vebaev130

I guess you deleted your comment. Anyway, bowtie-build is single-threaded. For the others, make sure you restart Galaxy, since the job_conf.xml file is only parsed when Galaxy starts.

ADD REPLYlink written 14 months ago by Devon Ryan1.9k

Yes , I couldnt paste my file config correctly as typing from my iPad. I restarted it. Let me see if I can paste it now....

<plugin id="local" type="runner" load="" workers="8"/>
<plugin id="multilocal" type="runner" load="" workers="2"/>
<handler id="main"/>.

<destinations default="local">
<destination id="local" runner="local"/>
<destination id="multicore8" runner="multilocal">
<param id="local_slots">8</param>

<tool id="bowtie2" destination="multicore8"/>
<tool id="bowtie" destination="multicore8"/>
<tool id="bowtie-build" destination="multicore8"/>
<tool id="fastq_groomer_parallel" destination="multicore8"/>
<tool id="deeptools" destination="multicore8"/>
<tool id="cufflinks" destination="multicore8"/>
<tool id="cuffdiff" destination="multicore8"/>
<tool id="cuffmerge" destination="multicore8"/>
<tool id="tophat2" destination="multicore8"/>

ADD REPLYlink written 14 months ago by vebaev130

I don't see anything obviously wrong with that, just make sure you restart galaxy.

ADD REPLYlink written 14 months ago by Devon Ryan1.9k
gravatar for Devon Ryan
14 months ago by
Devon Ryan1.9k
Devon Ryan1.9k wrote:

Make sure you're modifying job_conf.xml rather than jpb_conf.xml. Below is the relevant section of my job_conf.xml for running bowtie2:

 <destinations default="slurm">
    <destination id="slurm10threads" runner="slurm">
        <param id="request_cpus">10</param>
        <param id="embed_metadata_in_job">False</param>
        <param id="nativeSpecification">-p Galaxy -n 10</param>
        <env file="/data/galaxy/galaxy-central/galaxy_venv/bin/activate" />
    <tool id="bowtie2" destination="slurm10threads"/>

If you're using the "local" job runner, then you'll need something like:

    <destination id="local" runner="local">
        <param id="local_slots">4</param>

And then either set destination to local or just make that the default.

ADD COMMENTlink written 14 months ago by Devon Ryan1.9k
Please log in to add an answer.


Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 126 users visited in the last hour