Question: Passing job_resource_params_conf options to queue
0
david.a.morais • 110 wrote:
Hi All,
I am trying to use the job_resource_params_conf options to allow users to set parameters for some queues (i.e. walltime, ppn) and sometimes select an already pre-configured queue.
The web interface is showing the options that I want to choose from, but after selecting the option when I execute the tool the job runs with default parameter.
I only modified the job_resource_params_conf.xml and job_config.xml, (besides the galaxy.ini) so I am not sure if any other file/script needs to be modified in order to pass these parameters to the jobs.
I am using torque pbs
Below is my job_resource_params_conf.xml and and the job_config.xml
<param label="walltime" name="walltime" type="integer" size="3" min="1" max="120" value="" help="Maximum job time in hours, 'walltime' value (1-120). Leave blank to use default value."/>
<param label="Queue" name="queue" type="select" help="Modify this option if you need more walltime, cores or memory">
<option value="qwork">qwork 24 cores 32GB RAM walltime 72 Hours (default)</option>
<option value="qtest3_short">qtest3_short, 24 cores, 32 RAM, walltime 15 min (short jobs. Lowest waiting time)</option>
<option value="qwork_extended">qwork_extended 24 cores 32GB RAM walltime 120 Hours (CPU-time intensive)</option>
<option value="qfat256">qfat256, 48 cores, 256 RAM, walltime 120 Hours (High memory requirement)</option>
<option value="qfat512">qfat512, 48 cores, 512 RAM, walltime 48 Hours (Highest memory requirement. Limited walltime.)</option>
</param>
</parameters>
and job_config.xml
<?xml version="1.0"?>
<job_conf>
<plugins>
<plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner" worker="10"/>
<plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
</plugins>
<handlers default="handlers">
<handler id="handler0" tags="handlers"/>
<handler id="handler1" tags="handlers"/>
<handler id="handler2" tags="handlers"/>
</handlers>
<destinations default="qtest3">
<destination id="qtest3" runner="pbs" >
<param id="Resource_List">walltime=02:15:00</param>
<param id="destination">qtest3@localhost</param>
</destination>
<destination id="qtest3_short" runner="pbs" >
<param id="destination">qtest3@localhost</param>
<param id="Resource_List">walltime=00:15:00</param>
</destination>
<destination id="qtest" runner="pbs" >
<param id="Resource_List">walltime=00:15:00</param>
<param id="destination">qtest</param>
</destination>
<destination id="qwork" runner="pbs" >
<param id="Resource_List">walltime=120:00:00</param>
<param id="destination">qwork</param>
</destination>
<destination id="qfat256" runner="pbs" >
<param id="Resource_List">walltime=120:00:00</param>
<param id="destination">qfat256</param>
</destination>
<destination id="qfat512" runner="pbs" >
<param id="Resource_List">walltime=48:00:00</param>
<param id="destination">qfat512</param>
</destination>
<destination id="local" runner="local"/>
</destinations>
<tools>
<!-- Upload tools will run locally by default -->
<tool id="toolshed.g2.bx.psu.edu/repos/devteam/fastqtofasta/3571553aeb20/fastqtofasta" destination="qtest3_short" resources="walltime"/>
<tool id="toolshed.g2.bx.psu.edu/repos/geert-vandeweyer/sendmail/sendemail/0.0.1" destination="qtest3_short" resources="walltime"/>
<tool id="toolshed.g2.bx.psu.edu/repos/pjbriggs/trimmomatic/a60283899c6d/trimmomatic" destination="qtest3_short" resources="all"/>
</tools>
<resources default="default">
<!-- Group different parameters defined in job_resource_params_conf.xml
together and assign these groups ids. Tool section below can map
tools to different groups. This is experimental functionality!
-->
<group id="default">walltime</group>
<group id="walltime">walltime</group>
<group id="all">queue,walltime</group>
</resources>
</job_conf>
ADD COMMENT
• link
•
modified 2.9 years ago
by
jmchilton ♦ 1.1k
•
written
3.0 years ago by
david.a.morais • 110
Hello,
This seems worth a check: What is the setting of 'job_config_file' in your galaxy.ini?
Galaxy is not loading job_conf.xml ?
Hi Jennifer,
Galaxy is loading the job_conf.xml and the job_resource_params_conf.xml as well.
What was happening is that when a user was choosing a 'Job Resource Parameters' from the interface these parameters where not passed to the scheduler, instead the job was sent to default queue.
After digging on the internet and look at several ansible playbooks (unfortunately the documentation of job_resource_params_conf is not very clear) I realized that I had to write a destination.py under '../rules/' and create a dynamic destination which receives these 'Job Resource Parameters' and send them to the scheduler.
Now everything is working fine.
If ever someone has the same problem I will be more than glad to help those person to configure the job_resource_params_conf.xml , job_config.xml and to create a dynamic rule.
Thanks a lot
David Morais