Latest
Open
RNA-Seq
ChIP-Seq
SNP
Assembly
Forum
Planet
All »
Home
Log In
Question: Help configuring dynamic resource allocation
0
gravatar for jdv
2.5 years ago by
jdv • 20
jdv • 20 wrote:

I'm trying (for the first time) to set up dynamic resource allocation for some jobs on a local Galaxy instance. I've read the docs here and can get a toy example working, but I know very little Python and don't fully understand how to access the job parameters from within the rules script. The tool is the phyml wrapper - there's a pruned-down snippet here. My rules file looks like this:

from galaxy.jobs import JobDestination
import os

def phyml(job):
    # Allocate extra time/cores to bootstrap jobs
    #support = job.parameters.???? - how to access branchSupport value?
    #if support > 0:
        #param_str = "-l walltime=168:00:00 -l nodes=1:ppn=60 -q batch"
    #else:
    param_str = "-l walltime=24:00:00 -l nodes=1:ppn=1 -q batch"
    return JobDestination(runner="torque", params={"nativeSpecification": param_str})

This works but obviously doesn't do any actual decision-making as is, and even though I'm sure it's very basic python I can't figure out the correct syntax to access the "branchSupport" value from the job. I'd greatly appreciate any help.

Thanks, Jeremy
galaxy python • 694 views
ADD COMMENT • link •
modified 2.5 years ago • written 2.5 years ago by jdv • 20
1
gravatar for jdv
2.5 years ago by
jdv • 20
jdv • 20 wrote:

For the record, I came up with the following working solution:

from galaxy.jobs import JobDestination
import os
import ast 

def phyml(job):

    # Allocate extra time/cores to bootstrap jobs
    params  = dict( [ p.name, p.value) for p in job.parameters ] ) 
    support = ast.literal_eval(params['support_condition'])['branchSupport']
    if int(support) > 0:
        param_str = "-l walltime=168:00:00 -l nodes=1:ppn=60 -q batch"
    else:
        param_str = "-l walltime=168:00:00 -l nodes=1:ppn=1 -q batch"
    return JobDestination(runner="torque", params={"nativeSpecification": param_str})

I finally figured out that the ast.literal_eval() was necessary because the value of the "support_condition" jobParameter was a string representation of a dict and not an actual dict. Not sure if there is a better syntax or method - again I'm not a Python coder as such. I would welcome suggestions or a pointer to the proper API documentation.

Dynamic resource allocation is such a useful or even critical feature for cluster environments - nice work dev team.
ADD COMMENT • link modified 2.5 years ago • written 2.5 years ago by jdv • 20
Please log in to add an answer.

Similar posts • Search »

Content
Help
Access
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 171 users visited in the last hour