Latest
Open
RNA-Seq
ChIP-Seq
SNP
Assembly
Forum
Planet
All »
Home
Log In
Question: Using DRMAA from Python to submit & monitor SGE jobs
0
gravatar for steve
10 months ago by
steve • 30
NYC
steve • 30 wrote:

I am having issues configuring, submitting, and monitoring SGE HPC cluster jobs from within Python using DRMAA. I know that Galaxy uses Python and supports DRMAA, so I was trying to figure out how this is implemented there. I saw the docs here:

https://docs.galaxyproject.org/en/latest/admin/cluster.html?highlight=drmaa

and I was looking at the implementation here:

https://docs.galaxyproject.org/en/latest/_modules/galaxy/jobs/runners/drmaa.html#DRMAAJobRunner

It appears that Galaxy is using subprocess to issue qsub commands. Is that correct?

I was assuming that Galaxy would be using something like drmaa-python for this, but that appears to not be the case? I had tried that one and couldn't figure out how to properly configure my qsub jobs, but the developer hasn't replied so I was hoping that maybe the Galaxy team would have some insight into this.

Thanks
sge cluster qsub drmaa • 548 views
ADD COMMENT • link •
modified 10 months ago by Nate Coraor ♦ 3.2k • written 10 months ago by steve • 30
1
gravatar for Nate Coraor
10 months ago by
Nate Coraor ♦ 3.2k
United States
Nate Coraor ♦ 3.2k wrote:

It only uses the subprocess if you're using the "run-as-real-user" mode, and even then the default scripts still use drmaa, they just have to be run through some permissions-granting executable (e.g. sudo).

As per your question to python-drmaa, in Galaxy you can set these SGE-specific options in a <param id="nativeSpecification"></param> on the relevant <destination>.
ADD COMMENT • link written 10 months ago by Nate Coraor ♦ 3.2k
Please log in to add an answer.

Similar posts • Search »

Content
Help
Access
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 169 users visited in the last hour