Question: Using DRMAA from Python to submit & monitor SGE jobs
0
gravatar for steve
6 months ago by
steve30
NYC
steve30 wrote:

I am having issues configuring, submitting, and monitoring SGE HPC cluster jobs from within Python using DRMAA. I know that Galaxy uses Python and supports DRMAA, so I was trying to figure out how this is implemented there. I saw the docs here:

https://docs.galaxyproject.org/en/latest/admin/cluster.html?highlight=drmaa

and I was looking at the implementation here:

https://docs.galaxyproject.org/en/latest/_modules/galaxy/jobs/runners/drmaa.html#DRMAAJobRunner

It appears that Galaxy is using subprocess to issue qsub commands. Is that correct?

I was assuming that Galaxy would be using something like drmaa-python for this, but that appears to not be the case? I had tried that one and couldn't figure out how to properly configure my qsub jobs, but the developer hasn't replied so I was hoping that maybe the Galaxy team would have some insight into this.

Thanks

sge cluster qsub drmaa • 338 views
ADD COMMENTlink modified 6 months ago by Nate Coraor3.2k • written 6 months ago by steve30
1
gravatar for Nate Coraor
6 months ago by
Nate Coraor3.2k
United States
Nate Coraor3.2k wrote:

It only uses the subprocess if you're using the "run-as-real-user" mode, and even then the default scripts still use drmaa, they just have to be run through some permissions-granting executable (e.g. sudo).

As per your question to python-drmaa, in Galaxy you can set these SGE-specific options in a <param id="nativeSpecification"></param> on the relevant <destination>.

ADD COMMENTlink written 6 months ago by Nate Coraor3.2k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 112 users visited in the last hour