Question: Job was reported by drmaa as terminal but job state in SLURM is: PENDING, returning to monitor queue
1
gravatar for chadmatsalla
5 days ago by
chadmatsalla10
chadmatsalla10 wrote:

I have a brand new install of slurm with two nodes. I just managed to get slurm working.

Galaxy web reports no jobs.

Galaxy's main logs are filling with:

galaxy.jobs.runners.slurm WARNING 2017-12-06 15:50:59,820 (91/40) Job was reported by drmaa as terminal but job state in SLURM is: PENDING, returning to monitor queue galaxy.jobs.runners.drmaa DEBUG 2017-12-06 15:51:00,830 (91/40) state change: job finished, but failed

I can't seem to find out what to do about this.

 root@slurm-controller:~# squeue
         JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)

root@slurm-controller:~# sacct -s PD,R
   JobID    JobName  Partition    Account  AllocCPUS      State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
40           g91_uploa+    bioinfo                     1    PENDING      0:0

Is this a slurm problem? A Galaxy problem?

Thanks!

Chad Matsalla

ADD COMMENTlink modified 4 days ago • written 5 days ago by chadmatsalla10
0
gravatar for Nate Coraor
5 days ago by
Nate Coraor3.1k
United States
Nate Coraor3.1k wrote:

I'm not sure why the drmaa library is considering PENDING jobs as terminal. What version of slurm-drmaa are you using?

ADD COMMENTlink written 5 days ago by Nate Coraor3.1k
0
gravatar for chadmatsalla
4 days ago by
chadmatsalla10
chadmatsalla10 wrote:

Thanks for your help.

To fix this I used the well-worn strategy of "turning it off and turning it back on". I restarted the entire slurm apparatus and the entire galaxy apparatus.

The messages have stopped but sacct still shows that job as pending. I'm not really sure how to deal with that given that it's now showing in squeue.

ADD COMMENTlink written 4 days ago by chadmatsalla10
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 94 users visited in the last hour