Question: Job was reported by drmaa as terminal but job state in SLURM is: PENDING, returning to monitor queue
0
gravatar for chadmatsalla
11 months ago by
chadmatsalla40
chadmatsalla40 wrote:

I have a brand new install of slurm with two nodes. I just managed to get slurm working.

Galaxy web reports no jobs.

Galaxy's main logs are filling with:

galaxy.jobs.runners.slurm WARNING 2017-12-06 15:50:59,820 (91/40) Job was reported by drmaa as terminal but job state in SLURM is: PENDING, returning to monitor queue galaxy.jobs.runners.drmaa DEBUG 2017-12-06 15:51:00,830 (91/40) state change: job finished, but failed

I can't seem to find out what to do about this.

 root@slurm-controller:~# squeue
         JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)

root@slurm-controller:~# sacct -s PD,R
   JobID    JobName  Partition    Account  AllocCPUS      State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
40           g91_uploa+    bioinfo                     1    PENDING      0:0

Is this a slurm problem? A Galaxy problem?

Thanks!

Chad Matsalla

ADD COMMENTlink modified 11 months ago • written 11 months ago by chadmatsalla40
0
gravatar for Nate Coraor
11 months ago by
Nate Coraor3.2k
United States
Nate Coraor3.2k wrote:

I'm not sure why the drmaa library is considering PENDING jobs as terminal. What version of slurm-drmaa are you using?

ADD COMMENTlink written 11 months ago by Nate Coraor3.2k
0
gravatar for chadmatsalla
11 months ago by
chadmatsalla40
chadmatsalla40 wrote:

Thanks for your help.

To fix this I used the well-worn strategy of "turning it off and turning it back on". I restarted the entire slurm apparatus and the entire galaxy apparatus.

The messages have stopped but sacct still shows that job as pending. I'm not really sure how to deal with that given that it's now showing in squeue.

ADD COMMENTlink written 11 months ago by chadmatsalla40
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 169 users visited in the last hour