Usegalaxy main server down? (Queue not working)

Question: Usegalaxy main server down? (Queue not working) - PROBLEM FIXED

2.6 years ago by

Canada

yena.oh • 70 wrote:

Hi all,

Is the main usegalaxy server running, or is it just under a high work load? I've run cufflinks, and out of 12 files, 2 files had failed cufflinks the first time for unknown reasons:

This job failed for reasons that could not be determined.
Please click the bug icon to report this problem if you need help.

Since the other 10 files worked fine, I re-ran cufflinks and the jobs were highlighted yellow, and remained yellow for 8 hours. I ended up permanently deleting the files, thinking that perhaps the process froze.. and ran them once again. These jobs now have been highlighted grey with an exclamation mark for the past 14 hours, and haven't even started running. Is the server working properly? At this point, I am regretting permanently deleting the running files, but even so, 8 hours of "yellow," and 15 hours of "grey" seem way too long for cufflinks?

Thanks, Yena

rna-seq usegalaxy cufflinks server • 1.5k views

ADD COMMENT • link •

modified 2.6 years ago • written 2.6 years ago by yena.oh • 70

Hi! The same happens to me but with other tools. I am waiting for some runs about Bowtie and TopHat since yesterday, and they still are with the exclamation mark y grey color! So the better thing we can do is to wait, because that means that soon is going to be added to the queue? When I started working with galaxy it didn't last as much time as it does currently!

ADD REPLY • link written 2.6 years ago by iraia.maialen • 10

Hey, Did your jobs start running?

ADD REPLY • link written 2.6 years ago by yena.oh • 70

No, not yet... Both of them, Bowtie and TopHat are still with exclamation mark. And your job?

ADD REPLY • link written 2.6 years ago by iraia.maialen • 10

nope.. still hasn't changed and it's been 22 hours since my third attempt..

ADD REPLY • link written 2.6 years ago by yena.oh • 70

Hey! my job is ok now!

ADD REPLY • link written 2.6 years ago by iraia.maialen • 10

2.6 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

The first error message was a cluster problem, so a re-run was the correct fix (the only fix).

From there, it sounds as if you are not allowing jobs to fully queue and execute. Please review: https://wiki.galaxyproject.org/Support#Dataset_status_and_how_jobs_execute

Best, Jen, Galaxy team

ADD COMMENT • link written 2.6 years ago by Jennifer Hillman Jackson ♦ 25k

Hi Jen,

I have had to re-run cufflinks 3 times, because they were taking way too long (15hours, 19hours and 24hours each).. Should we be continuing to cancel and re-run, until jobs are added to queue within less than 10 hours? Thanks always for your help, Yena

ADD REPLY • link written 2.6 years ago by yena.oh • 70

2.6 years ago by

Martin Čech ♦♦ 4.9k

United States

Martin Čech ♦♦ 4.9k wrote:

Hi Yena, we are not aware of Main having problems running jobs. In fact the queue is not too full these days as we added more resources recently. Please click the bug icon on the red datasets to report this problem as the message states, it will provide us with more information regarding the failure.

You can also see the following for explanation of datasets color states https://wiki.galaxyproject.org/Histories#History_Panel_Datasets Grey datasets with exclamation mark are "new" which means that they have not been added to the queue yet, but will be soon.

ADD COMMENT • link written 2.6 years ago by Martin Čech ♦♦ 4.9k

Thanks Martin, Can datasets remain grey with exclamation marks for 16hours? Should I continue to wait until these are added to the queue? Yena

ADD REPLY • link written 2.6 years ago by yena.oh • 70

They shouldn't take that long. I would cancel it and try running the job again. Let us know how it goes. Sorry for the inconvenience.

ADD REPLY • link written 2.6 years ago by Martin Čech ♦♦ 4.9k

Hi Martin,

As you suggested, I cancelled the jobs and re-ran it and it's been 22 hours since then. It seems like I'm not the only one experiencing this problem currently:

is the main queue working?

meaning of Job status

how long cuffdiff takes

Given that the queue is not too full, perhaps there is an issue with the server?

Yena

ADD REPLY • link modified 2.6 years ago • written 2.6 years ago by yena.oh • 70

I was digging a bit more into this and form our sysadmin I leant that part of our resources hosted at IU are having connectivity problems. These are some of the resources that we added recently and that had and will have some hiccups before we smooth out the whole process. You cannot really test these well without being in production environment. :/

That said our best people are on this and we will get it fixed. I am sorry for the inconvenience, our goal is to make you wait less but in the meantime you might need to wait more :/

ADD REPLY • link written 2.6 years ago by Martin Čech ♦♦ 4.9k

Thanks for the update Martin, Please keep us informed once the issue is resolved, and if we should cancel and re-run(hopefully for the last time) the jobs that were in the grey exclamation state. Yena

ADD REPLY • link modified 2.6 years ago • written 2.6 years ago by yena.oh • 70

2.6 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello Yena & Others,

This is the latest update concerning jobs and http://usegalaxy.org

A: Jobs caught in long term grey zone (!)

Thanks again for your patience, Jen, Galaxy team

ADD COMMENT • link written 2.6 years ago by Jennifer Hillman Jackson ♦ 25k

Please log in to add an answer.

Similar posts • Search »