Question: Cloudman volume manipulation stuck
0
gravatar for squelchskin
2.8 years ago by
squelchskin0 wrote:

Hi there, 

I ran out of space on my cloud instance of Galaxy. Clicked on '+Grow' next to 'Disk status'. It has been almost a week, is that normal? I think it is stuck.

Would appreciate hearing from you.

Thanks.

The overlay on my CloudMan Console says:

Volume Manipulation In Progress

Creating a snapshot of the cluster's data volume is in progress. All of the cluster services have been stopped for the time being and will resume automatically upon completion of the process.
This message should go away after the process completes but if it does not, try refreshing the page then.

Snapshot status: configuring

galaxy • 809 views
ADD COMMENTlink modified 2.8 years ago by Dannon Baker3.7k • written 2.8 years ago by squelchskin0
0
gravatar for Dannon Baker
2.8 years ago by
Dannon Baker3.7k
United States
Dannon Baker3.7k wrote:

Sorry for the inconvenience here.  Depending on the size of the original volume snapshot and volume swap can take a while -- but it should never take this long.  This can happen when, for example, there is a user logged in via SSH and using the filesystem and preventing it from being unmounted.

The easiest way to fix this is generally to do a cluster restart, the 'Reboot the cluster' option, via the admin panel.  Cloudman should figure things out and come back correctly.

ADD COMMENTlink written 2.8 years ago by Dannon Baker3.7k

Thank you for that, the console is back. Should I try to grow the cluster again? Maybe I should use autoscaling instead?

ADD REPLYlink written 2.8 years ago by squelchskin0

Autoscaling is used for scaling the number of worker machines so it won't help with the disk. 

Try logging into the AWS web console (https://console.aws.amazon.com/ec2/home) and under Snapshots, see if there is a snapshot that was created at the time you initiated disk resizing (it should be in status 'completed'). If there is, go back to CloudMan and click on that '+ Grow' button again. Assuming you made no/minimal changes to the files, the process should be much quicker this time around. After the process completes and is successful, you can go back to the AWS console and delete the snapshot that was created a week ago.

Give this a shot and let us know how things go. If things don't go as expected, we'll try something else. BTW, how big is your original file system?

 

 

ADD REPLYlink written 2.8 years ago by Enis Afgan690

I have exactly the same problem as the original poster, but it's been over 2 weeks. Here's the particulars:

cloudman, using AWS cluster We ran out of space with 4TB. I pushed the Grow button and selected 8TB.

I can temporarily remove the overlay and hit the 'reboot the cluster' button. Will that work? Keep in mind we already ran out of disk space.

Any help is appreciated.

ADD REPLYlink written 16 months ago by dschwartzwald0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 180 users visited in the last hour