Cloudman volume manipulation stuck

Question: Cloudman volume manipulation stuck

2.8 years ago by

Hi there,

I ran out of space on my cloud instance of Galaxy. Clicked on '+Grow' next to 'Disk status'. It has been almost a week, is that normal? I think it is stuck.

Would appreciate hearing from you.

Thanks.

The overlay on my CloudMan Console says:

Volume Manipulation In Progress

Creating a snapshot of the cluster's data volume is in progress. All of the cluster services have been stopped for the time being and will resume automatically upon completion of the process.
This message should go away after the process completes but if it does not, try refreshing the page then.

Snapshot status: configuring

galaxy • 809 views

ADD COMMENT • link •

modified 2.8 years ago by Dannon Baker ♦ 3.7k • written 2.8 years ago by squelchskin • 0

2.8 years ago by

Dannon Baker ♦ 3.7k

United States

Dannon Baker ♦ 3.7k wrote:

Sorry for the inconvenience here. Depending on the size of the original volume snapshot and volume swap can take a while -- but it should never take this long. This can happen when, for example, there is a user logged in via SSH and using the filesystem and preventing it from being unmounted.

The easiest way to fix this is generally to do a cluster restart, the 'Reboot the cluster' option, via the admin panel. Cloudman should figure things out and come back correctly.

ADD COMMENT • link written 2.8 years ago by Dannon Baker ♦ 3.7k

Thank you for that, the console is back. Should I try to grow the cluster again? Maybe I should use autoscaling instead?

ADD REPLY • link written 2.8 years ago by squelchskin • 0

Autoscaling is used for scaling the number of worker machines so it won't help with the disk.

Try logging into the AWS web console (https://console.aws.amazon.com/ec2/home) and under Snapshots, see if there is a snapshot that was created at the time you initiated disk resizing (it should be in status 'completed'). If there is, go back to CloudMan and click on that '+ Grow' button again. Assuming you made no/minimal changes to the files, the process should be much quicker this time around. After the process completes and is successful, you can go back to the AWS console and delete the snapshot that was created a week ago.

Give this a shot and let us know how things go. If things don't go as expected, we'll try something else. BTW, how big is your original file system?

ADD REPLY • link written 2.8 years ago by Enis Afgan • 690

I have exactly the same problem as the original poster, but it's been over 2 weeks. Here's the particulars:

cloudman, using AWS cluster We ran out of space with 4TB. I pushed the Grow button and selected 8TB.

I can temporarily remove the overlay and hit the 'reboot the cluster' button. Will that work? Keep in mind we already ran out of disk space.

Any help is appreciated.

ADD REPLY • link written 16 months ago by dschwartzwald • 0

Similar posts • Search »