Best practice for backing up CloudMan Galaxy instance

Question: Best practice for backing up CloudMan Galaxy instance

4.5 years ago by

United States

Dear CloudMan Galaxy users,

What is the general best practices for maintaining and backing up Galaxy AWS instances and ensuring a smooth recovery in cases of crashes. I was thinking of implementing nightly images of the instance, de-registering old one after some time, but I would really love to hear about general practice.

We have deployed a small CloudMan cluster and are currently evaluating whether we will go with an AWS instance instead of a local server for our institute (25 labs, ~250 researchers, less then 10% potential users).

Going local would ensure that everything is backed up by our IT team, I am less sure what are the potential problems that an AWS instance is facing and what practices are out there.

Thanks

Marco

galaxy cloudman • 851 views

ADD COMMENT • link •

modified 4.5 years ago by Dannon Baker ♦ 3.7k • written 4.5 years ago by Marco Blanchette • 10

4.5 years ago by

Dannon Baker ♦ 3.7k

United States

Dannon Baker ♦ 3.7k wrote:

Incremental snapshots of the data volume would be a reasonable way to have a running backup of a galaxy cloud instance -- in a catastrophic (unexpected) failure you can recreate a functional instance from existing galaxyData snapshots with a little work.

One concern I'd have with your particular situation and running one galaxy cloud cluster would be disk space -- currently EBS volumes are limited to 1TB, so that's the maximum data volume size Cloudman can use. Depending on how heavily/frequently they're used, it may be more appropriate to manage multiple galaxy clusters, leaving them offline until a user needs to process/access data.

ADD COMMENT • link written 4.5 years ago by Dannon Baker ♦ 3.7k

Thanks Danon,

I was thinking of setting up software raid to break the 1T limit and was in fact a follow up question I had in mind... Any thoughts on that?

ADD REPLY • link written 4.5 years ago by Marco Blanchette • 10

That could work, but I haven't tried it and my guess is it wouldn't be supported by any of Cloudman's built in persistence without some extra poking I don't think.

While cloud instances don't run out of the box like this, it might also be worth considering the S3ObjectStore. I also need to personally do more testing of this, but it'd give you the persistence you want with a much smaller base volume. I know some others are using this in production, but I'd definitely test to make sure it works for your use case before jumping in head first.

ADD REPLY • link written 4.5 years ago by Dannon Baker ♦ 3.7k

Similar posts • Search »