Question: making galaxyIndices volume smaller in cloud by deleting genomes
1
gravatar for clarkd
3.7 years ago by
clarkd30
Canada
clarkd30 wrote:

I wish to remove most of the mnt/galaxyIndices/genomes files so I can have a smaller volume that contains those files and reduce AWS costs. I am new to this system. I have set up a Galaxy Cloudman account and used the tools on the web interface, and I can ssh the server using Ubuntu. So far, so good.

I tried the first approach here twice https://wiki.galaxyproject.org/CloudMan/CustomizeGalaxyCloud but the system hangs (>24h). I can’t follow the “manual” instructions on this page and I’m not sure it would result in a smaller volume anyway. So, I was trying to follow Enis’ steps from this thread to reduce the size of the galaxyIndices volume: http://dev.list.galaxyproject.org/Reducing-costs-in-Cloud-Galaxy-td4483448.html His instructions are:

“1. Start a new Galaxy cluster (ie, one you can easily delete later)

2. ssh into the master instance and delete whatever genomes you don't need/want (these are all located under /mnt/galaxyIndices)

3. Create a new EBS volume of size that'll fit whatever's left on the original volume, attach it and mount it

4. Copy over the data from the original volume to the new one while keeping the directory structure the same (rsync is probably the best tool for this)

5. Unmount & detach the new volume; create a snapshot from it

6. For the cluster you want to keep around (while it is terminated), edit persistent_data.yaml in it's bucket on S3 and replace the existing snap ID for the galaxyIndices with the snapshot ID you got in the previous step

7. Start that cluster and you should have a file system from the new snapshot mounted.

8. Terminate & delete the cluster you created in step 1”

 

Here are my questions to help me to get through step 4:

1) Step 1: Is a “cluster” the same thing as an “instance”?

2) Step 2: I deleted the directories for individual genomes using rm –rf . Is that the correct approach?

3) Step 3: Do I add the newly created EBS volume to the same instance where I deleted the genomes? Or is it added the instance I want to keep?

4) Step 3: I can see how to attach this newly created volume using the AWS EC2 management console, but how do I mount it? (and unmount it in Step 5?)

5) Step 4: what is the syntax for the rsync (or cp) command to copy directories/files from one volume to another volume (within the same instance, or in if they are in different instances)?

Any help would be gratefully received! Denise

galaxy • 1.1k views
ADD COMMENTlink written 3.7 years ago by clarkd30
1
gravatar for clarkd
3.7 years ago by
clarkd30
Canada
clarkd30 wrote:

I figured it out!  The "Getting started..." documentation on biocloudlinux.org filled in the conceptual gap I had about the relationship between volumes and file systems, and how to set up file systems on a newly added volume.  Basic stuff that is very helpful for newcomers.  -Denise
 

ADD COMMENTlink written 3.7 years ago by clarkd30

Thanks Denise for both posting back that this was resolved for you personally, but also for sharing where to find the appropriate help for others to read about. Very nice! Take care, Jen, Galaxy team

ADD REPLYlink written 3.7 years ago by Jennifer Hillman Jackson25k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 92 users visited in the last hour