Galaxy Cluster Nodes From The Cloudlaunch Site

5.9 years ago by

United States

Dannon Baker ♦ 3.7k wrote:

The root size for the worker nodes doesn't matter. The only thing relevant to Galaxy here are the available compute units and memory. The only storage and working space used is actually attached to the master node (the EBS volume you configure at initial launch) and shared over NFS with the workers. If you find yourself needing more space, you can expand this volume through the cloudman UI if you want. -Dannon .sanitized-message p.MsoNormal, .sanitized-message li.MsoNormal, .sanitized-message div.MsoNormal { margin : 0in; margin-bottom : .0001pt; font-size : 11.0pt; font-family : "Calibri","sans-serif"; } .sanitized-message a:link, .sanitized-message span.MsoHyperlink { color : blue; text-decoration : underline; } .sanitized-message a:visited, .sanitized-message span.MsoHyperlinkFollowed { color : purple; text-decoration : underline; } .sanitized-message span.EmailStyle17 { font-family : "Calibri","sans-serif"; color : windowtext; } .sanitized-message .MsoChpDefault { font-family : "Calibri","sans-serif"; } .sanitized-message div.WordSection1 { } Dear Galaxy users, I created galaxy instance using galaxy ami the first time. When I re- launch via cloudlaunch site the 2nd time, I have question on adding cluster nodes. There are several pull-down options for the nodes type, no matter which type I choose, they always generate nodes with only 15gb. Is there other option I can have option to define the root space for the nodes? Thank you, Kathryn ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

ADD COMMENT • link written 5.9 years ago by Dannon Baker ♦ 3.7k

Thanks for the information. Would you help providing further information on expanding ebs volume via cloudman UI? Thank you in advance, Kathryn To: Sun, Wenping [USA] Cc: galaxy-user@bx.psu.edu Subject: [External] Re: [galaxy-user] galaxy cluster nodes from the cloudlaunch site The root size for the worker nodes doesn't matter. The only thing relevant to Galaxy here are the available compute units and memory. The only storage and working space used is actually attached to the master node (the EBS volume you configure at initial launch) and shared over NFS with the workers. If you find yourself needing more space, you can expand this volume through the cloudman UI if you want. -Dannon .sanitized-message p.MsoNormal, .sanitized-message li.MsoNormal, .sanitized-message div.MsoNormal { margin : 0in; margin-bottom : .0001pt; font-size : 11.0pt; font-family : "Calibri","sans-serif"; } .sanitized-message a:link, .sanitized-message span.MsoHyperlink { color : blue; text-decoration : underline; } .sanitized-message a:visited, .sanitized-message span.MsoHyperlinkFollowed { color : purple; text-decoration : underline; } .sanitized-message span.EmailStyle17 { font-family : "Calibri","sans-serif"; color : windowtext; } .sanitized-message .MsoChpDefault { font-family : "Calibri","sans-serif"; } .sanitized-message div.WordSection1 { } Dear Galaxy users, I created galaxy instance using galaxy ami the first time. When I re- launch via cloudlaunch site the 2nd time, I have question on adding cluster nodes. There are several pull-down options for the nodes type, no matter which type I choose, they always generate nodes with only 15gb. Is there other option I can have option to define the root space for the nodes? Thank you, Kathryn ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

ADD REPLY • link written 5.9 years ago by Sun, Wenping [USA] • 230

In the main (non-admin) cloudman UI you'll see a little icon next to the volume size. Click this and you'll have the opportunity to enter a new size. Upon submitting this, galaxy will need to temporarily shut down, but it should be automatically restarted once the procedure is complete. Note that resizing can take quite some time (hours, even) depending on the size of the EBS volume in question.

ADD REPLY • link written 5.9 years ago by Dannon Baker ♦ 3.7k

Thank you for the help. I assume that if galaxy will shutting down while re-sizing, the current running jobs will be stopped. Could the running jobs be suspended which will be continuing after auto-restart? If not, I may need to wait my job finished. It will be very helpful if you can confirm this. Additional question, is there any way that I can adjust root, ebs volumes while initiating at cloudlaunch site? This would be help saving the time for the re-sizing as you mentioned. Thank you in advance, Kathryn To: Sun, Wenping [USA] Cc: galaxy-user@bx.psu.edu Subject: Re: [External] Re: [galaxy-user] galaxy cluster nodes from the cloudlaunch site In the main (non-admin) cloudman UI you'll see a little icon next to the volume size. Click this and you'll have the opportunity to enter a new size. Upon submitting this, galaxy will need to temporarily shut down, but it should be automatically restarted once the procedure is complete. Note that resizing can take quite some time (hours, even) depending on the size of the EBS volume in question.

ADD REPLY • link written 5.9 years ago by Sun, Wenping [USA] • 230

Correct, all running jobs will be stopped and there isn't currently any way to suspend running jobs. Jobs that haven't dispatched yet (not in the 'running' state) should, however, resume after the restart. And as far as resizing from cloudlaunch, no, currently resizing is only possible from a running cluster though this is an interesting idea. Thank you for the help. I assume that if galaxy will shutting down while re-sizing, the current running jobs will be stopped. Could the running jobs be suspended which will be continuing after auto-restart? If not, I may need to wait my job finished. It will be very helpful if you can confirm this. Additional question, is there any way that I can adjust root, ebs volumes while initiating at cloudlaunch site? This would be help saving the time for the re-sizing as you mentioned. Thank you in advance, Kathryn

ADD REPLY • link written 5.9 years ago by Dannon Baker ♦ 3.7k

Thanks for the answers. Thanks, Kathryn To: Sun, Wenping [USA] Cc: galaxy-user@bx.psu.edu Subject: Re: [External] Re: [galaxy-user] galaxy cluster nodes from the cloudlaunch site Correct, all running jobs will be stopped and there isn't currently any way to suspend running jobs. Jobs that haven't dispatched yet (not in the 'running' state) should, however, resume after the restart. And as far as resizing from cloudlaunch, no, currently resizing is only possible from a running cluster though this is an interesting idea. Thank you for the help. I assume that if galaxy will shutting down while re-sizing, the current running jobs will be stopped. Could the running jobs be suspended which will be continuing after auto-restart? If not, I may need to wait my job finished. It will be very helpful if you can confirm this. Additional question, is there any way that I can adjust root, ebs volumes while initiating at cloudlaunch site? This would be help saving the time for the re-sizing as you mentioned. Thank you in advance, Kathryn

ADD REPLY • link written 5.9 years ago by Sun, Wenping [USA] • 230

instance via aws site; or have the user be able to choose some major options if initiating from cloudlaunch site. I wonder either one would be the future concern for galaxy development. If so, I am looking forward to the capability. I'm not sure I understand, could you elaborate? Right now, all resizing (regardless of how you start the instance, via cloudlaunch or aws) requires you to use cloudman running on the instance.

ADD REPLY • link written 5.9 years ago by Dannon Baker ♦ 3.7k

Okay, sorry for the confusion. I meant that while initiating via aws, there are more options to choose the root and ebs volumes which it is very flexible. We do have the option to initiating the instance at the FIRST time via aws site (or from cloudlaunch site as you mentioned); whereas all the later launches have to go to cloudlaunch site. What I found was, the sizes that I initiated first at aws site were lost while relaunching from cloudlaunch. Moreover, it takes more time to re-size them. Hope this helps and thank you, Kathryn To: Sun, Wenping [USA] Cc: galaxy-user@bx.psu.edu Subject: Re: [External] Re: [galaxy-user] galaxy cluster nodes from the cloudlaunch site instance via aws site; or have the user be able to choose some major options if initiating from cloudlaunch site. I wonder either one would be the future concern for galaxy development. If so, I am looking forward to the capability. I'm not sure I understand, could you elaborate? Right now, all resizing (regardless of how you start the instance, via cloudlaunch or aws) requires you to use cloudman running on the instance.

ADD REPLY • link written 5.9 years ago by Sun, Wenping [USA] • 230

Ahh, I see the confusion. The instance root volume should *not* be changed in size, and no volumes should be manually added to instances at launch via the AWS console. Leave all of this at the default and let Cloudman manage the volumes. The reason this volume size changes back to the default after re-launching via cloudlaunch is that it's totally unnecessary and nothing will ever be written there so we use the default 15GB â all galaxy data are designed to be persisted on the single galaxyData EBS volume that is managed entirely by Cloudman. Okay, sorry for the confusion. I meant that while initiating via aws, there are more options to choose the root and ebs volumes which it is very flexible. We do have the option to initiating the instance at the FIRST time via aws site (or from cloudlaunch site as you mentioned); whereas all the later launches have to go to cloudlaunch site. What I found was, the sizes that I initiated first at aws site were lost while relaunching from cloudlaunch. Moreover, it takes more time to re-size them. Hope this helps and thank you, Kathryn To: Sun, Wenping [USA] Cc: galaxy-user@bx.psu.edu Subject: Re: [External] Re: [galaxy-user] galaxy cluster nodes from the cloudlaunch site master instance via aws site; or have the user be able to choose some major options if initiating from cloudlaunch site. I wonder either one would be the future concern for galaxy development. If so, I am looking forward to the capability. I'm not sure I understand, could you elaborate? Right now, all resizing (regardless of how you start the instance, via cloudlaunch or aws) requires you to use cloudman running on the instance.

ADD REPLY • link written 5.9 years ago by Dannon Baker ♦ 3.7k

Hello, I tried several time of re-sizing to create data volume as you indicated below. However, there is nothing change, either from galaxy log and aws console. Is there anything else that may miss from the your procedure below? Thanks, Kathryn To: Sun, Wenping [USA] Cc: galaxy-user@bx.psu.edu Subject: Re: [External] Re: [galaxy-user] galaxy cluster nodes from the cloudlaunch site In the main (non-admin) cloudman UI you'll see a little icon next to the volume size. Click this and you'll have the opportunity to enter a new size. Upon submitting this, galaxy will need to temporarily shut down, but it should be automatically restarted once the procedure is complete. Note that resizing can take quite some time (hours, even) depending on the size of the EBS volume in question.

ADD REPLY • link written 5.9 years ago by Sun, Wenping [USA] • 230

Similar posts • Search »