Cloudman - Sge / Cluster Questions

Question: Cloudman - Sge / Cluster Questions

6.9 years ago by

mailing list • 840 wrote:

Hi guys, So I'm trying to take a bioinformatics pipeline we currently run on our own SGE cluster (which seems to have a lot of customizations ! I'm finding out) and I want to run it on Amazon using CloudMan. So I'm trying to figure out what the differences are. Here are my questions so far: We seem to have 8 slots per node. Do you guys know if the same is true on CloudMan? I'm assuming more than one submitted qsub job can run on one node at one time? Also is there a way to reserve a certain amount of memory for a job submitted via qsub, or alternatively to request that the job have exclusive access to a node so it keeps all the memory available? Thanks, Greg

• 966 views

ADD COMMENT • link •

modified 6.9 years ago by Enis Afgan • 680 • written 6.9 years ago by mailing list • 840

6.9 years ago by

Enis Afgan • 680

Enis Afgan • 680 wrote:

Hi again Greg, Glad to hear this! Please let me know how it goes in the end. The number of slots per node depends on the type of instance used. CloudMan will configure an SGE instance to have as many slots as there are processing cores. So, for example, a large instance will have 2 slots, an extra large instance will have 4 slots. Each qsub job then gets submitted to one slot. There are no preconfigured provisions for this in the SGE configuration provided by CloudMan. However, this is a vanilla SGE setup so it is entirely possible to manually setup the queues as you'd like and have it do the various resource reservations. SGE is installed under /opt/sge

ADD COMMENT • link written 6.9 years ago by Enis Afgan • 680

Thanks. I'll be sure to keep you guys updated. So far everything is working great! -Greg

ADD REPLY • link written 6.9 years ago by mailing list • 840

Hey guys, Another cluster related question. Currently on our cluster I'm calling a job like this (from perl): system("qsub -N msgRun0-1.$$ -pe batch $update_nthreads -cwd -b y -V -sync n ./msgRun0-1.sh") Does anyone know what I should use for -pe? I couldn't tell from the man page what options are available. PE refers to parallel programming environment from my reading. Thanks, Greg

ADD REPLY • link written 6.9 years ago by mailing list • 840

Just an update on this. I tried taking out the whole "-pe batch $update_nthreads" part and it seemed to work fine. -Greg

ADD REPLY • link written 6.9 years ago by mailing list • 840

Similar posts • Search »