Scaling : from Paste to UWSGI, and several handlers

Question: Scaling : from Paste to UWSGI, and several handlers

2.7 years ago by

christophe.habib • 340

France

christophe.habib • 340 wrote:

Hello everyone,

I have a galaxy instance using Paste, I want to switch to UWSGI. I found this very interesting page :

https://wiki.galaxyproject.org/Admin/Config/Performance/Scaling#uWSGI

But I have a lot of questions :

From what I understand, even if I want to use uwsgi, will the handlers still use Paste ?
Is it the difference of responsiveness the reason why we need to set only 1 uwsgi when several server:web are required with Paste ?
I am working on a brand new instance of galaxy for my developments (from last week). I need to set PYTHONPATH as /path/to/galaxy/eggs/PasteDeploy-1.5.0-py2.7.egg in Supervisord. I can find it in my prod instance, but I don't even have eggs repertory in my dev instance. How should I set this PYTHONPATH ?
Still in supervisord, in the environment there is SGE_ROOT=/var/lib/gridengine. I am using HTCondor, so should I set SGE_ROOT=/var/lib/condor ? That sounds... bad to me (since condor is not sun gridengine).
Is it mandatory to set Nginx in supervisor as well ? I have seen it here : https://wiki.galaxyproject.org/Events/GCC2014/TrainingDay/AdminWalkthrough#Configure_Galaxy
Let's say I have set uwsgi, and 2 handlers in the galaxy.ini . In the end I will have 2 port for uwsgi (stats and socket), 2 ports for the 2 handlers... and what about the main section ? Does it still need his own port ? Why ?

I hope you can help me to answers this questions. Thank you for your attention.

Christ

uwsgi htcondor nginx handlers • 1.1k views

ADD COMMENT • link •

modified 2.6 years ago by romeu.alfred • 10 • written 2.7 years ago by christophe.habib • 340

2.6 years ago by

romeu.alfred • 10

romeu.alfred • 10 wrote:

Quite new to Galaxy deployment, but I'll try to answer some of your doubts

- Will the handlers still use Paste?
Yes, at least that's my understanding from the configuration you can find here or here

- uswgi = better performance?
Yes, uWSGI will give you a better performance, plus you can define a variable number of processes and threads to suit your needs

- no eggs in galaxy folder
Be sure to follow the steps in the guide. Virtual environment, pip install locally, nginx, supervisor, etc, and your folder eggs will appear

- condor or gridengine
Use gridengine if you have the choice. It seems condor could run with supervisord but there are not many online resources to help you out (maybe this?)

- Nginx mandatory in supervisor?
Recommended, if you want supervisord to manage everything at the same time (start/stop/restart), and if your server is serving just galaxy. If you serve other apps through nginx in the same machine you have galaxy, that's a different story

- 2 handlers + 2 port + sockets + ... Take a look at this tutorial from a bioit workshop, it will give you a better understanding on each step of the process and what does each option mean for uwsgi.

Please post your progress, some questions you made here are interesting (as using HTCondor)

Have fun! Alfredo

ADD COMMENT • link modified 2.6 years ago • written 2.6 years ago by romeu.alfred • 10

Similar posts • Search »