I am meeting a problem with the number of concurrent job running on my local instance of galaxy. Currently, I can run simultaneously at most 10 jobs. But it is clearly underusing the capability of my server. I am using the paste method with the default configuration (1 server 1 handler) and my runner is HTCondor.
I looked into simple solutions thanks to a conversation pvh_sa on the irc:
- set "concurrent_jobs" and "registered_user_concurrent_jobs" to 30 in the <limits> block of the job_conf.xml
- increasing the number of threadpool_workers to 20 in the galaxy.ini
- increasing the number of htcondor worker from 4 to 10.
But none of them worked. So I wonder what is limiting the number of concurrent jobs to 10 in my instance ? I was thinking about switching from paste to uwsgi to add servers and handlers, but since I don't know where is the limit of concurrent jobs I am not sure that would solve my problem. I would like to understand this before adding complexity.
Your help regarding the understanding of this problem would be very much appreciated.
Thank you !