My local Galaxy is quite slower than usegalaxy!?

Question: My local Galaxy is quite slower than usegalaxy!?

3.3 years ago by

vebaev • 130

vebaev • 130 wrote:

I was suprised to find out that my local Galaxy is quite slow than Main usegalaxy.

I have put a job on FASTQ groomer (2x12GB files) and my local Galaxy finiched about 5h and the Main finiched about 3h.

I'm using a machine with 48 cores, 128gb ram, 24tb hdd, during the job I did not use the machine for eny other stuff.

I thought that one of the pros of Galaxy local will be speed since the Main is shared for thousands of users at same time, but.....

galaxy local • 986 views

ADD COMMENT • link •

modified 3.3 years ago by Bjoern Gruening ♦ 5.1k • written 3.3 years ago by vebaev • 130

I also noticed that 1 job uses 1core cpu. For example the two groomers jobs use 2 cores and nor I run bowtie and uses 1core!?

ADD REPLY • link written 3.3 years ago by vebaev • 130

Also another slow issue - downloading via http// takes minutes, but trying connection with the same file with wget takes 5 seconds to download to the system...

ADD REPLY • link written 3.3 years ago by vebaev • 130

3.3 years ago by

Bjoern Gruening ♦ 5.1k

Germany

Bjoern Gruening ♦ 5.1k wrote:

Hi,

keep in mind that usegalaxy.org is running on a huge server farm with optimized CPU's, memory and storage. Usegalaxy.org is only slow if you need to wait until your job is running. If many users running jobs at the same time, your job will be queued and you need to wait until it starts. If it starts it should be fast.

You can configure your jobs/tools to use as many CPU's as you want in the job_conf.xml file. Have a look at it and read the corresponding wiki page.

Not sure about your http issue, if available try ftp://.

Thanks for using Galaxy,

Bjoern

ADD COMMENT • link written 3.3 years ago by Bjoern Gruening ♦ 5.1k

Thanks,

I have found an sample of the job_conf.xml from another post and I have pasted it into a newly created file, but galaxy failed to start, I will be happy if you can point where I am wrong, as I'm not so familiar creating these myself? Many thanks!

<?xml version="1.0"?>

<job_conf>
<plugins>
<plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="8"/>
<plugin id="multilocal" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="2"/>
</plugins>
<destinations default="local">
<destination id="local" runner="local"/>
<destination id="multicore8" runner="multilocal">
<param id="local_slots">8</param>
</destination>
</destinations>
<tools>
   <tool id="bowtie2" destination="multicore8" />
   <tool id="bowtie" destination="multicore8" />
   <tool id="deeptools" destination="multicore8" />
   <tool id="cufflinks" destination="multicore8" />
   <tool id="cuffdiff" destination="multicore8" />
   <tool id="cuffmerge" destination="multicore8" />
   <tool id="tophat2" destination="multicore8" />
</tools>
</job_conf>

ADD REPLY • link written 3.3 years ago by vebaev • 130

This is complex and highly dependent on your setup. I fear you need to learn this by reading the wiki page or using a preconfigured Galaxy instance in a VM or Docker container.

ADD REPLY • link written 3.3 years ago by Bjoern Gruening ♦ 5.1k

Do you know any good VM instances?

ADD REPLY • link written 3.3 years ago by vebaev • 130

ADD REPLY • link written 3.3 years ago by Bjoern Gruening ♦ 5.1k

Similar posts • Search »