Question: Docker Galaxy submitting jobs to UGE
0
gravatar for machinelearner
15 months ago by
machinelearner10 wrote:

Hi all,

Has anyone been able to submit jobs on a Docker Galaxy instance to SGE or UGE on a centOS7 machine?

After referring to https://github.com/bgruening/docker-galaxy-stable#Using-an-external-Grid-Engine-cluster we were still not able to have jobs passed on to UGE.

Going into the container, we are able to see that the environment variable was passed in and the act_qmaster file was present. So, these variables should be alright.

  1. Job configuration: we're using the sample docker configuration, except we replaced anything about slurm with UGE and took all the htcondor references out. We're trying to pass in the drmaa library. However, if we use this configuration, it seems like the application will not work and go to a 403 page, so there's something wrong here. Maybe someone could help point out what went wrong?

    <job_conf>
    <plugins workers="2">
        <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner">
            <param id="drmaa_library_path">/lib/lx-amd64/libdrmaa.so</param>
        </plugin>
        <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner"/>
    </plugins>
    
    <handlers default_from_environ="GALAXY_HANDLERS_DEFAULT" default="sge">
          <handler id="handler0" tags="sge"/>
          <handler id="handler1" tags="sge"/>
          <handler id="sge_handler">
              <plugin id="sge"/>
          </handler>
    </handlers>
    
    <destinations default_from_environ="GALAXY_DESTINATIONS_DEFAULT" default="sge">
            <destination id="sge" runner="sge">
            <param id="nativeSpecification">-V -pe threads 1 -l m_mem_free=1G</param>
    
            <env file="/galaxy_venv/bin/activate"/>
                    <param id="docker_enabled" from_environ="GALAXY_DOCKER_ENABLED">False</param>
        <param id="docker_sudo" from_environ="GALAXY_DOCKER_SUDO">False</param>
        <!-- The empty volumes from shouldn't affect Galaxy, set GALAXY_DOCKER_VOLUMES_FROM to use. -->
        <param id="docker_volumes_from" from_environ="GALAXY_DOCKER_VOLUMES_FROM"></param>
        <!-- For a stock Galaxy instance and traditional job runner $defaults will expand out as: $galaxy_root:ro,$tool_directory:ro,$working_directory:rw,$default_file_path:rw -->
        <param id="docker_volumes" from_environ="GALAXY_DOCKER_VOLUMES">$defaults</param>
        </destination>
    
    </destinations>
    <limits>
    </limits></job_conf>
    
  2. The next part of the readme says "Your Grid Engine needs to accept job submissions from inside the container." We're not sure how to do this. Does this mean that we have to add the hostname of the container? We tried this and UGE wasn't able to resolve the hostname of the container.

Thanks so much!

galaxy docker uge • 394 views
ADD COMMENTlink written 15 months ago by machinelearner10

I don't use SGE, but I've generally taken to compiling the scheduler inside the container. That way I'm sure it links correctly.

ADD REPLYlink written 15 months ago by Devon Ryan1.9k

Thanks for the reply! We are hesitant to do this because UGE is a license product and we don't want it in the container itself.

ADD REPLYlink written 15 months ago by machinelearner10

Hi all,

I'm running into the same issues. Can anyone advise?

Many thanks!

Konrad.

ADD REPLYlink written 12 months ago by konradpaszkiewicz0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 113 users visited in the last hour