TS dependencies install with conda doesn't work on galaxy Docker

Question: TS dependencies install with conda doesn't work on galaxy Docker

16 months ago by

i-guigon • 80 wrote:

Hi there

I created a Docker image of a Galaxy instance with some tools installed from the toolshed. It worked (almost) perfectly, except that the tools dependencies were not installed, I had to install them "manually" in my Dockerfile, which is quite tedious. I probably missed something with the configuration for automated dependencies install.

Here i'm making some tests with the bedtools, without manual installation of the bedtools, and my jobs are red saying that the tool is not found.

I started from bgruening/galaxy-stable, which is the version 17.01 so conda is supposed to be automatically installed and check for missing dependencies.

In the galaxy.ini file, I set the conda_auto_init and conda_auto_install param to True, and I un-commented the lines about conda_ensure_channels and conda_prefix. It did nothing.

After launching my instance I also tried to desinstall and reinstall the tool via the admin panel and I made sure that the box "When available, install Conda managed tool dependencies?" was checked. It still doesn't work. In the "manage install tools" panel, when I click on bedtools for more details, it still tells me that samtools and bedtools are not installed.

Here is the content of handler0.log after I launched a job: galaxy.tools.deps DEBUG 2017-07-17 12:58:31,199 Find dependency bedtools version 2.26.0gx
galaxy.tools.deps.conda_util DEBUG 2017-07-17 12:58:31,200 Executing command: /galaxy-central/<tool_dependency_dir>/_conda/bin/conda create -y --name __bedtools@2.26.0gx bedtools=2.26.0gx
galaxy.tools.deps.resolvers.conda DEBUG 2017-07-17 12:58:31,207 Removing failed conda install of bedtools, version '2.26.0gx'
galaxy.tools.deps.conda_util DEBUG 2017-07-17 12:58:31,208 Executing command: /galaxy-central/<tool_dependency_dir>/_conda/bin/conda create -y --name __bedtools@_uv_ bedtools
galaxy.tools.deps.resolvers.conda DEBUG 2017-07-17 12:58:31,216 Removing failed conda install of bedtools, version 'None'
galaxy.tools.deps DEBUG 2017-07-17 12:58:31,217 Dependency bedtools not found.
galaxy.jobs.command_factory INFO 2017-07-17 12:58:31,257 Built script [/export/galaxy-central/database/job_working_directory/000/3/tool_script.sh] for tool command [bedtools --version > /export/galaxy-central/database/files/GALAXY_VERSION_STRING_3 2>&1; sortBed -i '/export/galaxy-central/database/files/000/dataset_1.dat' -chrThenScoreA > '/export/galaxy-central/database/files/000/dataset_3.dat']
galaxy.tools.deps DEBUG 2017-07-17 12:58:31,353 Find dependency samtools version None
galaxy.tools.deps.conda_util DEBUG 2017-07-17 12:58:31,353 Executing command: /galaxy-central/<tool_dependency_dir>/_conda/bin/conda create -y --name __samtools@_uv_ samtools
galaxy.tools.deps.resolvers.conda DEBUG 2017-07-17 12:58:31,361 Removing failed conda install of samtools, version 'None'
galaxy.tools.deps.conda_util DEBUG 2017-07-17 12:58:31,361 Executing command: /galaxy-central/<tool_dependency_dir>/_conda/bin/conda create -y --name __samtools@_uv_ samtools
galaxy.tools.deps.resolvers.conda DEBUG 2017-07-17 12:58:31,369 Removing failed conda install of samtools, version 'None'
galaxy.tools.deps DEBUG 2017-07-17 12:58:31,369 Dependency samtools not found.

Can you help me?

dependencies conda docker • 551 views

ADD COMMENT • link •

modified 16 months ago • written 16 months ago by i-guigon • 80

You don't need to change anything in your galaxy.ini it should work out of the box. Can you verify that this also happens to other tools? I just checked with bgruening/galaxy-stable:17.05 and this works for me.

ADD REPLY • link written 16 months ago by Bjoern Gruening ♦ 5.1k

16 months ago by

i-guigon • 80

i-guigon • 80 wrote:

Hmm, well in fact it doesn't work all fine.
It's OK for the bedtools, but then I re-launched my Dockerfile with bgruening/galaxy-stable:17.05 and without installing manually my tools (which are bedtools, CEAS, deeptools and MACS2).
In the list of installed tools in the Admin panel I have a fair number of packages (and I couldn't say precisely what package goes with what tool) and nearly half of them have missing dependencies.

Here's the list of packages with missing dependencies: ceas
package_atlas_3_10
package_bx_python_0_7
package_cairo_1_12_14
package_fontconfig_2_11_2
package_freetype_2_5_2 (devteam)
package_freetype_2_5_2 (iuc)
package_libpng_1_6_7 (devteam)
package_libpng_1_6_7 (iuc)
package_libxml2_2_9_1
package_ncurses_5_9
package_numpy_1_7
package_pixman_0_32_4
package_r_3_1_2
package_readline_6_2
package_zlib_1_2_8

First of all I don't know why there is 2 versions for package_freetype_2_5_2 and package_libpng_1_6_7 but I guess it's another problem.

If I click on a package with missing dependencies it will tell me what package is missing (for instance package_zlib_1_2_8 complains that zlib was never installed (although I installed it manually in my Dockerfile, I had problems with that package during the install with version 17.01)) and it offers to install the missing dependencies.
Fine, I could repair a few packages (not all though) but why wasn't it done automatically?
It's really problematic for me to have to install some things manually, I want to have everything done in the Dockerfile...

ADD COMMENT • link written 16 months ago by i-guigon • 80

I noticed this myself for non-conda tool dependency installs and others have too. I and have asked our development team for feedback. My guess is that the configuration of the dependency_resolvers_conf.xml file needs to be in a specific format (ordering) - but that is ONLY a guess. Our admins will know more.

We'll have an updated reply soon. Much is going on during the server downtime over the next 24 hrs, so this may come after that is completed.

ADD REPLY • link written 16 months ago by Jennifer Hillman Jackson ♦ 25k

Thank you for your answer. I'll wait for more details :)

ADD REPLY • link written 16 months ago by i-guigon • 80

16 months ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

The 17_01 release included conda dependency resolution, but it was a first pass implementation. Try using the 17_05 release instead. There were several upgrades with respect to Conda dependency resolution and other admin functions/configurations:

https://galaxyproject.org/galaxy-updates/2017-06/#galaxy-docker-image-17-05

Thanks, Jen, Galaxy team

ADD COMMENT • link written 16 months ago by Jennifer Hillman Jackson ♦ 25k

16 months ago by

i-guigon • 80

i-guigon • 80 wrote:

I tried another tool (deeptools) on the 17.01 and I had the same error, the tool was not installed. I tried the version 17.05 without modifying galaxy.ini and it worked.

So the version 17.05 seems to solve the problem. Thank you both for your help :)

ADD COMMENT • link written 16 months ago by i-guigon • 80

Glad that using 17_05 worked!

ADD REPLY • link written 16 months ago by Jennifer Hillman Jackson ♦ 25k

Please log in to add an answer.

Similar posts • Search »