Question: Galaxy tool development - how do I get a working samtools?
0
gravatar for crashfrog
4 days ago by
crashfrog0
crashfrog0 wrote:

I'm trying to develop a Galaxy tool that has samtools 1.5 as a dependency, but nothing I declare in the "<requirements>" section succeeds in putting a working samtools on my PATH when the tool is executed by Galaxy.

With

<requirement type="package" version="1.5">samtools</requirement>

I get

samtools: error while loading shared libraries: libbz2.so.1.0: cannot open shared object file: No such file or directory

With

<requirement type="binary" version="1.5">samtools</requirement>

I get

Fatal error: Exit code 127 () ... /shared/000/399/tool_script.sh: line 9: samtools: command not found

Neither python-module or set_environment are appropriate here, and there are no other permitted values for requirement.type .

I know I'm not writing the first tool that has samtools as a dependency. How is this supposed to work?

galaxy samtools • 118 views
ADD COMMENTlink modified 4 days ago • written 4 days ago by crashfrog0
0
gravatar for y.hoogstrate
4 days ago by
y.hoogstrate450
Netherlands
y.hoogstrate450 wrote:
<requirement type="package" version="1.5">samtools</requirement>

should refer to the (i suppose bioconda) version of samtools 1.5

This is the way this should work I believe.

Can you run: <command>which -a samtools ; ldd `which samtools`</commmand> in your tool and post the output?

ADD COMMENTlink written 4 days ago by y.hoogstrate450
0
gravatar for crashfrog
4 days ago by
crashfrog0
crashfrog0 wrote:
CLASSPATH=/nfs/sw/apps/varscan/varscan.2.3.9/VarScan.v2.3.9.jar ;export CLASSPATH;LOADEDMODULES=galaxy-super ;export LOADEDMODULES;PATH=/nfs/sw/lang/java/jre1.8.0_112/bin:/nfs/sw/apps/smalt/bin:/nfs/sw/apps/spades/spades-3.9.0/bin:/nfs/sw/apps/sratoolkit/sratoolkit.2.8.0-centos_linux64/bin:/nfs/sw/apps/ascp:/nfs/sw/apps/bwa/bwa-0.7.15/:/nfs/sw/apps/blast/ncbi-blast-2.6.0+/bin:/nfs/sw/apps/samtools/samtools-1.3.1/bin:/nfs/sw/apps/bowtie/bowtie2-2.2.9/:/nfs/sw/apps/snp/snp-pipeline-0.7.0/bin:/nfs/sw/apps/galaxy-dev/galaxy/.venv/bin:/tmp/404.1.all.q:/usr/local/bin:/bin:/usr/bin ;export PATH;_LMFILES_=/nfs/sw/Modules/galaxy-super ;export _LMFILES_;varscan() { java -jar /nfs/sw/apps/varscan/varscan.2.3.9/VarScan.v2.3.9.jar;};/nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/samtools
    linux-vdso.so.1 =>  (0x00007ffdd58ce000)
    libz.so.1 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/libz.so.1 (0x00002b735a38e000)
    libm.so.6 => /lib64/libm.so.6 (0x00002b735a5a4000)
    libbz2.so.1.0 => not found
    liblzma.so.5 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/liblzma.so.5 (0x00002b735a8a6000)
    libcurl.so.4 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/libcurl.so.4 (0x00002b735aacb000)
    libcrypto.so.1.0.0 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/libcrypto.so.1.0.0 (0x00002b735ad3e000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b735b175000)
    libc.so.6 => /lib64/libc.so.6 (0x00002b735b391000)
    librt.so.1 => /lib64/librt.so.1 (0x00002b735b753000)
    libssh2.so.1 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/./libssh2.so.1 (0x00002b735b95b000)
    libssl.so.1.0.0 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/./libssl.so.1.0.0 (0x00002b735bb99000)
    libgssapi_krb5.so.2 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/./libgssapi_krb5.so.2 (0x00002b735be10000)
    libkrb5.so.3 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/./libkrb5.so.3 (0x00002b735c057000)
    libk5crypto.so.3 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/./libk5crypto.so.3 (0x00002b735c324000)
    libcom_err.so.3 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/./libcom_err.so.3 (0x00002b735c553000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00002b735c756000)
    /lib64/ld-linux-x86-64.so.2 (0x00002b735a16a000)
    libkrb5support.so.0 => /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/../lib/././libkrb5support.so.0 (0x00002b735c95a000)
    libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00002b735cb65000)
    libresolv.so.2 => /lib64/libresolv.so.2 (0x00002b735cd69000)
ADD COMMENTlink written 4 days ago by crashfrog0
0
gravatar for crashfrog
4 days ago by
crashfrog0
crashfrog0 wrote:

I know there's an issue in Conda's samtools package being built against dynamic libraries, and that the one in conda-forge is built against statics, but I've already changed galaxy.ini to use the recommended conda channel ordering, so it should be pulling the correct samtools dependencies.

ADD COMMENTlink written 4 days ago by crashfrog0

The path of samtools is /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda/envs/mulled-v1-ee953de1640e372d8d33733402c4e38671d17286210a6464d4805ae7ef8e67df/bin/samtools and some of the libraries are in there as well. I think your galaxy and tools are working fine but the bioconda package is (as you mentioned already) improperly linked.

I think the mulled builds are specific for bioconda, so you are not using samtools from condaforge. You could shutdown galaxy and move the bioconda data, and see if a clean env does the job correctly:

mv /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda /nfs/sw/apps/galaxy-dev/galaxy/database/dependencies/_conda.bak ; planemo serve ...xml

if this doesn't work you should move the directory back.

ADD REPLYlink written 4 days ago by y.hoogstrate450

I'm not using planemo, unfortunately. At this point I've changed the channel resolution order as specified in galaxy.ini (the default order has bioconda ahead of conda-forge) and reset the tool metadata, but it's still pulling mulled builds for samtools and causing the shared library error.

ADD REPLYlink written 3 days ago by crashfrog0

Not sure what your exact use-case is, but maybe you could use sambamba as drop-in replacement and revert back to bioconda. Changing the channel order, once it would work, may solve samtools but may break other tools. Since bioconda is supposed to be used as dependency resolver for Galaxy, this would be the handiest workaround I think. You need to pressure the bioconda ppl to solve it ;).


I've tested my instance of bioconda and got this:

conda create -n samtools150 samtools=1.5=0

conda create -n samtools151 samtools=1.5=1

conda create -n samtools152 samtools=1.5=2

source activate samtools150 ; ldd which samtools

libbz2.so.1.0 => /home/youri/anaconda2/envs/samtools150/bin/../lib/libbz2.so.1.0 (0x00007f31ee473000)

source activate samtools151 ; ldd which samtools

libbz2.so.1.0 => /home/youri/anaconda2/envs/samtools151/bin/../lib/libbz2.so.1.0 (0x00007fe51eaf4000)

source activate samtools152 ; ldd which samtools

libbz2.so.1.0 => /home/youri/anaconda2/envs/samtools152/bin/../lib/libbz2.so.1.0 (0x00007fdf796bf000)

Now I am a bit lost figuring out what the actual problem is because the appropriate linked libraries can be obtained via conda (at least on my instance).

ADD REPLYlink modified 3 days ago • written 3 days ago by y.hoogstrate450

The thing that doesn't make any sense is that

locate libbz2.so.1.0

returns

/usr/lib64/libbz2.so.1.0.6

so it's clear that libbz2 is present; but samtools can't find it anyway.

ADD REPLYlink written 2 days ago by crashfrog0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 124 users visited in the last hour