Unable to upload (.bam) data file to local Galaxy

Question: Unable to upload (.bam) data file to local Galaxy

3.9 years ago by

Singapore

jchen015 • 80 wrote:

Dear all,

I setup galaxy in my local workstation and when I tried to upload a (.bam) data file, it shows an error as attached in the link below:

Link: https://www.dropbox.com/sh/4cmk3is574pzkar/AADFBT1R7gaPdrkfzL1VhGrNa?dl=0

I had all the samtools as the public Galaxy Server has already, do I still need to install other tools to upload a (.bam) file?

Regards,

Julius

path galaxy samtools bam • 2.4k views

ADD COMMENT • link •

modified 3.9 years ago by Jennifer Hillman Jackson ♦ 25k • written 3.9 years ago by jchen015 • 80

Oh man. yes you are right. thanks alot Bjoern. you helped me alot.

Cheers

Julius

ADD REPLY • link written 3.9 years ago by jchen015 • 80

Cool - Enjoy Galaxy! Can you please mark the question as answered to help others to find it more quickly. Thanks!

ADD REPLY • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

Not sure how, but I had moved it to answer. Is that right?

Regard,

Julius

ADD REPLY • link written 3.9 years ago by jchen015 • 80

Partially, just the wrong answer :)

You can mark the real answer that was helpful. This will help others. Thanks!

ADD REPLY • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

Can you please copy the entire bug-report from this small 'bug' symbol here. I was not able to read the full message. Moreover, please make sure you have samtools < 1.0 on your $PATH. Nothing else should be required.

ADD REPLY • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

Hey bjoern Gruening, thanks for the reply.

The error is as shown below:

Dataset generation errors

Dataset 8: cut_L7_1_5.bam

Tool execution generated the following error message:

Traceback (most recent call last):
  File "/home/julius/galaxy-dist/galaxy-dist/tools/data_source/upload.py", line 407, in <module>
    __main__()
  File "/home/julius/galaxy-dist/galaxy-dist/tools/data_source/upload.py", line 396, in __main__
    add_file( dataset, registry, json_file, output_path )
  File "/home/julius/galaxy-dist/galaxy-dist/tools/data_source/upload.py", line 324, in add_file
    if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ):
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/binary.py", line 147, in dataset_content_needs_grooming
    version = self._get_samtools_version()
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/binary.py", line 129, in _get_samtools_version
    output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1]
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I have no idea what is the reason....

ADD REPLY • link written 3.9 years ago by jchen015 • 80

I am not sure whether this is the root of the problem, but I am unable to perform "make install" due to permission denied.

Please reference : https://www.dropbox.com/sh/4cmk3is574pzkar/AADFBT1R7gaPdrkfzL1VhGrNa?dl=0

Thanks...

ADD REPLY • link written 3.9 years ago by jchen015 • 80

3.9 years ago by

Bjoern Gruening ♦ 5.1k

Germany

Bjoern Gruening ♦ 5.1k wrote:

Hi,

you don't have samtools in your $PATH. Please install samtools < 1.0 on your server and make it accessible to Galaxy.

Cheers,

Bjoern

ADD COMMENT • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

Hi Bjoern,

i followed the following steps to install samtools ( http://biobits.org/samtools_primer.html ).

Things I had done:

1. I had successfully entered "make" in the samtools directory

2. I had successfully entered "make" in the bcftools-1.1 directory

3. I had added both samtools and bcftools to my $PATH.

The below image shows that they were added to my $PATH:

->https://www.dropbox.com/sh/4cmk3is574pzkar/AADFBT1R7gaPdrkfzL1VhGrNa?dl=0 (steps_taken.png) May I know what steps have i overlooked or not done yet??

ADD REPLY • link modified 3.9 years ago • written 3.9 years ago by jchen015 • 80

You need to install an older version of samtools. Samtools 0.19 should do it. Do you have the ToolShed enabled? Than you can install smatools with the ToolShed. Samtools should also be available in your package manager.

ADD REPLY • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

Ok noted. However, I did the following and the result is as stated below:

julius@julius-Aspire-4755:~/galaxy-dist/galaxy-dist$ sudo apt-get install samtools
Reading package lists... Done
Building dependency tree
Reading state information... Done
samtools is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

Upon executing this in cmd, do i have to perform any removal of previous samtools or bcftools ? Because I still have the same error which is...

Traceback (most recent call last):
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in queue_job
    job_wrapper.finish( stdout, stderr, exit_code )
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/jobs/__init__.py", line 1107, in finish
    dataset.datatype.set_meta( dataset, overwrite=False )  # call datatype.set_meta directly for the initial set_meta call during dataset creation
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/binary.py", line 250, in set_meta
    raise Exception, "Error Setting BAM Metadata: %s" % stderr
Exception: Error Setting BAM Metadata: Samtools-htslib-API: bam_index_build2() not yet implemented
Aborted (core dumped)

Regards,

Juliuys

ADD REPLY • link written 3.9 years ago by jchen015 • 80

If you just execute samtools you will see the version number. Please make sure the correct samtools version is on your $PATH.

ADD REPLY • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

3.9 years ago by

Bjoern Gruening ♦ 5.1k

Germany

Bjoern Gruening ♦ 5.1k wrote:

Hi,

after you have installed samtools from the ToolShed you have two possibilities to tell Galaxy to find samtools.

1. You add samtools to your $PATH and restart Galaxy.

2. You create a symbolic link from ~/tool_deps/samtools/0.19/foo/bar/ to ~/tool_deps/samtools/default

~/tool_deps/samtools/default should now contain a env.sh file and a /bin directory. Galaxy should now be able to find samtools.

Hope this helps,

Bjoern

ADD COMMENT • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

Hey Bjoern.

I added samtools 0.1.19 to my $PATH as shown below.

julius@julius-Aspire-4755:~/galaxy-dist/galaxy-dist$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/julius/galaxy-dist/galaxy-dist/samtools-0.1.19/

and after which, i start up my Galaxy by calling the sh run.sh command.

However, it still showed this error...

Traceback (most recent call last):
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 129, in queue_job
    job_wrapper.finish( stdout, stderr, exit_code )
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/jobs/__init__.py", line 1107, in finish
    dataset.datatype.set_meta( dataset, overwrite=False )  # call datatype.set_meta directly for the initial set_meta call during dataset creation
  File "/home/julius/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/binary.py", line 250, in set_meta
    raise Exception, "Error Setting BAM Metadata: %s" % stderr
Exception: Error Setting BAM Metadata: Samtools-htslib-API: bam_index_build2() not yet implemented
Aborted (core dumped)

What are the possibilities ??

Regards,

Julius

ADD REPLY • link written 3.9 years ago by jchen015 • 80

This message indicates that samtools > 1.0 is still in your $PATH. Can it be that samtools is located under /usr/bin too? What does a samtools --version tell you?

ADD REPLY • link written 3.9 years ago by Bjoern Gruening ♦ 5.1k

3.9 years ago by

nlemus • 0

Germany

nlemus • 0 wrote:

I have exactly the same problem. I have already installed the samtool by tool shed. It is installed but still i am not able to upload the file.

This is the bug information

Traceback (most recent call last):

  File "/Users/nlemus/galaxy-dist/tools/data_source/upload.py", line 407, in <module>
    __main__()
  File "/Users/nlemus/galaxy-dist/tools/data_source/upload.py", line 396, in __main__
    add_file( dataset, registry, json_file, output_path )
  File "/Users/nlemus/galaxy-dist/tools/data_source/upload.py", line 324, in add_file
    if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ):
  File "/Users/nlemus/galaxy-dist/lib/galaxy/datatypes/binary.py", line 147, in dataset_content_needs_grooming
    version = self._get_samtools_version()
  File "/Users/nlemus/galaxy-dist/lib/galaxy/datatypes/binary.py", line 129, in _get_samtools_version
    output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1]
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I would appreciate any help.

Nicolas

ADD COMMENT • link written 3.9 years ago by nlemus • 0

Please log in to add an answer.

Dataset generation errors

Similar posts • Search »