Question: Accessing bam file in history for visualization through Galaxy API
0
gravatar for asmariyaz23
3.4 years ago by
asmariyaz23 • 10
United States
asmariyaz23 • 10 wrote:

My goal is to use a JSON file (which is a product of a lab implemented pipeline ran on galaxy) for visualizing 2 BAM files (also produced by the pipeline) loaded in history.

For this purpose I started out by loading the BAM and JSON individually in history and working my way from there (as I haven't looked into generating API IDs and history ID dynamically through scripts).

I load a BAM file in history, extract it's history ID and API ID (which I later use in my JSON file to indicate where galaxy should find the file). Here is a small part of what the JSON file looks like:

{
  "CLL-HL_pilot.r1.fastq": {
    "DNA": "/api/histories/a799d38679e985db/contents/3f5830403180d620/display", 
    "DNA_INDEX": "test.dna.bai",
    "DNA_VCF": "test_DNA.vcf", 
    "FN": {
      "Chr1-144915561 (203)": {
        "Alt": "G", 
        "Chr": "1", 
        "Cov": "227", 
        "Cov_dna": "203", 
        "Loc": "144915561", 
        "Ref": "NA", 
        "Strand": "+"
      }, 

I want to use this JSON file to pass information into a JS which creates the IGV tracks and loads the BAM file. I keep ending up with a file not found error.

http://kcodevel.broadinstitute.org:8088/galaxy/visualization/show/api/histories/a799d38679e985db/contents/3f5830403180d620/display (404) NOT FOUND. 

When I go to the link above the actual BAM file gets downloaded so I know that the correct URL is being used, I just can't figure out how to provide a valid URL/path to the BAM file in history.

Note: I already have JSON data provider and visualization created in my local instance in galaxy which is configured with apache. Another test made was to try doing the same thing but without apache in place but I end up in the same problem. 

Appreciate any help I can get!

Asma

 

 

 

 

 

visualization galaxy json bam • 1.3k views
ADD COMMENT • link • modified 3.4 years ago • written 3.4 years ago by asmariyaz23 • 10
0
gravatar for carlfeberhard
3.4 years ago by
carlfeberhard ♦ 390
United States
carlfeberhard ♦ 390 wrote:

Hi, Asma

If I understand correctly, the url you listed:

http://kcodevel.broadinstitute.org:8088/galaxy/visualization/show/api/histories/a799d38679e985db/contents/3f5830403180d620/display

should, most likely be instead:

http://kcodevel.broadinstitute.org:8088/api/histories/a799d38679e985db/contents/3f5830403180d620/display

You might be able to correct this in a simple way by using an absolute url that begins with '/' for your 'DNA' attribute in the dictionary you posted above:

"CLL-HL_pilot.r1.fastq": {
    "DNA": "/api/histories/a799d38679e985db/contents/3f5830403180d620/display", 
    "DNA_INDEX": "test.dna.bai",
ADD COMMENT • link written 3.4 years ago by carlfeberhard ♦ 390

Upon your suggestion to add a "/" I did the following:

1)Extract API key and History ID of a newly loaded BAM file.

2) Include these 2 ids in the JSON object as below: 

"CLL-HL_pilot.r1.fastq": {
    "DNA": "/galaxy/api/histories/[history id]/contents/[API ID]/display", 
    "DNA_INDEX": "test.dna.bai",

3) Upload the modified JSON file into galaxy. But I end up in the same File not found error (I get this in the debugging window of Google Chrome)

--> http://kcodevel.broadinstitute.org:8088/galaxy/api/histories/f597429621d6eb2b/contents/c9468fdb6dc5c5f1/display 404 (NotFound)

Side note:

I have tested the above URL in a separate browser window and it immediately downloads the file on my local machine so I know that I am using the correct path info in the JSON file. 

I have also in the past given an absolute path of a BAM file which exists on the server but end up in the same file not found error.

Is this because apache is not serving these files?

 

ADD REPLY • link modified 3.4 years ago • written 3.4 years ago by asmariyaz23 • 10

Are you using a proxy for galaxy? In other words, is your homepage at 'http://kcodevel.broadinstitute.org:8088/galaxy' and not 'http://kcodevel.broadinstitute.org:8088'?

ADD REPLY • link written 3.4 years ago by carlfeberhard ♦ 390

Yes, I use a proxy so the URL is 'http://kcodevel.broadinstitute.org:8088/galaxy'. I have checked the error_log for apache and it looks like apache is trying to serve these bam files from htdocs area within apache folder. Here is the error log: 

File does not exist: /seq/../kcodevel/apache/httpd/htdocs/visualization, referer: http://kcodevel.broadinstitute.org:8088/galaxy/visualization/show/igv?dataset_id=2f94e8ae9edff68a

Any pointers on how to serve these bam files from galaxy's API? 

ADD REPLY • link written 3.4 years ago by asmariyaz23 • 10
/galaxy/api/histories/[history id]/contents/[API ID]/display

should work for BAM files.

I'm certainly not the best at Apache proxying - have you had a look at:

https://wiki.galaxyproject.org/Admin/Config/ApacheProxy

Specifically the sections on the rewrite rules ("Serving Galaxy at a sub directory (such as /galaxy") and XSendFile ("Sending files using Apache")? 

ADD REPLY • link written 3.4 years ago by carlfeberhard ♦ 390

Is there a way to access the .bai file associated with .bam file? The metadata has something of this sort:

"meta_files": [{"file_type": "bam_index"}]

when I access 

http://kcodevel.broadinstitute.org:8088/galaxy/api/histories/[history id]/contents/[API ID]/display

but have no clue how the file itself?

 

 

ADD REPLY • link written 3.4 years ago by asmariyaz23 • 10

It's not documented and may change, but for now the one I know of is:

localhost:8080/galaxy/dataset/get_metadata_file?hda_id=[API_ID]&metadata_name=bam_index

 

And, since we seem to be losing horizontal space for this conversation, I'll mention that the galaxy-dev mailing list is also a good resource for this kind of discussion:

https://lists.galaxyproject.org/listinfo/galaxy-dev

ADD REPLY • link modified 3.3 years ago • written 3.3 years ago by carlfeberhard ♦ 390

I used the URL for the bam index file, but returns the following: 

Request Range Not Satisfiable

The Range requested is not available. 
Invalid reference dataset id: get_metadata_file.

I will soon write to the galaxy-dev list too!

ADD REPLY • link written 3.3 years ago by asmariyaz23 • 10

I solved this issue by serving bam files through apache's modxsendfile (as suggested by carlfeberhard). IGV needs a Partial 206 content returned and Galaxy's server wasn't able to do it. 

ADD REPLY • link modified 3.3 years ago • written 3.3 years ago by asmariyaz23 • 10
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 173 users visited in the last hour