Question: Accessing bam file in history for visualization through Galaxy API
0
gravatar for asmariyaz23
3.4 years ago by
asmariyaz2310
United States
asmariyaz2310 wrote:

My goal is to use a JSON file (which is a product of a lab implemented pipeline ran on galaxy) for visualizing 2 BAM files (also produced by the pipeline) loaded in history.

For this purpose I started out by loading the BAM and JSON individually in history and working my way from there (as I haven't looked into generating API IDs and history ID dynamically through scripts).

I load a BAM file in history, extract it's history ID and API ID (which I later use in my JSON file to indicate where galaxy should find the file). Here is a small part of what the JSON file looks like:

{
  "CLL-HL_pilot.r1.fastq": {
    "DNA": "/api/histories/a799d38679e985db/contents/3f5830403180d620/display", 
    "DNA_INDEX": "test.dna.bai",
    "DNA_VCF": "test_DNA.vcf", 
    "FN": {
      "Chr1-144915561 (203)": {
        "Alt": "G", 
        "Chr": "1", 
        "Cov": "227", 
        "Cov_dna": "203", 
        "Loc": "144915561", 
        "Ref": "NA", 
        "Strand": "+"
      }, 

I want to use this JSON file to pass information into a JS which creates the IGV tracks and loads the BAM file. I keep ending up with a file not found error.

http://kcodevel.broadinstitute.org:8088/galaxy/visualization/show/api/histories/a799d38679e985db/contents/3f5830403180d620/display (404) NOT FOUND. 

When I go to the link above the actual BAM file gets downloaded so I know that the correct URL is being used, I just can't figure out how to provide a valid URL/path to the BAM file in history.

Note: I already have JSON data provider and visualization created in my local instance in galaxy which is configured with apache. Another test made was to try doing the same thing but without apache in place but I end up in the same problem. 

Appreciate any help I can get!

Asma

 

 

 

 

 

visualization galaxy json bam • 1.3k views
ADD COMMENTlink modified 3.4 years ago • written 3.4 years ago by asmariyaz2310
0
gravatar for carlfeberhard
3.4 years ago by
carlfeberhard390
United States
carlfeberhard390 wrote:

Hi, Asma

If I understand correctly, the url you listed:

http://kcodevel.broadinstitute.org:8088/galaxy/visualization/show/api/histories/a799d38679e985db/contents/3f5830403180d620/display

should, most likely be instead:

http://kcodevel.broadinstitute.org:8088/api/histories/a799d38679e985db/contents/3f5830403180d620/display

You might be able to correct this in a simple way by using an absolute url that begins with '/' for your 'DNA' attribute in the dictionary you posted above:

"CLL-HL_pilot.r1.fastq": {
    "DNA": "/api/histories/a799d38679e985db/contents/3f5830403180d620/display", 
    "DNA_INDEX": "test.dna.bai",
ADD COMMENTlink written 3.4 years ago by carlfeberhard390

Upon your suggestion to add a "/" I did the following:

1)Extract API key and History ID of a newly loaded BAM file.

2) Include these 2 ids in the JSON object as below: 

"CLL-HL_pilot.r1.fastq": {
    "DNA": "/galaxy/api/histories/[history id]/contents/[API ID]/display", 
    "DNA_INDEX": "test.dna.bai",

3) Upload the modified JSON file into galaxy. But I end up in the same File not found error (I get this in the debugging window of Google Chrome)

--> http://kcodevel.broadinstitute.org:8088/galaxy/api/histories/f597429621d6eb2b/contents/c9468fdb6dc5c5f1/display 404 (NotFound)

Side note:

I have tested the above URL in a separate browser window and it immediately downloads the file on my local machine so I know that I am using the correct path info in the JSON file. 

I have also in the past given an absolute path of a BAM file which exists on the server but end up in the same file not found error.

Is this because apache is not serving these files?

 

ADD REPLYlink modified 3.4 years ago • written 3.4 years ago by asmariyaz2310

Are you using a proxy for galaxy? In other words, is your homepage at 'http://kcodevel.broadinstitute.org:8088/galaxy' and not 'http://kcodevel.broadinstitute.org:8088'?

ADD REPLYlink written 3.4 years ago by carlfeberhard390

Yes, I use a proxy so the URL is 'http://kcodevel.broadinstitute.org:8088/galaxy'. I have checked the error_log for apache and it looks like apache is trying to serve these bam files from htdocs area within apache folder. Here is the error log: 

File does not exist: /seq/../kcodevel/apache/httpd/htdocs/visualization, referer: http://kcodevel.broadinstitute.org:8088/galaxy/visualization/show/igv?dataset_id=2f94e8ae9edff68a

Any pointers on how to serve these bam files from galaxy's API? 

ADD REPLYlink written 3.4 years ago by asmariyaz2310
/galaxy/api/histories/[history id]/contents/[API ID]/display

should work for BAM files.

I'm certainly not the best at Apache proxying - have you had a look at:

https://wiki.galaxyproject.org/Admin/Config/ApacheProxy

Specifically the sections on the rewrite rules ("Serving Galaxy at a sub directory (such as /galaxy") and XSendFile ("Sending files using Apache")? 

ADD REPLYlink written 3.4 years ago by carlfeberhard390

Is there a way to access the .bai file associated with .bam file? The metadata has something of this sort:

"meta_files": [{"file_type": "bam_index"}]

when I access 

http://kcodevel.broadinstitute.org:8088/galaxy/api/histories/[history id]/contents/[API ID]/display

but have no clue how the file itself?

 

 

ADD REPLYlink written 3.4 years ago by asmariyaz2310

It's not documented and may change, but for now the one I know of is:

localhost:8080/galaxy/dataset/get_metadata_file?hda_id=[API_ID]&metadata_name=bam_index

 

And, since we seem to be losing horizontal space for this conversation, I'll mention that the galaxy-dev mailing list is also a good resource for this kind of discussion:

https://lists.galaxyproject.org/listinfo/galaxy-dev

ADD REPLYlink modified 3.3 years ago • written 3.3 years ago by carlfeberhard390

I used the URL for the bam index file, but returns the following: 

Request Range Not Satisfiable

The Range requested is not available. 
Invalid reference dataset id: get_metadata_file.

I will soon write to the galaxy-dev list too!

ADD REPLYlink written 3.3 years ago by asmariyaz2310

I solved this issue by serving bam files through apache's modxsendfile (as suggested by carlfeberhard). IGV needs a Partial 206 content returned and Galaxy's server wasn't able to do it. 

ADD REPLYlink modified 3.3 years ago • written 3.3 years ago by asmariyaz2310
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 173 users visited in the last hour