Hello everyone,
I need to download a large amount of files from histories of different users. These files are named the same way so I can easily get their IDs with histories.show_matching_datasets with a regular expression. The code looks like this :
hl = gi.histories.get_histories()
for h in hl:
histID=h['id']
todl = gi.histories.show_matching_datasets(histID,name_filter=".*_thing.vcf")[0]['id']
gi.datasets.download_dataset(todl, file_path="./")
It works fine with VCF files, but when I try to do it with a bed files it doesn't work. I will try to give you all the details.
When I try to do it on the bed file I want to download, in a python terminal I have this error :
| >>> gi.datasets.download_dataset(GAP, file_path="./") | |
| Traceback (most recent call last): | |
| File "<stdin>", line 1, in <module> | |
| File "/usr/local/lib/python2.7/dist-packages/bioblend/galaxy/datasets/__init__.py", line 109, in download_dataset | |
| r.raise_for_status() | |
| File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 840, in raise_for_status | |
| raise HTTPError(http_error_msg, response=self) | |
| requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http://jacob.bct.aphp.fr/galaxydiagnostic/api/histories/1ec857d2695a147a/contents/d665ee5b3d1a2cec/display?to_ext=interval |
If I copy/paste the link in my browser it downloads the file (and the file is OK).
If I look into the uwsgi log I can read HistoryDatasetAssociation error (not accessible):
| 10.163.193.36 - - [06/sept./2016:17:11:46 +0200] "GET /galaxydiagnostic/api/datasets/d665ee5b3d1a2cec?hda_ldda=hda&key=KEY HTTP/1.1" 200 - "-" "python-requests/2.9.1" | |
| [pid: 62931|app: 0|req: 3087/5520] 10.143.10.11 () {44 vars in 733 bytes} [Tue Sep 6 17:11:46 2016] GET /galaxydiagnostic/api/datasets/d665ee5b3d1a2cec?hda_ldda=hda&key=5c6ff32e1cea0d1103b5791fd9dc611d => generated 5066 bytes in 113 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 1) | |
| galaxy.webapps.galaxy.api.datasets ERROR 2016-09-06 17:11:47,128 Error getting display data for dataset (d665ee5b3d1a2cec) from history (1ec857d2695a147a): HistoryDatasetAssociation is not accessible by user | |
| Traceback (most recent call last): | |
| File "lib/galaxy/webapps/galaxy/api/datasets.py", line 293, in display | |
| hda = self.hda_manager.get_accessible( decoded_content_id, trans.user ) | |
| File "lib/galaxy/managers/secured.py", line 33, in get_accessible | |
| return self.error_unless_accessible( item, user, **kwargs ) | |
| File "lib/galaxy/managers/secured.py", line 43, in error_unless_accessible | |
| raise exceptions.ItemAccessibilityException( "%s is not accessible by user" % ( self.model_class.__name__ ) ) | |
| ItemAccessibilityException: HistoryDatasetAssociation is not accessible by user |
If I try to download the file with Galaxy interface and then use "Get Data" to have it in another history, and then try to download it with bioblend, I have the exact same error.
Here is the file :
| chrX 101092448 101092628 - NXF5 | |
| chr12 1017619 1017979 + WNK1 | |
| chr19 10244847 10245027 - DNMT1 | |
| chr19 10246379 10246559 - DNMT1 | |
| chr5 10250057 10250237 + CCT5 | |
| chr19 10250677 10251037 - DNMT1 | |
| chr19 10252679 10252919 - DNMT1 | |
| chr19 10273232 10273503 - DNMT1 |
Can you reproduce this error ?
I just tested this on another instance, in another server, and it works. The instance where it doesn't works is located in a hospital where everything go through a proxy. I have already configurated the proxy in supervisor for uwsgi and handler.
When I try to get the file with wget I have this error :
So I guess something is missing in my configuration... Maybe NGINX needs to be configured as well for the hospital proxy ?
In my nginx config file I have add these lines as suggested here : https://wiki.galaxyproject.org/Admin/Config/nginxProxy to send files with Nginx
Do I need to add something related to the hospital proxy ? What I don't understand is why I have no problem for VCF files, and I have this error for this type of file only.
PS : i tried to remove these lines and restart nginx, it does not change anything