Question: Galaxy API file upload times out
1
gravatar for pstew
13 months ago by
pstew40
pstew40 wrote:

We have a Python script that uses the Galaxy API to upload files to a user's most recently used history on one of our institution's Galaxy instances. It has been working mostly fine for uploading from machines on our institution's network (it timed out on us one time), but it times out for our collaborators (outside of our institution) every time. They are able to successfully run a simple API request with our Galaxy instance, so to me this implies that there is an issue with the file upload. I was looking through an API tutorial here and found this in the step 6 script:

# NOTE: it may happen that the HDA will error! If that's the case - this will loop # forever. You could press 'ctrl+c' to stop this script - but it's a better # idea to add more code to handle the possibility. We'll be optimistic and # (for simplicity) assume everything will work.

Is there something broken with uploading files through the API? Should we add some lines to our Python script to wait a few seconds for the upload to time out and then retry the upload? These are small text files (kb to mb range), so I believe we have given them adequate time to upload.

The idea behind this script was to make it a Galaxy tool to allow users to transfer files from our collaborator's Galaxy instance to our Galaxy instance. Is there a better way to do this besides manually clicking on the download link for data in one Galaxy instance and then pasting that link into the upload file interface in another Galaxy instance?

api • 306 views
ADD COMMENTlink written 13 months ago by pstew40
2
gravatar for Martin Čech
13 months ago by
Martin Čech ♦♦ 4.7k
United States
Martin Čech ♦♦ 4.7k wrote:

Given it works within your institution and times out for externals I would suspect network problem as the culprit.

Is this python script already used as a Galaxy tool (i.e. are the external users executing a custom Galaxy tool to upload to the internal Galaxy) or is this a standalone script? Does your internal Galaxy have proxy server? What timeouts settings does it have if so? Do . you see any errors in the Galaxy (or proxy) log?

ADD COMMENTlink written 13 months ago by Martin Čech ♦♦ 4.7k

Thanks for writing. No proxy. It's being used as a standalone script for testing prior to being incorporated into a Galaxy tool. A friend from our institution tried the script outside of our institution's network and it works fine. We've also tried uploading different files and they work just fine.The script just hangs at the upload_file() step for our collaborator. The upload_file() step did time out inexplicably one time for us on our institution's network, but we are unable to reproduce this.

ADD REPLYlink written 13 months ago by pstew40

To me this seems more and more as a network problem on your collaborator's side. I can hardly imagine a situation where Galaxy would single out one user and fail their upload jobs.

You can try sharing your script and maybe we can spot something fishy together but given you are not able to reproduce my bet is on network.

ADD REPLYlink written 13 months ago by Martin Čech ♦♦ 4.7k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 87 users visited in the last hour