Uploading file to local Galaxy fails with "Failed: Bad Request (400)

Question: Uploading file to local Galaxy fails with "Failed: Bad Request (400)

14 months ago by

rltorgerson • 40 wrote:

Good evening,

I have recently configured a local instance of the Galaxy 17.01 software on a CentOS 6.9 server, and have got the service running enough that I can login as the administrative user I've defined and install software from a remote Tool Shed. However, whenever I attempt to upload a .bed file through the web GUI, it appears to be uploading fine and then ends with the status "Failed: Bad Request (400)". I've tried a number of different files and types including those from examples online, all of which give me the same status. The files are only a few KBs in size, so are by no means too large to upload through the web GUI, and in fact, I can upload them in the usegalaxy.org web GUI without issue.

I assume that I must have a misconfiguration somewhere that I need to resolve, but strangely, nothing is showing up in the galaxy.log file beyond the HTTP POST and the error code 400. There is no additional information to go on, so I'm at a loss where to look.

Any help that anyone can provide to help me debug this issue would be greatly appreciated!

Please let me know what other information I can provide to help in this process as well.

Thank you,

Bob

upload 17.01 galaxy local • 538 views

ADD COMMENT • link •

modified 14 months ago • written 14 months ago by rltorgerson • 40

Hello,

For the 400 error, this usually indicates a web browser issue. Chrome, Firefox, and Safari are recommended. Try clearing your browser cache and then uploading again.

Samtools should be installed in order to upload and work with BAM/SAM datasets, so make sure that has been done. Also, you might have problems with certain tool installs with Galaxy version 17.01. Some changes occurred since then with respect to how tool dependencies are managed. Upgrading to the latest Galaxy version is strongly recommended.

Release notes: https://docs.galaxyproject.org/en/master/releases/index.html

Let us know if this does not resolve the problem,

Thanks, Jen, Galaxy team

ADD REPLY • link written 14 months ago by Jennifer Hillman Jackson ♦ 25k

Thank you for your response Jennifer! I have installed the 17.05 branch of the Galaxy Git repository now with a fresh install and am using Google Chrome as my browser, which has been updated to the latest version available. I've cleared my cache and downloaded the package samtools version 0.1.19 from the Tool Shed. I am getting exactly the same results, it says that I have made a Bad Request (400) and no additional information about why it won't accept any uploaded data.

Please forgive me as I'm brand new to Galaxy and don't really know all of the "knobs" to turn in the configuration file to get more verbose logging or why this error would be happening as I have confirmed I can write to the directory that it should be placing user files. Is there a default set of tools that everyone should install as an administrator before trying to upload files?

Additionally, I can see that after I have started an upload, a new file appears in database/tmp with the correct number of bytes, current timestamp, and correct data inside. So it appears that it is properly writing the data to the temporary file location, but is failing to transfer that data into the user files?

Thank you in advance for your help!

Bob

ADD REPLY • link written 14 months ago by rltorgerson • 40

The Samtools package is not enough, the actual tools need to be installed (the package that contains the repository dependencies would be installed when the suite is installed, example suite_samtools_1_2.

However, the most current version of Samtools is recommended (version 2.0.1). There have been changes to the way dependencies are installed (through conda) and various bug fixes, allowing Galaxy to make use of the tools without other admin manipulations (specifically, adding Samtools to your $PATH).

Samtools 2.0.1 has not yet been bundled into a suite (but it is planned for near term). However, the individual tools can be installed and are needed for upload of BAM/SAM data and any manipulations that make use of those datatypes. Perhaps give this a try and test load a BAM/SAM dataset?

Also, to clarify, are other datatypes are still failing upload now when using Galaxy 17.05? If so, let us know. Meanwhile, I will recruit some help from our development team for further troubleshooting.

ADD REPLY • link modified 14 months ago • written 14 months ago by Jennifer Hillman Jackson ♦ 25k

I've got suite_samtools_1_2 installed on my server as well now, but I'm still having no luck uploading the bed file. I've also tried uploading a .fasta file but get exactly the same result. Things show up in database/tmp but I get the Bad Request (400) error.

Here are the files that I have been trying to load into my instance just for reference: http://bx.psu.edu/~clements/Events/GMOD2013/m.vannielli.TFBs.bed http://bx.psu.edu/~clements/Events/GMOD2013/m.vannielli.sequence.fasta http://bx.psu.edu/~clements/Events/GMOD2013/m.vannielli.wholegene.bed

Do you have any files that you think should work that you could provide to me for testing purposes to confirm that this is a more general problem in my instance than perhaps a few bad files?

Thanks,

Bob

ADD REPLY • link written 14 months ago by rltorgerson • 40

Ok, I see the problem. Entering multiple URLs with a direct copy/paste in a single upload query is presenting a problem with the error you state. Exactly why for this case is not clear yet (more testing Monday).

How the problem manifests:

Multiple URLs entered into a single Upload > Paste/Fetch data entry box with a copy/paste
Green datasets result
These are noted as having "emtpy" content
Message in the dataset comments ends with HTTP Error 400: Bad Request

Samtools is still needed to handle BAM/SAM data uploads (and other manipulations), but that is a secondary troubleshooting item.

ADD REPLY • link written 14 months ago by Jennifer Hillman Jackson ♦ 25k

Have you configured a proxy server by any chance? That could complicate uploading. Otherwise the upload should just work out of the box.

ADD REPLY • link written 14 months ago by Martin Čech ♦♦ 4.9k

Greetings Martin,

I'm going to directly to the port 8080 site that is being run via the Python paster script, so it should be as default a setup as possible. I plan to go through the Apache proxy configuration in the future, but have not done so yet.

Thanks,

Bob

ADD REPLY • link written 14 months ago by rltorgerson • 40

14 months ago by

rltorgerson • 40

rltorgerson • 40 wrote:

I finally got this figured out this afternoon!

It turns out that it is critically important not to create your own tool_config_file in the galaxy.ini without including the data available in the config/tool_conf.xml.sample file. I did not realize that without the functionality that is listed in this sample file that you would not be able to upload files at all, which is the case if you don't include the upload file tool configuration in that XML file at the very least. This might be something to document in the Galaxy admin guide for future iterations or to just have the upload tool pulled in by default. I can't imagine a situation where a user would want to not be able to upload files to their instance when the upload button is displayed on the tool bar of the Galaxy instance.

Additionally, some sort of error message on startup stating that the user is missing critical functionality or an error on failed upload beyond just "Failed: Bad Request (400)" would really be great for helping debug this sort of issue and expedite future admin's ability to diagnose and repair their instance.

To fix my instance, all I had to do was run this command from the working directory of Galaxy on my server: cp config/tool_conf.xml.sample config/tool_conf.xml and then add the following line in my galaxy.ini before restarting the Galaxy service:

tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml

My original tool_config_file was the config/shed_tool_conf.xml so I was able to have both the default tools that come installed with the instance and the tools that I had downloaded in trying to get this to work.

Thank you for all of your help in this matter!

Bob

ADD COMMENT • link written 14 months ago by rltorgerson • 40

Thanks Bob for posting back. The admin training here (https://github.com/galaxyproject/dagobah-training) covers how to set up the tool_conf.xml configuration. Detailed warnings about missing core functionality (upload is just one) is a good idea. If you want to submit a pull-request for that, it would be helpful. Or you can create an issue for the enhancement for others to consider taking on as a project. You decide. Either would be made against the galaxy repo here: https://github.com/galaxyproject/galaxy/

ADD REPLY • link written 14 months ago by Jennifer Hillman Jackson ♦ 25k

14 months ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

The issue is an Upload form data entry usage issue.

The error comes up when data is copy/pasted and "spaces" separate the URLs instead of an actual "return". This can be difficult to detect since both show up as whitespace. Aysam from our team figured it out and is making a change (to be deployed next update) that will allow any whitespace to be a recognized delimiter to make the tool friendlier.

Do this to upload URLs in batch:

Copy the URLs
Paste these into the upload tool's Paste/Fetch box (multiple URLs Ok)
Go through each line and clean up the whitespace. Specifically, at the start of each line use delete until the two URLs are joined, then use "return" to separate the lines.

Please give this a try! Thanks, Jen Galaxy team

ADD COMMENT • link modified 14 months ago • written 14 months ago by Jennifer Hillman Jackson ♦ 25k

Hey Jennifer,

Sorry for the late reply to this, but unfortunately, while the URLs not having proper return carriages may have been an additional problem, I can't even upload a single file at a time, or choose a local file and upload it. I can't even upload GIFs or PDFs like I can on usegalaxy.org so I believe that there must be a misconfiguration in my galaxy.ini file, but I'm not sure what needs to be changed. The permissions are correct enough to allow for the file to be uploaded to the database/tmp directory, but is not being placed into a "final" location for my Galaxy history to be able to use it.

Do you have any suggestions about where I might look in my galaxy.ini to set where these files are ending up? I have set the file_path and new_file_path directories and both have the correct permissions for the UID running the Galaxy instance, and since things are showing up in the new_file_path correctly, I'm not sure why it wouldn't be able to use the file_path?

Thank you for all of your help!

Bob

ADD REPLY • link written 14 months ago by rltorgerson • 40

I am still unable to upload files to our instance of Galaxy and I've tried moving to a CentOS 7 VM, changing the location of the file_path and new_file_path variables in the galaxy.ini, going through the Apache proxy configuration and trying to upload, and simply trying to upload through the port 8080 Python paster server. The permissions are correct on the directory that the Galaxy user is making use of, and I have even gone so far as to make the directories that Galaxy uses be world writable to ensure that this isn't a permission's issue. I'm at a loss for where to look next and could really use some help in debugging this issue.

I don't know if I've just missed some critical portion of the configuration that I'm not seeing or what is going on, but I'd be happy to send my galaxy.ini file or log files if that would help.

Thank you,

Bob

ADD REPLY • link modified 14 months ago • written 14 months ago by rltorgerson • 40

Hi Bob, I am sorry you are having such a hard time with this. :/

In case you did not try that yet, you can set debug = True in the config and that might give you more info in the logs.

Also does this error happen when you just paste some text in the textbox?

ADD REPLY • link written 14 months ago by Martin Čech ♦♦ 4.9k

Hey Martin,

I just got this figured out and as I expected, it was something I had done to the configuration that I didn't realize was an important step to have the upload functionality available to the instance. I'll post as an answer for anyone who might come across this same issue in the future.

Thanks,

Bob

ADD REPLY • link written 14 months ago by rltorgerson • 40

That would be great - if you post back as an answer, I'll mark it accepted. And thanks Martin for debugging advice! -Jen

ADD REPLY • link modified 14 months ago • written 14 months ago by Jennifer Hillman Jackson ♦ 25k

Please log in to add an answer.

Similar posts • Search »