Question: Can't preview/download a few files in history (but can from shared data)
0
gravatar for Devon Ryan
3.3 years ago by
Devon Ryan1.9k
Germany
Devon Ryan1.9k wrote:

We're currently running into a very strange issue wherein a certain group of files (linked into our local Galaxy instance and accessible via "shared data") can't be previewed or downloaded once they're imported into a history. If one tries to directly download these datasets via "shared data" then everything works and the resulting tar.gz files contain everything they should (and can be opened and the file contents match what's on the hard drive). To test if this might be some group/role permission problem (though I still personally have the same problem when I login as an administrator) I've tried making a few of the files public, but that also doesn't seem to resolve the issue.

So, in short, Galaxy seems to be able to see and access the files without issue, but something unexpected is happening if and only if one tries to interact with the files once they're in the history. I'd be more than happy to play around with the python code if needed to try to track down the source of this, but I'm hoping that someone has other suggestions :)

Before someone asks, no, this weird issue doesn't generally occur with files that we've linked in, just with a single recent dataset.

galaxy docker • 1.1k views
ADD COMMENTlink modified 3.3 years ago by Martin Čech ♦♦ 4.9k • written 3.3 years ago by Devon Ryan1.9k

Devon, does this only happen to `tar.gz` files? Can you reset metadata?

ADD REPLYlink written 3.3 years ago by Bjoern Gruening5.1k

No, those are fine. This is happening with BAM files and PDFs. At least with the "data libraries beta", you get a tar.gz (or tar.bz2, or zip) file if you download one or more datasets, which makes sense. Those files are all fine then, but if you import the same files into a history then you can't preview or download the file from the history.

ADD REPLYlink written 3.3 years ago by Devon Ryan1.9k

You mentioned "data libraries beta" is this specfic to the beta version?

ADD REPLYlink written 3.3 years ago by Bjoern Gruening5.1k

No clue, the beta is so much faster than the plain "data libraries" method that I tend to use only it. In any effect, that's the part that works properly...it's only once something is in the history that the problem arises (though I could check if it makes any difference if one imports into history via "data libraries" vs. "data libraries beta". I'll give that a whirl when I'm back in.
 

ADD REPLYlink written 3.3 years ago by Devon Ryan1.9k
1
gravatar for Martin Čech
3.3 years ago by
Martin Čech ♦♦ 4.9k
United States
Martin Čech ♦♦ 4.9k wrote:

I tried to replicate this on my local instance (latest dev) and after an hour of trying importing multiple things in multiple ways (including linking files) I was always able to download and previes files in history. Could you please elaborate on how you achieve this state? Thanks.

ADD COMMENTlink written 3.3 years ago by Martin Čech ♦♦ 4.9k

I wish I knew what was somehow different about this particular dataset. Every other dataset that we import into Galaxy from the file system by linking in (rather than copying in) presents no issues when imported into a history, it's only this particular one. I've also tried simply deleting it from Galaxy and reimporting (Admin -> Data Libraries -> click on the appropriate group and add a folder -> Add datasets -> set "Upload option" to "upload files from filesystem paths" and "Copy data into Galaxy?" to "Link to files without copying into Galaxy") and that reproduces the issue, though again only with this one particular dataset. This method works perfectly fine with the large number of other files/datasets we've imported in this method, just not this particular one. I should also note that I was able to import a different dataset yesterday using the same method and it's able to be viewed/downloaded after importing into the history, so this doesn't seem to be an issue wherein we just have a corrupted Galaxy file somewhere.

ADD REPLYlink written 3.3 years ago by Devon Ryan1.9k

How big is the file? Is it in some network-mount location? Does it have some obscure format?

ADD REPLYlink written 3.3 years ago by Martin Čech ♦♦ 4.9k

They were a mixture of BAM and TXT files (e.g., from featureCounts). Everything is accessed via a network mount, but so is everything else that's available to Galaxy (and they're all coming from the same utility storage server).

It seems that this was due to some sort of obscure permissions issue. I ended up doing a "chmod -R o+r *" and then reimported everything. That solved the problem, though why Galaxy could only partly access the files previously is beyond me. So I would consider this issue functionally solved.

ADD REPLYlink modified 3.2 years ago • written 3.2 years ago by Devon Ryan1.9k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 173 users visited in the last hour