Tool.Xml "&>" Console Redirection Within Command-Tag

Question: Tool.Xml "&>" Console Redirection Within Command-Tag

7.1 years ago by

Appelt, Uwe • 70 wrote:

Dear all, that has probably been asked 10e6 times, however, I don't find any comments on how to properly escape special characters in a toolxy.xml. What I want to achieve is the following: <command interpreter="bash">$myScript &> $logFile</command> where myScript is dynamically created (as part of the configfiles- section) and the logFile is where stderr and stdout should go to. Now the problem is obviously the "&"-character, because it causes the following error-msg on reloading the toolxy.xml: ExpatError: not well-formed (invalid token): line 5, column 50 where the 5/50 refers to the "&"-character. By the way: redirecting either stdout or stderr by specifying just ">" (instead of "&>") works fine. I tried a lot of different notations in the meanwhile, but neither something like "\&>" nor "\&>" works. Any suggestions? Many thanks in advance and Cheers! Uwe

• 1.3k views

ADD COMMENT • link •

modified 7.1 years ago by Peter Cock • 1.4k • written 7.1 years ago by Appelt, Uwe • 70

7.1 years ago by

Peter Cock • 1.4k

European Union

Peter Cock • 1.4k wrote:

On Mon, Oct 24, 2011 at 9:01 AM, Appelt, Uwe Don't you need to escape both the & and the > characters here? i.e. &> should work. Peter

ADD COMMENT • link written 7.1 years ago by Peter Cock • 1.4k

Hi Peter, thanks a lot for your quick reply. However "&>" doesn't seem to work as well. Also, I do have to admit that two different "doesn't work" exist. Notations that are not accepted during parsing of toolxy.xml: &> \&> \\&> Notations that are accepted, but don't produce any content written to "$logFile": &> &> &> Any other suggestions? Cheers, Uwe Von: Peter Cock [mailto:p.j.a.cock@googlemail.com] Gesendet: Montag, 24. Oktober 2011 10:06 An: Appelt, Uwe Cc: galaxy-user@lists.bx.psu.edu Betreff: Re: [galaxy-user] tool.xml "&>" console redirection within command-tag On Mon, Oct 24, 2011 at 9:01 AM, Appelt, Uwe Don't you need to escape both the & and the > characters here? i.e. &> should work. Peter

ADD REPLY • link written 7.1 years ago by Appelt, Uwe • 70

On Mon, Oct 24, 2011 at 9:42 AM, Appelt, Uwe Curious. Well, at least you seem to have valid XML now (double check this with an XML validator). Does Galaxy capture anything in the "info" box (stdout)? There are some variants like 2>&1 for redirecting stderr to stdout, try combining that with > logfile. My *guess* however is the interpreter="bash" may be to blame, try this after ensuring the script is executable and on the path or fully specified: <command>$myScript &> $logFile</command> Peter

ADD REPLY • link written 7.1 years ago by Peter Cock • 1.4k

Hi again, yes, you're right. The problem source is something else. As soon as I specify one of the following <command interpreter="bash">$hisapWrapperScript &> $logFile</command> <command interpreter="sh">$hisapWrapperScript &> $logFile</command> <command>bash $hisapWrapperScript &> $logFile</command> <command>sh $hisapWrapperScript &> $logFile</command> I get a according debug-line in the galaxy.log, reporting something like: galaxy.jobs.runners.local DEBUG 2011-10-24 11:26:55,360 executing: bash /home/extuser/www/galaxy/galaxy_working/database/job_working_dire ctory/2222/tmpZmFMfg &> /home/extuser/www/galaxy/galaxy_working/databa se/files/005/dataset_5410.dat So the "&>" definitely makes it through XML, Cheetah, etc. -parsers and finally to the cmd-line. However, neither BASH nor SH produce a log-file as intended. For some strange reason (Peter, you're right again) all log-lines appear in the green box. And even more, the green box turns of course red as soon as something is written to stderr. Aaaaaah, but the different redirection syntax works. So to conclude: <command interpreter="bash">$hisapWrapperScript > $logFile 2>&1</command> works like a charme - problem solved! Thanks Peter!!! Cheers, Uwe Von: Peter Cock [mailto:p.j.a.cock@googlemail.com] Gesendet: Montag, 24. Oktober 2011 11:07 An: Appelt, Uwe Cc: galaxy-user@lists.bx.psu.edu Betreff: Re: [galaxy-user] tool.xml "&>" console redirection within command-tag On Mon, Oct 24, 2011 at 9:42 AM, Appelt, Uwe Curious. Well, at least you seem to have valid XML now (double check this with an XML validator). Does Galaxy capture anything in the "info" box (stdout)? There are some variants like 2>&1 for redirecting stderr to stdout, try combining that with > logfile. My *guess* however is the interpreter="bash" may be to blame, try this after ensuring the script is executable and on the path or fully specified: <command>$myScript &> $logFile</command> Peter

ADD REPLY • link written 7.1 years ago by Appelt, Uwe • 70

Hi Uwe, On Mon, Oct 24, 2011 at 10:47 AM, Appelt, Uwe You didn't try the precise combination I meant, $hisapWrapperScript &> $logFile where $hisapWrapperScript is marked as executable and has a Unix hashbang line saying which shell to use, e.g. #!/usr/bin/bash Apologies if I was too vague. To be expected, Galaxy captures any stdout (assuming nothing else captured it first) and puts it in the info field. Yes, it is a long standing Galaxy bug that any stderr out is treated as an error condition, rather than looking at the return code: https://bitbucket.org/galaxy/galaxy-central/issue/325/ Glad you got there in the end :) Peter By the way - this whole discussion would have been better suited to the galaxy-dev list (CC'd) since it is about developing tools for Galaxy rather than Galaxy end users.

ADD REPLY • link written 7.1 years ago by Peter Cock • 1.4k

Well, no, I didn't try to specify neither an "interpreter=" nor the actual shell-binary as part of the command-tag itself, because I'm assembling the shell-script dynamically within a configfile-tag. Thus, I do not see any chance to apply an "executable"-flag to that script that is infact filed to disk by the galaxy-job-scheduler somewhen in the future. So it's not that you were too vague, I just that didn't know how to embed your idea into xml/shell-code. But as you already mentioned, I made it with the slightly more verbose notation and that's great. Thanks again, Peter! Cheers, Uwe Von: Peter Cock [mailto:p.j.a.cock@googlemail.com] Gesendet: Montag, 24. Oktober 2011 12:06 An: Appelt, Uwe Cc: galaxy-user@lists.bx.psu.edu; galaxy-dev@lists.bx.psu.edu Betreff: Re: [galaxy-user] tool.xml "&>" console redirection within command-tag Hi Uwe, On Mon, Oct 24, 2011 at 10:47 AM, Appelt, Uwe You didn't try the precise combination I meant, $hisapWrapperScript &> $logFile where $hisapWrapperScript is marked as executable and has a Unix hashbang line saying which shell to use, e.g. #!/usr/bin/bash Apologies if I was too vague. To be expected, Galaxy captures any stdout (assuming nothing else captured it first) and puts it in the info field. Yes, it is a long standing Galaxy bug that any stderr out is treated as an error condition, rather than looking at the return code: https://bitbucket.org/galaxy/galaxy-central/issue/325/ Glad you got there in the end :) Peter By the way - this whole discussion would have been better suited to the galaxy-dev list (CC'd) since it is about developing tools for Galaxy rather than Galaxy end users.

ADD REPLY • link written 7.1 years ago by Appelt, Uwe • 70

Similar posts • Search »