Latest
Open
RNA-Seq
ChIP-Seq
SNP
Assembly
Forum
Planet
All »
Home
Log In
Question: User Tool Does Not Initialize
1
gravatar for smrupp16
23 months ago by
smrupp16 • 10
United States
smrupp16 • 10 wrote:

I have been trying to get a program I wrote to work in Galaxy, but I can't seem to get it to initialize at all. I don't have any problems running the local Galaxy instance itself, but when I do my program is not available. As far as I can tell, I've followed the instruction on Galaxy's website closely so I'm not sure what is wrong.

Here is my xml config file up to the the tests section (I haven't gotten around to troubleshooting the outputs section yet):

<tool id="AP" name="AlignmentProcessor" version="1.2">

<description>will split a multi-fasta alignment, filter each gene, and convert the output to phylip or axt.</description>
<!-- Define command to be passed to program. -->
<command>python3 $__tool_directory__/AlignmentProcessor/AlignmentProcessor.py $input $output $reference --$outputType -p $percent --$retainStops </command>
<inputs>
    <param format="fasta" name="input" type="data" label="Source file" help="Fasta alignment file"/>
    <param name="reference" type="text" help="Name of reference species as it appears in the alignment"/>
    <!-- Specify phylip or axt. -->
    <conditional name="outputType">
        <param name="filetypeSelector" type="selector" label="Choose format to convert files to.">
            <option value="phylip" selected="True">Phylip format</option>
            <option value="axt">AXT format</option>
        </param>
    </conditional>
    <!-- Hide percentage and retainStops in expandable menu. -->
    <section name="adv" title="Advanced Options" expanded="False">
        <param name="percent" type="float" help="Minimum percent of nucleotides requided to retain filtered sequences."/>
        <param name="retainStops" type="boolean" selected="False" help="Keep sequences which contain premature stop codons."/>
    </section>
</inputs>
<outputs>
    <collection format="outputType" name="output"/>
</outputs>

</tool>

And here is the section definition I added to tool_conf.xml.main:

<toolbox> <section name="AlignmentProcessor" id="AP"> <tool file="tools/AlignmentProcessor/alignmentProcessor.xml"/>

</section>

Any help will be greatly appreciated.
galaxy • 1.3k views
ADD COMMENT • link •
modified 21 months ago • written 23 months ago by smrupp16 • 10
2
gravatar for y.hoogstrate
22 months ago by
y.hoogstrate • 460
Netherlands
y.hoogstrate • 460 wrote:

Hi,

I recommend you to use Planemo for development (pip install planemo).

It has the ability to track down top level XML problems and tool configuriation specific problems with planemo lint tool.xml. For example, it says about line 11: <param name="filetypeSelector" type="selector" label="Choose format to convert files to.">:

The value 'selector' is not an element of the set {'text', 'integer', 'float', 'color', 'boolean', 'genomebuild', 'library_data', 'select', 'data_column', 'hidden', 'baseurl', 'file', 'data', 'drill_down', 'data_collection'}, making it invalid for hosting.

You can confirm whether a tool works by running planemo serve tool.xml, which for the reason above gives the following warning:

galaxy.tools.toolbox.base ERROR 2017-01-11 11:43:59,100 Error reading tool from path: /tmp/a.xml Traceback (most recent call last): File "/usr/local/src/galaxy/lib/galaxy/tools/toolbox/base.py", line 554, in _load_tool_tag_set tool = self.load_tool(os.path.join(tool_path, path), use_cached=False) File "/usr/local/src/galaxy/lib/galaxy/tools/toolbox/base.py", line 736, in load_tool tool = self.create_tool( config_file=config_file, repository_id=repository_id, guid=guid, *kwds ) File "/usr/local/src/galaxy/lib/galaxy/tools/__init__.py", line 165, in create_tool tool = ToolClass( config_file, tool_source, self.app, guid=guid, repository_id=repository_id, *kwds ) File "/usr/local/src/galaxy/lib/galaxy/tools/__init__.py", line 341, in __init__ raise e ValueError: Tool parameter 'filetypeSelector' uses an unknown type 'selector'

If the two steps above go well, you can try to load them into Galaxy for real.

Youri
ADD COMMENT • link written 22 months ago by y.hoogstrate • 460

Thanks! This should help a lot.

ADD REPLY • link written 22 months ago by smrupp16 • 10
1
gravatar for y.hoogstrate
22 months ago by
y.hoogstrate • 460
Netherlands
y.hoogstrate • 460 wrote:

<param name="filetypeSelector" type="select" label="Choose format to convert files to."> Phylip format AXT format </param>

This needs options - https://docs.galaxyproject.org/en/latest/dev/schema.html#tool-inputs-param-option

<conditional name="outputType">
    <param name="filetypeSelector" type="select" label="Choose format to convert files to.">
        <option value="phylip">Phylip format</option>
        <option value="axt">AXT format</option>
    </param>
    <when value="phylip">
        <param name="outputType" type="text" argument="phylip"/>
    </when>
    <when value="axt">
        <param name="outputType" type="text" argument="axt"/>
    </when>
</conditional>

The planemo serve shell gives the following error on top of that:

galaxy.tools.toolbox.base ERROR 2017-02-01 16:53:36,722 Error reading tool from path: /tmp/tool.xml
Traceback (most recent call last):
  File "/home/youri/.local/src/galaxy/lib/galaxy/tools/toolbox/base.py", line 554, in _load_tool_tag_set
    tool = self.load_tool(os.path.join(tool_path, path), use_cached=False)
  File "/home/youri/.local/src/galaxy/lib/galaxy/tools/toolbox/base.py", line 733, in load_tool
    tool = self.create_tool( config_file=config_file, repository_id=repository_id, guid=guid, **kwds )
  File "/home/youri/.local/src/galaxy/lib/galaxy/tools/__init__.py", line 165, in create_tool
    tool = ToolClass( config_file, tool_source, self.app, guid=guid, repository_id=repository_id, **kwds )
  File "/home/youri/.local/src/galaxy/lib/galaxy/tools/__init__.py", line 341, in __init__
    raise e
ValueError: The settings for this field require a 'value' setting and optionally a default value which must be a real number

Your floats params need a value=0.0. I think fixing those is sufficient to get it running. Your test doesn't specify an output, so that's the next step to work on ;)

all best,

Youri
ADD COMMENT • link modified 22 months ago • written 22 months ago by y.hoogstrate • 460

Thanks again! I'll take another look at it.

MY program outputs a directory of files, so I'm not entirely sure what the best way to handle that is (which is why it isn't done yet). That should be the easy part, though.

ADD REPLY • link written 22 months ago by smrupp16 • 10

Good call on initializing the float. I'm not sure why I didn't get that error message. I got it load and look how I want using planemo serve. I can't execute a job, but I'm assuming I'm not supposed to be able to.

Now that it works with planemo serve, I'm in scenario 2, because it still won't load into galaxy itself. I can only guess that it doesn't like what I have for the output section. I've added the list type, but I still don't know if what I have is sufficient or appropriate for a directory.

<outputs>
    <collection name="output" type="list"/>
</outputs>
ADD REPLY • link written 22 months ago by smrupp16 • 10
0
gravatar for Jennifer Hillman Jackson
22 months ago by
Jennifer Hillman Jackson ♦ 25k
United States
Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

For reference, these are the instructions for creating/adding a local tool: https://wiki.galaxyproject.org/Admin/Tools/AddToolTutorial

Has the entry described in section "4. Make Galaxy aware of the new tool:" been created? And Galaxy restarted?

Let us know if you still have problems after reviewing, Jen, Galaxy team
ADD COMMENT • link written 22 months ago by Jennifer Hillman Jackson ♦ 25k

I've rewritten my xml script after diagnosing the errors with planemo. It passes the lint command, but when I run the serve command, I get the following error before galaxy starts:

planemo couldn't find a target test-data directory, you should likely create a test-data directory or pass an explicit path using --test_data.

I've tried moving the test files to the test-data directory and specifying the path from the galaxy directory to the files, and anythign I could think of, but I still can't seem to solve the problem.

ADD REPLY • link written 22 months ago by smrupp16 • 10
1

Choose the paths relative to the test-data directory. For a tool with the following directory structure:

├── hisat2_macros.xml
├── hisat2.xml
├── test-data
│   ├── hisat_input_1_forward.fastq
│   ├── hisat_input_1_reverse.fastq
│   ├── hisat_input_2_forward.fastq
│   ├── hisat_input_2_reverse.fastq
│   ├── hisat_output_1.bam
│   ├── hisat_output_2.bam
│   ├── hisat_output_3.bam
│   ├── phiX.fa
│   └── test_unaligned_reads.fasta

test-data files in this example are:

        <test>
            <param name="input_format_selector" value="fastq" />
            <param name="paired_selector" value="paired" />
            <param name="reference_genome_source" value="history" />
            <param ftype="fasta" name="history_item" value="phiX.fa" />
            <param ftype="fastqsanger" name="reads_f" value="hisat_input_1_forward.fastq" />
            <param ftype="fastqsanger" name="reads_r" value="hisat_input_1_reverse.fastq" />
            <output file="hisat_output_1.bam" ftype="bam" name="output_alignments" lines_diff="2" />
        </test>
        <test>
            <param name="input_format_selector" value="fastq" />
            <param name="paired_selector" value="paired" />
            <param name="reference_genome_source" value="history" />
            <param ftype="fasta" name="history_item" value="phiX.fa" />
            <param name="input_options_selector" value="advanced" />
            <param name="trim_three" value="15" />
            <param name="trim_five" value="15" />
            <param ftype="fastqsanger" name="reads_f" value="hisat_input_2_forward.fastq" />
            <param ftype="fastqsanger" name="reads_r" value="hisat_input_2_reverse.fastq" />
            <output file="hisat_output_2.bam" ftype="bam" name="output_alignments" lines_diff="2" />
        </test>

Try to avoid to use nested directories (test-data/test1/file.blabla). I don't recall exactly when but I've ran into problems with this.

ADD REPLY • link modified 22 months ago • written 22 months ago by y.hoogstrate • 460

Thanks! That was embarrassingly obvious...

ADD REPLY • link written 22 months ago by smrupp16 • 10

I've managed to get planemo serve to start without any errors, but my tool still isn't loading on galaxy. Are there any common issues with this, or is there any good way to diagnose the issue?

ADD REPLY • link written 22 months ago by smrupp16 • 10

I can interpret your question in 2 ways: (1) planemo serve starts, but does not load your tool. This usually means the tool is not correct. What does planemo lint tell you? If you're stuck, please drop the latest version of your tool XML and i'll have a quick look at it. (2) planemo serve starts, loads the tool, but once loading the tool in Galaxy (./run.sh) it does not pop-up. I have not encountered anything like this before. I would expect this is related to your Galaxy configuration and not to the tool. You could try to upload it to the testtoolshed and see if installing it via that route does make it appear.

ADD REPLY • link modified 22 months ago • written 22 months ago by y.hoogstrate • 460

1 is correct. Planemo serve loads without throwing any errors but does not load the tool. I'm sure it's a problem with the control file.

<tool id="AP" name="AlignmentProcessor" version="1.2"> <description>will split a multi-fasta alignment, filter each gene, and convert the output to phylip or axt.</description> <command>python3 $__tool_directory__/AlignmentProcessor/AlignmentProcessor.py $input $output $reference --$outputType -p $percent --$retainStops </command>

<inputs>
    <param format="fasta" name="input" type="data" label="Source file" help="Fasta alignment file"/>
    <param name="reference" type="text" help="Name of reference species as it appears in the alignment"/>
    
    <conditional name="outputType">
        <param name="filetypeSelector" type="select" label="Choose format to convert files to.">
            
            
        </param>
        <when value="phylip">
            <param name="outputType" type="text" argument="phylip"/>
        </when>
        <when value="axt">
            <param name="outputType" type="text" argument="axt"/>
        </when>
    </conditional>
    
    

        <param name="percent" type="float" help="Minimum percent of nucleotides requided to retain filtered sequences."/>
        <param name="retainStops" type="boolean" help="Keep sequences which contain premature stop codons."/>
    

</inputs>

<outputs>
    <collection name="output" type="data"/>
</outputs>

<tests>
    <test>
        
        <param name="input" value="codemlTest.fa"/>
        <param name="reference" value="anoCar2"/>
        <param name="outputType" value="phylip"/>
        
    </test>
    <test>
        
        <param name="input" value="kaksTest.fa"/>
        <param name="reference" value="anoCar2"/>
        <param name="outputType" value="axt"/>
        
    </test>
</tests>

<help>Splits, filters, and converts fasta alignments to phylip or axt.</help>
<citations>
    <citation type="bibtex">@ARTICLE{author = {Shawn Rupp and Melissa Wilson Sayres}, 
                                    title = {AlignmentProcessor}, 
                                    journal = {https://github.com/WilsonSayresLab/AlignmentProcessor}
    }</citation>
</citations>

</tool>

ADD REPLY • link written 22 months ago by smrupp16 • 10

Any thoughts on why this might load with planemo serve but not load in Galaxy?

ADD REPLY • link written 21 months ago by smrupp16 • 10

Did you configure it in ./config/tool_conf.xml with the correct PATH (relaltive to ./tools/, and the XML within there? ) or install via a toolshed?

ADD REPLY • link written 21 months ago by y.hoogstrate • 460
0
gravatar for smrupp16
21 months ago by
smrupp16 • 10
United States
smrupp16 • 10 wrote:

I did. On a hunch, I changed tool_conf.xml.main to tool_conf.xml and that seems to have worked.

Thanks for all the help! Now for the real troubleshooting...
ADD COMMENT • link written 21 months ago by smrupp16 • 10

This is where work starts to become fun ;)

ADD REPLY • link written 21 months ago by y.hoogstrate • 460

I've been trying to troubleshoot this for a while now, but I still can't get the program to execute. It passes planemo lint and loads in Galaxy itself, but after I upload data and hit execute, Galaxy just hangs.

I had been under the impression that Galaxy could not find the scripts called by the primary script, so I merged them into one script, but ran into the same error. I also tried to bypass any problem Galaxy might have with temporary files by running each gene through the program without making any temp files, but that still didn't help. Since I'm not getting an error message at all, I'm not really sure what the issue could be.

ADD REPLY • link written 17 months ago by smrupp16 • 10

Could you do something like:

<command><!--
echo "python3 ${__tool_directory__}/AlignmentProcessor/AlignmentProcessor.py $input $output $reference --$outputType -p $percent --$retainStops" > $output/file.txt
--></command>

and see if this hangs?

ADD REPLY • link written 17 months ago by y.hoogstrate • 460

It looks like that didn't do anything. I ran it without the comments and tried to send the output to my home folder as well, but that still didn't produce output. In the process of trying that out, I did run into an error where Galaxy could not find a tool with the given ID which turned out to be a minor formatting error. After I fixed that, the program doesn't hang anymore, but it just doesn't start now.

This program is a wrapper for three scripts, and I'm wondering if I need an xml file for each one? I did merge them all together at one point and the input and output is the same as the wrapper.

ADD REPLY • link written 17 months ago by smrupp16 • 10

Ooops, those comments should be replaced with CDATA tags (blame me!):

<command><![CDATA[
echo "python3 ${__tool_directory__}/AlignmentProcessor/AlignmentProcessor.py $input $output $reference --$outputType -p $percent --$retainStops" > $output/file.txt
]]></command>

"After I fixed that, the program doesn't hang anymore, but it just doesn't start now." Does the whole Galaxy wrapper not load, or does the command not get executed?

This program is a wrapper for three scripts, and I'm wondering if I need an xml file for each one? I did merge them all together at one point and the input and output is the same as the wrapper. There is no obvious anwer to this. It depends how much these scripts work together and what level of modularity you believe is ideal. If there might be a situation in which you might need to rerun only module 3, then it can be handy to have it as separate tool.

ADD REPLY • link written 17 months ago by y.hoogstrate • 460

Still nothing....

Galaxy loads, both with and without planemo, and I can load data and select commands for my program. When I click on "execute" it turns a slightly darker shade of blue so it does recognize that I clicked it. After that, though, nothing happens.

ADD REPLY • link written 17 months ago by smrupp16 • 10
Please log in to add an answer.

Similar posts • Search »

Content
Help
Access
Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 180 users visited in the last hour