Editing A Workflow - Hiding Intermediate Datasets

Question: Editing A Workflow - Hiding Intermediate Datasets

8.0 years ago by

Peter • 260

Peter • 260 wrote:

Hi all, I am quite quite puzzled about how hiding datasets is intended to work when editing a workflow. When editing a workflow, on the output of any tool (in the main part of the screen) next to the grab point for linking tools there is a "snowflake". This is either greyed out (which means hide the output when the tool is run) or yellow (which means the output is kept in the user's history), and you click on it to toggle this. Additionally, on the right hand side in the "Edit Step Actions" there can be a "Hide Dataset" action. My impression is the two UI mechanisms are intended to control the same thing - but toggling one does not affect the out (at least, not directly). For example, toggling the "snowflake" does seem to alter the right hand side (add/remove the "Hide Dataset"), but only after you save, close, and reload the workflow. Is this a bug? If so, the simplest way to solve this (from a UI perspective) might be to remove the "Hide Dataset" action from the right hand side. On a related point, if I extract a workflow from my history, all the steps have their "snowflakes" greyed out - even the final step in the history. Yet despite this, when I run the new workflow all the output is kept (nothing is hidden). After a little experimenting, my guess is that you have a special case: if all the outputs have greyed out snowflakes, rather than running the tools and hidding all the output, you hide nothing. I would expect a new workflow extracted from the history to have all the output kept (yellow snowflakes), or perhaps intermediate output hidden (greyed out snowflakes) with the output from the final history step shown (yellow). Regards, Peter

• 983 views

ADD COMMENT • link •

modified 8.0 years ago by Dannon Baker ♦ 3.7k • written 8.0 years ago by Peter • 260

8.0 years ago by

Dannon Baker ♦ 3.7k

United States

Dannon Baker ♦ 3.7k wrote:

Some clarification on Hide Dataset Actions and Workflow Outputs. Hide Dataset Actions- This action hides one individual dataset after a job has been completed. Workflow Outputs- The intent of these is to give workflows some notion of 'official' outputs. Clicking the asterisk, or snowflake, on a particular output dataset in the workflow editor flags the dataset as a workflow output- something you'd like to see as an end result of running a workflow. Ultimately, we'd like for this ability to formalize workflow outputs to lead to using workflows as one would use a tool, potentially embedding them within other workflows. Workflow outputs utilize Hide Dataset Actions to do the work of hiding datasets as jobs complete. That's why you see them added to your workflow whenever you save. Upon saving, if any workflow outputs are tagged, all non-tagged output steps have a hide dataset action added. Additionally, workflow outputs override individually set hide-dataset actions, and will remove them as appropriate. You are correct in your guess that if no workflow outputs are flagged, we do assume you'd rather see everything than nothing. This is why an extracted history with nothing flagged as an output, just like if you had an old workflow from before outputs were available, will show everything. I definitely understand some of the confusion here, and I agree that given the functionality available using just the new workflow outputs we should probably just remove manual control over Hide Dataset Actions, and represent this solely through the workflow output asterisks. I'll also soon be adding options for marking all / none as outputs to make working with large workflows easier. -Dannon

ADD COMMENT • link written 8.0 years ago by Dannon Baker ♦ 3.7k

How about doing this check when loading a workflow for editing - at that point if nothing is starred as an output, star everything. That way the editing UI matches the run time behaviour. +1 That sounds promising. Thanks for the clear explanation, Peter

ADD REPLY • link written 8.0 years ago by Peter • 260

Looks like that is done now on galaxy-central (so should be in the next update to galaxy-dist). I'll try to give the new code a go later this week: http://bitbucket.org/galaxy/galaxy-central/changeset/b725b8687aed Peter

ADD REPLY • link written 8.0 years ago by Peter • 260

Similar posts • Search »