Galaxy Architecture Documentation

Question: Galaxy Architecture Documentation

7.2 years ago by

Oren Livne • 130

Oren Livne • 130 wrote:

Dear All, I am a software developer, new to Galaxy. Is there a detailed document about the Galaxy code architecture, API, what runners are, and how to write a runner? I am especially interested in integrating Galaxy with an external scheduler software (I am aware of Galaxy's scheduling capabilities). I'd like to intercept Galaxy workflows and send their object model to my scheduler. I could not find this information on the Galaxy Wiki. Thank you so much in advance, Oren Livne

galaxy • 989 views

ADD COMMENT • link •

modified 7.2 years ago by Dave Clements ♦♦ 2.5k • written 7.2 years ago by Oren Livne • 130

7.2 years ago by

Dave Clements ♦♦ 2.5k

United States

Dave Clements ♦♦ 2.5k wrote:

Hi Oren, This book chapter http://userwww.service.emory.edu/~eafgan/content/Galaxy_eScienceGatewa y.pdf, includes a high-level overview of Galaxy. (Note: that chapter will be copied to the wiki soon.) There is some API documentation. ~/scripts/api/README, ~/scripts/api/workflow_ execute.py and ~/scripts/api/example_watch_folder.py in the distribution http://wiki.g2.bx.psu.edu/News%20Briefs/2010_07_16#Initial_implementat ion_of_the_Galaxy_Web_API http://wiki.g2.bx.psu.edu/News%20Briefs/2011_08_30#API http://wiki.g2.bx.psu.edu/Events/GCC2011 (*Galaxy Deployment and API*presentation) http://wiki.g2.bx.psu.edu/Admin/Sample%20Tracking/Next%20Gen http://wiki.g2.bx.psu.edu/Events/GCC2011/Workflows%20and%20API Sometime in the next week I'll compile all this into a single wiki page. As far as I know there is no "How to write a runner for Galaxy" document. The closest we have is http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster and http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Web%20Application%2 0Scaling Hope this helps, Dave C. -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

ADD COMMENT • link written 7.2 years ago by Dave Clements ♦♦ 2.5k

Dear Dave, This was very useful. Thank you so much. Specifically, I am trying to integrate Galaxy with Condor, which supports DRMAA. Is there an existing Condor job runner implementation, or any useful code for submitting Galaxy jobs to Condor? Thanks! Oren

ADD REPLY • link written 7.2 years ago by Oren Livne • 130

Hi Oren, I had no idea so I went to the Galaxy mailing list archive @ Nabble ( http://gmod.827538.n3.nabble.com/Galaxy-f829901.html) and searched for Condor ( http://gmod.827538.n3.nabble.com/template/NamlServlet.jtp?macro=search _page&node=829901&query=condor). It appears that Ravi Madduri @ Argonne has done some work with this. I was also at a meeting in March with someone from Wisconsin (where I think Condor is currently developed) and they indicated their might be interest in helping Galaxy work with Condor. I/you can pursue this option as well. Dave C. -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

ADD REPLY • link written 7.2 years ago by Dave Clements ♦♦ 2.5k

Hi Dave, That was me! I'm currently talking with the condor folks here in Madison WI. It would be great to get galaxy going with Condor since condor nows forms part of red hat. Besides the existing available docs that you listed below who do you think would be best as a contact for questions regarding this implementation? best, Victor

ADD REPLY • link written 7.2 years ago by Victor Ruotti • 90

Dear Dave, Victor, Is it possible to simply use the drmma job runner with the condor drmma lib.so? Thanks! Oren

ADD REPLY • link written 7.2 years ago by Oren Livne • 130

Oren, I do believe some folks have done this, I am a bit surprised none have replied. But yes, since Condor supports DRMAA, you should be able to point $DRMAA_LIBRARY_PATH at Condor's libdrmaa.so, enable the drmaa runner, and try it out. --nate

ADD REPLY • link written 7.2 years ago by Nate Coraor ♦ 3.2k

Dear All, We have a lot of sequencing data files whose locations and identifiers are managed by our home-grown webapp. We'd like to enable users to single-sign-on from our webapp into Galaxy (we will be using open ID on both systems for that). When the user opens a Galaxy window, we'd like them to be able to access the data files on our system which they have permissions to see. How should we go about that without copying or moving files? Should we make a new data library, or (preferably) create our identifier service and make it a remote data source/library that galaxy can see and that can be browsed like other data libraries? Is there an API we should write our service against, or wrap files with appropriate galaxy meta data, etc.? Thank you so much, Oren

ADD REPLY • link written 7.2 years ago by Oren Livne • 130

Hi Oren, Without knowing the full architecture it's a bit hard to say what the best solution is. However, Galaxy's API does allow for "upload" from locations on the filesystem or from an FTP/HTTP/HTTPS url to a data library, so you should be able to use this to do what you're talking about. Thanks, --nate

ADD REPLY • link written 7.2 years ago by Nate Coraor ♦ 3.2k

Similar posts • Search »