Question: Fetching Time Statistics from Galaxy Database
0
gravatar for marcoalbuquerque.sfu
3.1 years ago by
Canada
marcoalbuquerque.sfu50 wrote:

Hello BioStars,

I am trying to get some runtime statistics from some jobs I have submitted on a local instance of galaxy. Essentially I want the Job Start Time and the Job End Time as listed on each job information pane. 

I have investigated the database using the tutorial from the GCC2015. I also have used the mappings.py file to try and identify which table I should be obtaining these metrics from.

Currently I am doing the following:

For all the states that belong to a particular job of interest, I find a state title "running" and store the create_time as the start time. then I find the state titled "ok" and store the create_time as the end time.

This may very well be wrong. Times seem to be off by a few milli seconds when compared to the Galaxy Job information pane. I have also noticed that some jobs lack a running state (maybe because they are too short lived?).

Any how, I just want to know how to I obtain those two values, as seen on the Galaxy Job information pane from the galaxy database. (i.e. what create_time or update_time should I be looking for and from which table?)

Thanks,

schema galaxy database time • 644 views
ADD COMMENTlink modified 3.1 years ago by Jennifer Hillman Jackson25k • written 3.1 years ago by marcoalbuquerque.sfu50
0
gravatar for marcoalbuquerque.sfu
3.1 years ago by
Canada
marcoalbuquerque.sfu50 wrote:

I think I have figured this out.

The table : model.JobMetricNumeric.table

I grepped "time" in all python files present in galaxy/lib/job

I found a tool rule_helper.py) which uses this metric runtime_seconds from the table listed above.

I queried this table, and noticed there were a few other metrics that were of use: start_epoch and end_epoch.

I then grepped epoch. and found a core.py file that defined these values for me. I also googled epoch because I didn't know of this method of time representation.

So, in short, to find the start and end times of a job using the database. query model.JobMetricNumeric.table looking for metric_name equal to start_epoch or metric_name equal to end_epoch.

If someone from Galaxy Team can Approve! That would be Great,

Marco Albuquerque

 

ADD COMMENTlink written 3.1 years ago by marcoalbuquerque.sfu50
0
gravatar for Jennifer Hillman Jackson
3.1 years ago by
United States
Jennifer Hillman Jackson25k wrote:

Hello,

I am not as familiar with this part of the code as others from our team, but this is the source for time I also found. Specifically: https://github.com/galaxyproject/galaxy/blob/2b966ca4fae9e7348ac3a17de8d1567ebdacd21d/lib/galaxy/jobs/metrics/instrumenters/core.py

This is where the "Job Runtime (Wall Clock)" time is coming from on a dataset's "info" page. And is probably what you were looking for. If this is wrong, someone else from our team will correct.

Thanks! Jen, Galaxy team

ADD COMMENTlink modified 3.1 years ago • written 3.1 years ago by Jennifer Hillman Jackson25k
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 175 users visited in the last hour