Fetching Time Statistics from Galaxy Database

Question: Fetching Time Statistics from Galaxy Database

3.1 years ago by

Canada

Hello BioStars,

I am trying to get some runtime statistics from some jobs I have submitted on a local instance of galaxy. Essentially I want the Job Start Time and the Job End Time as listed on each job information pane.

I have investigated the database using the tutorial from the GCC2015. I also have used the mappings.py file to try and identify which table I should be obtaining these metrics from.

Currently I am doing the following:

For all the states that belong to a particular job of interest, I find a state title "running" and store the create_time as the start time. then I find the state titled "ok" and store the create_time as the end time.

This may very well be wrong. Times seem to be off by a few milli seconds when compared to the Galaxy Job information pane. I have also noticed that some jobs lack a running state (maybe because they are too short lived?).

Any how, I just want to know how to I obtain those two values, as seen on the Galaxy Job information pane from the galaxy database. (i.e. what create_time or update_time should I be looking for and from which table?)

Thanks,

schema galaxy database time • 644 views

ADD COMMENT • link •

modified 3.1 years ago by Jennifer Hillman Jackson ♦ 25k • written 3.1 years ago by marcoalbuquerque.sfu • 50

3.1 years ago by

marcoalbuquerque.sfu • 50

Canada

marcoalbuquerque.sfu • 50 wrote:

I think I have figured this out.

The table : model.JobMetricNumeric.table

I grepped "time" in all python files present in galaxy/lib/job

I found a tool rule_helper.py) which uses this metric runtime_seconds from the table listed above.

I queried this table, and noticed there were a few other metrics that were of use: start_epoch and end_epoch.

I then grepped epoch. and found a core.py file that defined these values for me. I also googled epoch because I didn't know of this method of time representation.

So, in short, to find the start and end times of a job using the database. query model.JobMetricNumeric.table looking for metric_name equal to start_epoch or metric_name equal to end_epoch.

If someone from Galaxy Team can Approve! That would be Great,

Marco Albuquerque

ADD COMMENT • link written 3.1 years ago by marcoalbuquerque.sfu • 50

3.1 years ago by

Jennifer Hillman Jackson ♦ 25k

United States

Jennifer Hillman Jackson ♦ 25k wrote:

Hello,

I am not as familiar with this part of the code as others from our team, but this is the source for time I also found. Specifically: https://github.com/galaxyproject/galaxy/blob/2b966ca4fae9e7348ac3a17de8d1567ebdacd21d/lib/galaxy/jobs/metrics/instrumenters/core.py

This is where the "Job Runtime (Wall Clock)" time is coming from on a dataset's "info" page. And is probably what you were looking for. If this is wrong, someone else from our team will correct.

Thanks! Jen, Galaxy team

ADD COMMENT • link modified 3.1 years ago • written 3.1 years ago by Jennifer Hillman Jackson ♦ 25k

Please log in to add an answer.

Similar posts • Search »