I am working with a large data set that records all jobs that have run for an ETL tool. There are about 1500 jobs that run per day and a lot of them can run for less than 10 seconds.
Data
What I am trying to do is get some statistics for run times of these run times.
Ideally I wanted a run time for all jobs per day and was running this query:
SELECT
RUNDATE = CAST(runstart AS date),
TOTALRUNTIME = CAST(DATEADD(ms, SUM(CAST(DATEDIFF(ms, '00:00:00', CAST(RUNTIME AS time)) AS BIGINT)), '00:00:00') AS Time)
FROM
"CADIS"."vwSolutionHistory"
GROUP BY
CAST(runstart AS date)
However this returned some weird results.
For example one day showed a total run time of 23:56:27.4833333, and on average we only have jobs running about 11 - 12 hours a day so a total run time of 23 hours didn't make sense. So if anyone has ideas on how to make that work that would be fantastic
Anyway I moved on and grouped the results by name as well so am getting a sum of the total time per job per day
SELECT
RUNDATE = CAST(runstart AS date),
NAME,
TOTALRUNTIME = CONVERT(varchar(50), DATEADD(ms, SUM(DATEDIFF(ms, '00:00:00.000', RUNTIME)), '00:00:00.000'), 108)
FROM
"CADIS"."vwSolutionHistory"
WHERE
TIMESINCELASTENTRY IS NULL
GROUP BY
name, CAST(runstart AS date)
ORDER BY
2, 1
What I would like to do in addition to this is to (1) get the total time of each time per day (per above query) (2) get the maximum time a job took per day (3) Get the median time a job took per day (4) Get the minimum time a job took per day
However, I'm not sure what how to get all of those 4 things in one query.
Related
I have table containing the columns:
1. ClockifyId,
2. StartTime EndTime of every Task
3. Date.
4. Duration
The image is attached below
My goal is to write query to calculate the total duration of every user(which is ClockifyId) of every date.
As One User can have multiple task in one day, I wanted to sum duration of all those task. In short,
I wanted to have total task duration of every user(which is clockifyid) of every date.
enter image description here
There are a couple of details missing here, but this should get you close enough.
The first thing you need to do is convert the StartTime and EndTime to datetime fields if they aren't already. Doing a DATEDIFF on them allows you to figure out per record what the difference in minutes is. You can change the unit of measure as needed.
Once you do that, you use the SUM() which is an aggregate function. This makes it necessary to use the GROUP BY. You then group by which ever fields, in this case the ClockifyId and the StartTime as a date. You have to do it as a date without the datetime or you will get multiple rows back for a single Clockify record in a day.
SELECT
ClockifyId
, SUM(DATEDIFF(mi, CAST(StartTime AS datetime), CAST(EndTime AS datetime))) AS DurationInMinutes
, CAST(StartTime AS date)
FROM TableName
GROUP BY
ClockifyId
, CAST(StartTime AS date)
It's worth noting that this assumes there is always a valid StartTime and EndTime. This will throw some errors if those fields have nulls.
Consider the following data showing the time it takes for engineers to travel to a job. The ChargeBand column shows the various rates charged at different hours of the day. Many engineers can attend one job.
I want to be able to cap the travel time for each engineer to one hour. So even if it takes longer than an hour to travel, the maximum to be paid is one hour travel cost, no more. This is fine and I can do this when the travel time is contained within one charge band using this CASE statement:
CASE
WHEN NumberOfHours > 1.0 AND CallIDStartDate >= '2019-11-01' Then cast((1 * isnull(x.Rate,1)) as
decimal(20,7))
ELSE cast((NumberOfHours * isnull(x.Rate,1)) as decimal(20,7))
END as LabourCost
However the problem is when an hour travel straddles 2 charge bands. This is the issue I am unable to resolve and would like help with. In the first two rows for example on Monday, the engineer travel from 07.46 til 09.41 but his travel cost should be capped at 8.46. So the first 14 mins are charged at 46.62 and the remaining 44 mins at 37.67. How do I do this?
The multiple tuesday entries signify multiple engineers attending. The challenge is also to identify two rows for each engineer as being the 07.46/44 and 08:00 StartTimes and cap the travel charge for one hour as per the Monday example.
I thought to partition the table by day so it was apparent that any StartTime less than the value in the previous row belongs to a different engineer attending the same job but this doesn't help with the calculation itself.
I also thought to use the LEAD() and LAG() functions to calculate the time or charge from the following row value, and perhaps the answer is with them, but I don't know how to apply in the code.
you can try with CTE and LEAD function. This example works if you have two records per employee for each workday
with cte as(
select employeeid, weekday, starttime, finishtime, chargeband, rate firstrate,
lead(starttime,1) over(partition by employeeid, weekday order by starttime) nextstarttime,
lead(finishtime,1) over(partition by employeeid, weekday order by starttime) nextfinishtime,
lead(rate,1) over(partition by employeeid, weekday order by starttime) nextrate
from ratetable)
select employeeid, weekday, case when firsttime>= 1 then firstrate
else firsttime*firstrate+ (case when secondtime>1 then 1-firsttime else secondtime-firsttime end)*nextrate end
from(
select employeeid, weekday, DATEDIFF(second, starttime, finishtime) / 3600.0 firsttime ,firstrate,
DATEDIFF(second, nextstarttime, nextfinishtime) / 3600.0 secondtime,nextrate
from cte where nextstarttime is not null) x
I have a booking system that allows a user to book places for 30 min timeslots (e.g. 1pm, 1:30pm, 2pm etc...)
In the sql database I may have one booking for 10am, a booking for 1pm and two for 2pm. I am trying to display a view of all 30 min booking slots in between a date time range displaying number of current bookings for each slot.
I am not storing each slot explicitly as it's not very efficient. Is there a way to make sql return 'empty' timeslots in a single query? I don't want to create a timeslot array then query each timeslot individually for the total count of bookings.
I am using sql server and asp.net mvc6 as my technology base. Some suggestions on technique would be appreciated.
Thanks.
you need to build a 30 minute interval time range table and do left join with your table to get all time slots
This query generates 30 minute interval times starting from startDate , total 12 time slots are generated, you can modify it accordingly.
declare #startDate datetime ='2014-01-12 12:00:00'
;with cte(value,nextval,n)
as
(
select CONVERT(VARCHAR(5),#startDate,108) as value,
dateadd(minute, datediff(minute, 0, #startDate)+30, 0) as nextval, 1 as n
union all
select CONVERT(VARCHAR(5),cte.nextval,108) as value,
dateadd(minute, datediff(minute, 0, cte.nextval)+30, 0) as nextval, n+1
from cte
where n <=12
)
select * from cte
left join Table1
on cte.nextval = Table1.timeslotvalue
Hi I have several tables in sql server that record data every 30 seconds. obviously after a while that gets a bit bulky, what I'd like to do is select 1 datapoint an hour over the past 24 hours and put that into a seperate tables for fast quesries, so I was thinking once every hour over a 24 hour period, 2 times a day over a week long period, and once a day over a month long period. I have datetime recorded on every data point we have.
I'd like something like this for the once an hour over 24 hours
Select * from MiscTBL Where Date >= (( Currentdatetime - 24 hh )) group by hh
thank you for any advice
Also I'm using sql server management studio it would be great if this were an automatically updating process so I had seperate tables I could use for faster queries of data over shorter pre quantified time periods
Something like this would return 1 sample per hour:
select *
from ActivityLog
where id in
(select max(id) maxID
from ActivityLog
where activityDateTime between #startDateTime and #endDateTime
group by DATEPART(hour, activityDateTime))
I would use this concept to build a stored proc that moved the data around and then I would schedule it to run as often as needed using a SQL Agent job.
I'm writing a report in Crystal Reports XI Developer that runs a stored procedure in a SQL Server 2005 database. The record set returns a summary from log table grouped by Day and Hour.
Currently my query looks something like this:
SELECT
sum(colA) as "Total 1",
day(convert(smalldatetime, convert(float, Timestamp) / 1440 - 1)) as "Date",
datepart(hh, convert(smalldatetime, convert(float, Timestamp) / 1440 - 1)) as "Hour"
`etc...`
GROUP BY
Day, Hour
Ignore the date insanity, I think the system designers were drinking heavily when they worked out how to store their dates.
My problem is this: since there are not always records from each hour of the day, then I end up with gaps, which is understandable, but I'd like Crystal to be able to report on the entire 24 hours regardless of whether there is data or not.
I know I can change this by putting the entire query in a WHILE loop (within the stored procedure) and doing queries on the individual hours, but something inside me says that one query is better than 24.
I'd like to know if there's a way to have Crystal iterate through the hours of the day as opposed to iterating through the rows in the table as it normally does.
Alternatively, is there a way to format the query so that it includes the empty hour rows without killing my database server?
Here's how I solved this problem:
Create a local table in your SQL-Server. Call it "LU_Hours".
This table will have 1 integer field (called "Hours") with 24 rows. Of course, the values would be 1 through 24.
Right join this onto your existing query.
You might need to tweak this to make sure the nulls of empty hours are handled to your satisfaction.
You could use a WITH clause to create the 24 hours, then OUTER JOIN it.
WITH hours AS (
SELECT 1 AS hour
UNION
SELECT 2 AS hour
...
SELECT 24 AS hour
)
SELECT *
FROM hours h
LEFT OUTER JOIN [your table] x ON h.hour=x.datepart(hh, convert(smalldatetime, convert(float, Timestamp) / 1440 - 1))
This SQL would need to added to a Command.
Group as necessary in the SQL or the report.