Receive a new formatted table via aggregation and group by - sql-server

Question

I am having a big issue with a SQL Server query here and I really don't know how to go on with it.
The aim is to receive a table differentiated by different time-intervals going from 00:00 - 00:29 to 23:30 - 23:59. In each of these intervals I want to sum up the total minutes of entities which waited during these times. This information can be received by a starttime, and endtime and the status of the entity, which looks like this:
startdate | finishdate | resourcestatus | id
2015-03-19 10:22:56.8490000 | 2015-03-19 10:32:56.8490000 | 8 | asdsdasdsad
As you see such an entity can have the status 8 from one interval (10:00 - 10:30) into another (10:30 - 11:00).
Until now I solved this by defining 4 groups of time-intervals (finish and start are both in interval, start out of interval but finish in, start in interval but finish out, both start and finish out of interval) these 4 groups are joined by the time-intervals.
I would post the code here but it is too much. My result looks like this. Here are the beginnings of the different parts of the query:
select zr.nr,
zr.interval,
case when outOfInterval.waittime is not null
then SUM(outOfInterval.waittime)
else 0
end
+
case when inInterval.waittime is not null
then SUM(inInterval.waittime)
else 0
end
+
case when startInInterval.waittime is not null
then SUM(startInInterval.waittime)
else 0
end
+
case when finishInInterval.waittime is not null
then sum(finishInInterval.waittime)
else 0
end
as waitingMinutes
From (select 1 as nr,'00:00 - 00:29' as interval, 0 as waittime
union select 2,'00:30 - 00:59', 0
union select 3,'01:00 - 01:29', 0 ...
) zr
left join (select case when CONVERT(time, rt.startedat, 8) < '00:00' and CONVERT(time, rt.finishedat , 8) > '00:30' then '00:00 - 00:29' end as inter, 30 as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) < DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 ))
...
) outOfInterval on outOfInterval.inter = zr.interval
left join (select case when CONVERT(time, rt.startedat, 8) >= '00:00' and CONVERT(time, rt.finishedat , 8) <= '00:30' then '00:00 - 00:29' end as inter, SUM(DATEDIFF(minute, rt.STARTEDAT, rt.FINISHEDAT)) as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) <= DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 )) group by rt.startedat, rt.finishedat
...
) inInterval on inInterval.inter = zr.interval
left join (select case when CONVERT(time, rt.startedat, 8) >= '00:00' and CONVERT(time, rt.startedat, 8) < '00:30'and CONVERT(time, rt.finishedat , 8) >= '00:30' then '00:00 - 00:29' end as inter, (30-DATEPART(minute, rt.STARTEDAT)) as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) <= DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 )) group by rt.startedat, rt.finishedat
...
) startInInterval on startInInterval.inter = zr.interval
left join (select case when CONVERT(time, rt.startedat, 8) >= '00:00' and CONVERT(time,rt.finishedat, 8) < '00:30'and CONVERT(time, rt.STARTEDAT , 8) < '00:00' then '00:00 - 00:29' end as inter, DATEPART(minute, rt.finishedat) as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) <= DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 )) group by rt.startedat, rt.finishedat
...
) finishInInterval on finishInInterval.inter = zr.interval
group by zr.interval, outOfInterval.waittime, inInterval.waittime, startInInterval.waittime, finishInInterval.waittime, zr.nr
And this is the result:
nr | interval | waitingMinutes
1 | 00:00 - 00:29 | 2
2 | 00:30 - 00:59 | 7
...
24 | 11:30 - 11:59 | 8
24 | 11:30 - 11:59 | 51
...
So as you see I have more then one of an interval in my result set.
Do you have any idea how to join the groups to one and sum the minutes up? I am really done with it, every kind of aggregate function did not work for me.
Thanks in advance!
#EDIT: If this was not difficult enough we need a second specification which I forgot to explain: We do not want to see all waitingtimes during the 48 time-intervals but the SUM of all those within a specific date-interval.
Let's say we want to know the summed up minutes from the last month. Then the result set should look like:
nr | interval | waitingMinutes
1 | 00:00 - 00:29 | 0
2 | 00:30 - 00:59 | 0
...
20 | 09:30 - 09:59 | 0
21 | 10:00 - 10:29 | 8
22 | 10:30 - 10:59 | 73
23 | 11:00 - 11:29 | 20
...
The minutes are summed up over all time-intervals of the last month. So for example from 11:00 - 11:29 in the last 30 days the entities waited 20 minutes in total (e.g. yesterday 10 minutes and the day before 10 minutes).
This is so difficult that I have really no clue anymore thinking that this is too much for SQL...
Any suggestions?

Answer 1

I would break you problem down something like this. I may have a few factors slightly off here but hopefully you can see where I'm going with this.
I'll break up the script with commentary, but the actual thing should be run as one single query:
declare #StartDate date
declare #EndDate date
select #StartDate = '20150202',#EndDate='20150508'
I've broken the start and end dates out as parameters as I guess these are subject to change and so this gives us one place to change them rather than many
;With Dates as (
select CAST(#StartDate as datetime) as Day
union all
select DATEADD(day,1,Day) from Dates where Day < #EndDate
)
First CTE, Dates, generates all dates within the period of interest. If you have a calendar table in your database, just select from it instead
, PMNs as (
select ROW_NUMBER() OVER (ORDER BY number)-1 as n
from master..spt_values
)
Next CTE, PMNs is my "poor man's numbers table" - if you have a real numbers table in your database, you can substitute that instead
, DateTimes as (
select
n+1 as nr,
DATEADD(minute,30*n,Day) as StartInclusive,
DATEADD(minute,30*(n+1),Day) as EndExclusive
from
Dates d
inner join
PMNs p
on
p.n between 0 and 47
)
Now, the real fun one. We combine the first two CTEs to generate DateTimes - the complete set of all half hour long periods across all dates of interest
select
nr,
CAST(time,StartInclusive) as StartTime,
CAST(time,EndInclusive) as EndTime,
SUM(
DATEDIFF(minute,
CASE WHEN dt.StartInclusive < rt.StartedAt THEN rt.StartedAt
ELSE dt.StartInclusive END,
CASE WHEN dt.EndExclusive > rt.finishedAt THEN rt.FinishedAt
ELSE dt.EndExclusive END
)) as TotalMinutes
from
DateTimes dt
inner join
T_resourcetracking rt
on
dt.StartInclusive < rt.finishedAt and
rt.startedAt < dt.EndExclusive
group by
nr,
CAST(time,StartInclusive),
CAST(time,EndInclusive)
And finally, we combine the data together. We find where a resourceTracking period overlaps one of our DateTimes periods (note the on clause for the join identifies all overlaps). And then a little manipulation inside some CASE expressions to work out the latter of the two start datetimes and the earlier of the two end datetimes - those are the two values we want to subtract.
If your T_resourcetracking isn't also (as with my DateTimes) computing intervals with a semi-open interval (inclusive start time, exclusive end time) you probably want to make some adjustments so that it does seem to be.

Answer 2

The idea is producing all 48 intervals with TALLY using CTE and joining to your data so that 2 intervals intersect. They intersect if any of vertice is between other vertices:
a-----------------b
c------------------------d
a-----------------b
c-----------------d
a------------------b
c----d
a------------------b
c----------d
The last select is just grouping and correct calculation depending on case.
DECLARE #t TABLE
(
sd DATETIME ,
ed DATETIME ,
st INT
)
INSERT INTO #t
VALUES ( '2015-03-19 10:31:56', '2015-03-19 10:42:56', 8 ),
( '2015-03-19 10:25:56', '2015-03-19 10:35:56', 8 ),
( '2015-03-19 10:31:56', '2015-03-19 11:10:56', 8 ),
( '2015-03-19 10:25:56', '2015-03-19 11:10:56', 8 );
WITH cte
AS ( SELECT DATEADD(mi,
30 * ( -1
+ ROW_NUMBER() OVER ( ORDER BY ( SELECT
1
) ) ),
CAST('00:00:00' AS TIME)) sp ,
DATEADD(mi,
-1 + 30
* ROW_NUMBER() OVER ( ORDER BY ( SELECT
1
) ),
CAST('00:00:00' AS TIME)) ep
FROM ( VALUES ( 1), ( 1), ( 1), ( 1), ( 1), ( 1), ( 1),
( 1) ) t1 ( n )
CROSS JOIN ( VALUES ( 1), ( 1), ( 1), ( 1),
( 1), ( 1) ) t2 ( n )
)
SELECT sp, ep,
SUM(CASE WHEN CAST(t.sd AS TIME) < c.sp
AND CAST (t.ed AS TIME) > c.ep THEN DATEDIFF(mi, sp, ep)
WHEN CAST(t.sd AS TIME) BETWEEN c.sp AND c.ep
AND CAST(t.ed AS TIME) BETWEEN c.sp AND c.ep
THEN DATEDIFF(mi, CAST(sd AS TIME), CAST(ed AS TIME))
WHEN CAST(t.sd AS TIME) BETWEEN c.sp AND c.ep
THEN DATEDIFF(mi, CAST(sd AS TIME), ep)
ELSE DATEDIFF(mi, sp, CAST(ed AS TIME))
END) AS Mi
FROM cte c
JOIN #t t ON CAST(t.sd AS TIME) BETWEEN c.sp AND c.ep
OR CAST(t.ed AS TIME) BETWEEN c.sp AND c.ep
OR c.sp BETWEEN CAST(t.sd AS TIME) AND CAST(t.ed AS TIME)
OR c.ep BETWEEN CAST(t.sd AS TIME) AND CAST(t.ed AS TIME)
GROUP BY sp, ep
Output:
sp ep Mi
10:00:00.0000000 10:29:00.0000000 8
10:30:00.0000000 10:59:00.0000000 73
11:00:00.0000000 11:29:00.0000000 20
Change JOIN to LEFT JOIN in order to get all intervals.
You should tweak this to get 0s using ISNULL on SUM. Also this considers only one day.

Answer 3

Please try this solution. You can use it even if the finishdate is on an other day than the startdate.
;with event_time as (
/*this is the input*/
select 1 id, convert(datetime,'2015-05-11 23:11') startdate, convert(datetime,'2015-05-12 00:15') finishdate
)
, event_time_convert as (
/*convert the input to calculation*/
select i.id, convert(time,i.startdate) startdate, DATEDIFF(MINUTE, i.startdate, i.finishdate) time_until_end
from event_time i
)
, intervall as (
/*create the intervall groups*/
select 1 id, CONVERT(time,'00:00') startdate, CONVERT(time,'00:29') finishdate
union all
select cs.id+1 id, DATEADD(minute,30,cs.startdate) startdate, DATEADD(minute,30,cs.finishdate) finishdate
from intervall cs
where cs.id<48
)
, event_time_in_intervall as (
/*calculate the waiting minutes in intervall*/
select i.id
, cs.id intervall_id
, case when DATEDIFF(minute,i.startdate, cs.finishdate) > i.time_until_end then i.time_until_end else DATEDIFF(minute,i.startdate, cs.finishdate) end time_in_intervall
, case when DATEDIFF(minute,i.startdate, cs.finishdate) > i.time_until_end then null else DATEADD(minute,1,cs.finishdate) end new_startdate
, case when DATEDIFF(minute,i.startdate, cs.finishdate) > i.time_until_end then 0 else i.time_until_end - DATEDIFF(minute,i.startdate, cs.finishdate)+1 end new_time_until_end
from event_time_convert i
join intervall cs on i.startdate between cs.startdate and cs.finishdate /*this is the first intervall*/
union all
select i.id
, cs.id intervall_id
, case when DATEDIFF(minute,i.new_startdate, cs.finishdate) > i.new_time_until_end then i.new_time_until_end else DATEDIFF(minute,i.new_startdate, cs.finishdate)+1 end time_in_intervall
, case when DATEDIFF(minute,i.new_startdate, cs.finishdate) > i.new_time_until_end then null else DATEADD(minute,1,cs.finishdate) end new_startdate
, case when DATEDIFF(minute,i.new_startdate, cs.finishdate) > i.new_time_until_end then 0 else i.new_time_until_end - DATEDIFF(minute,i.new_startdate, cs.finishdate)+1 end new_time_until_end
from event_time_in_intervall i
join intervall cs on i.new_startdate between cs.startdate and cs.finishdate
where i.new_time_until_end>0 /*if there is remaining time, I calculate with a recursion*/
)
/*the result*/
select i.id, CONVERT(varchar(5),i.startdate) + ' - ' + CONVERT(varchar(5), i.finishdate) intervall, s.sum_time_in_intervall waitingMinutes
from (
select i.intervall_id, SUM(i.time_in_intervall) sum_time_in_intervall
from event_time_in_intervall i
group by i.intervall_id
) s
join intervall i on s.intervall_id = i.id