Create SQL Pivot Table Depending On Different Periods of times - sql-server

I want to create pivot table for data of the due values of different customers, and i want to pivot my data to 3 pivoted periods of time Such that the first Column has the total notes of the period from today till 30 days from now in the future and the second one for the values due in the period of ((Now + 30)< Due <60)
and the next one has values of ((Now+ 60) < Due < 90) and the last one has the value due today.
This's My Code which get my raw data:
SELECT [ADD].AccountID
,SUM(convert(money,ADDN.Amount - ISNULL(CollectedValue,0))) AS [Total Rest Amount]
,ADDN.DueDate AS [Due Date]
FROM [Accounting].[AccDocumentDetailsNotes] ADDN
INNER JOIN Accounting.AccDocumentDetails [ADD]
ON ADDN.AccDocumentDetailID = [ADD].ID
INNER JOIN Accounting.AccDocumentHeader ADH
ON ADH.ID = [ADD].AccDocumentHeaderID
INNER JOIN [Accounting].[AccNotesCollectors] ANC
ON ANC.NoteID = ADDN.ID
INNER JOIN Accounting.AccAccounts AA
ON AA.ID = [ADD].AccountID
GROUP BY [ADD].AccountID,ADDN.DueDate,[CodeTypePart],ADDN.Amount,CollectedValue
HAVING [CodeTypePart] = 'NR' AND convert(money,ADDN.Amount - ISNULL(CollectedValue,0)) > 0
And This's a Historical sample from the result:
AccountID Total Rest Amount Due Date
----------- --------------------- -----------------------
25 6800.00 2017-02-23 17:31:09.000
25 1700.00 2017-02-23 17:31:09.000
25 10602.00 2017-05-28 16:28:14.000
27 14500.00 2017-02-28 14:53:57.000
30 120150.00 2017-02-24 00:23:20.000
30 117050.00 2017-02-24 00:23:20.000
33 2000.00 2017-04-04 20:48:51.193
45 39500.00 2017-04-18 20:13:46.000
45 31300.00 2017-04-18 20:13:46.000
45 9000.00 2017-04-18 20:13:46.000
45 32200.00 2017-04-22 16:38:47.803
46 32500.00 2017-02-23 20:14:24.000
46 15910.00 2017-02-23 20:14:24.000
And I want to seems as:

So you need to break down your data into groups by how overdue it is, and then pivot on that. Then to get the total, you can add together all the sub-columns.
select
AccountID,
isnull([90+],0)+isnull( [today 61-90],0)+ isnull( [today 31-60],0)+isnull( [today-30],0) total,
[90+], [today 61-90], [today 31-60], [today-30]
from
(
select AccountId, Amount,
CASE
WHEN datediff(d, duedate, getdate()) <= 30 THEN 'today-30'
when datediff(d, duedate, getdate()) between 31 and 60 then 'today 31-60'
when datediff(d, duedate, getdate()) between 61 and 90 then 'today 61-90'
else '90+'
END as daysoverdue
from #t
) src
pivot
( sum(Amount) for daysoverdue in ([90+], [today 61-90], [today 31-60], [today-30] ))p

Try this:
;with data as (
select
Today = cast(getdate() as date),
Plus30 = dateadd(d, 30, cast(getdate() as date) ),
Plus60 = dateadd(d, 60, cast(getdate() as date) ),
Plus90 = dateadd(d, 90, cast(getdate() as date) ),
EndOfTime = cast('21991231' as date),
t.*
from #t as t
)
select
AccountId,
Total = sum(Amount),
Due0To30 = sum(pvt.Due0To30),
Due31To60 = sum(pvt.Due31To60),
Due61To90 = sum(pvt.Due61To90),
Due91Plus = sum(pvt.Due91Plus)
from data
cross apply (values
(Today, Plus30, Amount, 0, 0, 0),
(Plus30, Plus60, 0, Amount, 0, 0),
(Plus60, Plus90, 0, 0, Amount, 0),
(Plus90, EndOfTime, 0, 0, 0, Amount)
)pvt(StartDate,EndDate,Due0To30, Due31To60, Due61To90, Due91Plus)
where [Due Date] >= pvt.StartDate
and [Due Date] < pvt.EndDate
group by AccountID

Related

Calculate time between startdate and enddate and subtracting days that have no worktime

My goal is to check if an email is answered within 24 hours during workdays. de definition of a workday is if there is time registered in another table. this because we sometimes work on a Saturday or a Sunday or to exclude holidays. I made a view from that table that gives a 1 if the date has worktime or a 0 if there is no worktime registered.
DateWorked
HasWorked
2021-04-01 00:00:00.000
1
2021-04-02 00:00:00.000
1
2021-04-03 00:00:00.000
1
2021-04-04 00:00:00.000
0
2021-04-05 00:00:00.000
1
So for example a few situations:
1. MailIncoming: 2021-04-01 16:30:00, MailAnswering: 2021-04-02 14:00:00
This one is easy, I don't have to subtract anything and the mail is answered within 24 hours.
2. MailIncoming: 2021-04-01 09:30:00, MailAnswering: 2021-04-03 14:00:00
This one is also easy, I don't have to subtract anything and the mail is not answered within 24 hours.
3. MailIncoming: 2021-04-03 12:30:00, MailAnswering: 2021-04-05 10:00:00
There is 1 day where no one has worked, so I need to subtract 1 whole day from the total time, and in that case the email is answered within 24 hours during workdays.
4. MailIncoming: 2021-04-04 11:00:00, MailAnswering: 2021-04-05 18:00:00
The remaining 13 hours from 04 do not count toward the '24 hours during workdays' so the email is answered within 24 during workdays.
Also, there can be multiple dates with zero after each other.
So the outcome I'm looking for is:
MailIncoming
MailAnswering
TotalTime
TotalTimeWithoutDaysNotWorked
2021-04-04 11:00:00.000
2021-04-05 18:00:00.000
31
18
How can I calculate this last column? Or am I approaching this in the wrong way?
The query needs a way to generate calculated dates between MailIncoming and MailAnswering so there can be a LEFT JOIN (or INNER JOIN) to the WorkingDay table. In this case the query uses dbo.fnTally which is known to be a fast and efficient way to generate rows.
tables
drop table if exists #WorkingDay;
go
create table #WorkingDay(
DateWorked Date,
HasNotWorked int);
drop table if exists #MailIncoming;
go
create table #MailIncoming(
MailIncoming DateTime,
MailAnswering DateTime);
insert into #WorkingDay values
('2021-04-01', 0),
('2021-04-02', 0),
('2021-04-03', 0),
('2021-04-04', 1),
('2021-04-05', 0),
('2021-04-06', 0);
insert into #MailIncoming values
('2021-04-01 16:30:00', '2021-04-02 14:00:00'),
('2021-04-01 09:30:00', '2021-04-03 14:00:00'),
('2021-04-03 12:30:00', '2021-04-05 10:00:00'),
('2021-04-04 11:00:00', '2021-04-05 18:00:00');
dbo.fnTally
CREATE FUNCTION [dbo].[fnTally]
/**********************************************************************************************************************
Jeff Moden Script on SSC: https://www.sqlservercentral.com/scripts/create-a-tally-function-fntally
**********************************************************************************************************************/
(#ZeroOrOne BIT, #MaxN BIGINT)
RETURNS TABLE WITH SCHEMABINDING AS
RETURN WITH
H2(N) AS ( SELECT 1
FROM (VALUES
(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
,(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)
)V(N)) --16^2 or 256 rows
, H4(N) AS (SELECT 1 FROM H2 a, H2 b) --16^4 or 65,536 rows
, H8(N) AS (SELECT 1 FROM H4 a, H4 b) --16^8 or 4,294,967,296 rows
SELECT N = 0 WHERE #ZeroOrOne = 0 UNION ALL
SELECT TOP(#MaxN)
N = ROW_NUMBER() OVER (ORDER BY N)
FROM H8
;
query
select mi.MailIncoming, mi.MailAnswering,
avg(datediff(hour, MailIncoming, MailAnswering)) hrs_to_ans,
sum(case when w.HasNotWorked=1 and
v.calc_dt > mi_dt.inc_dt and
v.calc_dt < mi_dt.ans_dt
then -24
when w.HasNotWorked=1
then datediff(hour, dateadd(day, 1, mi_dt.inc_dt), mi.MailIncoming)
else 0 end) hrs_to_sub
from #MailIncoming mi
cross apply (values (cast(MailIncoming as date),
cast(MailAnswering as date))) mi_dt(inc_dt, ans_dt)
cross apply dbo.fnTally(0, datediff(day, mi.MailIncoming, mi.MailAnswering)) fn
cross apply (values (dateadd(day, fn.n, mi_dt.inc_dt))) v(calc_dt)
left join #WorkingDay w on v.calc_dt=w.DateWorked
group by mi.MailIncoming, mi.MailAnswering
order by mi.MailIncoming;
MailIncoming MailAnswering hrs_to_ans hrs_to_sub
2021-04-01 09:30:00.000 2021-04-03 14:00:00.000 53 0
2021-04-01 16:30:00.000 2021-04-02 14:00:00.000 22 0
2021-04-03 12:30:00.000 2021-04-05 10:00:00.000 46 -24
2021-04-04 11:00:00.000 2021-04-05 18:00:00.000 31 -13
I suggest you to use a column HasNotWorked, so the tables are
create table WorkingDay(DateWorked Date, HasNotWorked int);
create table MailIncoming(MailIncoming DateTime, MailAnswering DateTime);
and the rows
insert into WorkingDay values('2021-04-01', 0);
insert into WorkingDay values('2021-04-02', 0);
insert into WorkingDay values('2021-04-03', 0);
insert into WorkingDay values('2021-04-04', 1);
insert into WorkingDay values('2021-04-05', 0);
insert into WorkingDay values('2021-04-06', 0);
insert into MailIncoming values('2021-04-04 11:00:00.000', '2021-04-06 18:00:00.000');
I want calculate the start date. If is in working day, we must consider the hour of the mail, else the first working day with
case when
(select HasNotWorked from WorkingDay where DateWorked = convert(date, MailIncoming)) = 1 then
(select min(DateWorked) from WorkingDay where DateWorked > MailIncoming and HasNotWorked = 0)
else MailIncoming end as startDate
and discard the day that are not working day
((select sum(HasNotWorked) from WorkingDay where DateWorked between convert(date, startDate)
and convert(date, MailAnswering)
) * 24) as numNotWorkingDay
so the query could be
select startDate, MailAnswering, MailIncoming, hour, numNotWorkingDay, hour - numNotWorkingDay hourWitoutWorkingDay
from (
select
MailAnswering, startDate, MailIncoming,
DateDiff("hh", startDate, MailAnswering) hour,
((select sum(HasNotWorked) from WorkingDay where DateWorked between convert(date, startDate)
and convert(date, MailAnswering)
) * 24) as numNotWorkingDay
from (
select *,
case when
(select HasNotWorked from WorkingDay where DateWorked = convert(date, MailIncoming)) = 1 then
(select min(DateWorked) from WorkingDay where DateWorked > MailIncoming and HasNotWorked = 0)
else MailIncoming end as startDate
from MailIncoming) as startCalc
) as calcTable;
sqlfiddle

SQL Server 2014 - Sum Working Hour group by Day/Night Time, Week/Weekend, Regular/Overtime

I have a list of tasks,
for each I have a TaskID, startTime and StopTime as milliseconds from 1-1-1970 and a list of users (#Tasks).
I need to calculate the time spent by each user on the task splitted by day/night time, week or weekend, regular/overtime considering nighttime hours from 10:00 PM to 06:00 AM.
Surely there's a better solution but so far I got this:
IF OBJECT_ID('tempdb..#Tasks') IS NULL
BEGIN
create table #Tasks
(
TaskID nvarchar(50),
DateStart bigint,
DateStop bigint,
Staff nvarchar(100)
)
insert into #Tasks values
('C001',1554181200000,1554190200000,'john,jack'),
('C002',1554202800000,1554212700000,'tom,john'),
('C003',1554228000000,1554246900000,'john,franck'),
('C004',1554613200000,1554626700000,'john')
END
GO
declare
#UserName nvarchar(50)='john',
#DateFrom datetime='2019-04-01',
#DateTo datetime='2019-04-30',
#nStart time='06:00:00',
#nStop time='22:00:00'
select
startday as [Day],
sum([WeekDay]) as [WeekDay],
sum([WeekNight]) as [WeekNight],
sum([WeekendDay]) as [WeekendDay],
sum([WeekendNight]) as [WeekendNight],
sum([TotalMinutes]) as [TotalMinutes],
0 WeekDayOverTime,
0 WeekNightOvertime,
0 WeekendDayOvertime,
0 WeekendNightOvertime,
[UserName]
,timeframe
from
(
select
iif(isWeekend=1,NightMinutes,0) WeekendNight,
iif(isWeekend=0,NightMinutes,0) WeekNight,
iif(isWeekend=1,DayMinutes,0) WeekendDay,
iif(isWeekend=0,DayMinutes,0) [WeekDay],
TotalMinutes,
username,
startday,
timeframe
from
(
select
iif(Before6>0,Before6,0)+ iif(After22>0,After22,0) NightMinutes,
TotalMinutes-iif(Before6>0,Before6,0)- iif(After22>0,After22,0) DayMinutes,
TotalMinutes,
startday,
isWeekend,
UserName,
timeframe
from
(
Select
(t.datestop-t.datestart)/60000 TotalMinutes,
datediff(n,convert(time,DATEADD(SECOND,t.DateStart/1000,'1970-01-01')),#nStart) Before6,
datediff(n,#nStop,convert(time,DATEADD(SECOND,t.DateStop/1000,'1970-01-01'))) After22,
iif((((DATEPART(DW, convert(datetime,DATEADD(SECOND,t.DateStart/1000,'1970-01-01'))) - 1 ) + ##DATEFIRST ) % 7) IN (0,6),1,0) isWeekend,
convert(varchar(10),DATEADD(SECOND,t.DateStart/1000,'1970-01-01'),126) startday,
STUFF(( SELECT distinct ' ' + convert(varchar(5),DATEADD(SECOND,t.DateStart/1000,'1970-01-01'),108)+'-'+convert(varchar(5),DATEADD(SECOND,t.DateStop/1000,'1970-01-01'),108) AS [text()]
FROM #Tasks tt
--WHERE tt.taskID=t.TaskID
FOR XML PATH('') ), 1, 1, '' ) AS [timeframe],
#UserName UserName
FROM #Tasks t
WHERE t.Staff like '%'+#UserName+'%'
and DATEADD(SECOND,t.DateStart/1000,'1970-01-01') between #DateFrom and #DateTo
) z
) zz
) zzz
group by startday,username,timeframe
order by startday
I need now to :
1) group result by day, summing up WeekDay,WeekNight,WeekendDay,WeekendNight and TotalMinutes, and concatenating timeframe so to have for example on 2nd April "05:00-07:30|11:00-13:45|18:00-23:00"
2) Not sum up time between 12:00 and 12:30 (if applicable) since it is lunch time
3) considering that after 8 hours daily it has to be calculated as overtime, I have to split the total minutes between in time and overtime, but depending if overtime is by daytime or nighttime or on the weekend
4) eventually using a holiday table
in other words we should have this:
Day TotalMinutes WeekDay WeekNight WeekendDay WeekendNight WeekDayOverTime WeekNightOvertime WeekendDayOvertime WeekendNightOvertime UserName timeframe
02/04/2019 630 420 60 0 0 45 75 0 0 john 05:00-07:30|11:00-13:45|18:00-23:00
07/04/2019 225 0 0 165 60 0 0 0 0 john 05:00-08:45
because (on 2nd April) we have:
First Task:
60 minutes of Regular NightTime
90 minutes of Regular DayTime
Second Task:
165 minutes of Regular DayTime, but have to count only 135 due to lunch time
Third Task:
240 DayTime
75 NightTime
but since with Task 1 and 2 we sum up 285 minutes, only the first 185 Minutes of Third task are Regular DayTime: the remaining 45 are Overtime DayTime, and the following 75 of NightTime are actually OvertimeNightTime
In this approach the first CTE (properDates) get the Start and Stop Datetimes, then you don't need to repeat that formula over the query.
The second CTE(splittedMinutes) is to get the same data you get in your current approach, except for the first CROSS APPLY, which is splitting the timeframes crossing with lunch time. The second CROSS APPLY gets the number of minutes and isWeekend value.
In the third CTE(qualifiedMinutes) I am using a window partition to get the accumulated minutes and generate the overtimes when applies.
At the end I used a selective SUM to separate weekdays and weekends in the aggregates
;with properDates AS (
SELECT TaskID, DATEADD(SECOND,t.DateStart/1000,'1970-01-01') as DateStart,DATEADD(SECOND,t.DateStop/1000,'1970-01-01') as DateStop, Staff
FROM #Tasks t
WHERE Staff LIKE '%' + #UserName + '%'
), splittedMinutes AS (
select
CAST(p.DateStart AS DATE) as [Day],
TotalMinutes,
SUM(TotalMinutes) OVER (PARTITION BY CAST(p.DateStart AS DATE) ORDER BY b.start) AS cumulate,
TotalMinutes - EarlyMinutes - LateMinutes as DayTime,
EarlyMinutes + LateMinutes as NightTime,
isWeekend,
CONVERT(VARCHAR(5),b.Start,108) + '-' + CONVERT(VARCHAR(5),b.Stop,108) as [timeframe]
from properdates p
cross apply (
select CAST(p.DateStart As TIME) AS Start, #bStart as Stop WHERE CAST(p.DateStart AS TIME) < #bStart and CAST(p.DateStop AS TIME) > #bStart
union
select #bStop as Start, CAST(DateStop AS TIME) AS Stop WHERE CAST(p.DateStop AS TIME) > #bStop and CAST(p.DateStart AS TIME) < #bStop
union
select CAST(p.DateStart AS TIME) AS Start, CAST(p.DateStop AS TIME) AS Stop WHERE NOT (CAST(p.DateStart AS TIME) < #bStart and CAST(p.DateStop AS TIME) > #bStart) AND NOT (CAST(p.DateStop AS TIME) > #bStop and CAST(p.DateStart AS TIME) < #bStop)
) b
cross apply (
select
DATEDIFF(Minute, b.Start, b.Stop) as TotalMinutes,
(DATEDIFF(Minute, CAST(b.Start AS TIME), #nStart) + ABS(DATEDIFF(Minute, CAST(b.Start AS TIME), #nStart))) / 2 as EarlyMinutes,
(DATEDIFF(Minute, #nStop, CAST(b.Stop AS TIME)) + ABS(DATEDIFF(Minute, #nStop, CAST(b.Stop AS TIME)))) / 2 as LateMinutes,
CASE WHEN DATEPART(DW, p.DateStart) IN (1,7) THEN 1 ELSE 0 END AS isWeekend
) c
), qualifiedMinutes As (
SELECT Day, TotalMinutes, RegularDay, RegularNight, OvertimeDay, OvertimeNight, isWeekend, timeframe
FROM splittedMinutes
OUTER APPLY (
SELECT RegularDay = CASE WHEN cumulate <= #maxTime THEN DayTime WHEN DayTime - (cumulate - TotalMinutes - #maxTime) > 0 THEN ABS(cumulate - TotalMinutes - #maxTime) ELSE 0 END
) RD
OUTER APPLY (
SELECT OvertimeDay = DayTime - RegularDay
) OWD
OUTER APPLY (
SELECT RegularNight = CASE WHEN cumulate <= #maxTime THEN NightTime WHEN (cumulate - TotalMinutes - #maxTime + RegularDay) < 0 THEN NightTime + (cumulate - TotalMinutes - #maxTime + RegularDay) ELSE 0 END
) RWN
OUTER APPLY (
SELECT OvertimeNight = NightTime - RegularNight
) OWN
)
SELECT
Day,
#UserName and UserName,
SUM(TotalMinutes) AS TotalMinutes,
SUM(CASE WHEN isWeekend = 0 THEN RegularDay ELSE 0 END) AS WeekDay,
SUM(CASE WHEN isWeekend = 0 THEN RegularNight ELSE 0 END) AS WeekNight,
SUM(CASE WHEN isWeekend = 1 THEN RegularDay ELSE 0 END) AS WeekendDay,
SUM(CASE WHEN isWeekend = 1 THEN RegularNight ELSE 0 END) AS WeekendNight,
SUM(CASE WHEN isWeekend = 0 THEN OvertimeDay ELSE 0 END) AS WeekDayOverTime,
SUM(CASE WHEN isWeekend = 0 THEN OvertimeNight ELSE 0 END) AS WeekNightOvertime,
SUM(CASE WHEN isWeekend = 1 THEN OvertimeDay ELSE 0 END) AS WeekendDayOverTime,
SUM(CASE WHEN isWeekend = 1 THEN OvertimeNight ELSE 0 END) AS WeekendNightOvertime,
STUFF((SELECT '|' + timeframe FROM qualifiedMinutes tt WHERE tt.Day = q.Day ORDER BY timeframe FOR XML PATH('') ), 1, 1, '' ) AS [timeframe]
FROM qualifiedMinutes q
GROUP BY Day

SQL Select Sequential Dates with Additional Lookup Values

I am trying to grab a series of dates and the corresponding values (if any) that exist in my database.
I have two parameters - today (date using getDate()) - and a number of days (integer). For this example, I'm using the value 10 for the days.
Code to get the sequential dates for 10 days after today:
SELECT top 10 DATEADD(DAY, ROW_NUMBER()
OVER (ORDER BY object_id), REPLACE(getDate(),'-','')) as Alldays
FROM sys.all_objects
I now need to look up several values for each day in the sequential days code, which may or may not exist in the time table (we assume 8 hours for all dates, unless otherwise specified). The lookup would be on the field recordDateTime. If no "hours" value exists in the table cap_time for that date, I need to return a default value of 8 as the number of hours. Here's the base query:
SELECT u.FullName as UserName, d2.department,
recordDateTime, ISNULL(hours,8) as hours
FROM cap_time c
left join user u on c.userID = u.userid
left join dept d2 on u.deptID = d2.DeptID
WHERE c.userid = 38 AND u.deptID = 1
My end result for the next 10 days should be something like:
Date (sequential), Department, UserName, Number of Hours
I can accomplish this using TSQL and a temp table, but I'd like to see if this can be done in a single statement. Any help is appreciated.
Without any DDL or sample data it's hard to determine exactly what you need.
I think this will get you pretty close (note my comments):
-- sample data
------------------------------------------------------------------------------------------
DECLARE #table TABLE
(
fullName varchar(10),
department varchar(10),
[hours] tinyint,
somedate date
);
INSERT #table VALUES
('bob', 'sales', 5, getdate()+1),
('Sue', 'marketing', 3, getdate()+2),
('Sue', 'sales', 12, getdate()+4),
('Craig', 'sales', 4, getdate()+8),
('Joe', 'sales', 18, getdate()+9),
('Fred', 'sales', 10, getdate()+10);
--SELECT * FROM #table
;
-- solution
------------------------------------------------------------------------------------------
WITH alldays([day]) AS -- logic to get your dates for a LEFT date table
(
SELECT TOP (10)
CAST(DATEADD
(
DAY,
ROW_NUMBER() OVER (ORDER BY object_id),
getdate()
) AS date)
FROM sys.all_objects
)
SELECT d.[day], t.fullName, department, [hours] = ISNULL([hours], 8)
FROM alldays d
LEFT JOIN #table t ON d.[day] = t.somedate;
Results:
day fullName department hours
---------- ---------- ---------- -----
2017-04-12 bob sales 5
2017-04-13 Sue marketing 3
2017-04-14 NULL NULL 8
2017-04-15 Sue sales 12
2017-04-16 NULL NULL 8
2017-04-17 NULL NULL 8
2017-04-18 NULL NULL 8
2017-04-19 Craig sales 4
2017-04-20 Joe sales 18
2017-04-21 Fred sales 10
Maybe a subquery and the in statement, like:
SELECT u.FullName as UserName, d2.department,
recordDateTime, ISNULL(hours,8) as hours
FROM cap_time c
left join user u on c.userID = u.userid
left join dept d2 on u.deptID = d2.DeptID
WHERE c.userid = 38 AND u.deptID = 1 and recordDateTime in
(SELECT top 10 DATEADD(DAY, ROW_NUMBER()
OVER (ORDER BY object_id), REPLACE(getDate(),'-','')) as Alldays
FROM sys.all_objects)

Need Top N Row for Large Dataset. Query is taking a long time. How to optimize?

I have two tables (SalesforceTasks and SalesforceContacts) that I am using for a scoring system project. A simple SELECT statement with a ROW_NUMBER() calculation is taking a very long time to run and actually stops querying once it hits a certain number of rows. The query doesn't stop executing, but it stops returning data.
Here is the query in question. It is a very vanilla process, where I need to get the newest date in the SalesforceTasks table and link it to the contact ID in the SalesforceContacts table. The SalesforceTasks table has 2,091,946 rows and the SalesforceContacts table has 446,772 rows.
Here is the query in question:
SELECT SC.ID
,CASE
WHEN DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 360 AND 1500
THEN 15
WHEN DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 181 AND 360
THEN 10
WHEN DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 60 AND 180
THEN 5
ELSE 0
END AS Score
,ROW_NUMBER() OVER (PARTITION BY ST.ACCOUNTID ORDER BY ACTIVITYDATE) AS LastCall
FROM Salesforce.dbo.SalesforceTasks AS ST
JOIN Salesforce.dbo.SalesforceContacts AS SC
ON ST.ACCOUNTID = SC.ACCOUNTID
WHERE STATUS = 'Completed'
AND TYPE LIKE 'Call%'
What is the best plan of attack here? As stated, the query is taking a very, very long time to run. Is there a better way to get the newest date from the SalesforceTasks table?
You could try breaking the statement down in to a 2 step process.
First filter records into #temp table and get the datediff without the CASE:
SELECT SC.ID
,DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) AS ScoreDiff
,ROW_NUMBER() OVER (PARTITION BY ST.ACCOUNTID ORDER BY ACTIVITYDATE) AS LastCall
INTO #TEMP
FROM Salesforce.dbo.SalesforceTasks AS ST
JOIN Salesforce.dbo.SalesforceContacts AS SC
ON ST.ACCOUNTID = SC.ACCOUNTID
WHERE STATUS = 'Completed'
AND TYPE LIKE 'Call%'
AND DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 60 AND 1500
With the reduced dataset, you then perform the Scoring operation:
SELECT Id,
CASE ScoreDiff
WHEN BETWEEN 360 AND 1500
THEN 15
WHEN BETWEEN 181 AND 360
THEN 10
WHEN BETWEEN 60 AND 180
THEN 5
ELSE 0
END AS Score,
LastCall
FROM #temp
If purpose is just to get latest one then you can try this else need to find out other way
SELECT SC.ID,CASE
WHEN DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 360 AND 1500
THEN 15
WHEN DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 181 AND 360
THEN 10
WHEN DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 60 AND 180
THEN 5
ELSE 0
END AS Score,
SFC.ACTIVITYDATE
FROM Salesforce.dbo.SalesforceTasks AS ST
JOIN Salesforce.dbo.SalesforceContacts AS SC
CROSS APPLY
(
SELECT MAX(SFC.ID) AS SCID,MAX(SFC.ACTIVITYDATE) AS ACTIVITYDATE FROM Salesforce.dbo.SalesforceContacts SFC
WHERE SFC.ACCOUNTID=SC.ACCOUNTID
GROUP BY BY SFC.ACCOUNTID
HAVING MAX(SFC.ID)= SC.ID
)
ON ST.ACCOUNTID = SC.ACCOUNTID
WHERE STATUS = 'Completed'
AND TYPE LIKE 'Call%'
AND DATEDIFF(DD, ST.CREATEDDATE, GETDATE()) BETWEEN 60 AND 1500

Receive a new formatted table via aggregation and group by

I am having a big issue with a SQL Server query here and I really don't know how to go on with it.
The aim is to receive a table differentiated by different time-intervals going from 00:00 - 00:29 to 23:30 - 23:59. In each of these intervals I want to sum up the total minutes of entities which waited during these times. This information can be received by a starttime, and endtime and the status of the entity, which looks like this:
startdate | finishdate | resourcestatus | id
2015-03-19 10:22:56.8490000 | 2015-03-19 10:32:56.8490000 | 8 | asdsdasdsad
As you see such an entity can have the status 8 from one interval (10:00 - 10:30) into another (10:30 - 11:00).
Until now I solved this by defining 4 groups of time-intervals (finish and start are both in interval, start out of interval but finish in, start in interval but finish out, both start and finish out of interval) these 4 groups are joined by the time-intervals.
I would post the code here but it is too much. My result looks like this. Here are the beginnings of the different parts of the query:
select zr.nr,
zr.interval,
case when outOfInterval.waittime is not null
then SUM(outOfInterval.waittime)
else 0
end
+
case when inInterval.waittime is not null
then SUM(inInterval.waittime)
else 0
end
+
case when startInInterval.waittime is not null
then SUM(startInInterval.waittime)
else 0
end
+
case when finishInInterval.waittime is not null
then sum(finishInInterval.waittime)
else 0
end
as waitingMinutes
From (select 1 as nr,'00:00 - 00:29' as interval, 0 as waittime
union select 2,'00:30 - 00:59', 0
union select 3,'01:00 - 01:29', 0 ...
) zr
left join (select case when CONVERT(time, rt.startedat, 8) < '00:00' and CONVERT(time, rt.finishedat , 8) > '00:30' then '00:00 - 00:29' end as inter, 30 as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) < DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 ))
...
) outOfInterval on outOfInterval.inter = zr.interval
left join (select case when CONVERT(time, rt.startedat, 8) >= '00:00' and CONVERT(time, rt.finishedat , 8) <= '00:30' then '00:00 - 00:29' end as inter, SUM(DATEDIFF(minute, rt.STARTEDAT, rt.FINISHEDAT)) as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) <= DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 )) group by rt.startedat, rt.finishedat
...
) inInterval on inInterval.inter = zr.interval
left join (select case when CONVERT(time, rt.startedat, 8) >= '00:00' and CONVERT(time, rt.startedat, 8) < '00:30'and CONVERT(time, rt.finishedat , 8) >= '00:30' then '00:00 - 00:29' end as inter, (30-DATEPART(minute, rt.STARTEDAT)) as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) <= DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 )) group by rt.startedat, rt.finishedat
...
) startInInterval on startInInterval.inter = zr.interval
left join (select case when CONVERT(time, rt.startedat, 8) >= '00:00' and CONVERT(time,rt.finishedat, 8) < '00:30'and CONVERT(time, rt.STARTEDAT , 8) < '00:00' then '00:00 - 00:29' end as inter, DATEPART(minute, rt.finishedat) as waittime from T_resourcetracking rt where rt.resource_id is not null and rt.resourcestatus = 8 AND CONVERT(Date, rt.startedat) >= '02.02.2015' AND CONVERT(Date, rt.finishedat) <= DateAdd(day,1,CONVERT ( datetime , '08.05.2015', 120 )) group by rt.startedat, rt.finishedat
...
) finishInInterval on finishInInterval.inter = zr.interval
group by zr.interval, outOfInterval.waittime, inInterval.waittime, startInInterval.waittime, finishInInterval.waittime, zr.nr
And this is the result:
nr | interval | waitingMinutes
1 | 00:00 - 00:29 | 2
2 | 00:30 - 00:59 | 7
...
24 | 11:30 - 11:59 | 8
24 | 11:30 - 11:59 | 51
...
So as you see I have more then one of an interval in my result set.
Do you have any idea how to join the groups to one and sum the minutes up? I am really done with it, every kind of aggregate function did not work for me.
Thanks in advance!
#EDIT: If this was not difficult enough we need a second specification which I forgot to explain: We do not want to see all waitingtimes during the 48 time-intervals but the SUM of all those within a specific date-interval.
Let's say we want to know the summed up minutes from the last month. Then the result set should look like:
nr | interval | waitingMinutes
1 | 00:00 - 00:29 | 0
2 | 00:30 - 00:59 | 0
...
20 | 09:30 - 09:59 | 0
21 | 10:00 - 10:29 | 8
22 | 10:30 - 10:59 | 73
23 | 11:00 - 11:29 | 20
...
The minutes are summed up over all time-intervals of the last month. So for example from 11:00 - 11:29 in the last 30 days the entities waited 20 minutes in total (e.g. yesterday 10 minutes and the day before 10 minutes).
This is so difficult that I have really no clue anymore thinking that this is too much for SQL...
Any suggestions?
I would break you problem down something like this. I may have a few factors slightly off here but hopefully you can see where I'm going with this.
I'll break up the script with commentary, but the actual thing should be run as one single query:
declare #StartDate date
declare #EndDate date
select #StartDate = '20150202',#EndDate='20150508'
I've broken the start and end dates out as parameters as I guess these are subject to change and so this gives us one place to change them rather than many
;With Dates as (
select CAST(#StartDate as datetime) as Day
union all
select DATEADD(day,1,Day) from Dates where Day < #EndDate
)
First CTE, Dates, generates all dates within the period of interest. If you have a calendar table in your database, just select from it instead
, PMNs as (
select ROW_NUMBER() OVER (ORDER BY number)-1 as n
from master..spt_values
)
Next CTE, PMNs is my "poor man's numbers table" - if you have a real numbers table in your database, you can substitute that instead
, DateTimes as (
select
n+1 as nr,
DATEADD(minute,30*n,Day) as StartInclusive,
DATEADD(minute,30*(n+1),Day) as EndExclusive
from
Dates d
inner join
PMNs p
on
p.n between 0 and 47
)
Now, the real fun one. We combine the first two CTEs to generate DateTimes - the complete set of all half hour long periods across all dates of interest
select
nr,
CAST(time,StartInclusive) as StartTime,
CAST(time,EndInclusive) as EndTime,
SUM(
DATEDIFF(minute,
CASE WHEN dt.StartInclusive < rt.StartedAt THEN rt.StartedAt
ELSE dt.StartInclusive END,
CASE WHEN dt.EndExclusive > rt.finishedAt THEN rt.FinishedAt
ELSE dt.EndExclusive END
)) as TotalMinutes
from
DateTimes dt
inner join
T_resourcetracking rt
on
dt.StartInclusive < rt.finishedAt and
rt.startedAt < dt.EndExclusive
group by
nr,
CAST(time,StartInclusive),
CAST(time,EndInclusive)
And finally, we combine the data together. We find where a resourceTracking period overlaps one of our DateTimes periods (note the on clause for the join identifies all overlaps). And then a little manipulation inside some CASE expressions to work out the latter of the two start datetimes and the earlier of the two end datetimes - those are the two values we want to subtract.
If your T_resourcetracking isn't also (as with my DateTimes) computing intervals with a semi-open interval (inclusive start time, exclusive end time) you probably want to make some adjustments so that it does seem to be.
The idea is producing all 48 intervals with TALLY using CTE and joining to your data so that 2 intervals intersect. They intersect if any of vertice is between other vertices:
a-----------------b
c------------------------d
a-----------------b
c-----------------d
a------------------b
c----d
a------------------b
c----------d
The last select is just grouping and correct calculation depending on case.
DECLARE #t TABLE
(
sd DATETIME ,
ed DATETIME ,
st INT
)
INSERT INTO #t
VALUES ( '2015-03-19 10:31:56', '2015-03-19 10:42:56', 8 ),
( '2015-03-19 10:25:56', '2015-03-19 10:35:56', 8 ),
( '2015-03-19 10:31:56', '2015-03-19 11:10:56', 8 ),
( '2015-03-19 10:25:56', '2015-03-19 11:10:56', 8 );
WITH cte
AS ( SELECT DATEADD(mi,
30 * ( -1
+ ROW_NUMBER() OVER ( ORDER BY ( SELECT
1
) ) ),
CAST('00:00:00' AS TIME)) sp ,
DATEADD(mi,
-1 + 30
* ROW_NUMBER() OVER ( ORDER BY ( SELECT
1
) ),
CAST('00:00:00' AS TIME)) ep
FROM ( VALUES ( 1), ( 1), ( 1), ( 1), ( 1), ( 1), ( 1),
( 1) ) t1 ( n )
CROSS JOIN ( VALUES ( 1), ( 1), ( 1), ( 1),
( 1), ( 1) ) t2 ( n )
)
SELECT sp, ep,
SUM(CASE WHEN CAST(t.sd AS TIME) < c.sp
AND CAST (t.ed AS TIME) > c.ep THEN DATEDIFF(mi, sp, ep)
WHEN CAST(t.sd AS TIME) BETWEEN c.sp AND c.ep
AND CAST(t.ed AS TIME) BETWEEN c.sp AND c.ep
THEN DATEDIFF(mi, CAST(sd AS TIME), CAST(ed AS TIME))
WHEN CAST(t.sd AS TIME) BETWEEN c.sp AND c.ep
THEN DATEDIFF(mi, CAST(sd AS TIME), ep)
ELSE DATEDIFF(mi, sp, CAST(ed AS TIME))
END) AS Mi
FROM cte c
JOIN #t t ON CAST(t.sd AS TIME) BETWEEN c.sp AND c.ep
OR CAST(t.ed AS TIME) BETWEEN c.sp AND c.ep
OR c.sp BETWEEN CAST(t.sd AS TIME) AND CAST(t.ed AS TIME)
OR c.ep BETWEEN CAST(t.sd AS TIME) AND CAST(t.ed AS TIME)
GROUP BY sp, ep
Output:
sp ep Mi
10:00:00.0000000 10:29:00.0000000 8
10:30:00.0000000 10:59:00.0000000 73
11:00:00.0000000 11:29:00.0000000 20
Change JOIN to LEFT JOIN in order to get all intervals.
You should tweak this to get 0s using ISNULL on SUM. Also this considers only one day.
Please try this solution. You can use it even if the finishdate is on an other day than the startdate.
;with event_time as (
/*this is the input*/
select 1 id, convert(datetime,'2015-05-11 23:11') startdate, convert(datetime,'2015-05-12 00:15') finishdate
)
, event_time_convert as (
/*convert the input to calculation*/
select i.id, convert(time,i.startdate) startdate, DATEDIFF(MINUTE, i.startdate, i.finishdate) time_until_end
from event_time i
)
, intervall as (
/*create the intervall groups*/
select 1 id, CONVERT(time,'00:00') startdate, CONVERT(time,'00:29') finishdate
union all
select cs.id+1 id, DATEADD(minute,30,cs.startdate) startdate, DATEADD(minute,30,cs.finishdate) finishdate
from intervall cs
where cs.id<48
)
, event_time_in_intervall as (
/*calculate the waiting minutes in intervall*/
select i.id
, cs.id intervall_id
, case when DATEDIFF(minute,i.startdate, cs.finishdate) > i.time_until_end then i.time_until_end else DATEDIFF(minute,i.startdate, cs.finishdate) end time_in_intervall
, case when DATEDIFF(minute,i.startdate, cs.finishdate) > i.time_until_end then null else DATEADD(minute,1,cs.finishdate) end new_startdate
, case when DATEDIFF(minute,i.startdate, cs.finishdate) > i.time_until_end then 0 else i.time_until_end - DATEDIFF(minute,i.startdate, cs.finishdate)+1 end new_time_until_end
from event_time_convert i
join intervall cs on i.startdate between cs.startdate and cs.finishdate /*this is the first intervall*/
union all
select i.id
, cs.id intervall_id
, case when DATEDIFF(minute,i.new_startdate, cs.finishdate) > i.new_time_until_end then i.new_time_until_end else DATEDIFF(minute,i.new_startdate, cs.finishdate)+1 end time_in_intervall
, case when DATEDIFF(minute,i.new_startdate, cs.finishdate) > i.new_time_until_end then null else DATEADD(minute,1,cs.finishdate) end new_startdate
, case when DATEDIFF(minute,i.new_startdate, cs.finishdate) > i.new_time_until_end then 0 else i.new_time_until_end - DATEDIFF(minute,i.new_startdate, cs.finishdate)+1 end new_time_until_end
from event_time_in_intervall i
join intervall cs on i.new_startdate between cs.startdate and cs.finishdate
where i.new_time_until_end>0 /*if there is remaining time, I calculate with a recursion*/
)
/*the result*/
select i.id, CONVERT(varchar(5),i.startdate) + ' - ' + CONVERT(varchar(5), i.finishdate) intervall, s.sum_time_in_intervall waitingMinutes
from (
select i.intervall_id, SUM(i.time_in_intervall) sum_time_in_intervall
from event_time_in_intervall i
group by i.intervall_id
) s
join intervall i on s.intervall_id = i.id

Resources