Related
I have a table like this:
Date
Consec_Days
2015-01-01
1
2015-01-03
1
2015-01-06
1
2015-01-07
2
2015-01-09
1
2015-01-12
1
2015-01-13
2
2015-01-14
3
2015-01-17
1
I need to Sum the max value (days) for each of the consecutive groupings where Consec_Days are > 1. So the correct result would be 5 days.
This is a type of gaps-and-islands problem.
There are many solutions, here is one simple one
Get the start points of each group using LAG
Calculate a grouping ID using a windowed conditional count
Group by that ID and take the highest sum
WITH StartPoints AS (
SELECT *,
IsStart = CASE WHEN LAG(Consec_Days) OVER (ORDER BY Date) = 1 THEN 1 END
FROM YourTable t
),
Groupings AS (
SELECT *,
GroupId = COUNT(IsStart) OVER (ORDER BY Date)
FROM StartPoints
WHERE Consec_Days > 1
)
SELECT TOP (1)
SUM(Consec_Days)
FROM Groupings
GROUP BY
GroupId
ORDER BY
SUM(Consec_Days) DESC;
db<>fiddle
with cte as (
select Consec_Days,
coalesce(lead(Consec_Days) over (order by Date), 1) as next
from YourTable
)
select sum(Consec_Days)
from cte
where Consec_Days <> 1 and next = 1
db<>fiddle
In my table, I have a primary key and a date. What I'd like to achieve is to have an incremental label based on whether or not there is a break between the dates - column Goal.
Now, below is an example. The break column was calculated using LEAD function (I thought it might help).
I am able to solve it using T-SQL, but this would be last resort. Nothing I tried has worked so far. I am using MSSQL 2014.
PK | Date | break | Goal |
-------------------------------
1 | 03/2017 | 0 | 1 |
1 | 04/2017 | 0 | 1 |
1 | 08/2017 | 1 | 2 |
1 | 09/2017 | 0 | 2 |
1 | 10/2017 | 0 | 2 |
1 | 02/2018 | 1 | 3 |
1 | 03/2018 | 0 | 3 |
Here is a code to reproduce this example:
CREATE TABLE #test
(
ConsumerId INT,
FullDate DATE,
Goal INT
)
INSERT INTO #test (ConsumerId, FullDate, Goal) VALUES (1,'2017-03-01',1)
INSERT INTO #test (ConsumerId, FullDate, Goal) VALUES (1,'2017-04-01',1)
INSERT INTO #test (ConsumerId, FullDate, Goal) VALUES (1,'2017-08-01',2)
INSERT INTO #test (ConsumerId, FullDate, Goal) VALUES (1,'2017-09-01',2)
INSERT INTO #test (ConsumerId, FullDate, Goal) VALUES (1,'2017-10-01',2)
INSERT INTO #test (ConsumerId, FullDate, Goal) VALUES (1,'2018-02-01',3)
INSERT INTO #test (ConsumerId, FullDate, Goal) VALUES (1,'2018-03-01',3)
SELECT ConsumerId,
FullDate,
CASE WHEN (datediff(month,
isnull(
LEAD (FullDate,1) OVER (PARTITION BY ConsumerId ORDER BY FullDate DESC),
FullDate),
FullDate) > 1)
THEN 1
ELSE 0
END AS break,
Goal
FROM #test
ORDER BY FullDate ASC
EDIT
This is apparently a famous problem "Islands and gaps" as pointed out in the comments. And Google offers many solutions as well as other questions here at SO.
Try this...
WITH
cte_TestGap AS (
SELECT
t.ConsumerId, t.FullDate,
Gap = CASE
WHEN DATEDIFF(mm, t.FullDate, LAG(t.FullDate, 1) OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)) = -1
THEN 0
ELSE ROW_NUMBER() OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)
END
FROM
#test t
),
cte_SmearGap AS (
SELECT
tg.ConsumerId, tg.FullDate,
GV = MAX(tg.Gap) OVER (PARTITION BY tg.ConsumerId ORDER BY tg.FullDate ROWS UNBOUNDED PRECEDING)
FROM
cte_TestGap tg
)
SELECT
sg.ConsumerId, sg.FullDate,
GroupValue = DENSE_RANK() OVER (PARTITION BY sg.ConsumerId ORDER BY sg.GV)
FROM
cte_SmearGap sg;
An explanation of the code an how it works...
The 1st query, in cte_TestGap, uses the LAG function along with ROW_NUMBER() function to mark the location of gap in the data. We can see that by breaking it out and looking at it's results...
WITH
cte_TestGap AS (
SELECT
t.ConsumerId, t.FullDate,
Gap = CASE
WHEN DATEDIFF(mm, t.FullDate, LAG(t.FullDate, 1) OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)) = -1
THEN 0
ELSE ROW_NUMBER() OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)
END
FROM
#test t
)
SELECT * FROM cte_TestGap;
cte_TestGap results...
ConsumerId FullDate Gap
----------- ---------- --------------------
1 2017-03-01 1
1 2017-04-01 0
1 2017-08-01 3
1 2017-09-01 0
1 2017-10-01 0
1 2018-02-01 6
1 2018-03-01 0
At this point we want the 0 values to take on the value of the preceding non-0 values, allowing them to be grouped together. This is done in the 2nd query (cte_SmearGap) using the MAX function with a "window frame". So if we look at the output of cte_SmearGap, we can see that...
WITH
cte_TestGap AS (
SELECT
t.ConsumerId, t.FullDate,
Gap = CASE
WHEN DATEDIFF(mm, t.FullDate, LAG(t.FullDate, 1) OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)) = -1
THEN 0
ELSE ROW_NUMBER() OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)
END
FROM
#test t
),
cte_SmearGap AS (
SELECT
tg.ConsumerId, tg.FullDate,
GV = MAX(tg.Gap) OVER (PARTITION BY tg.ConsumerId ORDER BY tg.FullDate ROWS UNBOUNDED PRECEDING)
FROM
cte_TestGap tg
)
SELECT * FROM cte_SmearGap;
cte_SmearGap results...
ConsumerId FullDate GV
----------- ---------- --------------------
1 2017-03-01 1
1 2017-04-01 1
1 2017-08-01 3
1 2017-09-01 3
1 2017-10-01 3
1 2018-02-01 6
1 2018-03-01 6
At this point All of the rows are in distinct groups... but... We'd like to have our group numbers in a contiguous sequence (1,2,3) as opposed to (1,3,6).
Of course that's easy enough to fix using the DENSE_Rank() function, which is what's happening in the final select...
WITH
cte_TestGap AS (
SELECT
t.ConsumerId, t.FullDate,
Gap = CASE
WHEN DATEDIFF(mm, t.FullDate, LAG(t.FullDate, 1) OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)) = -1
THEN 0
ELSE ROW_NUMBER() OVER (PARTITION BY t.ConsumerId ORDER BY t.FullDate)
END
FROM
#test t
),
cte_SmearGap AS (
SELECT
tg.ConsumerId, tg.FullDate,
GV = MAX(tg.Gap) OVER (PARTITION BY tg.ConsumerId ORDER BY tg.FullDate ROWS UNBOUNDED PRECEDING)
FROM
cte_TestGap tg
)
SELECT
sg.ConsumerId, sg.FullDate,
GroupValue = DENSE_RANK() OVER (PARTITION BY sg.ConsumerId ORDER BY sg.GV)
FROM
cte_SmearGap sg;
The end result...
ConsumerId FullDate GroupValue
----------- ---------- --------------------
1 2017-03-01 1
1 2017-04-01 1
1 2017-08-01 2
1 2017-09-01 2
1 2017-10-01 2
1 2018-02-01 3
1 2018-03-01 3
The comment from David Browne was actually extremely useful. If you google "Islands and Gaps", there are many variations of the solution. Below is the one I liked the most.
In the end, I needed the Goal column to be able to group the dates into MIN/MAX. This solution skips this step and directly creates the aggregated range.
Here is the source.
SELECT MIN(FullDate) AS range_start,
MAX(FUllDate) AS range_end
FROM (
SELECT FullDate,
DATEADD(MM, -1 * ROW_NUMBER() OVER(ORDER BY FullDate), FullDate) AS grp
FROM #test
) a
GROUP BY a.grp
And the output:
range_start | range_end |
--------------------------
2017-03-01 | 2017-04-01 |
2017-08-01 | 2017-10-01 |
2018-02-01 | 2018-03-01 |
Here is how the data looks like currently
OrderNo, OrderDate, Order_PROD1, Order_Unit1, Order_IP1_Date, Order_PROD2,
1 12/20/2017 17383 894YU 12/23/2017 49348
Order_Unit2, Order_IP2_Date ...... Order_PROD30, Order_Unit30,
489UI 11/12/2015
The way i want to transform is
OrderNo, OrderDate, Order_Prod, Order_Unit, Order_IP_Date
1 12/20/2017 17383 894YU 12/23/2017
1 12/20/2017 49348 489UI 11/12/2015
1 12/20/2017 Order_Prod3* Order_Unit3* Order_IP3_Date*
1 12/20/2017 Order_Prod4* Order_Unit4* Order_IP4_Date*
Order_Prod3* = Value of column Order_Prod3
Order_Prod4* = Value of column Order_Prod4
Here is the query i have so far
select Orderid, OrderDate, Order_Prod, Order_unit, Order_IP_Date
from tbl
unpivot
(
Order_Prod ??????
for Order_Prod in (Order_Prod1, Order_Prod2, Order_Prod3)???
) unpiv;
Not sure how to un-pivot on multiple columns..
A Dynamic version (Notice I added a second record as an illustration)
Declare #YourTable table (OrderNo int,OrderDate date,Order_PROD1 varchar(25),Order_Unit1 varchar(25),Order_IP1_Date date,Order_PROD2 varchar(25),Order_Unit2 varchar(25),Order_IP2_Date date,Order_PROD3 varchar(25),Order_Unit3 varchar(25),Order_IP3_Date date)
Insert Into #YourTable values
(1,'2017-12-20','17383','894YU','2017-12-23','9999','AAA-894YU','2017-12-31','a9999','bAAA-894YU','2017-12-28'),
(2,'2017-12-22','17999','89999','2017-12-27','8888','BBB-894YU','2017-12-29','b8888','bBBB-894YU','2017-12-30')
Declare #XML xml = (Select *,RN=Row_Number() over (Partition By OrderNo Order By OrderNo) from #YourTable for XML RAW)
Select OrderNo
,OrderDate
,OrderRow = Replace(Substring(Item,PatIndex('%[0-9]%',Item),2),'_','')
,Order_Prod = max(case when Item Like 'Order_Prod%' then Value else null end)
,Order_Unit = max(case when Item Like 'Order_Unit%' then Value else null end)
,Order_IP_Date = max(case when Item Like 'Order_IP%' then Value else null end)
From (
Select OrderNo = r.value('#OrderNo','int')
,OrderDate = r.value('#OrderDate','date')
,RN = r.value('#RN','int')
,Item = attr.value('local-name(.)','varchar(100)')
,Value = attr.value('.','varchar(max)')
From #XML.nodes('/row') as A(r)
Cross Apply A.r.nodes('./#*') AS B(attr)
Where attr.value('local-name(.)','varchar(100)') not in ('OrderNo','OrderDate','RN')
) A
Group By OrderNo,OrderDate,RN,Replace(Substring(Item,PatIndex('%[0-9]%',Item),2),'_','')
Returns
OrderNo OrderDate OrderRow Order_Prod Order_Unit Order_IP_Date
1 2017-12-20 1 17383 894YU 2017-12-23
1 2017-12-20 2 9999 AAA-894YU 2017-12-31
1 2017-12-20 3 a9999 bAAA-894YU 2017-12-28
2 2017-12-22 1 17999 89999 2017-12-27
2 2017-12-22 2 8888 BBB-894YU 2017-12-29
2 2017-12-22 3 b8888 bBBB-894YU 2017-12-30
Perhaps a Cross Apply may help here. UnPivot has greater performance, but you will have a little more flexibility.
Declare #YourTable table (OrderNo int,OrderDate date,Order_PROD1 varchar(25),Order_Unit1 varchar(25),Order_IP1_Date date,Order_PROD2 varchar(25),Order_Unit2 varchar(25),Order_IP2_Date date,Order_PROD3 varchar(25),Order_Unit3 varchar(25),Order_IP3_Date date)
Insert Into #YourTable values
(1,'2017-12-20','17383','894YU','2017-12-23','9999','AAA-894YU','2017-12-31','a9999','bAAA-894YU','2017-12-28'),
(2,'2017-12-22','17999','89999','2017-12-27','8888','BBB-894YU','2017-12-29','b8888','bBBB-894YU','2017-12-30')
Select A.OrderNo
,A.OrderDate
,B.*
From #YourTable A
Cross Apply ( Values (1,A.Order_Prod1,A.Order_Unit1,A.Order_IP1_Date)
,(2,A.Order_Prod2,A.Order_Unit2,A.Order_IP2_Date)
,(3,A.Order_Prod3,A.Order_Unit3,A.Order_IP3_Date)
-- ...
--,(30,A.Order_Prod30,A.Order_Unit30,A.Order_IP30_Date)
) B (OrderRow,Order_Prod,Order_Unit,Order_IP_Date)
Returns
OrderNo OrderDate OrderRow Order_Prod Order_Unit Order_IP_Date
1 2017-12-20 1 17383 894YU 2017-12-23
1 2017-12-20 2 9999 AAA-894YU 2017-12-31
1 2017-12-20 3 a9999 bAAA-894YU 2017-12-28
2 2017-12-22 1 17999 89999 2017-12-27
2 2017-12-22 2 8888 BBB-894YU 2017-12-29
2 2017-12-22 3 b8888 bBBB-894YU 2017-12-30
Let's assume I have a table which has columns From and To which are dates and a bit type column which identifies whether it is a cancel (1 = cancel). Also an Id which is a PK and CancelId which references what is cancelled.
Let's say I have records which look like:
Id From To IsCancel CancelId
1 2015-01-01 2015-01-31 0 NULL
2 2015-01-03 2015-01-09 1 1
3 2015-01-27 2015-01-31 1 1
I am expecting the result to show what intervals of then non-cancel records are still uncancelled:
Id From To
1 2015-01-01 2015-01-02
1 2015-01-10 2015-01-26
I can make it so it would split each record into dates, then subtract cancelled dates from the records then merge the intervals but since I have quite a lot of records, I find this very inefficient and am pretty sure that I am overlooking something simple.
The task you want to achieve is non trivial. A possible solution involves placing all From / To dates in an ordered sequence. The following UNPIVOT operation:
SELECT ID, EventDate, StartStop,
ROW_NUMBER() OVER (ORDER BY ID, EventDate, StartStop) AS EventRowNum,
IsCancel
FROM
(SELECT ID, IsCancel, [From], [To]
FROM Event) Src
UNPIVOT (
EventDate FOR StartStop IN ([From], [To])
) AS Unpvt
produces this result set:
ID EventDate StartStop EventRowNum IsCancel
--------------------------------------------------
1 2015-01-01 From 1 0
2 2015-01-03 From 2 1
2 2015-01-09 To 3 1
3 2015-01-27 From 4 1
3 2015-01-31 To 5 1
1 2015-01-31 To 6 0
Using a CTE, you can subsequently simulate LEAD function (available from SQL Server 2012 onwards) in order to place in a single record the current and the next date from the sequence above:
;WITH StretchEventDates AS
(
-- above query goes here
), CTE AS
(
SELECT s.ID, s.EventDate, s.StartStop, s.IsCancel,
sLead.EventDate As LeadEventDate, sLead.StartStop AS LeadStartStop, sLead.IsCancel AS LeadIsCancel
FROM StretchEventDates AS s
LEFT JOIN StretchEventDates AS sLead ON s.EventRowNum + 1 = sLead.EventRowNum
)
The above produces the following result set:
ID EventDate StartStop IsCancel LeadEventDate LeadStartStop LeadIsCancel
--------------------------------------------------------------------------------------
1 2015-01-01 From 0 2015-01-03 From 1
2 2015-01-03 From 1 2015-01-09 To 1
2 2015-01-09 To 1 2015-01-27 From 1
3 2015-01-27 From 1 2015-01-31 To 1
3 2015-01-31 To 1 2015-01-31 To 0
1 2015-01-31 To 0 NULL NULL NULL
Using CASE statements you can filter these records in order to get the desired output.
Putting it all together:
;WITH StretchEventDates AS
(
SELECT ID, EventDate, StartStop,
ROW_NUMBER() OVER (ORDER BY EventDate, StartStop) AS EventRowNum,
IsCancel
FROM
(SELECT ID, IsCancel, [From], [To]
FROM Event) Src
UNPIVOT (
EventDate FOR StartStop IN ([From], [To])
) AS Unpvt
), CTE AS
(
SELECT s.ID, s.EventDate, s.StartStop, s.IsCancel,
sLead.EventDate As LeadEventDate, sLead.StartStop AS LeadStartStop, sLead.IsCancel AS LeadIsCancel
FROM StretchEventDates AS s
LEFT JOIN StretchEventDates AS sLead ON s.EventRowNum + 1 = sLead.EventRowNum
), CTE_FINAL AS
(SELECT *,
CASE WHEN StartStop = 'From' AND IsCancel = 0 THEN EventDate
WHEN StartStop = 'To' AND IsCancel = 1 THEN DATEADD(d, 1, EventDate)
END AS [From],
CASE WHEN LeadStartStop = 'From' AND LeadIsCancel = 1 THEN DATEADD(d, -1, LeadEventDate)
WHEN LeadStartStop = 'To' AND LeadIsCancel = 0 THEN LeadEventDate
END AS [To]
FROM CTE
)
SELECT ID, [From], [To]
FROM CTE_FINAL
WHERE [From] IS NOT NULL AND [To] IS NOT NULL AND [From] <= [To]
You may have to add additional CASEs in the query above to handle additional combinations of 'cancelations' following 'non-canceled' (and vice-versa) events.
With the data provided in the OP the above yields the following output:
ID From To
---------------------------
1 2015-01-01 2015-01-02
2 2015-01-10 2015-01-26
I have a SQL statement.
SELECT
ID, LOCATION, CODE,MAX(DATE),FLAG
FROM
TABLE1
WHERE
DATE <= CONVERT(DATETIME,'11-11-2012')
AND EXISTS (SELECT * FROM #TEMP_CODE WHERE TABLE1.CODE = #TEMP_CODE.CODE)
AND ID IN (14, 279)
GROUP BY
ID, LOCATION, CODE
I need rows with the nearest date to the 11-11-2012, but the table returns all the values. What am I doing wrong. Thanks
ID LOCATION CODE DATE FLAG
-------------------------------------------------------------------
14 CAR STREET,UDUPI 234 2012-08-08 00:00:00.000 0
14 CAR STREET,UDUPI 234 2012-08-10 00:00:00.000 1
14 CAR STREET,UDUPI 234 2012-08-14 00:00:00.000 0
279 MADHUGIRI 234 2012-08-08 00:00:00.000 1
279 MADHUGIRI 234 2012-08-11 00:00:00.000 0
I want to show only the rows with dates less than or equal to the given date. The required result is
ID LOCATION CODE DATE FLAG
-------------------------------------------------------------------
14 CAR STREET,UDUPI 234 2012-08-10 00:00:00.000 1
279 MADHUGIRI 234 2012-08-11 00:00:00.000 0
;WITH x AS
(
SELECT ID, Location, Code, Date, Flag,
rn = ROW_NUMBER() OVER
(PARTITION BY ID, Location, Code ORDER BY [Date] DESC)
FROM dbo.TABLE1 AS t1
WHERE [Date] <= '20121111'
AND ID IN (14, 279) -- sorry, missed this
AND EXISTS (SELECT 1 FROM #TEMP_CODE WHERE CODE = t1.CODE)
)
SELECT ID, Location, Code, Date, Flag
FROM x WHERE rn = 1;
This yields:
ID LOCATION CODE [Date] FLAG
--- ---------------- ---- ---------- ----
14 CAR STREET,UDUPI 234 2012-08-14 0
279 MADHUGIRI 234 2012-08-11 0
This disagrees with your required results, but I think those are wrong and I think you should check them.
Use a subquery to get the max date for each ID, and then join that to your table:
SELECT
ID, LOCATION, CODE, DATE, FLAG
FROM
TABLE1
JOIN (
SELECT ID AS SubID, MAX(DATE) AS SubDATE
FROM TABLE1
WHERE DATE < '11/11/2012'
AND EXISTS (SELECT * FROM #TEMP_CODE WHERE TABLE1.CODE = #TEMP_CODE.CODE)
AND ID IN (14, 279)
GROUP BY ID
) AS SUB ON ID = SubID AND DATE = SubDATE
add a Order BY DATE LIMIT 0,2
With the order by you will make the date order by the closest to your condition in where and with the limit will return only the top 2 values!
SET ROWCOUNT 2
SELECT
ID, LOCATION, CODE,MAX(DATE),FLAG
FROM
TABLE1
WHERE
DATE <= CONVERT(DATETIME,'11-11-2012')
AND EXISTS (SELECT * FROM #TEMP_CODE WHERE TABLE1.CODE = #TEMP_CODE.CODE)
AND ID IN (14, 279)
GROUP BY
ID, LOCATION, CODE
ORDER BY DATE