I was facing this problem and spend a lot of time today. So, i thought to share it here:
I have a table where we store debitDate and we have a stored procedure where every month we set the debit date to next month in the table.
So, if its debit date is 29th Jan, 2020 -> 29th Feb, 2020 -> 29th March, 2020 - so it should go on like this. I am using DATEADD() function in the stored procedure.
But for 30th & 31st i am facing issue. It should work like below in upcoming years:
Desired Behaviour:
30th Jan, 2020 -> 29th Feb, 2020 -> 30th Mar, 2020 -> 30th Apr, 2020
30th Jan, 2021 -> 28th Feb, 2021 -> 30th Mar, 2021 -> 30th Apr, 2021
31st Jan, 2020 -> 29th Feb, 2020 -> 31st Mar, 2020 -> 30th Apr, 2020
Issue:
30th Jan, 2020 -> 29th Feb, 2020 -> 29th Mar, 2020 -> 29th Apr, 2020
30th Jan, 2021 -> 28th Feb, 2021 -> 28th Mar, 2021 -> 28th Apr, 2021
31st Jan, 2020 -> 29th Feb, 2020 -> 29th Mar, 2020 -> 29th Apr, 2020
Solution 1:
For solution i have thought i can add a new column to the table as previousDebitDate and when we update the debit date we will check, if previousDebitDate day is 30 or 31.
If true then
DATEADD(MONTH, 2, #previousDebitDate)
else
DATEADD(MONTH, 1, #debitDate)
If anyone has a better solution please feel free to post your answer.
Solution 2:
For this issue a better solution is to add debitDay as a new column to the table and save only day part (ex: 30) and calculate each month debit date on the fly.
I think Solution 2 is better! Thanks #Arvo!!!
Maybe I 've understand very well & maybe not, but here's what I think you're looking for
CREATE TABLE Data
(
Dates DATE
);
INSERT Data(Dates) VALUES
('2020-01-30');
WITH CTE AS
(
SELECT Dates,
DATEADD(Month, 1, Dates) NextMonth,
DAY(EOMONTH(DATEADD(Month, 1, Dates))) LastDay
FROM Data
UNION ALL
SELECT DATEADD(Month, 1, Dates),
DATEADD(Month, 1, NextMonth),
DAY(EOMONTH(DATEADD(Month, 1, NextMonth)))
FROM CTE
WHERE Dates <= '2021-12-31'
)
SELECT Dates, NextMonth, DATEFROMPARTS(YEAR(Dates), MONTH(NextMonth),
CASE WHEN LastDay > 30 THEN 30 ELSE LastDay END) Value
FROM CTE;
Which 'll returns:
+------------+------------+------------+
| Dates | NextMonth | Value |
+------------+------------+------------+
| 2020-01-30 | 2020-02-29 | 2020-02-29 |
| 2020-02-29 | 2020-03-29 | 2020-03-30 |
| 2020-03-29 | 2020-04-29 | 2020-04-30 |
| 2020-04-29 | 2020-05-29 | 2020-05-30 |
| 2020-05-29 | 2020-06-29 | 2020-06-30 |
| 2020-06-29 | 2020-07-29 | 2020-07-30 |
| 2020-07-29 | 2020-08-29 | 2020-08-30 |
| 2020-08-29 | 2020-09-29 | 2020-09-30 |
| 2020-09-29 | 2020-10-29 | 2020-10-30 |
| 2020-10-29 | 2020-11-29 | 2020-11-30 |
| 2020-11-29 | 2020-12-29 | 2020-12-30 |
| 2020-12-29 | 2021-01-29 | 2020-01-30 |
| 2021-01-29 | 2021-02-28 | 2021-02-28 |
| 2021-02-28 | 2021-03-28 | 2021-03-30 |
| 2021-03-28 | 2021-04-28 | 2021-04-30 |
| 2021-04-28 | 2021-05-28 | 2021-05-30 |
| 2021-05-28 | 2021-06-28 | 2021-06-30 |
| 2021-06-28 | 2021-07-28 | 2021-07-30 |
| 2021-07-28 | 2021-08-28 | 2021-08-30 |
| 2021-08-28 | 2021-09-28 | 2021-09-30 |
| 2021-09-28 | 2021-10-28 | 2021-10-30 |
| 2021-10-28 | 2021-11-28 | 2021-11-30 |
| 2021-11-28 | 2021-12-28 | 2021-12-30 |
| 2021-12-28 | 2022-01-28 | 2021-01-30 |
| 2022-01-28 | 2022-02-28 | 2022-02-28 |
+------------+------------+------------+
Much better
WITH CTE AS
(
SELECT 1 N, Dates, Dates ExpectedValue
FROM Data
UNION ALL
SELECT N+1, DATEADD(Month, 1, Dates), DATEFROMPARTS(YEAR(ExpectedValue), MONTH(DATEADD(Month, 1, ExpectedValue)),
CASE WHEN DAY(EOMONTH(DATEADD(Month, 1, ExpectedValue))) > 30 THEN 30
ELSE DAY(EOMONTH(DATEADD(Month, 1, ExpectedValue)))
END)
FROM CTE
WHERE N < 15
)
SELECT *
FROM CTE
ORDER BY N;
Returns:
+----+------------+---------------+
| N | Dates | ExpectedValue |
+----+------------+---------------+
| 1 | 2020-01-30 | 2020-01-30 |
| 2 | 2020-02-29 | 2020-02-29 |
| 3 | 2020-03-29 | 2020-03-30 |
| 4 | 2020-04-29 | 2020-04-30 |
| 5 | 2020-05-29 | 2020-05-30 |
| 6 | 2020-06-29 | 2020-06-30 |
| 7 | 2020-07-29 | 2020-07-30 |
| 8 | 2020-08-29 | 2020-08-30 |
| 9 | 2020-09-29 | 2020-09-30 |
| 10 | 2020-10-29 | 2020-10-30 |
| 11 | 2020-11-29 | 2020-11-30 |
| 12 | 2020-12-29 | 2020-12-30 |
| 13 | 2021-01-29 | 2020-01-30 |
| 14 | 2021-02-28 | 2020-02-29 |
| 15 | 2021-03-28 | 2020-03-30 |
+----+------------+---------------+
Here is a db<>fiddle
Due to a software bug that was unfortunately not obvious enough in the develop environment to be recognized, it happened that we created massive loads of SQL records we do not actually need. The records do not harm data integrity or anything else, but they are simply unnecessary.
We are looking at a database schema like the following:
entity_static (just some static data that won't change):
id | val1 | val2 | val3
-----------------------
1 | 50 | 183 | 93
2 | 60 | 823 | 123
entity_dynamic (some dynamic data we need a historical record of):
id | entity_static_id | val1 | val2 | valid_from | valid_to
-------------------------------------------------------------------------------
1 | 1 | 50 | 75 | 2018-01-01 00:00:00 | 2018-01-01 00:59:59
2 | 1 | 50 | 75 | 2018-01-01 01:00:00 | 2018-01-01 01:59:59
3 | 1 | 50 | 75 | 2018-01-01 02:00:00 | 2018-01-01 02:59:59
4 | 1 | 50 | 75 | 2018-01-01 03:00:00 | 2018-01-01 03:59:59
5 | 2 | 60 | 75 | 2018-01-01 00:00:00 | 2018-01-01 00:59:59
6 | 2 | 60 | 75 | 2018-01-01 01:00:00 | 2018-01-01 01:59:59
7 | 2 | 60 | 75 | 2018-01-01 02:00:00 | 2018-01-01 02:59:59
8 | 2 | 60 | 75 | 2018-01-01 03:00:00 | 2018-01-01 03:59:59
There are some more columns besides val1 and val2, this is just an example.
The entity_dynamic table describes what parameters were valid for a given period of time. It is not a recording for a certain point in time (like sensor data).
Therefor all equal records could easily be aggregated into one record like the following:
id | entity_static_id | val1 | val2 | valid_from | valid_to
-------------------------------------------------------------------------------
1 | 1 | 50 | 75 | 2018-01-01 00:00:00 | 2018-01-01 03:59:59
5 | 2 | 60 | 75 | 2018-01-01 00:00:00 | 2018-01-01 03:59:59
It is possible that the data in the valid_to column is NULL.
My question is now, with what query am I able to aggregate similar records with consecutive validity ranges into one record. Grouping should be done by the foreign key on entity_static_id.
with entity_dynamic as
(
select
*
from
(values
('1','1','50','75',' 2018-01-01 00:00:00 ',' 2018-01-01 00:59:59')
,('2','1','50','75',' 2018-01-01 01:00:00 ',' 2018-01-01 01:59:59')
,('3','1','50','75',' 2018-01-01 02:00:00 ',' 2018-01-01 02:59:59')
,('4','1','50','75',' 2018-01-01 03:00:00 ',' 2018-01-01 03:59:59')
,('5','2','60','75',' 2018-01-01 00:00:00 ',' 2018-01-01 00:59:59')
,('6','2','60','75',' 2018-01-01 01:00:00 ',' 2018-01-01 01:59:59')
,('7','2','60','75',' 2018-01-01 02:00:00 ',' 2018-01-01 02:59:59')
,('8','2','60','75',' 2018-01-01 03:00:00 ',' 2018-01-01 03:59:59')
,('9','1','60','75',' 2018-01-01 04:00:00 ',' 2018-01-01 04:59:59')
,('10','1','60','75',' 2018-01-01 05:00:00 ',' 2018-01-01 05:59:59')
,('11','2','70','75',' 2018-01-01 04:00:00 ',' 2018-01-01 04:59:59')
,('12','2','70','75',' 2018-01-01 05:00:00 ',' 2018-01-01 05:59:59')
,('13','2','60','75',' 2018-01-01 06:00:00 ',' 2018-01-01 06:59:59')
)
a(id , entity_static_id , val1 , val2 , valid_from , valid_to)
)
,
First add rownumbers for the unique combinations of val1 and val2 for each entity_static_id (unique group), add a row number for entity_static_id. Order by valid_from descending
step1 as
(
select
id , entity_static_id , val1 , val2 , valid_from , valid_to
,row_number() over (partition by entity_static_id,val1,val2 order by valid_from) valrn
,ROW_NUMBER() over (partition by entity_static_id order by valid_from desc) rn
from entity_dynamic
)
This gives:
+----------------------------------------------------------------------------------------+
|id|entity_static_id|val1|val2|valid_from |valid_to |unique_group|rn|
+----------------------------------------------------------------------------------------+
|10|1 |60 |75 | 2018-01-01 05:00:00 | 2018-01-01 05:59:59|2 |1 |
|9 |1 |60 |75 | 2018-01-01 04:00:00 | 2018-01-01 04:59:59|1 |2 |
|4 |1 |50 |75 | 2018-01-01 03:00:00 | 2018-01-01 03:59:59|4 |3 |
|3 |1 |50 |75 | 2018-01-01 02:00:00 | 2018-01-01 02:59:59|3 |4 |
|2 |1 |50 |75 | 2018-01-01 01:00:00 | 2018-01-01 01:59:59|2 |5 |
|1 |1 |50 |75 | 2018-01-01 00:00:00 | 2018-01-01 00:59:59|1 |6 |
|13|2 |60 |75 | 2018-01-01 06:00:00 | 2018-01-01 06:59:59|5 |1 |
|12|2 |70 |75 | 2018-01-01 05:00:00 | 2018-01-01 05:59:59|2 |2 |
|11|2 |70 |75 | 2018-01-01 04:00:00 | 2018-01-01 04:59:59|1 |3 |
|8 |2 |60 |75 | 2018-01-01 03:00:00 | 2018-01-01 03:59:59|4 |4 |
|7 |2 |60 |75 | 2018-01-01 02:00:00 | 2018-01-01 02:59:59|3 |5 |
|6 |2 |60 |75 | 2018-01-01 01:00:00 | 2018-01-01 01:59:59|2 |6 |
|5 |2 |60 |75 | 2018-01-01 00:00:00 | 2018-01-01 00:59:59|1 |7 |
+----------------------------------------------------------------------------------------+
Step2 is to add the rownumber for each unique group and the overall row num, since the last is descending, row with equal values following each other vil have the same sum, called tar in this example
,step2 as
(
select
*
,unique_group+rn tar
from step1
)
Step 2 gives:
+--------------------------------------------------------------------------------------------+
|id|entity_static_id|val1|val2|valid_from |valid_to |unique_group|rn|tar|
+--------------------------------------------------------------------------------------------+
|10|1 |60 |75 | 2018-01-01 05:00:00 | 2018-01-01 05:59:59|2 |1 |3 |
|9 |1 |60 |75 | 2018-01-01 04:00:00 | 2018-01-01 04:59:59|1 |2 |3 |
|4 |1 |50 |75 | 2018-01-01 03:00:00 | 2018-01-01 03:59:59|4 |3 |7 |
|3 |1 |50 |75 | 2018-01-01 02:00:00 | 2018-01-01 02:59:59|3 |4 |7 |
|2 |1 |50 |75 | 2018-01-01 01:00:00 | 2018-01-01 01:59:59|2 |5 |7 |
|1 |1 |50 |75 | 2018-01-01 00:00:00 | 2018-01-01 00:59:59|1 |6 |7 |
|13|2 |60 |75 | 2018-01-01 06:00:00 | 2018-01-01 06:59:59|5 |1 |6 |
|12|2 |70 |75 | 2018-01-01 05:00:00 | 2018-01-01 05:59:59|2 |2 |4 |
|11|2 |70 |75 | 2018-01-01 04:00:00 | 2018-01-01 04:59:59|1 |3 |4 |
|8 |2 |60 |75 | 2018-01-01 03:00:00 | 2018-01-01 03:59:59|4 |4 |8 |
|7 |2 |60 |75 | 2018-01-01 02:00:00 | 2018-01-01 02:59:59|3 |5 |8 |
|6 |2 |60 |75 | 2018-01-01 01:00:00 | 2018-01-01 01:59:59|2 |6 |8 |
|5 |2 |60 |75 | 2018-01-01 00:00:00 | 2018-01-01 00:59:59|1 |7 |8 |
+--------------------------------------------------------------------------------------------+
Finally, you can find the valid_from and vallid_to dates by using min and maxm and group by the correct values:
select
min(id) id
,entity_static_id
,val1
,val2
,min(valid_from) valid_from
,max(valid_to) valid_to
from step2
group by entity_static_id,val1
,val2
,tar
order by entity_static_id,valid_from
In totality the code is:
with entity_dynamic as
(
select
*
from
(values
('1','1','50','75',' 2018-01-01 00:00:00 ',' 2018-01-01 00:59:59')
,('2','1','50','75',' 2018-01-01 01:00:00 ',' 2018-01-01 01:59:59')
,('3','1','50','75',' 2018-01-01 02:00:00 ',' 2018-01-01 02:59:59')
,('4','1','50','75',' 2018-01-01 03:00:00 ',' 2018-01-01 03:59:59')
,('5','2','60','75',' 2018-01-01 00:00:00 ',' 2018-01-01 00:59:59')
,('6','2','60','75',' 2018-01-01 01:00:00 ',' 2018-01-01 01:59:59')
,('7','2','60','75',' 2018-01-01 02:00:00 ',' 2018-01-01 02:59:59')
,('8','2','60','75',' 2018-01-01 03:00:00 ',' 2018-01-01 03:59:59')
,('9','1','60','75',' 2018-01-01 04:00:00 ',' 2018-01-01 04:59:59')
,('10','1','60','75',' 2018-01-01 05:00:00 ',' 2018-01-01 05:59:59')
,('11','2','70','75',' 2018-01-01 04:00:00 ',' 2018-01-01 04:59:59')
,('12','2','70','75',' 2018-01-01 05:00:00 ',' 2018-01-01 05:59:59')
,('13','2','60','75',' 2018-01-01 06:00:00 ',' 2018-01-01 06:59:59')
)
a(id , entity_static_id , val1 , val2 , valid_from , valid_to)
)
,step1 as
(
select
id , entity_static_id , val1 , val2 , valid_from , valid_to
,row_number() over (partition by entity_static_id,val1,val2 order by valid_from) unique_group
,ROW_NUMBER() over (partition by entity_static_id order by valid_from desc) rn
from entity_dynamic
)
,step2 as
(
select
*
,dense_rank() over (partition by entity_static_id order by unique_group) f
,unique_group+rn tar
from step1
)
select
min(id) id
,entity_static_id
,val1
,val2
,min(valid_from) valid_from
,max(valid_to) valid_to
from step2
group by entity_static_id,val1
,val2
,tar
order by entity_static_id,valid_from
The result is
+------------------------------------------------------------------------+
|id|entity_static_id|val1|val2|valid_from |valid_to |
+------------------------------------------------------------------------+
|1 |1 |50 |75 | 2018-01-01 00:00:00 | 2018-01-01 03:59:59|
|10|1 |60 |75 | 2018-01-01 04:00:00 | 2018-01-01 05:59:59|
|5 |2 |60 |75 | 2018-01-01 00:00:00 | 2018-01-01 03:59:59|
|11|2 |70 |75 | 2018-01-01 04:00:00 | 2018-01-01 05:59:59|
|13|2 |60 |75 | 2018-01-01 06:00:00 | 2018-01-01 06:59:59|
+------------------------------------------------------------------------+
If group is defined by entity_dynamic then this should be all you need
with entity_dynamic as
( select *
from (values ('1' ,'1','50','75',' 2018-01-01 00:00:00 ',' 2018-01-01 00:59:59')
,('2' ,'1','50','75',' 2018-01-01 01:00:00 ',' 2018-01-01 01:59:59')
,('3' ,'1','50','75',' 2018-01-01 02:00:00 ',' 2018-01-01 02:59:59')
,('4' ,'1','50','75',' 2018-01-01 03:00:00 ',' 2018-01-01 03:59:59')
,('5' ,'2','60','75',' 2018-01-01 00:00:00 ',' 2018-01-01 00:59:59')
,('6' ,'2','60','75',' 2018-01-01 01:00:00 ',' 2018-01-01 01:59:59')
,('7' ,'2','60','75',' 2018-01-01 02:00:00 ',' 2018-01-01 02:59:59')
,('8' ,'2','60','75',' 2018-01-01 03:00:00 ',' 2018-01-01 03:59:59')
) a(id , entity_static_id , val1 , val2 , valid_from , valid_to)
)
, entity_dynamicPlus as
( select *
, ROW_NUMBER() over (partition by entity_static_id order by valid_to asc ) as rnA
, ROW_NUMBER() over (partition by entity_static_id order by valid_to desc) as rnD
from entity_dynamic
)
select eStart.id, eStart.entity_static_id, eStart.val1, eStart.val2, eStart.valid_from, eEnd.valid_to
, eEnd.valid_to
from entity_dynamicPlus as eStart
join entity_dynamicPlus as eEnd
on eStart.entity_static_id = eEnd.entity_static_id
and eStart.rnA = 1
and eEnd.rnD = 1
order by eStart.entity_static_id