SQL Server query to see what changed from month to month - sql-server

Question

I am struggling with developing a query to compare changes in a single table from month to month, example data -
+-----------------------------------------------------------+
| TaxGroupDetails |
+-----------+--+----------+--+-----------+--+---------------+
| Tax Group | | Tax Type | | Geocode | | EffectiveDate |
+-----------+--+----------+--+-----------+--+---------------+
| 2001 | | 1D | | 440011111 | | 1120531 |
| 2001 | | X1 | | 440011111 | | 1120531 |
| 2001 | | D3 | | 440011111 | | 1120531 |
| 2001 | | DGH | | 440011111 | | 1120531 |
| 2001 | | 1D | | 440011111 | | 1130101 |
| 2001 | | X1 | | 440011111 | | 1130101 |
| 2001 | | D3 | | 440011111 | | 1130101 |
| 2001 | | 1D | | 440011111 | | 1140201 |
| 2001 | | X1 | | 440011111 | | 1140201 |
| 2001 | | D3 | | 440011111 | | 1140201 |
| 2001 | | Z9 | | 440011111 | | 1140201 |
+-----------+--+----------+--+-----------+--+---------------+
I want to see the changes in the table, what was added or removed from a taxgroup, between the top two effective dates.
The results I am trying to obtain based on the sample data would be Z9 (added) if I was running the query in February (1140201) of this year.
If I was running the query in January (1130101) of last year I would expect to see DGH (removed)
I would expect two seperate queries, one to show what was added and another to show what was removed.
I have tried multiple avenues to come up with these two queries but cant seem to obtain the correct results. Can anyone point me in the right direction ?

Answer 1

SELECT
Current.TaxGroup,
Current.TaxType,
Current.GeoCode,
'Added'
FROM
TaxGroupDetails AS Current
WHERE
Current.EffectiveDate = #CurrentPeriod AND
NOT EXISTS
(
SELECT *
FROM TaxGroupDetails As Previous
WHERE
Previous.EffectiveDate = #PreviousPeriod
Current.TaxGroup = Previous.TaxGroup and
Current.TaxType = Previous.TaxType and
Current.GeoCode = Previous.GeoCode
)
UNION ALL
SELECT
Current.TaxGroup,
Current.TaxType,
Current.GeoCode,
'Added'
FROM
TaxGroupDetails AS Previous
WHERE
Previous.EffectiveDate = #PreviousPeriod AND
NOT EXISTS
(
SELECT *
FROM TaxGroupDetails As Current
WHERE
Current.EffectiveDate = #CurrentPeriod
Current.TaxGroup = Previous.TaxGroup and
Current.TaxType = Previous.TaxType and
Current.GeoCode = Previous.GeoCode
)

Answer 2

As you say you need two queries, one to select each of the two groups of data you want to compare.
SELECT [Tax Group], [Tax Type], [Geocode], [EffectiveDate]
FROM TaxGroupDetails
WHERE EffectiveDate = 1120531
SELECT [Tax Group], [Tax Type], [Geocode], [EffectiveDate]
FROM TaxGroupDetails
WHERE EffectiveDate = 1140201
You then need to join these two together using some form of key, the combination of tax group and tax type seems sensible here.
SELECT *
FROM
(
SELECT [Tax Group], [Tax Type], [Geocode], [EffectiveDate]
FROM TaxGroupDetails
WHERE EffectiveDate = 1120531
) AS FirstGroup
FULL OUTER JOIN
(
SELECT [Tax Group], [Tax Type], [Geocode], [EffectiveDate]
FROM TaxGroupDetails
WHERE EffectiveDate = 1140201
) AS SecondGroup
ON FirstGroup.[Tax Group] = SecondGroup.[Tax Group]
AND FirstGroup.[Tax Type] = SecondGroup.[Tax Type]
The FULL OUTER JOIN here tells SQL to include the remaining row when the other doesn't exist.
Finally let's tidy up and order the columns and not use a *:
SELECT COALESCE(FirstGroup.[Tax Group], SecondGroup.[Tax Group]),
COALESCE(FirstGroup.[Tax Type], SecondGroup.[Tax Type]),
FirstGroup.Geocode, SecondGroup.Geocode,
FirstGroup.EffectiveDate, SecondGroup.EffectiveDate
FROM
.
.
.
COALESCE removes the NULLs from the first matched columns and as we are saying these muct be equal there is no point showing both copies.

Answer 3

The set-based solution: take the difference between the whole table and the whole table with all dates projected forward by one time interval. That will eliminate all rows except the ones with "new" codes.
SELECT
[TaxGroup],
[Tax Type],
[EffectiveDate]
FROM TaxGroupDetails t
EXCEPT
SELECT
[TaxGroup],
[Tax Type],
( SELECT MIN([EffectiveDate])
FROM TaxGroupDetails
WHERE [EffectiveDate] > t.[EffectiveDate]
AND [TaxGroup] = t.[TaxGroup]
)
FROM TaxGroupDetails t
To see what got deleted, project backwards instead. Change the subquery to:
SELECT MAX([EffectiveDate])
FROM TaxGroupDetails
WHERE [EffectiveDate] < t.[EffectiveDate]
AND [TaxGroup] = t.[TaxGroup]

Answer 4

If you have SQL2012:
WITH t AS (
SELECT *,
ROW_NUMBER() OVER(PARTITION BY [TaxGroup], [Tax Type] ORDER BY [EffectiveDate] ASC) rownum
FROM [TaxGroup]
)
SELECT *
FROM t
WHERE rownum = 1
AND [EffectiveDate] = #Date
To get the other query, change ASC to DESC

Answer 5

Try this / you could start from this [partial] solution:
DECLARE #MyTable TABLE (
ID INT IDENTITY PRIMARY KEY,
[Tax Group] SMALLINT NOT NULL,
[Tax Type] VARCHAR(3) NOT NULL,
[Geocode] INT NOT NULL,
[EffectiveDate] INT NOT NULL
);
INSERT #MyTable
SELECT 2001, '1D ', 440011111, 1120531
UNION ALL SELECT 2001, 'X1 ', 440011111, 1120531
UNION ALL SELECT 2001, 'D3 ', 440011111, 1120531
UNION ALL SELECT 2001, 'DGH', 440011111, 1120531
UNION ALL SELECT 2001, '1D ', 440011111, 1130101
UNION ALL SELECT 2001, 'X1 ', 440011111, 1130101
UNION ALL SELECT 2001, 'D3 ', 440011111, 1130101
UNION ALL SELECT 2001, '1D ', 440011111, 1140201
UNION ALL SELECT 2001, 'X1 ', 440011111, 1140201
UNION ALL SELECT 2001, 'D3 ', 440011111, 1140201
UNION ALL SELECT 2001, 'Z9 ', 440011111, 1140201;
DECLARE #Results TABLE (
ID INT NOT NULL,
Rnk INT NOT NULL,
EffectiveYear SMALLINT NOT NULL,
PRIMARY KEY (Rnk, EffectiveYear)
);
INSERT #Results
SELECT x.ID,
DENSE_RANK() OVER(ORDER BY x.[Tax Group], x.[Tax Type], x.[Geocode]) AS Rnk,
x.EffectiveDate / 10000 AS EffectiveYear
FROM #MyTable x;
SELECT
crt.*,
prev.*,
CASE
WHEN crt.ID IS NOT NULL AND prev.ID IS NOT NULL THEN '-' -- No change
WHEN crt.ID IS NULL AND prev.ID IS NOT NULL THEN 'D' -- Deleted
WHEN crt.ID IS NOT NULL AND prev.ID IS NULL THEN 'I' -- Inserted
END AS RowStatus
FROM #Results crt FULL OUTER JOIN #Results prev ON crt.Rnk = prev.Rnk
AND crt.EffectiveYear - 1 = prev.EffectiveYear
ORDER BY ISNULL(crt.EffectiveYear - 1, prev.EffectiveYear), crt.Rnk;
Sample output:
---- ---- ------------- ---- ---- -------------
| Current data | | Previous data |
---- ---- ------------- ---- ---- ------------- ---------
ID Rnk EffectiveYear ID Rnk EffectiveYear RowStatus
---- ---- ------------- ---- ---- ------------- ---------
1 1 112 NULL NULL NULL I -- Current vs. previous: current row hasn't a previous row
3 2 112 NULL NULL NULL I -- the same thing
4 3 112 NULL NULL NULL I -- the same thing
2 4 112 NULL NULL NULL I -- the same thing
NULL NULL NULL 4 3 112 D <-- Deleted: ID 4 = 'DGH'
5 1 113 1 1 112 - -- there is no change
7 2 113 3 2 112 -
6 4 113 2 4 112 -
8 1 114 5 1 113 -
10 2 114 7 2 113 -
9 4 114 6 4 113 -
11 5 114 NULL NULL NULL I <-- Inserted: ID 11 = 'Z9'
NULL NULL NULL 8 1 114 D
NULL NULL NULL 10 2 114 D
NULL NULL NULL 9 4 114 D
NULL NULL NULL 11 5 114 D
Note: I assume that there are no duplicated rows (x.[Tax Group], x.[Tax Type], x.[Geocode]) within a year.