Finding dublets and variations in mssql - sql-server

Question

I have a table with a whole bunch of data. In that table there is a row, with not unique ids, so there can be dublets of them - I have found them by doing this query:
SELECT theid FROM thetable
GROUP BY theid
HAVING COUNT(*) > 1
In the table there is also columns like street1,street2,city1,city2
In the list of rows from the first query where I found dublets, in those I need to check if street1 is different from street2 and city1 from city2, in any of the dublets of the given id from the first query - makes sense?
So lets say we have two rows with the same ids - in those i need to check if street1 is different from street1 in all rows with the specific id
Any tips, pointers on how to do this, I am getting blind starring at this problem, and cant seem to find the right query for that.
Thanks a bunch

Answer 1

Using a CTE will help:
;WITH CTE AS
(
SELECT theID,
Street1,
Street2,
Street3,
City,
State,
Zip,
rn = ROW_NUMBER() OVER(PARTITION BY theID ORDER BY theID)
FROM Table
-- add joins if necessary
)
SELECT oldestID = c1.theID,
oldestStreet1 = c1.Street1,
newestStreet1 = c2.Street1,
newestID = c2.theID
FROM CTE c1
INNER JOIN CTE c2 ON c2.rn = c1.rn + 1
You could also add a case statement to display matches vs. non-matches. This will help to manually identify typos (1337 Test St. vs 1337 Test Street):
;WITH CTE AS
(
SELECT theID,
Street1,
Street2,
Street3,
City,
State,
Zip,
rn = ROW_NUMBER() OVER(PARTITION BY theID ORDER BY theID)
FROM Table
-- add joins if necessary
)
SELECT oldestID = c1.theID,
oldestStreet1 = CASE WHEN c1.Street1 = c2.Street1 THEN 'Match' ELSE c1.Street1 END,
newestStreet1 = CASE WHEN c1.Street1 = c2.Street1 THEn 'Match' ELSE c2.Street1 END,
newestID = c2.theID
FROM CTE c1
INNER JOIN CTE c2 ON c2.rn = c1.rn + 1
Or you could return just the items that do not match by adding it to your INNER JOIN clause:
;WITH CTE AS
(
SELECT theID,
Street1,
Street2,
Street3,
City,
State,
Zip,
rn = ROW_NUMBER() OVER(PARTITION BY theID ORDER BY theID)
FROM Table
-- add joins if necessary
)
SELECT oldestID = c1.theID,
oldestStreet1 = c1.Street1,
newestStreet1 = c2.Street1,
newestID = c2.theID
FROM CTE c1
INNER JOIN CTE c2 ON c2.rn = c1.rn + 1 AND c1.Street1 <> c2.Street1 -- add as many of these as you need.
Keep in mind these are exact matches. You could implement static-fuzzy logic LEFT(Zip, 5) to match only on the first 5 digits of Zip Code (in case some have a zip + 4 and some don't.)

Answer 2

you can also analyse like this,
;WITH CTE AS
(
SELECT theID,
Street1,
Street2,
Street3,
City,
State,
Zip,
rn = ROW_NUMBER() OVER(PARTITION BY theID ORDER BY theID)
FROM Table
-- add joins if necessary
)
,
CTE1 as
(
select *,ROW_NUMBER()
OVER(PARTITION BY theID,Street1,Street2,City,State,Zip
oRDER BY theID) rn2 from cte where rn>2
)
select * from cte1