SQL WHERE clause performance degradation - sql-server

Question

I have a query that I am working on and it is displaying performance issues that I would not have expected. Here is the query so far.
INSERT INTO #Bridge (PolicyNumber, ProducerCode, BridgeDate, EffectiveDate, FirstName, LastName, LicenseNumber, BirthDate, Address, City, State, ZipCode)
SELECT tab.col.value('#PolicyNumber', 'VARCHAR(10)') AS PolicyNumber,
tab.col.value('#ProducerCode','VARCHAR(10)') as ProducerCode,
tab.col.value('#BridgeDate','DATETIME') AS BridgeDate,
tab.col.value('#EffectiveDate', 'DATETIME') as EffectiveDate,
tab.col.value('#FirstName', 'VARCHAR(200)') as FirstName,
tab.col.value('#LastName', 'VARCHAR(200)') as LastName,
CASE
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%0000%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%1111%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%2222%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%3333%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%4444%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%5555%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%6666%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%7777%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%8888%' THEN NULL
WHEN tab.col.value('#LicenseNumber','VARCHAR(50)') LIKE '%9999%' THEN NULL
ELSE tab.col.value('#LicenseNumber','VARCHAR(50)')
END as LicenseNumber,
tab.col.value('#BirthDate','DATETIME') as BirthDate,
REPLACE(tab.col.value('#Address1','VARCHAR(300)'), ' APT ',' #') as Address1,
tab.col.value('#City','VARCHAR(300)') as City,
tab.col.value('#State','VARCHAR(5)') as State,
tab.col.value('#ZipCode','VARCHAR(10)') as Zip
FROM #xml.nodes('//rows/datarow') as tab(col)
SELECT B.PolicyNumber,
B.ProducerCode,
B.BridgeDate,
B.EffectiveDate,
H.current_policy,
H.cancel_date,
H.first_eff_date,
H.display_address,
H.city,
H.state,
H.zip
FROM #Bridge B
LEFT JOIN (
SELECT P.policy_id,
P.current_policy,
CASE
WHEN A.pobox <> '' THEN 'PO BOX ' + REPLACE(A.pobox,'PO BOX ','')
ELSE RTRIM(A.house_num + ' ' + A.street_name + ' ' + CASE
WHEN A.apt_num = '' THEN ''
ELSE '#' + A.apt_num
END)
END as display_address,
A.pobox,
A.house_num,
A.street_name,
A.apt_num,
A.city,
MAX(A.policyimage_num) as policimage_num, --this is just to limit the results to the most recent
S.state,
A.zip,
P.first_eff_date,
P.cancel_date
FROM Diamond.dbo.Policy P WITH (NOLOCK)
LEFT JOIN Diamond.dbo.Address A WITH (NOLOCK)
ON P.policy_id = A.policy_id
AND A.nameaddresssource_id = 3
LEFT JOIN Diamond.dbo.State S WITH (NOLOCK)
ON A.state_id = S.state_id
WHERE A.state_id IS NOT NULL
AND P.current_policy NOT IN (SELECT PolicyNumber FROM #Bridge)
GROUP BY P.policy_id,
P.current_policy,
P.cancel_date,
P.first_eff_date,
A.pobox,
A.house_num,
A.street_name,
A.apt_num,
A.city,
S.state,
A.zip) AS H
ON B.Address = H.display_address
AND B.State = H.state
AND B.City = H.city
AND SUBSTRING(B.ZipCode,1,5) = SUBSTRING(H.Zip,1,5)
AND B.PolicyNumber != H.current_policy
WHERE H.current_policy IS NOT NULL
This query, run by itself, finishes in about 1:30 seconds. But if I add the following to the WHERE clause
AND B.EffectiveDate != H.first_eff_date
Suddenly the query takes far longer to return results. (We are at over 15 minutes and still going while I am writing this) I would think that simply having a clause to weed out a few additional rows wouldn't have such a drastic effect, but apparently it does. I how to get around it, I am just curious if anyone has any ideas as to why it has this effect?

Answer 1

Without having a hands on I can only guess at this, but here are some places I think you can tidy up and probably shave off run time.
1, You duplicate the effort required to make sure policy numbers don't match. Pick one of the two you have, not both. I would suggest trying both see which is faster.
i.e. this:
AND P.current_policy NOT IN (SELECT PolicyNumber FROM #Bridge)
Will do the same as this, you don't need both.
AND B.PolicyNumber != H.current_policy
2, It's worth a try to remove all that grouping from your sub query - you don't actually use policimage_num for anything. So why do the grouping? If you are worried that many rows are returned from Address, then you can use DISTINCT on your column set instead, that may be faster.
3, Is A.state_id a nullable value? If not consider trying an INNER JOIN to Address and removing the null check.
4, In all honesty I'm not seeing an obvious reason for that subquery at all, it seems to be over-complicating matters. Can you not simply join the tables together without it (again using DISTINCT if required)?
In other words get tweaking, I bet you can get it below the original run time if you try a few of these ideas.