Design change not to use Dynamic SQL? - sql-server

Question

There is a sporadic performance problem with a batch process (nested sprocs) in SQL Server 2012. Sometimes, it takes much longer than usual.
The process rebuilds certain tables based on input parameters. RULE table has a statement column that has 70+ different statements (involving 20+ different columns) to be used in the WHERE clause in dynamic sql.
So, the DELETE, at each process run, not only has different parameters, but also different types and number of columns in the WHERE clause.
What would you recommend other than managing stats and index tuning? The dev team is open to code and schema changes.
SELECT rul.SQL_STATEMENT
FROM APP_RULE rul
LEFT JOIN APP_RULE_EXCEPTION exc
ON rul.RULE_ID = exc.RULE_ID
WHERE rul.APP_ID = #AppId
AND (exc.RULE_ID IS NULL OR exc.RULE_ID NOT IN (
SELECT RULE_ID FROM APP_RULE_EXCEPTION ))
SET #SQLStatement =
'DELETE FROM EMPLOYEE ' +
'WHERE APP_ID = ' + CAST(#AppId AS VARCHAR(10)) +
' AND EMPLOYEE_ID NOT IN (' +
'SELECT EMPLOYEE_ID FROM EMPLOYEE_NEW ' +
'WHERE '+ #SQLStatement + ')'
EXEC (#SQLStatement)
SET #SQLStatement =
'DELETE FROM ' + #TableName + ' ' +
'WHERE EMPLOYEE_ID NOT IN (' +
'SELECT EMPLOYEE_ID FROM EMPLOYEE_STG ' +
'WHERE APP_ID = ' + CAST(#AppId AS VARCHAR(10)) +''')'
EXEC (#SQLStatement)
SET #SQLStatement =
'INSERT INTO ' + #DestinationTable + ' '+'( [APP_ID], ' + #AttributeList + ')'+
'SELECT ' + CAST(#AppId AS VARCHAR(10)) + ',''' + #ExposedAttributeList +
'FROM ' + #SourceTable + ' ' +
'WHERE [DATE] = ''' + #TDate + ''''
EXEC (#SQLStatement)
The following are two sample DELETEs generated by dynamic sql.
DELETE FROM EMPLOYEE WHERE APP_ID = 103 AND APP_TYPE = 'IE' AND EMPLOYEE_ID NOT IN (
SELECT EMPLOYEE_ID FROM EMPLOYEE_NEW WHERE APP_TYPE = 'IE' )
DELETE FROM EMPLOYEE WHERE APP_ID = 103 AND APP_TYPE = 'IE' AND COUNTRY='USA' AND EMPLOYEE_ID NOT IN (
SELECT EMPLOYEE_ID FROM EMPLOYEE_NEW WHERE APP_TYPE = 'IE' AND EMPLOYEE_ID IN (
SELECT EMPLOYEE_ID FROM EMPLOYEE_EMAIL WHERE ISNULL(EMAIL_ADDRESS,'')<>'' and EType='OFFICE' ))
Thanks,
Kuzey

Answer 1

First thing you should do is to understand what the problem is. When something runs a long time randomly, it's usually case of getting non-optimal query plan, assuming you have already checked there is no blocking happening at the same time.
You should look at plan cache and see what is the difference between the plans and the CPU & IO measurements of the different cases.
Here's short SQL you can use to look at plan cache:
select top 100
SUBSTRING(t.text, (s.statement_start_offset/2)+1,
((CASE s.statement_end_offset WHEN -1 THEN DATALENGTH(t.text) ELSE s.statement_end_offset END
- s.statement_start_offset)/2) + 1) as statement_text,
t.text,
s.total_logical_reads, s.total_logical_reads / s.execution_count as avg_logical_reads,
s.total_worker_time, s.total_worker_time / s.execution_count as avg_worker_time,
s.execution_count,
max_logical_reads,
creation_time,
last_execution_time
--,cast(p.query_plan as xml) as query_plan
from sys.dm_exec_query_stats s
cross apply sys.dm_exec_sql_text (sql_handle) t
--cross apply sys.dm_exec_text_query_plan (plan_handle, statement_start_offset, statement_end_offset) p
order by s.total_logical_reads desc
That will show you all the measurements collected from plans that are still in cache. When the plan is dropped, also the measurements will be deleted. The measurements are for all the executions for the plan since it was created.
The commented out part is the plan for the statements, from there you can see what are the operators and estimated row counts. Don't look at the percentages in the plans, those are based on estimates and can be totally wrong.