Email data from several tables which is joined with inserted table - sql-server

I have four tables:
Activity:
ActivityID(PK) ActivityName CustomerID(FK) UserId(FK)
1 Lead Gen 1st 50 U1
2 Lead Gen 2nd 60 U2
Customer:
CustomerID(PK) CustomerNumber CustomerName
50 C0150 cust50 ltd
60 C0160 cust60 ltd
User:
UserID(PK) UserName Email
U1 Mr. X X#cat.com
U2 Mr. Y Y#cat.com
UserActivity:
UserActivityID(PK) UserID(FK) ActivityID(FK)
888 U1 1
889 U2 2
I want to send an email (i.e. Email:X#cat.com) to the users related to the activity (i.e. ActivityId:1) if any insert happens in Activity Table (SQL Server 2008-R2).
The email body should contain the ActivityId, ActivityName, CustomerNumber and CustomerName.
The trigger has to do the above mentioned and the result should be like this in the email:
ActivityID:1, ActivityName:Lead Gen 1st created for CustomerNumber: C0150 & CustomerName: cust50 ltd
Here is my code:
CREATE TRIGGER [dbo].[Activity_Insert_Mail_Notification]
ON [dbo].[Activity]
AFTER INSERT
AS
BEGIN
DECLARE #ActivityID varchar(2000)
DECLARE #ActivityName varchar (2000)
Select #ActivityID=inserted.ActivityID,#ActivityName=inserted.ActivityName
From inserted
DECLARE #CustomerNo varchar(2000)
DECLARE #CustomerName varchar(2000)
Select #CustomerNo = B.[CustomerNumber]
,#CustomerName= B.[CustomerName]
from [dbo].[Activity] A
inner join [dbo].[Customer] B
on A.[CustomerID]=B.[CustomerID]
DECLARE #email VARCHAR(2000)
SELECT #email = RTRIM(U.[Email]) + ';'
FROM [dbo].[Activity] A
left join [dbo].[UserActivity] UA
inner join [dbo].[User] U
on UA.[UserID]=U.[UserID]
on A.[ActivityID]=UA.[ActivityID]
WHERE U.[Email]<> ''
DECLARE #content varchar (2000)
= 'ActivityID:' + #ActivityId + ' '
+ ',ActivityName:' + #ActivityName + ' '
+ 'has been created for' + 'CustomerNumber: ' + #CustomerNo
+ ' ' + '&CustomerName: ' + #CustomerName
EXEC msdb.dbo.sp_send_dbmail
#profile_name = 'LEADNOTIFY'
,#recipients = #email
,#subject = 'New Lead Found'
,#body = #content
,#importance ='HIGH'
END
The problem is in my code that I can't fetch the customer data and email from the respective tables properly.

I have written some code below which will loop through all the effected rows and send an email for each.
However before you read that, I would highly recommend (as #HABO commented) on using a different approach. Triggers are fine for some tasks, but 2 key things you want to keep in mind when using triggers:
Ensure its obvious to anyone developing the system that there are triggers - there is nothing worse as a developer than finding stuff magically happening on what seems like a simple CRUD operation.
Do whatever you do in your trigger as fast as possible because you are holding locks which not only affect the current session, but could easily affect other users as well. Ideally therefore you want to be performing set-based operations, not RBAR (Row By Agonising Row) operations.
Sending emails is a terrible thing to do inside a trigger because its not uncommon to be forced to wait for an SMTP server to respond. If you wish to trigger emails, a better way is to use the trigger to insert the email data into a queuing table and then have a service elsewhere which de-queues these emails and sends them.
All that aside the following code shows one way to handle the Inserted pseudo-table when you want to perform RBAR operations. Because in SQL Server the Inserted pseudo-table (and the Deleted pseudo-table) will contain the number of rows effected by the operation i.e. 0-N. Also I've hopefully joined your tables to correctly obtain the required information.
CREATE TRIGGER [dbo].[Activity_Insert_Mail_Notification]
ON [dbo].[Activity]
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
DECLARE #ActivityID VARCHAR(2000), #ActivityName VARCHAR(2000), #CustomerNo VARCHAR(2000), #CustomerName VARCHAR(2000), #Email VARCHAR(2000), #Content VARCHAR(2000);
-- Get all the relevant information into a temp table
SELECT ActivityID, ActivityName, C.CustomerNumber, C.CustomerName, RTRIM(U.[Email]) + ';' Email, CONVERT(BIT, 0) Handled
INTO #ActivityTriggerTemp
FROM Inserted I
INNER JOIN Customer C on C.CustomerID = I.CustomerID
INNER JOIN UserActivity UA on UA.ActivityID = I.ActivityID
INNER JOIN [USER] U on U.UserID = UA.UserID;
-- Loop through the temp table sending an email for each row, then setting the row as 'handled' to avoid sending it again.
WHILE EXISTS (SELECT 1 FROM #ActivityTriggerTemp WHERE Handled = 0) BEGIN
SELECT TOP 1 #ActivityID = ActivityID, #ActivityName = ActivityName, #CustomerNumber = CustomerNumber, #CustomerName = CustomerName, #Email = Email
FROM #ActivityTriggerTemp
WHERE Handled = 0;
-- Build the body of the email
set #Content = 'ActivityID:' + #ActivityId + ' '
+ ',ActivityName:' + #ActivityName + ' '
+ 'has been created for' + 'CustomerNumber: ' + #CustomerNo
+ ' ' + '&CustomerName: ' + #CustomerName;
-- Send the email
EXEC msdb.dbo.sp_send_dbmail
#profile_name = 'LEADNOTIFY'
, #recipients = #Email
, #subject = 'New Lead Found'
, #body = #Content
, #importance ='HIGH';
UPDATE #ActivityTriggerTemp SET
Handled = 1
WHERE ActivityID = #ActivityID AND ActivityName = #ActivityName AND CustomerNumber = #CustomerNumber AND CustomerName = #CustomerName AND Email = #Email;
END;
END

Related

Building extended properties from existing table

I need to document metadata for the fields of multiple tables. I intend to do this by appending the tables with extended properties. The only way I've done this in the past is using the built-in stored procedure which looks like this:
ALTER procedure [sys].[sp_addextendedproperty]
#name sysname,
#value sql_variant = NULL,
#level0type varchar(128) = NULL,
#level0name sysname = NULL,
#level1type varchar(128) = NULL,
#level1name sysname = NULL,
#level2type varchar(128) = NULL,
#level2name sysname = NULL
as
declare #ret int
if datalength(#value) > 7500
begin
raiserror(15097,-1,-1)
return 1
end
if #name is null
begin
raiserror(15600,-1,-1,'sp_addextendedproperty')
return (1)
end
execute #ret = sys.sp_validname #name
if (#ret <> 0)
begin
raiserror(15600,-1,-1,'sp_addextendedproperty')
return (1)
end
BEGIN TRANSACTION
begin
EXEC %%ExtendedPropertySet().AddValue(Name = #name, Value =
#value, Level0type = #level0type, Level0name = #level0name, Level1type =
#level1type, Level1name = #level1name, Level2type = #level2type,
Level2name = #level2name)
IF ##error <> 0
begin
COMMIT TRANSACTION
return (1)
end
end
COMMIT TRANSACTION
return (0)
The way I've done this in the past is by just passing strings to the variables:
EXEC sys.sp_addextendedproperty
#name=N'desc_en',
#value=N'Sex. 1 denotes male, 2 denotes female.' ,
#level0type=N'SCHEMA',
#level0name=N'dbo',
#level1type=N'TABLE',
#level1name=N'MyTable',
#level2type=N'COLUMN',
#level2name=N'sex'
GO
But now that I have to do this for multiple tables (and ideally write some re-usable procedure), I'm attempting to write a function that takes dynamic (i.e. SELECT) arguments.
For instance, take the following tables. The first has a couple of fields that need extended properties, whereas the other table has two fields containing that information:
dbo.users:
id | name | sex | year
-------------------------
1 | 'john' | 1 | 2019
2 | 'jane' | 2 | 2019
dbo.metadata:
id | property_en | desc_en
-------------------------
1 | sex | Sex. 1 denotes male, 2 denotes female
2 | year | The year of the event record
desired ep:
EXEC sys.sp_addextendedproperty
#name=N'desc_en',
#value=N'Sex. 1 denotes male, 2 denotes female' ,
#level0type=N'SCHEMA',
#level0name=N'dbo',
#level1type=N'TABLE',
#level1name=N'users',
#level2type=N'COLUMN',
#level2name=N'sex'
Any suggestions to derive EPs directly from a source and metadata table like that as an iteration? The first set of arguments have to come from the source table itself (i.e., information about the schema, table name and column name), whereas the second from the metadata table itself (i.e. property_en and desc_en).
Essentially, the first table (t1) provides:
#level0name
#level1name
And t2:
#name
#value
And the join is on #level2name, in other words where t1.clmns.name = t2.property_en
Here we go. Dynamic sql like this is a bit tricky if you haven't done it very much. In essence I am just using the system tables to get the information needed based on the data in your metadata table. Then we just build up a big sql string with a bunch of stored proc calls in it. Finally we just fire it off to run.
Please note, this will find any column that matches in any table based on the name in metadata. If you need to refine that for something like only tables with both columns or something you will need to adjust this a little bit.
create table metadata
(
id int
, property_en sysname
, desc_en varchar(500)
)
insert metadata values
(1, 'sex', 'Sex. 1 denotes male, 2 denotes female')
, (2, 'year', 'The year of the event record')
declare #SQL nvarchar(max) = ''
select #SQL = #SQL + 'EXEC sys.sp_addextendedproperty #name=N''' + d.property_en + ''', #value=N''' + d.desc_en + ''', #level0type=N''SCHEMA'', #level0name=N''' + s.name + ''', #level1type=N''TABLE'', #level1name=N''' + t.name + ''', #level2type=N''COLUMN'', #level2name=N''' + d.property_en + ''';'
from metadata d
join sys.columns c on c.name = d.property_en
join sys.tables t on t.object_id = c.object_id
join sys.schemas s on s.schema_id = t.schema_id
select #SQL
--once you are comfortable that the dynamic sql works uncomment the line below.
--exec sp_executesql #SQL
Here is an 'instead of' trigger based mechanism based on prior work by Cade Roux and "Phil Factor" (credits in the comments)
https://github.com/phrrngtn/rule4/blob/main/sql/extended_properties.sql
This creates some views RULE4.extended_property and RULE4.extended_property_aux that you can treat as normal table and insert/update/delete extended properties. The instead of trigger proxies this onto calls to the corresponding extended properties sproc.

Automates sending email using SQL Server 2008 T-SQL, with custom message title and body

This question is the follow-up of this thread
Given that I've created two simple tables User and Item:
User (Id, Name, Email)
Item (Id, CreatedDate, EmailSent, UserId)
I am able to create an SQL Server Agent job that periodically runs the following script:
USE test
DECLARE #emails varchar(1000)
SET #emails = STUFF((SELECT ';' + u.Email FROM [user] u JOIN [item] i
ON u.ID = i.UserId
WHERE i.EmailSent = 0 AND DATEDIFF(day, i.CreatedDate, GETDATE()) >= 30
FOR XML PATH('')),1,1,'')
/** send emails **/
EXEC msdb.dbo.sp_send_dbmail
#profile_name='test',
#recipients=#emails,
#subject='Test message',
#body='This is the body of the test message.'
The purpose is to get any item that has been created for >= 30 days and then send a reminder email to its user. The EmailSent is checked first to exclude items already reminded.
I want to improve this script as follows:
There's a separate query that does the SELECT part and stores the result into a variable so that it can be reused in the email sending query, and in an UPDATE query that sets selected items' EmailSent to True (done after emails sent).
Customise the title and body of the message with user's Name and item's Id. Something like: Dear Name, the item Id has been created for 30 days or more.
Anybody has any idea of how these improvement could be done?
I've managed to have this so far
USE test
-- 1) Update selected items to state 1
UPDATE [PickingList]
SET ReminderState = 1
WHERE ReminderState = 0 AND DATEDIFF(day, CreatedDate, GETDATE()) >= 30
DECLARE #userId INT
DECLARE #email NVARCHAR(100)
DECLARE #plId INT
DECLARE #getId CURSOR
-- 2) Process those having state 1
SET #getId = CURSOR FOR
SELECT u.ID, u.Email, pl.ID
FROM [User] u JOIN [PickingList] pl
ON u.ID = pl.UserId
WHERE pl.ReminderState = 1
OPEN #getId
FETCH NEXT
FROM #getId INTO #userId, #email, #plId
WHILE ##FETCH_STATUS = 0
BEGIN
/** send emails **/
DECLARE #pId VARCHAR(1000)
DECLARE #title VARCHAR(1000)
DECLARE #body VARCHAR(8000)
SET #pId = CAST(#plId AS VARCHAR(16))
SET #title = #pId + ' was created more than 30 days ago'
SET #body = 'The following picking list ' + #pId + ' blah blah'
DECLARE #code INT
SET #code = -1
EXEC #code = msdb.dbo.sp_send_dbmail
#profile_name='test',
#recipients=#email,
#subject=#title,
#body=#body
-- 3) Log the email sent and update state to 2
-- Below is what I want to do, but ONLY when the it can be sure that
-- the email has been delivered
INSERT INTO [PickingListEmail] (UserId, PickingListId, SentOn)
VALUES (#userId, #plId, GETDATE())
UPDATE [PickingList] SET ReminderState = 2 WHERE ReminderState = 1
FETCH NEXT
FROM #getId INTO #userId, #email, #plId
END
CLOSE #getId
DEALLOCATE #getId
In step (3), before saving the email sent and update the item to state 2 (processed), I would want to make sure that the email has been sent, based on the data fetched from sysmail_log, as sp_send_dbmail would only care about whether it can send the mail to the SMTP server, so will return the success code 0 even when the sending fails.
Something like this:
meaning of the values of sent_status on msdb.dbo.sysmail_mailitems
or
Check If sp_send_dbmail Was Successful

SQL Server: stored procedure and send dbmail

The reason of creating a stored procedure is to schedule a job to send a biweekly report to our staff (coordinators) using SQL Server db mail.
I'm having problems with getting it to work the right way. I don't usually work with cursors but couldn't find other choices.
Here's the issue. I tested the query by set criteria to send only to one Coordinator with one record "if #Coord_Email = 'lamez.sw1#gmail.com', where n.id = '43422546'". However the query been running over 5 minutes so i had to cancel it.
ALTER PROCEDURE [dbo].[sp_MZ_Coord_rpt_s9]
AS
BEGIN
DECLARE #Member_ID VARCHAR(20)
DECLARE Report_S9 CURSOR FOR
SELECT id
FROM name
WHERE status = 'a'
OPEN Report_S9
FETCH NEXT FROM Report_S9 INTO #member_ID
WHILE ##FETCH_STATUS = 0
BEGIN
DECLARE #Coord_ID Varchar(20)
DECLARE #CO_ID Varchar(20)
DECLARE #Coord_Name Varchar(50)
DECLARE #Coord_Email Varchar(50)
SELECT #CO_ID = ID
FROM Relationship
WHERE id = #Member_ID
SELECT #Coord_ID = target_id
FROM Relationship
WHERE RELATION_TYPE = 'CO'
SELECT #Coord_Name = FULL_NAME
FROM Name
WHERE ID = #Coord_ID
SELECT #Coord_Email = email
FROM Name
WHERE id = #Coord_ID
IF #Coord_Email <> ''
BEGIN
SELECT
n.id, n.CO_ID, n.FULL_NAME, a.TRANSACTION_DATE, a.UF_1, r.TARGET_ID
FROM name n
INNER JOIN activity a ON n.id = a.id
INNER JOIN Tops_Profile tp ON a.id = tp.ID
INNER JOIN Relationship r ON n.CO_ID = r.ID
WHERE
n.id = #member
AND UF_1 <> ''
AND (DATEDIFF(dd, TRANSACTION_DATE, GETDATE()) < 2)
AND r.RELATION_TYPE = 'co'
ORDER BY
TRANSACTION_DATE
EXEC msdb..sp_send_dbmail
#profile_name = 'TOPS.ADMIN',
#recipients = #Coord_Email,
--#blind_copy_recipients = ,
#subject = 'S9 Report'
End
FETCH NEXT FROM Report_S9 INTO #member_ID
END
CLOSE Report_S9
DEALLOCAT Report_S9
End
Any help is greatly appreciated.
The FETCH NEXT should be outside of your check for null. You need to continue the loop, even when there is nothing to do.

Retrieve column definition for stored procedure result set

I'm working with stored procedures in SQL Server 2008 and I've come to learn that I have to INSERT INTO a temp table that has been predefined in order to work with the data. That's fine, except how do I figure out how to define my temp table, if I'm not the one that wrote the stored procedure other than listing its definition and reading through the code?
For example, what would my temporary table look like for `EXEC sp_stored_procedure'? That is a simple stored procedure, and I could probably guess at the data types, but it seems there must be a way to just read the type and length of the columns returned from executing the procedure.
So let's say you have a stored procedure in tempdb:
USE tempdb;
GO
CREATE PROCEDURE dbo.my_procedure
AS
BEGIN
SET NOCOUNT ON;
SELECT foo = 1, bar = 'tooth';
END
GO
There is a quite convoluted way you can go about determining the metadata that the stored procedure will output. There are several caveats, including the procedure can only output a single result set, and that a best guess will be made about the data type if it can't be determined precisely. It requires the use of OPENQUERY and a loopback linked server with the 'DATA ACCESS' property set to true. You can check sys.servers to see if you already have a valid server, but let's just create one manually called loopback:
EXEC master..sp_addlinkedserver
#server = 'loopback',
#srvproduct = '',
#provider = 'SQLNCLI',
#datasrc = ##SERVERNAME;
EXEC master..sp_serveroption
#server = 'loopback',
#optname = 'DATA ACCESS',
#optvalue = 'TRUE';
Now that you can query this as a linked server, you can use the result of any query (including a stored procedure call) as a regular SELECT. So you can do this (note that the database prefix is important, otherwise you will get error 11529 and 2812):
SELECT * FROM OPENQUERY(loopback, 'EXEC tempdb.dbo.my_procedure;');
If we can perform a SELECT *, we can also perform a SELECT * INTO:
SELECT * INTO #tmp FROM OPENQUERY(loopback, 'EXEC tempdb.dbo.my_procedure;');
And once that #tmp table exists, we can determine the metadata by saying (assuming SQL Server 2005 or greater):
SELECT c.name, [type] = t.name, c.max_length, c.[precision], c.scale
FROM sys.columns AS c
INNER JOIN sys.types AS t
ON c.system_type_id = t.system_type_id
AND c.user_type_id = t.user_type_id
WHERE c.[object_id] = OBJECT_ID('tempdb..#tmp');
(If you're using SQL Server 2000, you can do something similar with syscolumns, but I don't have a 2000 instance handy to validate an equivalent query.)
Results:
name type max_length precision scale
--------- ------- ---------- --------- -----
foo int 4 10 0
bar varchar 5 0 0
In Denali, this will be much, much, much easier. Again there is still a limitation of the first result set but you don't have to set up a linked server and jump through all those hoops. You can just say:
DECLARE #sql NVARCHAR(MAX) = N'EXEC tempdb.dbo.my_procedure;';
SELECT name, system_type_name
FROM sys.dm_exec_describe_first_result_set(#sql, NULL, 1);
Results:
name system_type_name
--------- ----------------
foo int
bar varchar(5)
Until Denali, I suggest it would be easier to just roll up your sleeves and figure out the data types on your own. Not just because it's tedious to go through the above steps, but also because you are far more likely to make a correct (or at least more accurate) guess than the engine will, since the data type guesses the engine makes will be based on runtime output, without any external knowledge of the domain of possible values. This factor will remain true in Denali as well, so don't get the impression that the new metadata discovery features are a be-all end-all, they just make the above a bit less tedious.
Oh and for some other potential gotchas with OPENQUERY, see Erland Sommarskog's article here:
http://www.sommarskog.se/share_data.html#OPENQUERY
It looks like in SQL 2012 there is a new SP to help with this.
exec sp_describe_first_result_set N'PROC_NAME'
https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-describe-first-result-set-transact-sql
A less sophisticated way (that could be sufficient in some cases): edit your original SP, after the final SELECT and before the FROM clause add INSERT INTO tmpTable to save the SP result in tmpTable.
Run the modified SP, preferably with meaningful parameters in order to get actual data. Restore the original code of the procedure.
Now you can get the script of tmpTable from SQL server management studio or query sys.columns to get fields descriptions.
Here is some code that I wrote. The idea is (as someone else stated) is to get the SP code, modify it and execute it. However, my code does not change the original SP.
First step, get the definition of the SP, strip the 'Create' part out and get rid of the 'AS' after the declaration of parameters, if exists.
Declare #SPName varchar(250)
Set nocount on
Declare #SQL Varchar(max), #SQLReverse Varchar(MAX), #StartPos int, #LastParameterName varchar(250) = '', #TableName varchar(36) = 'A' + REPLACE(CONVERT(varchar(36), NewID()), '-', '')
Select * INTO #Temp from INFORMATION_SCHEMA.PARAMETERS where SPECIFIC_NAME = 'ADMIN_Sync_CompareDataForSync'
if ##ROWCOUNT > 0
BEGIN
Select #SQL = REPLACE(ROUTINE_DEFINITION, 'CREATE PROCEDURE [' + ROUTINE_SCHEMA + '].[' + ROUTINE_NAME + ']', 'Declare')
from INFORMATION_SCHEMA.ROUTINES
where ROUTINE_NAME = #SPName
Select #LastParameterName = PARAMETER_NAME + ' ' + DATA_TYPE +
CASE WHEN CHARACTER_MAXIMUM_LENGTH is not null THEN '(' +
CASE WHEN CHARACTER_MAXIMUM_LENGTH = -1 THEN 'MAX' ELSE CONVERT(varchar,CHARACTER_MAXIMUM_LENGTH) END + ')' ELSE '' END
from #Temp
WHERE ORDINAL_POSITION =
(Select MAX(ORDINAL_POSITION)
From #Temp)
Select #StartPos = CHARINDEX(#LastParameterName, REPLACE(#SQL, ' ', ' '), 1) + LEN(#LastParameterName)
END
else
Select #SQL = REPLACE(ROUTINE_DEFINITION, 'CREATE PROCEDURE [' + ROUTINE_SCHEMA + '].[' + ROUTINE_NAME + ']', '') from INFORMATION_SCHEMA.ROUTINES where ROUTINE_NAME = #SPName
DROP TABLE #Temp
Select #StartPos = CHARINDEX('AS', UPPER(#SQL), #StartPos)
Select #SQL = STUFF(#SQL, #StartPos, 2, '')
(Note the creation of a new table name based on a unique identifier)
Now find the last 'From' word in the code assuming this is the code that does the select that returns the result set.
Select #SQLReverse = REVERSE(#SQL)
Select #StartPos = CHARINDEX('MORF', UPPER(#SQLReverse), 1)
Change the code to select the resultset into a table (the table based on the uniqueidentifier)
Select #StartPos = LEN(#SQL) - #StartPos - 2
Select #SQL = STUFF(#SQL, #StartPos, 5, ' INTO ' + #TableName + ' FROM ')
EXEC (#SQL)
The result set is now in a table, it does not matter if the table is empty!
Lets get the structure of the table
Select * from INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #TableName
You can now do your magic with this
Don't forget to drop that unique table
Select #SQL = 'drop table ' + #TableName
Exec (#SQL)
Hope this helps!
In order to get queryable resultset sys.dm_exec_describe_first_result_set(SQL Server 2012) could be used:
SELECT column_ordinal, name, system_type_name
FROM sys.dm_exec_describe_first_result_set(N'EXEC stored_procedure_name', NULL, 0);
db<>fiddle demo
This soultion has few limitations though for instance SP cannot use temporary tables.
If you are working in an environment with restricted rights where things like loopback linked server seems black magic and are definitely "no way!", but you have a few rights on schema and only a couple of stored procedure to process there is a very simple solution.
You can use the very helpful SELECT INTO syntax, which will create a new table with result set of a query.
Let's say your procedure contains the following Select query :
SELECT x, y, z
FROM MyTable t INNER JOIN Table2 t2 ON t.id = t2.id...
Instead replace it by :
SELECT x, y, z
INTO MyOutputTable
FROM MyTable t INNER JOIN Table2 t2 ON t.id = t2.id...
When you will execute it, it will create a new table MyOutputTable with the results returned by the query.
You just have to do a right click on its name to get the table definition.
That's all !
SELECT INTO only require the ability to create new tables and also works with temporary tables (SELECT... INTO #MyTempTable), but it could be harder to retrieve the definition.
However of course if you need to retrieve the output definition of a thousands SP, it's not the fastest way :)

Why does ad-hoc SQL run faster than executing the same code in a stored procedure?

I have a stored procedure that processes phones and addresses in a batch in SQL server 2005
If I execute the stored procedure in takes 2 hours. But if I run the same code and the same batch ad hoc it takes 2 seconds.
I have try the following steps to make it faster but they have not worked:
Re indexing the entire database
SET ANSI_NULLS ON;
DBCC FreeProcCache
DBCC DROPCLEANBUFFERS
Here is the basic code
USE [MyDB]
GO
/****** Object: StoredProcedure [myschema].[ProccesBatch] Script Date: 06/30/2011 10:37:33 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [myschema].[ProccesBatch]
-- Add the parameters for the stored procedure here
(#BatchId int)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SET ANSI_NULLS ON;
-- AD Hoc TESTING ONLY. This gets uncommented when running ad hoc.
-- DECLARE #BatchId Int
-- SET #BatchId = 59
DECLARE #MyList AS VARCHAR (500)
DECLARE #MySICList AS VARCHAR (500)
DECLARE #MyType AS CHAR (1)
DECLARE #MyProvider AS VARCHAR (500)
DECLARE #MyState AS VARCHAR (2)
DECLARE #MyCityList AS VARCHAR (500)
DECLARE #MyZipList AS VARCHAR (500)
DECLARE #MyStyle AS VARCHAR (1)
DECLARE #MySource AS VARCHAR (150)
DECLARE #MyStartDate AS DATETIME
DECLARE #MyEndDate AS DATETIME
DECLARE #MyCampaign AS BIT
DECLARE #CheckExist AS INT
SET #CheckExist = 0
--
-- 1. Check if Campaign Exist.
--
SELECT
#CheckExist = Id
FROM myschema.Destination
WHERE Id = #BatchId
IF #CheckExist > 0
BEGIN
RAISERROR('Creation has already been processed', 16, 1)
RETURN
END
--
-- 2. Get Header and parameters for controlling process.
--
SELECT
#MyList = ISNULL(LeadBatchHeaderList,''),
#MySICList = ISNULL(SICCodeList,''),
#MyType = ISNULL(MyType,''),
#MyProvider = ISNULL(LDCList,''),
#MyState = ISNULL([State],''),
#MyCityList = ISNULL(CityList,''),
#MyZipList = ISNULL(ZipCodeList,''),
#MyStyle = ISNULL(Commodities,''),
#MySource = ISNULL(LeadSource,''),
#MyStartDate = ISNULL(HeaderCreationStart,''),
#MyEndDate = ISNULL(HeaderCreationEnd,''),
#MyCampaign = ISNULL(AllCampaign ,'')
FROM myschema.Header
WHERE ID = #BatchId
IF ##ROWCOUNT < 1
BEGIN
RAISERROR('header id was not found', 16, 1)
RETURN
END
-- Place Commas for charindex
IF #MyList > ''
SET #MyList = ',' + #MyList + ','
IF #MySICList > ''
SET #MySICList = ',' + #MySICList + ','
IF #MyProvider > ''
SET #MyProvider = ',' + #MyProvider + ','
IF #MyCityList > ''
SET #MyCityList = ',' + #MyCityList + ','
IF #MyZipList > ''
SET #MyZipList = ',' + #MyZipList + ','
--
-- 3. Add qualifying leads.
--
INSERT INTO myschema.Destination
(Id, LeadBatchDetailId, CustomerIdOne, CustomerIdTwo, MyProviderOne, MyProviderTwo, SicCode, SicDesc, SicCode2, SicDesc2,
MyType, Company, CompanyURL, Title, Salutation, Suffix, FullName, FirstName, MiddleInitial,
LastName, Email, MyPhone, Work, Cell, Home, Fax, Ext, Address1, Address2, City, [State],
Zip5, Zip4, County, TSR, EmployeeSize, Revenue, MyProviderOne, MyProviderTwo, CustomerUsageOne, CustomerUsageTwo, MyExpenses, Remarks, Decline,
WhyLeft, PCC, RCC, PCC, SCC)
SELECT
#BatchId, d.ID, d.CustomerIdOne, d.CustomerIdTwo, d.MyProviderOne, d.MyProviderTwo, d.SicCode, d.SicDesc, d.SicCode2, d.SicDesc2,
d.MyType, d.Company, d.CompanyURL, d.Title, d.Salutation, d.Suffix, d.FullName, d.FirstName, d.MiddleInitial,
d.LastName, d.Email, d.MyPhone, d.Work, d.Cell, d.Home, d.Fax, d.Ext, d.Address1, d.Address2, d.City, d.[State],
d.Zip5, d.Zip4, d.County, d.TSR, d.EmployeeSize, d.Revenue, d.MyProviderOne, d.MyProviderTwo,d.CustomerUsageOne, d.CustomerUsageTwo, d.MyExpenses, d.Remarks, d.Decline,
d.WhyLeft, d.PCC, d.RCC, d.PCC, d.SCC
FROM myschema.Source as d
JOIN myschema.Summary as h ON d.MyId = h.ID
JOIN myschema.source AS s ON h.Id = s.ID
WHERE
-- MyId.
(#MyList = '' OR (charindex(',' + CAST(d.MyId AS VARCHAR) + ',', #MyList) > 0)) AND
-- SIC Code.
(#MySICList = '' OR (charindex(',' + CAST(d.SicCode AS VARCHAR) + ',', #MySICList) > 0)) AND
-- My Types
(#MyType = '' OR #MyType = 'A' OR d.MyType = #MyType OR h.DefaultMyType = #MyType) AND
-- MYProviders
((#MyProvider = '' OR (charindex(',' + CAST(d.MyProviderOne AS VARCHAR) + ',', #MyProvider) > 0)) OR
(#MyProvider = '' OR (charindex(',' + CAST(d.MyProviderTwo AS VARCHAR) + ',', #MyProvider) > 0))) AND
-- State.
(#MyState = '' OR d.[State] = #MyState) AND
-- City.
(#MyCityList = '' OR (charindex(',' + d.City + ',', #MyCityList) > 0)) AND
-- Zip Code.
(#MyZipList = '' OR (charindex(',' + d.Zip5 + ',', #MyZipList) > 0)) AND
-- LeadSource
(#MySource = '' OR s.MySource = #MySource) AND
-- Between Dates
(#MyStartDate = '' AND #MyEndDate = '' OR h.CreationDate BETWEEN #MyStartDate AND #MyEndDate) AND
-- Mystyle
((#MyStyle = 'A' AND (d.MyProviderOne IS NOT NULL OR d.MyProviderOne > 0 OR d.CustomerUsageOne > 0)) OR
(#MyStyle = 'B' AND (d.MyProviderTwo IS NOT NULL OR d.MyProviderTwo > 0 OR d.CustomerUsageTwo > 0)) OR
(#MyStyle = '' OR #MyStyle IS NULL)) AND
-- Source parameters are important. Only processed finished batches.
(h.UseThisRecord = 1) AND
(h.[status] = 'Finished') AND
(d.MyDuplicate IS NULL) AND
(d.DoNotUseFlag IS NULL) AND
(d.DoNotUseIFlag IS NULL) AND
(d.CustomerHome IS NULL) AND
(d.CustomerWork IS NULL) AND
(d.LeadDuplicate IS NULL) AND
(d.MyPhone >'' OR d.MyPhone <> NULL) AND
((CAST(FLOOR( CAST( h.ExpirationDate AS FLOAT ) )AS DATETIME) > CAST(FLOOR( CAST( GETDATE() AS FLOAT ) )AS DATETIME)) OR
h.ExpirationDate IS NULL)
--
-- 4. Flag Phone Duplicates inside myschema.Destination
--
UPDATE T1
SET DeleteFlag = 1
FROM myschema.Destination T1, myschema.Destination T2
WHERE
T1.MyPhone = T2.MyPhone AND
T1.FullName = T2.FullName AND
T1.Address1 = T2.Address1 AND
T1.City = T2.City AND
T1.[State] = T2.[State] AND
T1.Zip5 = T2.Zip5 AND
T1.MyPhone <> '' AND
T1.Id = T2.Id AND -- This will flag the batch itself
T1.Id = #BatchId AND
T1.Id < T2.Id -- This will leave the highest Id unflagged (latest record)
--
-- 5. Duplicate Contact Flag. All Records
--
IF #MyCampaign = 1
UPDATE T1
SET DeleteFlag = 1
FROM myschema.Destination T1, myschema.Destination T2
WHERE
T1.MyPhone = T2.MyPhone AND
T1.FullName = T2.FullName AND
T1.Address1 = T2.Address1 AND
T1.City = T2.City AND
T1.[State] = T2.[State] AND
T1.Zip5 = T2.Zip5 AND
T1.MyPhone <> '' AND
T1.Id = #BatchId AND
T1.Id <> T2.Id -- Process against other batches
--
-- 6. Active Flag
--
IF #MyCampaign <> 1
UPDATE T1
SET DeleteFlag = 1
FROM myschema.Destination T1, myschema.Destination T2
JOIN myschema.Header H ON T2.Id = H.ID
WHERE
T1.MyPhone = T2.MyPhone AND
T1.FullName = T2.FullName AND
T1.Address1 = T2.Address1 AND
T1.City = T2.City AND
T1.[State] = T2.[State] AND
T1.Zip5 = T2.Zip5 AND
T1.MyPhone <> '' AND
T1.Id = #BatchId AND
T1.Id <> T2.Id AND -- Process against other batches
H.ActiveBatch = 1 -- Only Active
--
-- 7. Delete DeleteFlag rows. Check for Id just in case
--
IF #BatchId > 0
DELETE FROM myschema.Destination
WHERE
(DeleteFlag = 1) AND (Id = #BatchId)
--
-- 8. Update header with date last run
--
UPDATE myschema.Header
SET DateLastRun = GETDATE()
WHERE ID = #BatchId
END
GO
Thanks,
Christian
This often happens when you use parameters in the proc and constants ad-hoc
SQL Server tries to generate a re-usable plan usually. With constants it doesn't need to because it can never be re-used for different constants.
There are other options such as SET options for the stored procedure or datatype mismatches.. but we have no further information to go on.
1 - Show the code.
2 - without seeing the code, I'm going to guess you have multiple parameters you are passing to the stored proc, and you are having a parameter sniffing issue.
This may be due to parameter sniffing. Have you compared the execution plans? If you grab the free Plan Explorer from http://www.sqlsentry.net/plan-explorer/sql-server-query-view.asp you can see quite clearly, among other things, the runtime vs. compiled values for the parameters to your stored procedure. Sometimes adding WITH RECOMPILE to the procedure can help prevent bad plans from sticking around (at the potential cost of slightly higher CPU usage every time the procedure is executed). But it's not always the best answer - the plan that's currently cached may be atypical and RECOMPILE every time may be overkill. Sometimes the answer is in how the parameters are defined - e.g. if you pass a parameter in you can sometimes defeat parameter sniffing by declaring a local variable and passing the input param to that and using the local variable later in the code. Sometimes the answer is turning on the optimize for ad hoc setting. Tough to say without more details.
Investigating the plan is a good first step, though. If you see that the plans are different post back with the differences and we can help guide you.
It would help to see the sproc and the query but I'm guessing something 'bad' is cached about the stored procedure. Try clearing SQL's cache (assuming SQL Server from your tags). See this
There is probably a bad query plan for the stored procedure. Try dropping and recreating the stored procedure.
it may be that you put the driving query inside the loop in a funny way - making it execute too many times.
i agree with all others - show the code.

Resources