I am using a FileTable in SQL Server 2014 and need to run an executable that parses the file name of any inserted/updated/deleted file and then in turn the executable inserts into other tables on the database the information that was parsed from the name. I do not expect the .exe to run long at all but if it runs into issues, I do not want to lock it for an extended period of time.
For instance:
CREATE PROCEDURE filename_parser
#name nvarchar(255)
AS
BEGIN
DECLARE #exe nvarchar(255)
SET #exe = 'c:\test\my.exe "' + #name + '"'
EXEC master..xp_cmdshell #exe
END
GO
If I run the stored procedure from an INSERT or UPDATE trigger, for instance:
USE [db_1]
GO
CREATE TRIGGER [dbo].[i_table_a]
ON
[dbo].[table_a]
AFTER
INSERT
AS
DECLARE #file nvarchar(255)
SELECT TOP 1
#file = name
FROM
inserted
EXEC filename_parser #name = #file
will I end up locking table_a until the executable completes? Sorry, if the answer is obvious. I have not found a straight forward answer. Any help/pointing in the appropriate direction is appreciated.
Related links:
Do stored procedures lock tables/rows?
SQL Server - How to lock a table until a stored procedure finishes
Microsoft docs say xp_cmdshell will run synchronously. Triggers run synchronously too. So, if your exe gets stuck, it will hang the trigger, which will hang the insert, and other stuff. msdn.microsoft.com/en-us/library/ms175046.aspx#remarks
I've read that using Dynamic SQL in a stored procedure can hurt performance of your stored procedures. I guess the theory is that the store procedure won't store an execution plan for SQL executed via EXEC or sp_executesql.
I want to know if this is true. If it is true, do I have the same problem with multiple nested IF blocks, each one with a different "version" of my SQL statement?
If you have multiple nested IF blocks then SQL Server will be able to store execution plans.
I'm assuming that the IFs are straightforward, eg. IF #Parameter1 IS NOT NULL
SchmitzIT's answer is correct that SQL Server can also store execution paths for Dynamic SQL. However this is only true if the sql is properly built and executed.
By properly built, I mean explicitly declaring the parameters and passing them to sp_executesql. For example
declare #Param1 nvarchar(255) = 'foo'
,#Param2 nvarchar(255) = 'bar'
,#sqlcommand nvarchar(max)
,#paramList nvarchar(max)
set #paramList = '#Param1 nvarchar(255), #Param2 nvarchar(255)'
set #sqlcommand = N'Select Something from Table where Field1 = #Param1 AND Field2 = #Param2'
exec sp_executesql #statement = #sqlcommand
,#params = #paramList
,#Param1 = #Param1
,#Param2 = #Param2
As you can see the sqlcommand text does not hardcode the paramer values to use. They are passed separately in the exec sp_executesql
If you write bad old dynamic sqL
set #sqlcommand = N'Select Something from Table where Field1 = ' + #Param1 + ' AND Field2 = ' + #Param2
exec sp_executesql #sqlcommand
then SQL Server won't be able to store execution plans
This is what MSDN has to say about it. I highlighted the relevant bits to your question
sp_executesql has the same behavior as EXECUTE with regard to batches,
the scope of names, and database context. The Transact-SQL statement
or batch in the sp_executesql #stmt parameter is not compiled until
the sp_executesql statement is executed. The contents of #stmt are
then compiled and executed as an execution plan separate from the
execution plan of the batch that called sp_executesql. The
sp_executesql batch cannot reference variables declared in the batch
that calls sp_executesql. Local cursors or variables in the
sp_executesql batch are not visible to the batch that calls
sp_executesql. Changes in database context last only to the end of the
sp_executesql statement.
sp_executesql can be used instead of stored procedures to execute a
Transact-SQL statement many times when the change in parameter values
to the statement is the only variation. Because the Transact-SQL
statement itself remains constant and only the parameter values
change, the SQL Server query optimizer is likely to reuse the
execution plan it generates for the first execution.
http://msdn.microsoft.com/en-us/library/ms188001.aspx
A server running SQL Server 2005 was converted into a virtual machine. The original server had 16 logical cores. The new virtual server only has 4 cores, but is supposed to be faster.
Some stored procedures (that may call views or UDFs) are taking longer to run. This could be due to less parallelism. But, could the query plans still be optimized for 16 cores or do they automatically re-optimize after a hardware change?
If I need to force all plans to be recalculated, what is the best way? Other thoughts?
Parallel Query Processing shows that the saved query plans allow for parallel processing, but are not specifically tied a particular number of threads.
There may be other reasons to periodically compile new query plans, e.g. after updating statistics. A stored procedure can be scheduled to mark all stored procedures for recompilation. I've had some success with the following:
create procedure [dbo].[INUpdateStatistics]
as
set nocount on
create table #Tables ( Table_Qualifier sysname, Table_Owner sysname, Table_Name sysname, Table_Type VarChar(32), Remarks VarChar(254) )
declare CTable cursor local for select Table_Name, Table_Owner, Table_Type from #Tables order by Table_Name
declare #TableName as sysname
declare #TableOwner as sysname
declare #TableType as sysname
-- Get the list of tables in the database.
insert into #Tables exec sp_tables
open CTable
fetch next from CTable into #TableName, #TableOwner, #TableType
-- For each table ... .
while ##Fetch_Status = 0
begin
if #TableOwner = 'dbo' and #TableType = 'TABLE'
begin
-- Update statistics for all user tables.
execute( 'update statistics [' + #TableName + '] with fullscan, all' )
-- Recompile all stored procedures and triggers when they are next executed.
exec sp_recompile #objname = #TableName
-- Throttle the loop.
waitfor delay '00:00:01'
end
fetch next from CTable into #TableName, #TableOwner, #TableType
end
-- Houseclean.
close CTable
deallocate CTable
drop table #Tables
You can clear the plan cache using:
DBCC FREEPROCCACHE;
However I would inspect some of your plans first, for some of these "slower" queries, to see if they have any parallel operations in the first place.
I am currently trying to script a job on SQL Server 2005 that will automate the DBCC CHECKDB process. Basically, I am using a cursor to run through and run DBCC CHECKDB on every database on an instance. Sometimes it works, running through every database and logging the errors in a table I have designed for that purpose and sometimes it only runs through a few of the databases and stop. Does anyone have any idea what is going on? I have included the code that I use for the cursor.
DECLARE #DbName varchar(100)
DECLARE
GetDbName CURSOR
LOCAL
FORWARD_ONLY
OPTIMISTIC
FOR
SELECT
name
FROM
sys.databases
ORDER BY
name
OPEN GetDbName
FETCH NEXT FROM GetDbName
INTO #DbName
WHILE (##fetch_status = 0)
BEGIN
print #DbName
INSERT INTO
TempLog
EXEC('DBCC CHECKDB ('+ #DbName +') WITH NO_INFOMSGS, TABLERESULTS')
FETCH NEXT FROM GetDbName
INTO #DbName
END
CLOSE GetDbName
DEALLOCATE GetDbName
Errors of a sufficiently high (20+) severity -- in my experience these are most often corruptions such as invalid text pointers -- will stop whatever SQL job is currently running with extreme prejudice, ignoring try/catch constructs and killing the connection. I would suggest moving this task out to an external process with a new connection for each DBCC CHECKDB command so that it can continue in the face of high-severity errors.
use the undocumented sp_MSforeachdb stored proc
exec sp_msforeachdb 'use ?; dbcc checkdb'
i want to ensure that all stored procedures are still syntactically valid. (This can happen if someone renames/deletes a table/column).
Right now my solution to check the syntax of all stored procedures is to go into Enterprise Manager, select the first stored procedure in the list, and use the procedure:
Enter
Alt+C
Escape
Escape
Down Arrow
Goto 1
It works, but it's pretty tedious. i'd like a stored procedure called
SyntaxCheckAllStoredProcedures
like the other stored procedure i wrote that does the same thing for views:
RefreshAllViews
For everyone's benefit, RefreshAllViews:
RefreshAllViews.prc
CREATE PROCEDURE dbo.RefreshAllViews AS
-- This sp will refresh all views in the catalog.
-- It enumerates all views, and runs sp_refreshview for each of them
DECLARE abc CURSOR FOR
SELECT TABLE_NAME AS ViewName
FROM INFORMATION_SCHEMA.VIEWS
OPEN abc
DECLARE #ViewName varchar(128)
-- Build select string
DECLARE #SQLString nvarchar(2048)
FETCH NEXT FROM abc
INTO #ViewName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #SQLString = 'EXECUTE sp_RefreshView '+#ViewName
PRINT #SQLString
EXECUTE sp_ExecuteSQL #SQLString
FETCH NEXT FROM abc
INTO #ViewName
END
CLOSE abc
DEALLOCATE abc
For everyone's benefit, a stored procedure to mark all stored procedure as needing a recompile (marking a stored procedure for recompile will not tell you if it's syntactically valid):
RecompileAllStoredProcedures.prc
CREATE PROCEDURE dbo.RecompileAllStoredProcedures AS
DECLARE abc CURSOR FOR
SELECT ROUTINE_NAME
FROM INFORMATION_SCHEMA.routines
WHERE ROUTINE_TYPE = 'PROCEDURE'
OPEN abc
DECLARE #RoutineName varchar(128)
-- Build select string once
DECLARE #SQLString nvarchar(2048)
FETCH NEXT FROM abc
INTO #RoutineName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #SQLString = 'EXECUTE sp_recompile '+#RoutineName
PRINT #SQLString
EXECUTE sp_ExecuteSQL #SQLString
FETCH NEXT FROM abc
INTO #RoutineName
END
CLOSE abc
DEALLOCATE abc
For completeness sake, the UpdateAllStatistics procedure. This will update all statistics in the database by doing a full data scan:
RefreshAllStatistics.prc
CREATE PROCEDURE dbo.RefreshAllStatistics AS
EXECUTE sp_msForEachTable 'UPDATE STATISTICS ? WITH FULLSCAN'
You can also do this "in-place" - without getting all the create statements.
In addition to setting NOEXEC ON, you will also need to set your favorite SHOWPLAN_* ON (I use SHOWPLAN_TEXT). Now you can get rid of your step 2 and just execute each procedure you retrieved in step 1.
Here is a sample using an individual stored procedure. You can work it into your favorite loop:
create procedure tests #bob int as
select * from missing_table_or_view
go
set showplan_text on;
go
set noexec on
exec tests
set noexec off
go
set showplan_text off;
go
drop procedure tests
go
The above sample should generate the following output:
Msg 208, Level 16, State 1, Procedure tests, Line 2
Invalid object name 'missing_table_or_view'.
The check suggested by KenJ is definitely the best one, since the recreate/alter-approaches does not find all errors. E.g.
impossible execution plans due to query-hints
I even had an SP referencing a non-existing table that went through without the error being detected.
Please find my version that checks all existing SPs at once with KenJ's method below. AFAIK, it will detect every error that will keep the SP from being executed.
--Forces the creation of execution-plans for all sps.
--To achieve this, a temporary SP is created that calls all existing SPs.
--It seems like the simulation of the parameters is not necessary. That makes things a lot easier.
DECLARE #stmt NVARCHAR(MAX) = 'CREATE PROCEDURE pTempCompileTest AS ' + CHAR(13) + CHAR(10)
SELECT #stmt = #stmt + 'EXEC [' + schemas.name + '].[' + procedures.name + '];'
FROM sys.procedures
INNER JOIN sys.schemas ON schemas.schema_id = procedures.schema_id
WHERE schemas.name = 'dbo'
ORDER BY procedures.name
EXEC sp_executesql #stmt
GO
--Here, the real magic happens.
--In order to display as many errors as possible, XACT_ABORT is turned off.
--Unfortunately, for some errors, the execution stops anyway.
SET XACT_ABORT OFF
GO
--Showplan disables the actual execution, but forces t-sql to create execution-plans for every statement.
--This is the core of the whole thing!
SET SHOWPLAN_ALL ON
GO
--You cannot use dynamic SQL in here, since sp_executesql will not be executed, but only show the string passed in in the execution-plan
EXEC pTempCompileTest
GO
SET SHOWPLAN_ALL OFF
GO
SET XACT_ABORT ON
GO
--drop temp sp again
DROP PROCEDURE pTempCompileTest
--If you have any errors in the messages-window now, you should fix these...
If you are using sql 2008 r2 or below then do not use
SET NOEXEC ON
It only checks the syntax and not for potential errors like the existence of tables or columns.
Instead use:
SET FMTONLY ON
it will do a full compile as it tries to return the meta data of the stored procedure.
For 2012 and you will need to use stored procedure:
sp_describe_first_result_set
Also you can do a complete script in Tsql that checks all sp and views, its just a bit of work.
UPDATE
I wrote a complete solution for in tsql that goes through all user defined stored proceedures and checks there syntax. the script is long winded but can be found here http://chocosmith.wordpress.com/2012/12/07/tsql-recompile-all-views-and-stored-proceedures-and-check-for-error/
In addition you might want to consider using Visual Studio Team System 2008 Database Edition which, among other things, does a static verification of all stored procedures in the project on build, thus ensuring that all are consistent with the current schema.
I know this is way old, but I created a slightly different version that actually re-creates all stored procedures, thus throwing errors if they cannot compile. This is something you do not achieve by using the SP_Recompile command.
CREATE PROCEDURE dbo.UTL_ForceSPRecompilation
(
#Verbose BIT = 0
)
AS
BEGIN
--Forces all stored procedures to recompile, thereby checking syntax validity.
DECLARE #SQL NVARCHAR(MAX)
DECLARE #SPName NVARCHAR(255)
DECLARE abc CURSOR FOR
SELECT NAME, OBJECT_DEFINITION(o.[object_id])
FROM sys.objects AS o
WHERE o.[type] = 'P'
ORDER BY o.[name]
OPEN abc
FETCH NEXT FROM abc
INTO #SPName, #SQL
WHILE ##FETCH_STATUS = 0
BEGIN
--This changes "CREATE PROCEDURE" to "ALTER PROCEDURE"
SET #SQL = 'ALTER ' + RIGHT(#SQL, LEN(#SQL) - (CHARINDEX('CREATE', #SQL) + 6))
IF #Verbose <> 0 PRINT #SPName
EXEC(#SQL)
FETCH NEXT FROM abc
INTO #SPName, #SQL
END
CLOSE abc
DEALLOCATE abc
END
I know this is a old question but this is my solution when I could not find any suiting.
I required to validate my stored procedures and views after alot of changes in the database.
Basicly what i wanted was to try to do a ALTER PROCEDURE and ALTER VIEW using the current procedures and view (not actually changing them).
I have written this that works fairly well.
Note! Do not perform on live database, make a copy to validate and then fix the things need fixing. Also sys.sql_modules can be inconsistent so take extra care. I do not use this to actually make the changes, only to check which are not working properly.
DECLARE #scripts TABLE
(
Name NVARCHAR(MAX),
Command NVARCHAR(MAX),
[Type] NVARCHAR(1)
)
DECLARE #name NVARCHAR(MAX), -- Name of procedure or view
#command NVARCHAR(MAX), -- Command or part of command stored in syscomments
#type NVARCHAR(1) -- Procedure or view
INSERT INTO #scripts(Name, Command, [Type])
SELECT P.name, M.definition, 'P' FROM sys.procedures P
JOIN sys.sql_modules M ON P.object_id = M.object_id
INSERT INTO #scripts(Name, Command, [Type])
SELECT V.name, M.definition, 'V' FROM sys.views V
JOIN sys.sql_modules M ON V.object_id = M.object_id
DECLARE curs CURSOR FOR
SELECT Name, Command, [Type] FROM #scripts
OPEN curs
FETCH NEXT FROM curs
INTO #name, #command, #type
WHILE ##FETCH_STATUS = 0
BEGIN
BEGIN TRY
IF #type = 'P'
SET #command = REPLACE(#command, 'CREATE PROCEDURE', 'ALTER PROCEDURE')
ELSE
SET #command = REPLACE(#command, 'CREATE VIEW', 'ALTER VIEW')
EXEC sp_executesql #command
PRINT #name + ' - OK'
END TRY
BEGIN CATCH
PRINT #name + ' - FAILED: ' + CAST(ERROR_NUMBER() AS NVARCHAR(MAX)) + ' ' + ERROR_MESSAGE()
--PRINT #command
END CATCH
FETCH NEXT FROM curs
INTO #name, #command, #type
END
CLOSE curs
A bit of a drawn-out option:
Create a copy of the database
(backup and restore). You could do this on the target database, if your confidence level is high.
Use SSMS to script out all the
stored procedures into a single script file
DROP all the procedures
Run the script to recreate them. Any that can't be created will error out.
Couple of fussy gotchas in here, such as:
You want to have the "if proc exists
then drop proc GO create proc ... GO"
syntax to separte each procedure.
Nested procedures will fail if they
call a proc that has not yet been
(re)created. Running the script several
times should catch that (since
ordering them properly can be a real
pain).
Other and more obscure issues might crop up, so be wary.
To quickly drop 10 or 1000 procedures, run
SELECT 'DROP PROCEDURE ' + schema_name(schema_id) + '.' + name
from sys.procedures
select the output, and run it.
This assumes you're doing a very infrequent task. If you have to do this regularly (daily, weekly...), please let us know why!
There is no way to do it from T-SQL, or Enterprise Manager, so i had to write something from client code. i won't post all the code here, but the trick is to:
1) Get a list of all stored procedures
SELECT ROUTINE_NAME AS StoredProcedureName
FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_TYPE = 'PROCEDURE' --as opposed to a function
ORDER BY ROUTINE_NAME
2) Get the stored procedure create T-SQL:
select
c.text
from dbo.syscomments c
where c.id = object_id(N'StoredProcedureName')
order by c.number, c.colid
option(robust plan)
3) Run the create statement with NOEXEC on, so that the syntax is checked, but it doesn't actually try to create the stored procedure:
connection("SET NOEXEC ON", ExecuteNoRecords);
connection(StoredProcedureCreateSQL, ExecuteNoRecords);
connection("SET NOEXEC ON", ExecuteNoRecords);
Here is an amendment which deals with multiple schemas
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[RefreshAllViews] AS
-- This sp will refresh all views in the catalog.
-- It enumerates all views, and runs sp_refreshview for each of them
DECLARE abc CURSOR FOR
SELECT TABLE_SCHEMA+'.'+TABLE_NAME AS ViewName
FROM INFORMATION_SCHEMA.VIEWS
OPEN abc
DECLARE #ViewName varchar(128)
-- Build select string
DECLARE #SQLString nvarchar(2048)
FETCH NEXT FROM abc
INTO #ViewName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #SQLString = 'EXECUTE sp_RefreshView ['+#ViewName+']'
PRINT #SQLString
EXECUTE sp_ExecuteSQL #SQLString
FETCH NEXT FROM abc
INTO #ViewName
END
CLOSE abc
DEALLOCATE abc
GO