SSMS: 2008 R2
We are having our software system updated, which may contain an unknown number of undocumented changes to the way data is entered and stored in our database. We have asked for documentation, but only have schema compares for "physical" changes to the database, not the way the data is treated. They may change in the future, but for now we have to assume not.
In order to check that our stored procedures work as expected after the update, we would like to run a sample of procedures using a sample of parameters before and after the update to compare the actual data results. The stored procedures here all take a single Id as the parameter (they are used to make SSRS reports within the software system)
I have set some things up, but I am having problems with my approach and would welcome any suggestions about either a better way to do things, or how to fix my approach. The problem is that an error is returned whenever a called stored procedure uses a temporary table. Here is what I have done:
Made a script to get a random sample of Ids for paramaters (only one table used at the moment - that's fine).
ALTER PROC [dbo].[UpdateValidation_GET_RandomIdSample](#TestSizePercent DECIMAL(6,3))
AS
-- This table is already created and will persist both sides of the update
--CREATE TABLE Live_Companion.dbo.UpdateValidationIds
--( Id INT IDENTITY(1,1)NOT NULL PRIMARY KEY CLUSTERED
-- ,MyTableId NT NULL)
IF #TestSizePercent > 100 RAISERROR('Do you even percent, bro?',16,1)
DECLARE #SQL VARCHAR(255)
TRUNCATE TABLE UpdateValidationIds
SET #SQL =
'INSERT dbo.UpdateValidationIds(Id)
SELECT TOP ' + CONVERT(VARCHAR(10),#TestSizePercent) + ' PERCENT ID FROM Live.dbo.MyTable ORDER BY NEWID()'
EXEC (#SQL)
Made a second script to run a stored procedure for each Id in the table:
ALTER PROC [dbo].[UpdateValidation_GET_ProcedureResultsManyTimes](#Procedure_Name VARCHAR(255))
AS
--DECLARE #Procedure_Name VARCHAR(255) = 'Live_Companion.dbo.MyProc'
DECLARE #ID INT
DECLARE #GET_ID CURSOR
DECLARE #SQL VARCHAR(MAX) = ''
DECLARE #MyTableId INT
DECLARE #FirstRun BIT = 1
SET #GET_ID = CURSOR FOR
SELECT Id FROM Live_Companion.dbo.UpdateValidationIds
WHERE MyTableId IS NOT NULL
OPEN #GET_ID
FETCH NEXT FROM #GET_ID INTO #ID
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #MyTableId = MyTableId FROM Live_Companion.dbo.UpdateValidationIds
WHERE Id = #ID
IF #FirstRun = 1
BEGIN
SET #SQL = 'SELECT * INTO #ProcedureOutput FROM OPENROWSET(''SQLNCLI'',''Server=SQL1;Trusted_Connection=yes;'',''EXEC ' + #Procedure_Name + ' ' + CONVERT(VARCHAR(50),#MyTableId) + ''');'
SET #FirstRun = 0
END
ELSE
BEGIN
SET #SQL = #SQL + '
INSERT #ProcedureOutput SELECT * FROM OPENROWSET(''SQLNCLI'',''Server=SQL1;Trusted_Connection=yes;'',''EXEC ' + #Procedure_Name + ' ' + CONVERT(VARCHAR(50),#MyTableId) + ''');'
END
FETCH NEXT FROM #GET_ID INTO #ID
END
SET #SQL = #SQL + '
SELECT * FROM #ProcedureOutput
DROP TABLE #ProcedureOutput'
EXEC (#SQL)
CLOSE #GET_ID
DEALLOCATE #GET_ID
So now I should be able to execute the second procedure for various stored procedures and output the results to file over a range of Ids, then repeat using the saved (initially random) Ids again after the update and compare the results.
The trouble is, it fails when any of the called procedures use a temporary table:
EDIT:
Error Message returned:
Cannot process the object "EXEC Live_Companion.dbo.MyProc 12345". The
OLE DB provider "SQLNCLI10" for linked server "(null)" indicates that
either the object has no columns or the current user does not have
permissions on that object.
Any suggestions or ideas for how to proceed?
Related
I have a problem that works in SQL Server 2017 but not in SQL Server 2019. It is related to tempdb caching. This has to do with creating temporary tables in stored procedures and changing its structure using dynamic SQL. We have a need to do that for various dynamic reporting needs. The first time it is called, the structure is cached and subsequent call to the procedure fails or returns invalid results. How do I prevent caching of such tables? Below is some sample code and how come it works in 2017. Help appreciated.
CREATE PROCEDURE [dbo].[tempDBCachingCheck]
#yearList varchar(max)
AS
BEGIN
SET NOCOUNT ON
DECLARE #yearCount int
DECLARE #yearCounter INT
DECLARE #yearValue INT
DECLARE #sql nvarchar(max)
-- With table variable
DECLARE #tempYearList TABLE (id INT IDENTITY(1,1), rpt_yr int)
INSERT INTO #tempYearList (rpt_yr)
SELECT value FROM STRING_SPLIT(#yearList, ',');
SELECT * FROM #tempYearList
--------------------------------------------------------------------
--With temporary table, since we will be altering this with dynamic sql
CREATE TABLE #returnTable (id INT IDENTITY(1,1))
-- Tried adding a named constraint to not make it cache, but does not work
ALTER TABLE #returnTable
ADD CONSTRAINT UC_ID UNIQUE (id);
SELECT #yearCount = COUNT(*) FROM #tempYearList
-- Add the years as columns to the return table to demostrate the problem
SET #sql = N'ALTER TABLE #returnTable ADD '
SET #yearCounter = 1
WHILE #yearCounter <= #yearCount
BEGIN
SELECT #yearValue = rpt_yr FROM #tempYearList WHERE id = #yearCounter
IF #yearCounter > 1
SET #Sql = #Sql + N', '
SET #sql = #sql + N' [' + convert(varchar(20), #yearValue) + N'] float'
SET #yearCounter = #yearCounter + 1
END
EXECUTE sp_executesql #sql
SELECT * FROM #returnTable
-- No need to drop the temporary tables but doing just in case
DROP TABLE #returnTable
END
GO
-- run these statements and you will see the second call with return the cached #returnTable
EXEC tempDBCachingCheck '2019,2020'
EXEC tempDBCachingCheck '2017,2018,2019,2020'
GO
-- Clear temp table cache and call in reverse order, then will hit an error
-- 'A severe error occurred on the current command. The results, if any, should be discarded.'
USE tempDB
GO
DBCC FREEPROCCACHE
GO
EXEC tempDBCachingCheck '2017,2018,2019,2020'
EXEC tempDBCachingCheck '2019,2020'
GO
It seems this has been fixed in one of cummulative update. The description seems to match:
KB4538853:
When you repeatedly run a stored procedure that uses temporary table with indexes on SQL Server 2019, the client may receive an unexpected error with message "A severe error occurred on the current command" and an access violation exception is recorded on the SQL Server. If the same workload is executed on any previous major version of SQL Server, this issue does not occur.
Dan Guzman's recommendation to install newest CU is the way to go.
Using: EXEC tempDBCachingCheck '2017,2018,2019,2020' WITH RECOMPILE could help as well.
I'm using a search object query (found on the internet, wish I could give credit to the developer) to search database for the columns needed when I write queries. The output search object query allows me to enter the type of table to look in (partial name) as well as the column name (partial name) I'm trying to find. I've been attempting to modify the search object query so it returns the 1st value (Top 1) it finds. This would help me to easily see at a glance if the column has the particular type of data I'm looking for.
I've attempted to write it both as a stored procedure that I could pass two parameters (partial table and partial column name) and I've also tried using dynamic SQL (my first attempt at using it, so I'm a novice when it comes to use it). I had moderate success with the use of dynamic SQL, but can only get it to produce one result rather than be called multiple times for all the results in my search object output. The code I used is shown here:
-- This is the search object query found on internet
Use masterdb
Select a.name, b.name
From sysobjects a
Inner Join syscolumns b On a.id = b.id
Where b.name like '%Result%'
And a.name like '%Lab%'
Order By a.name, b.name
-- This is a separate query I used to test calling the data with dynamic SQL
DECLARE #value VARCHAR(100), #tablename VARCHAR(100)
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
If I use the search object query and search for tables that have lab and column names that have result, I might get output like this:
LabMain,ResultID
LabSpecimen,ResultCategory
LabSpecimen,ResultDate
LabSpecimen,Results
I would like to have the search object query pull data from the table in the first column and the column name in the 2nd column and return the first value it finds to give me a sample output for the given column name/table. Output would look like this:
LabMain,ResultID,E201812310001
LabSpecimen,ResultCategory,ExampleCategory
LabSpecimen,ResultDate,20181231
LabSpecimen,Results,34.20
Okay, I really didn't want to have to post an answer to this, but here goes.
So, the first, really-really-huge thing is: SQL Injection. SQL Injection is the #1 security vulnerability for something like a dozen years running, per OWASP. Basically, SQL Injection is where you use dynamic SQL that has any fragment of the sql command being populated by a user. So in the OP's case, this section here:
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
... if the end incarnation would be that #tableName and #value are populated by the user as part of their search? Then the user can do a 'search' that ends up injecting sql statements that the server runs directly; for a cheap example, imagine this for #value:
3' ; drop table #tableName --
... which would go ahead and drop every table that matches the #tablename you passed in.
Anyway, so, as we go through this problem, we're going to keep SQL Injection in mind at every step.
Problem #1: How to get the tables/columns that match.
You pretty much already nailed this. The only thing missing is to put it into a temp table so that you can loop through it (and limit it down to U-types, since otherwise you'll get stored procs and system tables.) I went ahead and had it also hit the Schema information - that way, if you have tables in different schemas, it'll still be able to get the results.
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
Now, notice that while #tableNameFragment and #columnNameFragment are used, they're not used in a dynamic query. It doesn't matter if the user puts in something malicious into those values
Problem #2 - How to loop through your table
Basically, you're going to need a cursor. I hate cursors, but sometimes (like this one), they're necessary.
Problem #3 - How to actually do a dynamic query and get a result back
This is actually trickier than it looks. You can't do a raw EXEC() for a return value, nor can you simply have the cmd you're executing populating a variable - because EXEC (and SP_ExecuteSql operate in a different context, so they can't populate variables outside in your script.)
You need to use SP_ExecuteSQL, but specify a return variable getting populated by the interior sql command. For example:
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select #retVal=1'
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
select #dynamicReturn
Problem #4 - How to write your Dynamic command
Here's where things get dicey, since it's where we're using a dynamic SQL command. The important thing here is: you cannot use anything the user provided as an input. Which means, you can't use the variables #tableNameFragment or #columnNameFragment. You can use the values in the #entityMatches table, though. Why? Because the user didn't populate them. They got populated by the data in the sys tables - it doesn't matter if the user puts something nefarious in the input variables, that #entityMatches data simply holds the existing table/column names that match.
Also important: When you're working on code that could be a problem if a future dev down the line tweaks or copies/pastes - you should put comment warnings to illuminate the issue.
So, putting it all together? You'll have something that looks like this:
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
declare #returnResults TABLE (TableName varchar(200), ColName varchar(128), FirstValue varchar(max))
declare Cur Cursor For select TableName,ColName from #entityMatches
declare #cursorTable varchar(200), #cursorColumn varchar(128)
open Cur
fetch Next from cur into #cursorTable,#cursorColumn
while ##FETCH_STATUS = 0
begin
-- Note: the variables #cursorTable, #cursorColumn are NOT user populated
-- but instead are populated from the Sys tables. Because of this,
-- this dynamic sql below is not SQL-Injection vulnerable (the entries
-- are not populated from user entry of any sort.)
-- Be very careful modifying the lines below to make sure you don't
-- introduce a vulnerability.
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select top 1 #retVal=[' + #cursorColumn + '] from ' + #cursorTable
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
insert into #returnResults values (#cursorTable, #cursorColumn, #dynamicReturn)
fetch Next from cur into #cursorTable,#cursorColumn
End
close cur
deallocate cur
select * from #returnResults
Create a stored procedure like below mention stored procedure.
Get the table and column name from sysobject & syscolumn and add it in hash table on the base of parameter of stored procedure. After that declare a cursor and in loop of cursor create a dynamic query of column and table name and get first row of current column from table of cursor loop. After that execute the query and update the result in the hash table. At the end of lookup select the Record from hash table. Check the below stored procedure. I hope that its helpful for you.
Create procedure Sp_GetSampleData
#TName varchar(200) = ''
as
Select
a.name TableName, b.name ColumnName,
CAST('' as varchar(max)) as SampleValue
into
#Tbl
from
sysobjects a
inner join
syscolumns b on a.id = b.id
where
(#TName='' or a.name = #TName)
order ny
a.name, b.name
declare #TableName varchar(200), #ColumnName varchar(200),
#sqlText nvarchar(max), #Val varchar(max)
declare Cur Cursor For
select TableName, ColumnName
from #Tbl
open Cur
fetch Next from cur into #TableName,#ColumnName
while ##FETCH_STATUS =0
begin
set #sqlText=''
set #Val=''
SET #sqlText = N'SELECT Top 1 #Val=[' + #ColumnName + '] FROM testndb.dbo.' + #TableName
EXEC Sp_executesql
#sqlText,
N'#Val varchar(max) output',
#Val output
print #sqlText
update #Tbl set SampleValue=#Val where TableName=#TableName and ColumnName =#ColumnName
fetch Next from cur into #TableName,#ColumnName
End
close cur
deallocate cur
select * from #Tbl
I have filtering SQL that returns query with uncertain number of columns, and want to use results in stored procedure.
DECLARE #RecordSelectionSql VARCHAR(MAX)
SET #RecordSelectionSql = (SELECT SQLQUERY FROM RecordSelection WHERE Id = #Id) + ' AND ([Active] = 1)'
DECLARE #sql NVARCHAR(MAX) = N'';
SELECT #sql += ',' + CHAR(13) + CHAR(10) + CHAR(9) + name + ' ' + system_type_name
FROM sys.dm_exec_describe_first_result_set(#RecordSelectionSql, NULL, 0);
SELECT #sql = N'CREATE TABLE #TmpImport
(' + STUFF(#sql, 1, 1, N'') + '
);';
EXEC (#sql)
INSERT INTO #TmpImport
EXEC (#RecordSelectionSql)
However I am getting error
Invalid object name '#TmpImport'.
How to properly code this part?
EDIT: added missing condition on RecordSelection
EDIT2:
I cannot use code below because #TmpImport destroyed after #RecordSelectionSql being executed.
DECLARE #RecordSelectionSql AS VARCHAR(MAX)
SET #RecordSelectionSql = 'SELECT X.* INTO #TmpImport FROM ('
+ (SELECT SQLQUERY FROM RecordSelection WHERE Id = #Id) + ' AND ([Active] = 1) AS X'
EXEC (#RecordSelectionSql)
SELECT * FROM #TmpImport
Gives the same error
Invalid object name '#TmpImport'.
Temporary tables are only available within the session that created them. With Dynamic SQL this means it is not available after the Dynamic SQL has run. Your options here are to:
Create a global temporary table, that will persist outside your session until it is explicitly dropped or cleared out of TempDB another way, using a double hash: create table ##GlobalTemp
--To incorporate Radu's very relevant comment below: Because this table persists outside your session, you need to make sure you don't create two of them or have two different processes trying to process data within it. You need to have a way of uniquely identifying the global temp table you want to be dealing with.
You can create a regular table and remember to drop it again afterwards.
Include whatever logic that needs to reference the temp table within the Dynamic SQL script
For your particular instance though, you are best off simply executing a select into which will generate your table structure from the data that is selected.
It's much easier to select into your temp table.
For example
SELECT * INTO #TmpImport FROM SomeTable
I have db A and db B. At the beginning of a stored procedure I want to back up all rows from B.mytable to B.mytablebackup. The rest of the stored procedure runs against tables on db A (which gathers data and writes it to B.mytable).
So I check to see if B.mytablebackup exists
IF EXISTS(SELECT 1 FROM B.dbo.mytablebackup)
and if it does, the stored procedure does an
INSERT INTO B..mytablebackup SELECT * FROM B..mytable
If it doesn't exist it does a
SELECT * INTO B..mytablebackup from B..mytable
But when I execute the stored procedure I get the error
There is already an object named 'mytablebackup' in the database
I added a Print statement and execution is taking the "does not exist" branch of the IF.
What am I doing wrong?
For SQL Server, you should use system view sys.tables to check if table exists.
IF EXISTS(SELECT 1 FROM B.sys.tables WHERE name = 'mytablebackup')
OBJECT_ID can be used too:
IF OBJECT_ID('B.dbo.mytablebackup') IS NOT NULL
You can directly check from the given DB,SCHEMA and TABLE parameters (For dynamic database, schema and table use)
DECLARE #targetdatabase NVARCHAR(MAX),
#SchemaName NVARCHAR(MAX),
#TableName NVARCHAR(MAX)
DECLARE #TempTableName NVARCHAR(MAX) = QUOTENAME(#targetdatabase) + '.' +
QUOTENAME(#SchemaName) + '.' + QUOTENAME(#TableName)
IF OBJECT_ID(#TempTableName) IS NULL
BEGIN
PRINT #TempTableName
END
In SQL Server 2008, is there a way to access an object based on the OBJECT_ID?
[Edit] The example described below was solved using Andrew's suggestion in the comment, but I'm still curious about the general case. Can an object itself be retrieved using OBJECT_ID, or can it only be accessed indirectly by using the object name via sp_executesql?
My specific case is a stored procedure that uses several temporary tables. At the end of the procedure I want to dump the data from the temporary tables into actual tables for analysis (only if a debug switch is enabled).
The code for dumping the data is similar to this:
IF OBJECT_ID('tempdb..#MyTempTable', 'U') IS NOT NULL
BEGIN
IF OBJECT_ID('Debug_MyTempTable', 'U') IS NOT NULL
DROP TABLE Debug_MyTempTable
SELECT * INTO Debug_MyTempTable FROM #MyTempTable
END
This code block is repeated for each temporary table, so I would prefer to put it in a procedure and call it with a table name:
EXEC [dbo].[CreateDebugTable]
#tableName = 'MyTempTable'
I imagine the procedure would look something like:
CREATE PROCEDURE [dbo].[CreateDebugTable]
#tableName VARCHAR(50)
AS
BEGIN
IF OBJECT_ID('tempdb..#' + #tableName, 'U') IS NOT NULL
BEGIN
IF OBJECT_ID('dbo.Debug_' + #tableName, 'U') IS NOT NULL
DROP TABLE <Debug_TempTable>
SELECT * INTO <Debug_TempTable> FROM <#TempTable>
END
END
The procedure depends on being able to translate the OBJECT_ID of DebugTempTable and #TempTable into the actual tables (shown with <> in the code above). Is this possible?
[Edit]
This is the altered procedure using sp_executesql instead of explicit tables.
CREATE PROCEDURE [dbo].[CreateDebugTable]
#tableName VARCHAR(50)
AS
BEGIN
DECLARE #tmpTable VARCHAR(50) = '#' + #tableName
DECLARE #dboTable VARCHAR(50) = 'Debug_' + #tableName
DECLARE #sql NVARCHAR(100)
IF OBJECT_ID('tempdb..' + #tmpTable, 'U') IS NOT NULL
BEGIN
IF OBJECT_ID('dbo.' + #dboTable, 'U') IS NOT NULL
BEGIN
SET #sql = 'DROP TABLE ' + #dboTable
EXECUTE sp_executesql #sql
END
SET #sql = 'SELECT * INTO ' + #dboTable + ' FROM ' + #tmpTable
EXECUTE sp_executesql #sql
END
END
The object_id is just used as a key in the various metadata views. There is no TSQL syntax to SELECT from (or otherwise manipulate) objects based on their object_id.
If you have an object_id then in general you could use
SELECT QUOTENAME(OBJECT_SCHEMA_NAME(#object_id[,database_id])) +
'.' +
QUOTENAME(OBJECT_NAME(#object_id[,database_id]) )
To get the 2 part name of the object but for #temp tables this returns the long internal name rather than the short one that you can actually use in queries.