I'm using a search object query (found on the internet, wish I could give credit to the developer) to search database for the columns needed when I write queries. The output search object query allows me to enter the type of table to look in (partial name) as well as the column name (partial name) I'm trying to find. I've been attempting to modify the search object query so it returns the 1st value (Top 1) it finds. This would help me to easily see at a glance if the column has the particular type of data I'm looking for.
I've attempted to write it both as a stored procedure that I could pass two parameters (partial table and partial column name) and I've also tried using dynamic SQL (my first attempt at using it, so I'm a novice when it comes to use it). I had moderate success with the use of dynamic SQL, but can only get it to produce one result rather than be called multiple times for all the results in my search object output. The code I used is shown here:
-- This is the search object query found on internet
Use masterdb
Select a.name, b.name
From sysobjects a
Inner Join syscolumns b On a.id = b.id
Where b.name like '%Result%'
And a.name like '%Lab%'
Order By a.name, b.name
-- This is a separate query I used to test calling the data with dynamic SQL
DECLARE #value VARCHAR(100), #tablename VARCHAR(100)
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
If I use the search object query and search for tables that have lab and column names that have result, I might get output like this:
LabMain,ResultID
LabSpecimen,ResultCategory
LabSpecimen,ResultDate
LabSpecimen,Results
I would like to have the search object query pull data from the table in the first column and the column name in the 2nd column and return the first value it finds to give me a sample output for the given column name/table. Output would look like this:
LabMain,ResultID,E201812310001
LabSpecimen,ResultCategory,ExampleCategory
LabSpecimen,ResultDate,20181231
LabSpecimen,Results,34.20
Okay, I really didn't want to have to post an answer to this, but here goes.
So, the first, really-really-huge thing is: SQL Injection. SQL Injection is the #1 security vulnerability for something like a dozen years running, per OWASP. Basically, SQL Injection is where you use dynamic SQL that has any fragment of the sql command being populated by a user. So in the OP's case, this section here:
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
... if the end incarnation would be that #tableName and #value are populated by the user as part of their search? Then the user can do a 'search' that ends up injecting sql statements that the server runs directly; for a cheap example, imagine this for #value:
3' ; drop table #tableName --
... which would go ahead and drop every table that matches the #tablename you passed in.
Anyway, so, as we go through this problem, we're going to keep SQL Injection in mind at every step.
Problem #1: How to get the tables/columns that match.
You pretty much already nailed this. The only thing missing is to put it into a temp table so that you can loop through it (and limit it down to U-types, since otherwise you'll get stored procs and system tables.) I went ahead and had it also hit the Schema information - that way, if you have tables in different schemas, it'll still be able to get the results.
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
Now, notice that while #tableNameFragment and #columnNameFragment are used, they're not used in a dynamic query. It doesn't matter if the user puts in something malicious into those values
Problem #2 - How to loop through your table
Basically, you're going to need a cursor. I hate cursors, but sometimes (like this one), they're necessary.
Problem #3 - How to actually do a dynamic query and get a result back
This is actually trickier than it looks. You can't do a raw EXEC() for a return value, nor can you simply have the cmd you're executing populating a variable - because EXEC (and SP_ExecuteSql operate in a different context, so they can't populate variables outside in your script.)
You need to use SP_ExecuteSQL, but specify a return variable getting populated by the interior sql command. For example:
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select #retVal=1'
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
select #dynamicReturn
Problem #4 - How to write your Dynamic command
Here's where things get dicey, since it's where we're using a dynamic SQL command. The important thing here is: you cannot use anything the user provided as an input. Which means, you can't use the variables #tableNameFragment or #columnNameFragment. You can use the values in the #entityMatches table, though. Why? Because the user didn't populate them. They got populated by the data in the sys tables - it doesn't matter if the user puts something nefarious in the input variables, that #entityMatches data simply holds the existing table/column names that match.
Also important: When you're working on code that could be a problem if a future dev down the line tweaks or copies/pastes - you should put comment warnings to illuminate the issue.
So, putting it all together? You'll have something that looks like this:
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
declare #returnResults TABLE (TableName varchar(200), ColName varchar(128), FirstValue varchar(max))
declare Cur Cursor For select TableName,ColName from #entityMatches
declare #cursorTable varchar(200), #cursorColumn varchar(128)
open Cur
fetch Next from cur into #cursorTable,#cursorColumn
while ##FETCH_STATUS = 0
begin
-- Note: the variables #cursorTable, #cursorColumn are NOT user populated
-- but instead are populated from the Sys tables. Because of this,
-- this dynamic sql below is not SQL-Injection vulnerable (the entries
-- are not populated from user entry of any sort.)
-- Be very careful modifying the lines below to make sure you don't
-- introduce a vulnerability.
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select top 1 #retVal=[' + #cursorColumn + '] from ' + #cursorTable
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
insert into #returnResults values (#cursorTable, #cursorColumn, #dynamicReturn)
fetch Next from cur into #cursorTable,#cursorColumn
End
close cur
deallocate cur
select * from #returnResults
Create a stored procedure like below mention stored procedure.
Get the table and column name from sysobject & syscolumn and add it in hash table on the base of parameter of stored procedure. After that declare a cursor and in loop of cursor create a dynamic query of column and table name and get first row of current column from table of cursor loop. After that execute the query and update the result in the hash table. At the end of lookup select the Record from hash table. Check the below stored procedure. I hope that its helpful for you.
Create procedure Sp_GetSampleData
#TName varchar(200) = ''
as
Select
a.name TableName, b.name ColumnName,
CAST('' as varchar(max)) as SampleValue
into
#Tbl
from
sysobjects a
inner join
syscolumns b on a.id = b.id
where
(#TName='' or a.name = #TName)
order ny
a.name, b.name
declare #TableName varchar(200), #ColumnName varchar(200),
#sqlText nvarchar(max), #Val varchar(max)
declare Cur Cursor For
select TableName, ColumnName
from #Tbl
open Cur
fetch Next from cur into #TableName,#ColumnName
while ##FETCH_STATUS =0
begin
set #sqlText=''
set #Val=''
SET #sqlText = N'SELECT Top 1 #Val=[' + #ColumnName + '] FROM testndb.dbo.' + #TableName
EXEC Sp_executesql
#sqlText,
N'#Val varchar(max) output',
#Val output
print #sqlText
update #Tbl set SampleValue=#Val where TableName=#TableName and ColumnName =#ColumnName
fetch Next from cur into #TableName,#ColumnName
End
close cur
deallocate cur
select * from #Tbl
As I have seen so far, people suggested using dynamic SQL.
For example:
How to pass schema as parameter to a stored procedure in sql server?
How to pass schema name as parameter in stored procedure
However, dynamic SQL has the risk of SQL injection. Hence, I want to know if there are any other safe alternatives?
Basically, this stored procedure that I am creating will be called at runtime. There will be 2 possible schemas to be passed in. And the table name will be passed in as well.
Something like below: (It does not work)
CREATE PROCEDURE [EFM].[usp_readApexTable]
#SCHEMANAME VARCHAR(20) = NULL,
#TABLENAME VARCHAR(100) = NULL
AS
BEGIN
SET NOCOUNT ON;
SELECT *
FROM [#SCHEMANAME].[#TABLENAME];
END
GO
This is just an example of READ action. My plan is to create for CRUD, which requires 4 different stored procedures.
You can use QUOTENAME to avoid any SQL injection and build your dynamic query like the following:
CREATE PROCEDURE [EFM].[usp_readApexTable]
#SCHEMANAME VARCHAR(20) = NULL,
#TABLENAME VARCHAR(100) = NULL
AS
BEGIN
SET NOCOUNT ON;
DECLARE #SQL VARCHAR(MAX)=N'SELECT * FROM '
+ QUOTENAME(#SCHEMANAME) + '.' + QUOTENAME(#TABLENAME)
EXEC (#SQL)
END
GO
Note: If you have any plan to add parameters also for your WHERE clause, in that case QUOTENAME will not help much, I suggest to to use sp_executesql by passing appropriate parameters used in WHERE clause.
Still you need to use QUOTENAME for schema and table name as SQL excepts it only as literal, you can't use variable names for table and schema.
For example.
declare #sql nvarchar(max)
set #sql = N'select * from ' + quotename(#SCHEMANAME ) + '.' + quotename(#TABLENAME )
+ ' where (City = #City)'
exec sp_executesql
#sql,
N'#City nvarchar(50)',
#City
You can find more details here
You need to use dynamic sql to do this operation
CREATE PROCEDURE [EFM].[usp_readApexTable]
#SCHEMANAME VARCHAR(20) = NULL,
#TABLENAME VARCHAR(100) = NULL
AS
BEGIN
SET NOCOUNT ON;
DECLARE #sqlCommand nvarchar(MAX)
SET #sqlCommand='SELECT * FROM ['+#SCHEMANAME+'].['+#TABLENAME+'];'
--Create Your Temp Table where you can set the records after executing the dynamic query
CREATE TABLE #tmpTbl(
Column1 [datatype]
Column2 [datatype]
.
.
ColumnN
)
INSERT INTO #tmpTbl EXEC sp_executesql #sqlCommand --Copy data to #tmpTbl table
SELECT * FROM #tmpTbl
DROP TABLE #tmpTbl
END
GO
I am creating a stored procedure within a stored procedure and am looking to insert dynamically generated values within a stored procedure to another stored procedure, I tried going to many places but still not find out my answer or they are not near to my given scenario please check below
Below 6 lines are part of above stored procedure and based on these six lines I would like to create another stored procedure where I am calling them for insert statement these six lines are doing the job I wanted but how to print those or call them within a stored procedure so that it shows the insert statement.
I tried output and table variable still not able to populate the insert statement. Please help me achieve this thanks
declare #DD varchar(max)
set #DD='select '+isnull(#allcol,'')+ ' from '+#Table1+' a '+'inner join '+#Table2+' b on a.'+#KeyColumn1+'=b.'+#KeyColumn2+isnull(' inner join '+#Table3+' c on a.'+#KeyColumn1+'=c.'+#KeyColumn3,'')
--print #DD
--exec (#DD)
-- Declare your stored procedure name here
DECLARE #create_stored_procedure nvarchar(max)
DECLARE #TableName nvarchar(max)
-- Please provide name to your stored procedure
set #TableName = 'staging_ABC'
SET #create_stored_procedure = N'
CREATE PROCEDURE [dbo].[sproc_' + #TableName + ']
AS
BEGIN
-- If stage table exist drop it and then insert into [staging_ABC]
IF EXISTS (SELECT * FROM sys.objects WHERE NAME = ''' + #TableName + ''')
BEGIN
DROP TABLE [dbo].[Staging_ABC]
END
-- We need to populate insert statement here from #dd
--print #DD
END '
I am doing work for a company that stores each of their client's info in a different database. When a table needs modification, I have to go to each database and run the ALTER TABLE script. Is there a way I can use a prepared statement to run through all 100+ DBO names?
ALTER TABLE ?.dbo.profileTable
ADD COLUMN profileStatus int
where ? = 'CompanyA, CompanyB, CompanyC' or something similar?
Use Sp_MSforeachdb
EXECUTE master.sys.sp_MSforeachdb 'USE [?]; alter query'
[?] is used as a placeholder for the heretofore unspecified database name
You can modify the query as per your needs ,to exclude system databases use like below..
EXECUTE master.sys.sp_MSforeachdb 'USE [?]; IF DB_ID(''?'') > 4 begin yourquery end'
This will exclude any database that does not have the table you are looking for including system databases.
Declare #TableName Varchar(8000) = 'ProfileTable'
Declare #Sql Varchar(8000)
Select #Sql = Stuff(
(Select ';', 'Alter Table ' + Name + SqlText
From sys.databases
Cross Apply (Select '.dbo.profileTable ADD profileStatus int' SqlText) CA
Where Case When State_Desc = 'ONLINE'
Then Object_Id (QuoteName(Name) + '.[dbo].' + #TableName, 'U')
End Is Not Null
FOR XML PATH('')
),1,1,'')
Exec (#Sql)
This ? before is database ([database].[schema].[table]). Thus you can use sp_MSforeachdb or, as I prefer, use sys.databases view to prepare dynamic queries.
Beware, both methods can interfere with system databases.
Take a look at this solution:
DECLARE #query nvarchar(MAX)='';
SELECT #query = #query + 'USE '+QUOTENAME(name)+';ALTER TABLE dbo.profileTable ADD profileStatus int;'
FROM sys.databases
WHERE OBJECT_ID(QUOTENAME(name)+'.dbo.profileTable', 'U') IS NOT NULL
EXEC(#query)
It adds column col1 int to each dbo.profileTable in every database.
SSMS: 2008 R2
We are having our software system updated, which may contain an unknown number of undocumented changes to the way data is entered and stored in our database. We have asked for documentation, but only have schema compares for "physical" changes to the database, not the way the data is treated. They may change in the future, but for now we have to assume not.
In order to check that our stored procedures work as expected after the update, we would like to run a sample of procedures using a sample of parameters before and after the update to compare the actual data results. The stored procedures here all take a single Id as the parameter (they are used to make SSRS reports within the software system)
I have set some things up, but I am having problems with my approach and would welcome any suggestions about either a better way to do things, or how to fix my approach. The problem is that an error is returned whenever a called stored procedure uses a temporary table. Here is what I have done:
Made a script to get a random sample of Ids for paramaters (only one table used at the moment - that's fine).
ALTER PROC [dbo].[UpdateValidation_GET_RandomIdSample](#TestSizePercent DECIMAL(6,3))
AS
-- This table is already created and will persist both sides of the update
--CREATE TABLE Live_Companion.dbo.UpdateValidationIds
--( Id INT IDENTITY(1,1)NOT NULL PRIMARY KEY CLUSTERED
-- ,MyTableId NT NULL)
IF #TestSizePercent > 100 RAISERROR('Do you even percent, bro?',16,1)
DECLARE #SQL VARCHAR(255)
TRUNCATE TABLE UpdateValidationIds
SET #SQL =
'INSERT dbo.UpdateValidationIds(Id)
SELECT TOP ' + CONVERT(VARCHAR(10),#TestSizePercent) + ' PERCENT ID FROM Live.dbo.MyTable ORDER BY NEWID()'
EXEC (#SQL)
Made a second script to run a stored procedure for each Id in the table:
ALTER PROC [dbo].[UpdateValidation_GET_ProcedureResultsManyTimes](#Procedure_Name VARCHAR(255))
AS
--DECLARE #Procedure_Name VARCHAR(255) = 'Live_Companion.dbo.MyProc'
DECLARE #ID INT
DECLARE #GET_ID CURSOR
DECLARE #SQL VARCHAR(MAX) = ''
DECLARE #MyTableId INT
DECLARE #FirstRun BIT = 1
SET #GET_ID = CURSOR FOR
SELECT Id FROM Live_Companion.dbo.UpdateValidationIds
WHERE MyTableId IS NOT NULL
OPEN #GET_ID
FETCH NEXT FROM #GET_ID INTO #ID
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #MyTableId = MyTableId FROM Live_Companion.dbo.UpdateValidationIds
WHERE Id = #ID
IF #FirstRun = 1
BEGIN
SET #SQL = 'SELECT * INTO #ProcedureOutput FROM OPENROWSET(''SQLNCLI'',''Server=SQL1;Trusted_Connection=yes;'',''EXEC ' + #Procedure_Name + ' ' + CONVERT(VARCHAR(50),#MyTableId) + ''');'
SET #FirstRun = 0
END
ELSE
BEGIN
SET #SQL = #SQL + '
INSERT #ProcedureOutput SELECT * FROM OPENROWSET(''SQLNCLI'',''Server=SQL1;Trusted_Connection=yes;'',''EXEC ' + #Procedure_Name + ' ' + CONVERT(VARCHAR(50),#MyTableId) + ''');'
END
FETCH NEXT FROM #GET_ID INTO #ID
END
SET #SQL = #SQL + '
SELECT * FROM #ProcedureOutput
DROP TABLE #ProcedureOutput'
EXEC (#SQL)
CLOSE #GET_ID
DEALLOCATE #GET_ID
So now I should be able to execute the second procedure for various stored procedures and output the results to file over a range of Ids, then repeat using the saved (initially random) Ids again after the update and compare the results.
The trouble is, it fails when any of the called procedures use a temporary table:
EDIT:
Error Message returned:
Cannot process the object "EXEC Live_Companion.dbo.MyProc 12345". The
OLE DB provider "SQLNCLI10" for linked server "(null)" indicates that
either the object has no columns or the current user does not have
permissions on that object.
Any suggestions or ideas for how to proceed?