I'm using a search object query (found on the internet, wish I could give credit to the developer) to search database for the columns needed when I write queries. The output search object query allows me to enter the type of table to look in (partial name) as well as the column name (partial name) I'm trying to find. I've been attempting to modify the search object query so it returns the 1st value (Top 1) it finds. This would help me to easily see at a glance if the column has the particular type of data I'm looking for.
I've attempted to write it both as a stored procedure that I could pass two parameters (partial table and partial column name) and I've also tried using dynamic SQL (my first attempt at using it, so I'm a novice when it comes to use it). I had moderate success with the use of dynamic SQL, but can only get it to produce one result rather than be called multiple times for all the results in my search object output. The code I used is shown here:
-- This is the search object query found on internet
Use masterdb
Select a.name, b.name
From sysobjects a
Inner Join syscolumns b On a.id = b.id
Where b.name like '%Result%'
And a.name like '%Lab%'
Order By a.name, b.name
-- This is a separate query I used to test calling the data with dynamic SQL
DECLARE #value VARCHAR(100), #tablename VARCHAR(100)
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
If I use the search object query and search for tables that have lab and column names that have result, I might get output like this:
LabMain,ResultID
LabSpecimen,ResultCategory
LabSpecimen,ResultDate
LabSpecimen,Results
I would like to have the search object query pull data from the table in the first column and the column name in the 2nd column and return the first value it finds to give me a sample output for the given column name/table. Output would look like this:
LabMain,ResultID,E201812310001
LabSpecimen,ResultCategory,ExampleCategory
LabSpecimen,ResultDate,20181231
LabSpecimen,Results,34.20
Okay, I really didn't want to have to post an answer to this, but here goes.
So, the first, really-really-huge thing is: SQL Injection. SQL Injection is the #1 security vulnerability for something like a dozen years running, per OWASP. Basically, SQL Injection is where you use dynamic SQL that has any fragment of the sql command being populated by a user. So in the OP's case, this section here:
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
... if the end incarnation would be that #tableName and #value are populated by the user as part of their search? Then the user can do a 'search' that ends up injecting sql statements that the server runs directly; for a cheap example, imagine this for #value:
3' ; drop table #tableName --
... which would go ahead and drop every table that matches the #tablename you passed in.
Anyway, so, as we go through this problem, we're going to keep SQL Injection in mind at every step.
Problem #1: How to get the tables/columns that match.
You pretty much already nailed this. The only thing missing is to put it into a temp table so that you can loop through it (and limit it down to U-types, since otherwise you'll get stored procs and system tables.) I went ahead and had it also hit the Schema information - that way, if you have tables in different schemas, it'll still be able to get the results.
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
Now, notice that while #tableNameFragment and #columnNameFragment are used, they're not used in a dynamic query. It doesn't matter if the user puts in something malicious into those values
Problem #2 - How to loop through your table
Basically, you're going to need a cursor. I hate cursors, but sometimes (like this one), they're necessary.
Problem #3 - How to actually do a dynamic query and get a result back
This is actually trickier than it looks. You can't do a raw EXEC() for a return value, nor can you simply have the cmd you're executing populating a variable - because EXEC (and SP_ExecuteSql operate in a different context, so they can't populate variables outside in your script.)
You need to use SP_ExecuteSQL, but specify a return variable getting populated by the interior sql command. For example:
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select #retVal=1'
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
select #dynamicReturn
Problem #4 - How to write your Dynamic command
Here's where things get dicey, since it's where we're using a dynamic SQL command. The important thing here is: you cannot use anything the user provided as an input. Which means, you can't use the variables #tableNameFragment or #columnNameFragment. You can use the values in the #entityMatches table, though. Why? Because the user didn't populate them. They got populated by the data in the sys tables - it doesn't matter if the user puts something nefarious in the input variables, that #entityMatches data simply holds the existing table/column names that match.
Also important: When you're working on code that could be a problem if a future dev down the line tweaks or copies/pastes - you should put comment warnings to illuminate the issue.
So, putting it all together? You'll have something that looks like this:
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
declare #returnResults TABLE (TableName varchar(200), ColName varchar(128), FirstValue varchar(max))
declare Cur Cursor For select TableName,ColName from #entityMatches
declare #cursorTable varchar(200), #cursorColumn varchar(128)
open Cur
fetch Next from cur into #cursorTable,#cursorColumn
while ##FETCH_STATUS = 0
begin
-- Note: the variables #cursorTable, #cursorColumn are NOT user populated
-- but instead are populated from the Sys tables. Because of this,
-- this dynamic sql below is not SQL-Injection vulnerable (the entries
-- are not populated from user entry of any sort.)
-- Be very careful modifying the lines below to make sure you don't
-- introduce a vulnerability.
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select top 1 #retVal=[' + #cursorColumn + '] from ' + #cursorTable
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
insert into #returnResults values (#cursorTable, #cursorColumn, #dynamicReturn)
fetch Next from cur into #cursorTable,#cursorColumn
End
close cur
deallocate cur
select * from #returnResults
Create a stored procedure like below mention stored procedure.
Get the table and column name from sysobject & syscolumn and add it in hash table on the base of parameter of stored procedure. After that declare a cursor and in loop of cursor create a dynamic query of column and table name and get first row of current column from table of cursor loop. After that execute the query and update the result in the hash table. At the end of lookup select the Record from hash table. Check the below stored procedure. I hope that its helpful for you.
Create procedure Sp_GetSampleData
#TName varchar(200) = ''
as
Select
a.name TableName, b.name ColumnName,
CAST('' as varchar(max)) as SampleValue
into
#Tbl
from
sysobjects a
inner join
syscolumns b on a.id = b.id
where
(#TName='' or a.name = #TName)
order ny
a.name, b.name
declare #TableName varchar(200), #ColumnName varchar(200),
#sqlText nvarchar(max), #Val varchar(max)
declare Cur Cursor For
select TableName, ColumnName
from #Tbl
open Cur
fetch Next from cur into #TableName,#ColumnName
while ##FETCH_STATUS =0
begin
set #sqlText=''
set #Val=''
SET #sqlText = N'SELECT Top 1 #Val=[' + #ColumnName + '] FROM testndb.dbo.' + #TableName
EXEC Sp_executesql
#sqlText,
N'#Val varchar(max) output',
#Val output
print #sqlText
update #Tbl set SampleValue=#Val where TableName=#TableName and ColumnName =#ColumnName
fetch Next from cur into #TableName,#ColumnName
End
close cur
deallocate cur
select * from #Tbl
Related
I have two databases in SQL Server. Let's say they are DB1 and DB2.
When the system starts, the current database is always DB1. BTW, I have to use DB2 for another table.
For the reason, I want to give a table name as a variable like #tablename and want to select a database name for the #tablename. Would it be possible to pull the database name associated with #tablename?
Also, I want to save the database name to a variable like #databasename to print it out.
When I tried to find a database name from the code below, I could get the database name of the table, ExampleTable, among DB1 and DB2.
EXEC sys.sp_msforeachdb
'SELECT ''?'' DatabaseName, name FROM [?].sys.Tables WHERE Name = ''ExampleTable'''
However, I can't go forward to process how to make a code using a variable #table instead of a fixed table name, ExampleTable.
I will use list of tables to input #tablename into the query one by one from the list.
DECLARE #table sysname = 'TableNames';
DECLARE #database_name sysname = 'dbo';
DECLARE #DatabaseName VARCHAR(50)
-- tbl_01 in dbo.DB1
-- tbl_02 in dbo.DB2
-- tbl_03 in dbo.DB1
-- tbl_04 in dbo.DB2
/*
I need the code block
(1) To input a table using #table
(2) To save the database name to a variable like #database_name
EXEC sys.sp_msforeachdb
'SELECT ''?'' DatabaseName, name FROM [?].sys.Tables WHERE Name ='+#table
*/
Please help me to create a script for my work.
The below code contains a set of tools you can use to do what you want to do. I expect your requirements may change a bit as you do this, so that's why I'm giving you the tools first.
These tools are written as simple checks - easy to understand and relatively quick. They currently just do a SELECT 'Yes' if the database exists/etc - but of course you can change that as needed.
DECLARE #DBName_1 nvarchar(100) = N'DB1';
DECLARE #DBName_2 nvarchar(100) = N'DB2';
DECLARE #TableName nvarchar(100) = N'MyTable';
DECLARE #TableNameToCheck nvarchar(200);
-- Is the current database 'DB1'?
IF DB_Name() = #DBName_1 SELECT 'Yes' ELSE SELECT 'No';
-- Check if the database 'DB2' exists
IF DB_ID(#DBName_2) IS NOT NULL SELECT 'Yes' ELSE SELECT 'No';
-- Check if table is in first database
SET #TableNameToCheck = QUOTENAME(#DBName_1) + N'.[dbo].' + QUOTENAME(#TableName);
IF OBJECT_ID(#TableNameToCheck, 'U') IS NOT NULL SELECT 'Yes' ELSE SELECT 'No';
-- Check if table is in second database (note it uses #DBName_2)
SET #TableNameToCheck = QUOTENAME(#DBName_2) + N'.[dbo].' + QUOTENAME(#TableName);
IF OBJECT_ID(#TableNameToCheck, 'U') IS NOT NULL SELECT 'Yes' ELSE SELECT 'No';
Note that you do not actually need the variable #TableNameToCheck - you can just construct it within the OBJECT_ID() function e.g., OBJECT_ID(#DBName_2 + N'.dbo.' + #TableName , 'U'). However, it can be useful to set it first to check/ensure it's correct, and it has infinitesimal impact on performance.
For your actual task (recording whether the table exists in DB1 or DB2) you can do the following
DECLARE #DBName_1 nvarchar(100) = N'DB1';
DECLARE #DBName_2 nvarchar(100) = N'DB2';
DECLARE #TableName nvarchar(100) = N'MyTable';
DECLARE #DatabaseWithTable nvarchar(100);
IF OBJECT_ID(QUOTENAME(#DBName_1) + N'.[dbo].' + QUOTENAME(#TableName), 'U') IS NOT NULL
BEGIN
SET #DatabaseWithTable = #DBName_1;
END
ELSE IF OBJECT_ID(QUOTENAME(#DBName_2) + N'.[dbo].' + QUOTENAME(#TableName), 'U') IS NOT NULL
BEGIN
SET #DatabaseWithTable = #DBName_2;
END
ELSE
BEGIN
SET #DatabaseWithTable = N'Not Found';
END;
SELECT #DatabaseWithTable;
Edit: Added QUOTENAME() as per suggestion/comment from #HABO
We are using SQL Server 2014 Enterprise with many databases. I have to execute query and get reports / data from every database with EXACT SAME Schema and database starts with Cab
When a new company is added in our ERP project a new database is created with exact schema starting with Cab and incremented number is assigned to it like:
Cab1
Cab2
Cab3
Cab5
Cab10
I can get the database names as:
SELECT name
FROM master.sys.databases
where [name] like 'Cab%' order by [name]
I have to create a Stored Procedure to get data from tables of every database.
How to do that using a Stored Procedure as the databases are created dynamically starting with Cab?
You can use EXEC(#Statement) or EXEC SP_EXECUTESQL if you have to pass parameters.
CREATE OR ALTER PROCEDURE dbo.GetDataFromAllDatabases
AS
BEGIN
DECLARE #T TABLE (id INT NOT NULL IDENTITY(1, 1), dbName VARCHAR(256) NOT NULL)
INSERT INTO #T
SELECT NAME FROM MASTER.SYS.DATABASES WHERE [NAME] LIKE 'Cab%' ORDER BY [NAME]
CREATE TABLE #AllData (......)
DECLARE #Id INT, #DbName VARCHAR(128)
SELECT #Id = MIN(Id) FROM #T
WHILE #Id IS NOT NULL
BEGIN
SELECT #DbName = dbName FROM #T WHERE Id = #Id
DECLARE #Statement NVARCHAR(MAX)
SET #Statement = CONCAT(N'INSERT INTO #AllData (...) SELECT .... FROM ', #DbName, '.dbo.[TableName]')
EXEC(#Statement);
--YOU CAN USE BELOW LINE TOO IF YOU NEED TO PASS VARIABLE
--EXEC SP_EXECUTESQL #Statement, '#Value INT', #Value = 128
SET #Id = (SELECT MIN(Id) FROM #T WHERE Id > #Id)
END
END
A quick and easy dynamic SQL solution would be something like this:
DECLARE #Sql nvarchar(max);
SET #Sql = STUFF((
SELECT ' UNION ALL SELECT [ColumnsList], '''+ [name] + ''' As SourceDb FROM '+ QUOTENAME([name]) + '.[SchemaName].[TableName]' + char(10)
FROM master.sys.databases
WHERE [name] LIKE 'Cab%'
FOR XML PATH('')
), 1, 10, '');
--When dealing with dynamic SQL, print is your best friend...
PRINT #Sql
-- Once the #Sql is printed and you can see it looks OK, you can run it.
--EXEC(#Sql)
Notes:
Use quotename to protect against "funny" chars in identifiers names.
Replace [ColumnsList] with the actual list of columns you need.
There's no need for loops of any kind, just a simple stuff + for xml to mimic string_agg (which was only introduced in 2017).
I've thrown in the source database name as a "bonus", if you don't want it that's fine.
The Order by clause in the query that generates the dynamic SQL is meaningless for the final query, so I've removed it.
Using MS Sql Server
I'm executing the following code:
SELECT #Fieldname = COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #TableName
AND ORDINAL_POSITION = #FIELD
It assigns IsPrimary to #Fieldname, which is the expected answer. But when I execute this line:
SET #MyField = (SELECT #Fieldname FROM inserted WHERE FileID = #FileID)
#MyField is also assigned the value IsPrimary, instead 1, the value contained in the IsPrimary field.
(Note: Executing SELECT IsPrimary FROM inserted WHERE FileID = #FileID works correctly and returns the expected value of 1)
What's the correct sql statement?
It doesn't work that way. You cannot use variables to represent portions of your query.
You can, however, build your query string dynamically into a VARCHAR variable and use the EXEC function to execute it. It comes with tradeoffs of all kinds (different security context, risk of sql injection attacks to account for, etc... ), but it will work the way you expect.
To elaborate on Crono's answer, and fix my previous answer, the "correct (T-)SQL statement" would be like this:
SELECT FileID = CAST(1 AS int), CreatedAt = GETDATE()
INTO #inserted;
DECLARE #Fieldname varchar(30),
#MyField datetime,
#FileID int,
#sql nvarchar(4000);
SELECT #Fieldname = 'CreatedAt',
#FileID = 1;
SELECT #sql = 'SELECT #MyField = ' + #Fieldname + ' FROM #inserted WHERE FileID = ' + CAST(#FileID AS varchar) + ';';
EXEC sp_executesql #sql, N'#MyField datetime OUTPUT', #MyField OUTPUT;
SELECT [#MyField] = #MyField;
DROP TABLE #inserted;
I added the code to create the temp table #inserted just so you can run the example code as-is.
I'm glad I mentioned Erland's article below, as it states something important about dynamic SQL that explains why my previous answer didn't work:
Within the block of dynamic SQL, you cannot access local variables (including table variables) or parameters of the calling stored procedure. But you can pass parameters – in and out – to a block of dynamic SQL if you use sp_executesql.
See (the classic) The Curse and Blessings of Dynamic SQL by Erland Sommarskog for loads of more info about using dynamic SQL (i.e. generating SQL and then executing it).
I am writing a stored procedure designed to be run against a number of similar tables. Since we archive our databases each year, it takes parameters of not only table name but also database name.
The problem is, I want to check that the table exists, but I do not know the database name ahead of time. Ordinarily I could do this:
IF EXISTS (
SELECT *
FROM [Database].sys.objects
WHERE object_id = OBJECT_ID(#TableName)
AND type in (N'U'))
But having the DB name passed to it as a variable means I'm pretty sure sys.objects is going to get kind of ratty with me.
Is this possible?
You need dynamic sql. Notice the space before each new statement to act like new line
declare #db varchar(50) = 'mydbname'
declare #TableName varchar(50) = 'mytablename'
declare #sql varchar(max)
set #sql = 'use '+#db
set #sql = #sql + ' IF EXISTS (SELECT * FROM sys.objects
WHERE object_id = OBJECT_ID('''+#TableName+''') AND type in (''U''))
begin
select 1
end'
exec (#sql)
The following code generates the primaey key for the new record to be inserted and inserts the record into a table, whose name and the values to be inserted are given as parameters to the stored procedure. I am getting a runtime error. I am using Visual Studio 2005 to work with SQL Server 2005 Express Edition
ALTER PROCEDURE spGenericInsert
(
#insValueStr nvarchar(300),
#tblName nvarchar(10)
)
AS
DECLARE #sql nvarchar(400)
DECLARE #params nvarchar(200)
DECLARE #insPrimaryKey nvarchar(10)
DECLARE #rowCountVal integer
DECLARE #prefix nvarchar(5)
--following gets the rowcount of the table--
SELECT #rowCountVal = ISNULL(SUM(spart.rows), 0)
FROM sys.partitions spart
WHERE spart.object_id = object_id(#tblName) AND spart.index_id < 2
SET #rowCountVal = #rowCountVal+1
--Following Creates the Primary Key--
IF #tblName = 'DEFECT_LOG'
SET #prefix='DEF_'
ELSE IF #tblName='INV_Allocation_DB'
SET #prefix='INV_'
ELSE IF #tblName='REQ_Master_DB'
SET #prefix='REQ_'
ELSE IF #tblName='SW_Master_DB'
SET #prefix='SWI_'
ELSE IF #tblName='HW_Master_DB'
SET #prefix='HWI_'
SET #insPrimaryKey= #prefix + RIGHT(replicate('0',5)+ convert(varchar(5),#rowCountVal),5) -- returns somethin like 'DEF_00005'
-- Following is for inserting into the table --
SELECT #sql = N' INSERT INTO #tableName VALUES ' +
N' ( #PrimaryKey , #ValueStr )'
SELECT #params = N'#tableName nvarchar(10), ' +
N'#PrimaryKey nvarchar(10), ' +
N'#ValueStr nvarchar(300)'
EXEC sp_executesql #sql, #params, #tableName=#tblName, #PrimaryKey=#insPrimaryKey, #ValueStr=#insValueStr
Output Message:
Running [dbo].[spGenericInsert] ( #insValueStr = 2,"Hi",1/1/1987, #tblName = DEFECT_LOG ).
Must declare the table variable "#tableName".
No rows affected.
(0 row(s) returned)
#RETURN_VALUE = 0
Finished running [dbo].[spGenericInsert].
You are going to have to concatenate the table name directly into the string, as this cannot be parameterized:
SELECT #sql = N' INSERT INTO [' + #tblName + '] VALUES ' +
N' ( #PrimaryKey , #ValueStr )'
SELECT #params = N'#PrimaryKey nvarchar(10), ' +
N'#ValueStr nvarchar(300)'
To prevent injection attacks, you should white-list this table name. This also isn't robust if the table has other non-nullable columns, etc.
note: Personally, though, I don't think this is a good use of TSQL; it might be more appropriate to construct the command in the client (C# or whatever), and execute it as a parameterized command. There are use-cases for dynamic-SQL, but I'm not sure this is a good example of one.
Better yet, use your preferred ORM tool (LINQ-to-SQL, NHibernate, LLBLGen, Entity Framework, etc) to do all this for you, and concentrate on your actual problem domain.
White list essentially means make sure that the table being passed in is a valid table that you want them to be able to insert into. Let's just say for arguments sake that table name is user provided, the user could then start inserting records into system tables.
You can do a white list check by bouncing the table name of the sysobjects table:
select * from sysobjects where name=#tblname and xType='U'
However as Marc suggested this is not a good use of TSQL, and your better off handling this in the app tier as a paramatized query.
Agree with Marc- overall this is an extremely poor idea. Generic inserts/updates or deletes cause problems for the database eventually.
Another point is that this process will have problems when two users run simulutaneously against the same table as they will try to insert the same Primary Key.