Can't query and put data inside a cursor when using variable inside the query - sql-server

I have to put a result of a query (single column and value is being pulled) into a variable. I'm trying to use a cursor however I choose the database to query based on a variable here is my query
SELECT productName, price FROM #ShopName.dbo.Products WHERE ProductName = #ProductName
#ShopName variable is being pulled from the database first and assigned to the variable using a cursor. #ProductName variable is being populated by an input parameter coming from API. I have to get ProductName from a specific database (there are multiple databases with products), but the query above throws syntax errors. Additionally when I tried ad hoc query assigned to a variable:
SET #Sql = N'SELECT productName, price FROM ' + QUOTENAME(#ShopName) + '.dbo.Products WHERE ProductName = ' + #ProductName
It doesn't allow to use it in
DECLARE cursorT CURSOR
FOR
#Sql
This throws Incorrect syntax near '#Sql', Expecting '(', SELECT, or WITH
Is there any way to make it possible to use that query in cursor while using the variable with database name in it?

Cursors should be right at the bottom of your bag of techniques, used sparingly and with great care, only when necessary. I can't tell if it's necessary in your case, there's not enough code to know. But I wanted to get that out before continuing.
As a point of purely academic interest, yes, there are some ways you can do this. Two main ways:
Declare a cursor in the dynamic SQL, as Dale suggested. You can still use the cursor in static code which follows the declaration if the cursor is global.
Use dynamic SQL to drop the results into something with scope outside of the dynamic sql, like a temp table. The cursor over the temp table.
1 is just bad. It is likely to result in code which is extremely difficult to understand in future. I include it for curiosity only. 2 is reasonable.
Examples:
-- some dummy schema and data to work with
create table t(i int);
insert t values(1), (2);
-- option 1: declare a cursor dynamically, use it statically (don't do this)
declare #i int;
exec sp_executesql N'declare c cursor global for select i from t';
open c;
fetch next from c into #i;
while (##fetch_status = 0)
begin
print #i;
fetch next from c into #i;
end
close c;
deallocate c;
-- option 2: dynamically dump data to a table, eg a temp table
create table #u(i int);
exec sp_executesql N'insert #u (i) select i from t';
declare c cursor local for select i from #u;
declare #i int;
open c;
fetch next from c into #i;
while (##fetch_status = 0)
begin
print #i;
fetch next from c into #i;
end
close c;
deallocate c;

Related

How to systematically get stored procedure (SP) parameter names and their values INSIDE the SP execution

As described in title, I am trying to systematically get stored procedure pararameter names and their corresponding values inside the execution of the proper stored procedure.
First point, which is taking stored procedure parameter names, is easy using table [sys].[all_parameters] and the stored procedure name. However, getting the actual values of these parameters is the difficult part, specially when you are not allowed to use table [sys].[dm_exec_input_buffer] (as a developer, I am not allowed to read this table, since it is a system administrator table).
Here is the code I have so far, which I am sure can serve you as a template:
CREATE PROCEDURE [dbo].[get_proc_params_demo]
(
#number1 int,
#string1 varchar(50),
#calendar datetime,
#number2 int,
#string2 nvarchar(max)
)
AS
BEGIN
DECLARE #sql NVARCHAR(MAX);
DECLARE #ParameterNames NVARCHAR(MAX) = ( SELECT STRING_AGG([Name], ',') FROM [sys].[all_parameters] WHERE OBJECT_ID = OBJECT_ID('[dbo].[get_proc_params_demo]') )
SET #sql = N'SELECT ' + #ParameterNames;
DECLARE GetParameterValues CURSOR FOR
SELECT DISTINCT [Name] FROM [sys].[all_parameters] WHERE OBJECT_ID = OBJECT_ID('[dbo].[get_proc_params_demo]');
OPEN GetParameterValues;
DECLARE #param_values NVARCHAR(MAX) = NULL
DECLARE #StoredProcedureParameter NVARCHAR(MAX)
FETCH NEXT FROM GetParameterValues INTO #StoredProcedureParameter;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #param_values = 'ISNULL('+#param_values+','')'+#StoredProcedureParameter+','
EXEC(#param_values)
FETCH NEXT FROM GetParameterValues INTO #StoredProcedureParameter;
END;
CLOSE GetParameterValues;
DEALLOCATE GetParameterValues;
SET #param_values = LEFT(#param_values, LEN(#param_values) - 1)
EXEC sp_executesql #sql,#ParameterNames,#param_values;
END
EXEC [dbo].[get_proc_params_demo]
#number1=42,
#string1='is the answer',
#calendar='2019-06-19',
#number2=123456789,
#string2='another string'
This is my approach trying to dynamically get parameter actual values inside a cursor, but it does not work, and I am clueless so far. I know it is quite rudimentary, and I am happy to hear other approaches. To be fair, I don't know if this problem is even possible to solve without system tables, but it would be great.
EDIT: This is an attempt to get a generic code that works on any stored procedure. You do not want to hardcode any parameter name. The only input you have is the stored procedure name via OBJECT_NAME(##PROCID)

How to create a SQL function which splits comma separated value?

I want to create a function in SQL Server which takes a comma separated string a parameter, splits it and returns one value at a time. The basic idea here is to call that function from a query. Something like this.
CREATE FUNCTION SPLIT_VALUE(#IN_CSV)
RETURN VARCHAR AS
-- LOGIC TO RETURN A SINGLE VALUE FROM CSV
END
I want to call this function from a stored procedure.
CREATE PROCEDURE DEMO_PROC #IN_CSV VARCHAR(5000), #OUT VARCHAR(5000) OUTPUT AS
BEGIN
SELECT #OUT= CONCAT(A.VALUE1,B.VALUE2) FROM TABLE1 A INNER JOIN TABLE2 B ON A.ID=B.ID WHERE A.ID
IN(--CALL THE FUNCTION AND GET ONE VALUE);
END;
I have to create a loop or cursor to point to a particular value every time. Is this practically possible to? If yes then how can I do that?
Like I mention, you'll have to use a CURSOR to do this, however, the fact you want to do it this way infers a (large) design flaw:
DECLARE #value varchar(8000)
DECLARE Delimited_Values CURSOR FAST_FORWARD
FOR
SELECT [value]
FROM STRING_SPLIT('a,b,c,d,e',',')
OPEN Delimited_Values;
FETCH NEXT FROM Delimited_Values
INTO #value;
WHILE ##FETCH_STATUS = 0 BEGIN
SELECT #value; --Do your stuff here
FETCH NEXT FROM Delimited_Values
INTO #value;
END;
CLOSE Delimited_Values;
DEALLOCATE Delimited_Values;

Calling dynamic SQL or stored procedure

I'm using a search object query (found on the internet, wish I could give credit to the developer) to search database for the columns needed when I write queries. The output search object query allows me to enter the type of table to look in (partial name) as well as the column name (partial name) I'm trying to find. I've been attempting to modify the search object query so it returns the 1st value (Top 1) it finds. This would help me to easily see at a glance if the column has the particular type of data I'm looking for.
I've attempted to write it both as a stored procedure that I could pass two parameters (partial table and partial column name) and I've also tried using dynamic SQL (my first attempt at using it, so I'm a novice when it comes to use it). I had moderate success with the use of dynamic SQL, but can only get it to produce one result rather than be called multiple times for all the results in my search object output. The code I used is shown here:
-- This is the search object query found on internet
Use masterdb
Select a.name, b.name
From sysobjects a
Inner Join syscolumns b On a.id = b.id
Where b.name like '%Result%'
And a.name like '%Lab%'
Order By a.name, b.name
-- This is a separate query I used to test calling the data with dynamic SQL
DECLARE #value VARCHAR(100), #tablename VARCHAR(100)
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
If I use the search object query and search for tables that have lab and column names that have result, I might get output like this:
LabMain,ResultID
LabSpecimen,ResultCategory
LabSpecimen,ResultDate
LabSpecimen,Results
I would like to have the search object query pull data from the table in the first column and the column name in the 2nd column and return the first value it finds to give me a sample output for the given column name/table. Output would look like this:
LabMain,ResultID,E201812310001
LabSpecimen,ResultCategory,ExampleCategory
LabSpecimen,ResultDate,20181231
LabSpecimen,Results,34.20
Okay, I really didn't want to have to post an answer to this, but here goes.
So, the first, really-really-huge thing is: SQL Injection. SQL Injection is the #1 security vulnerability for something like a dozen years running, per OWASP. Basically, SQL Injection is where you use dynamic SQL that has any fragment of the sql command being populated by a user. So in the OP's case, this section here:
SET #value = 'Result'
SET #tablename = 'LabSpecimen'
DECLARE #sqlText NVARCHAR(1000);
SET #sqlText = N'SELECT Top 1 ' + #value + ' FROM testndb.dbo.' + #tablename
EXEC (#sqlText)
... if the end incarnation would be that #tableName and #value are populated by the user as part of their search? Then the user can do a 'search' that ends up injecting sql statements that the server runs directly; for a cheap example, imagine this for #value:
3' ; drop table #tableName --
... which would go ahead and drop every table that matches the #tablename you passed in.
Anyway, so, as we go through this problem, we're going to keep SQL Injection in mind at every step.
Problem #1: How to get the tables/columns that match.
You pretty much already nailed this. The only thing missing is to put it into a temp table so that you can loop through it (and limit it down to U-types, since otherwise you'll get stored procs and system tables.) I went ahead and had it also hit the Schema information - that way, if you have tables in different schemas, it'll still be able to get the results.
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
Now, notice that while #tableNameFragment and #columnNameFragment are used, they're not used in a dynamic query. It doesn't matter if the user puts in something malicious into those values
Problem #2 - How to loop through your table
Basically, you're going to need a cursor. I hate cursors, but sometimes (like this one), they're necessary.
Problem #3 - How to actually do a dynamic query and get a result back
This is actually trickier than it looks. You can't do a raw EXEC() for a return value, nor can you simply have the cmd you're executing populating a variable - because EXEC (and SP_ExecuteSql operate in a different context, so they can't populate variables outside in your script.)
You need to use SP_ExecuteSQL, but specify a return variable getting populated by the interior sql command. For example:
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select #retVal=1'
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
select #dynamicReturn
Problem #4 - How to write your Dynamic command
Here's where things get dicey, since it's where we're using a dynamic SQL command. The important thing here is: you cannot use anything the user provided as an input. Which means, you can't use the variables #tableNameFragment or #columnNameFragment. You can use the values in the #entityMatches table, though. Why? Because the user didn't populate them. They got populated by the data in the sys tables - it doesn't matter if the user puts something nefarious in the input variables, that #entityMatches data simply holds the existing table/column names that match.
Also important: When you're working on code that could be a problem if a future dev down the line tweaks or copies/pastes - you should put comment warnings to illuminate the issue.
So, putting it all together? You'll have something that looks like this:
declare #tableNameFragment varchar(100) -- note: these top 4 lines will eventually
declare #columnNameFragment varchar(100) -- be changed to stored proc args
set #tableNameFragment = 'Performance' -- and populated by the user calling
set #columnNameFragment = 'status' -- the proc (instead of hard-coded.)
declare #entityMatches TABLE (TableName varchar(200), ColName varchar(128))
insert into #entityMatches
Select sch.TABLE_SCHEMA + '.' + sysobj.name as TableName, syscol.name as ColName
From sysobjects sysobj
Join syscolumns syscol On sysobj.id = syscol.id
Join INFORMATION_SCHEMA.TABLES sch on sch.TABLE_NAME = sysobj.name
where sysobj.xtype = 'U'
and (sysobj.name like '%' + isnull(#tableNameFragment,'') + '%')
and (syscol.name like '%' + isnull(#columnNameFragment,'') + '%')
declare #returnResults TABLE (TableName varchar(200), ColName varchar(128), FirstValue varchar(max))
declare Cur Cursor For select TableName,ColName from #entityMatches
declare #cursorTable varchar(200), #cursorColumn varchar(128)
open Cur
fetch Next from cur into #cursorTable,#cursorColumn
while ##FETCH_STATUS = 0
begin
-- Note: the variables #cursorTable, #cursorColumn are NOT user populated
-- but instead are populated from the Sys tables. Because of this,
-- this dynamic sql below is not SQL-Injection vulnerable (the entries
-- are not populated from user entry of any sort.)
-- Be very careful modifying the lines below to make sure you don't
-- introduce a vulnerability.
declare #sqlCmd nvarchar(max)
declare #dynamicReturn varchar(max)
set #sqlCmd = 'select top 1 #retVal=[' + #cursorColumn + '] from ' + #cursorTable
EXEC Sp_executesql #sqlCmd,
N'#retVal varchar(max) output',
#dynamicReturn output
insert into #returnResults values (#cursorTable, #cursorColumn, #dynamicReturn)
fetch Next from cur into #cursorTable,#cursorColumn
End
close cur
deallocate cur
select * from #returnResults
Create a stored procedure like below mention stored procedure.
Get the table and column name from sysobject & syscolumn and add it in hash table on the base of parameter of stored procedure. After that declare a cursor and in loop of cursor create a dynamic query of column and table name and get first row of current column from table of cursor loop. After that execute the query and update the result in the hash table. At the end of lookup select the Record from hash table. Check the below stored procedure. I hope that its helpful for you.
Create procedure Sp_GetSampleData
#TName varchar(200) = ''
as
Select
a.name TableName, b.name ColumnName,
CAST('' as varchar(max)) as SampleValue
into
#Tbl
from
sysobjects a
inner join
syscolumns b on a.id = b.id
where
(#TName='' or a.name = #TName)
order ny
a.name, b.name
declare #TableName varchar(200), #ColumnName varchar(200),
#sqlText nvarchar(max), #Val varchar(max)
declare Cur Cursor For
select TableName, ColumnName
from #Tbl
open Cur
fetch Next from cur into #TableName,#ColumnName
while ##FETCH_STATUS =0
begin
set #sqlText=''
set #Val=''
SET #sqlText = N'SELECT Top 1 #Val=[' + #ColumnName + '] FROM testndb.dbo.' + #TableName
EXEC Sp_executesql
#sqlText,
N'#Val varchar(max) output',
#Val output
print #sqlText
update #Tbl set SampleValue=#Val where TableName=#TableName and ColumnName =#ColumnName
fetch Next from cur into #TableName,#ColumnName
End
close cur
deallocate cur
select * from #Tbl

Use of cursor in SQL

I want to know what is the use of cursor? i search on google and i read that cursor is used for manipulate data like in this example cursor is use .. in this example the select statement Select firstName, lastName FROM myTable returns rows
and when i execute whole query with cursor then this returns same as select statement return so what is the difference? we may use only select instead of cursor? here cursor what is the use ? can anyone explain please in simple words
DECLARE #fName varchar(50), #lName varchar(50)
DECLARE cursorName CURSOR -- Declare cursor
LOCAL SCROLL STATIC
FOR
Select firstName, lastName FROM myTable
OPEN cursorName -- open the cursor
FETCH NEXT FROM cursorName
INTO #fName, #lName
PRINT #fName + ' ' + #lName -- print the name
WHILE ##FETCH_STATUS = 0
BEGIN
FETCH NEXT FROM cursorName
INTO #fName, #lName
PRINT #fName + ' ' + #lName -- print the name
END
CLOSE cursorName -- close the cursor
DEALLOCATE cursorName -- Deallocate the cursor
Many DBA's and developers have love/hate relationship with Cursors. Some will tell you not to use it, others that there is no danger and you can use it. As with many other tools, a cursor is just another tool to be used on some specific scenarios, correctly used can be an awesome tool. But incorrectly used can cause big performance issues.
Cursors work on a row basis and are a perfect sample of the RBAR "Row By Agonizing Row" instead of sets-based operations where tsql shines. The sample you provided is a really bad sample of correct cursor usage, yes, it can show you how a cursor works, but as you commented, that same action can be done with a simple SELECT.
If you do a quick search on your prefered search engine will find lot of good references about cursors, here are some to read:
https://www.brentozar.com/sql-syntax-examples/cursor-example/
RBAR vs. Set based programming for SQL
https://www.red-gate.com/simple-talk/sql/t-sql-programming/rbar-row-by-agonizing-row/

SQL server cursor slow performance

I'm getting started with my first use of a cursor in a stored procedure in sql server 2008. I've done some preliminary reading and I understand that they have significant performance limitations. In my current case I think they're necessary (I want to run multiple stored procedures for each stock symbol in a symbols table.
Edit:
The sprocs I'll be calling on each symbol will for the most part be insert operations to calculate symbol- dependent values, such as 5 day moving average, average daily volume, ATR (average true range). Most of these values will be calculated from data from a daily pricing and volume table... I'd like to streamline the retrieval of data values that would be retrieved redundantly otherwise... for example, I'd like to get for each symbol the daily pricing and volume data into a table variable... that temp table will then be passed in to the stored procedure that calls each of the aggregated functions I just mentioned. Hope that makes sense...
So my initial "outer loop" cursor- based stored procedure is below.. it times out after several minutes, without returning anything to the output window.
ALTER PROCEDURE dbo.sprocSymbolDependentAggsDriver2
AS
DECLARE #symbol nchar(10)
DECLARE symbolCursor CURSOR
STATIC FOR
SELECT Symbol FROM tblSymbolsMain ORDER BY Symbol
OPEN symbolCursor
FETCH NEXT FROM symbolCursor INTO #symbol
WHILE ##FETCH_STATUS = 0
SET #symbol = #symbol + ': Test.'
FETCH NEXT FROM symbolCursor INTO #symbol
CLOSE symbolCursor
DEALLOCATE symbolCursor
When I run it without the #symbol local variable and eliminate the assignment to it in the while loop, it seems to run ok. Is there a clear violation of performance best- practices within that assignment? Thanks..
"In my current case I think they're necessary (I want to run multiple
stored procedures for each stock symbol in a symbols table."
Cursors are rarely necessary.
From your example above, I think a simple WHILE loop will easily take the place of your cursor. Adapted from SQL Cursors - How to avoid them (one of my favorite SQL bookmarks)
-- Create a temporary table...
CREATE TABLE #Symbols (
RowID int IDENTITY(1, 1),
Symbol(nvarchar(max))
)
DECLARE #NumberRecords int, #RowCount int
DECLARE #Symbol nvarchar(max)
-- Get your data that you want to loop over
INSERT INTO #Symbols (Symbol)
SELECT Symbol
FROM tblSymbolsMain
ORDER BY Symbol
-- Get the number of records you just grabbed
SET #NumberRecords = ##ROWCOUNT
SET #RowCount = 1
-- Just do a WHILE loop. No cursor necessary.
WHILE #RowCount <= #NumberRecords
BEGIN
SELECT #Symbol = Symbol
FROM #Symbols
WHERE RowID = #RowCount
EXEC <myProc1> #Symbol
EXEC <myProc2> #Symbol
EXEC <myProc3> #Symbol
SET #RowCount = #RowCount + 1
END
DROP TABLE #Symbols
You don't really need all that explicit cursor jazz to build a string. Here is probably a more efficient way to do it:
DECLARE #symbol NVARCHAR(MAX) = N'';
SELECT #symbol += ': Test.'
FROM dbo.tblSymbolsMain
ORDER BY Symbol;
Though I suspect you actually wanted to see the names of the symbol, e.g.
DECLARE #symbol NVARCHAR(MAX) = N'';
SELECT #symbol += N':' + Symbol
FROM dbo.tblSymbolsMain
ORDER BY Symbol;
One caveat is that while you will typically observe the order to be observed, it is not guaranteed. So if you want to stick to the cursor, at least declare the cursor as follows:
DECLARE symbolCursor CURSOR
LOCAL STATIC READ_ONLY FORWARD_ONLY
FOR
...
Also it seems to me like NCHAR(10) is not sufficient to hold the data you're trying to stuff into it, unless you only have one row (which is why I chose NVARCHAR(MAX) above).
And I agree with Abe... it is quite possible you don't need to fire a stored procedure for every row in the cursor, but to suggest ways around that (which will almost certainly be more efficient), we'd have to understand what those stored procedures actually do.
you need an begin end here:
WHILE ##FETCH_STATUS = 0 BEGIN
SET #symbol = #symbol + ': Test.'
FETCH NEXT FROM symbolCursor INTO #symbol
END
also try DECLARE symbolCursor CURSOR LOCAL READ_ONLY FORWARD_ONLY instead of STATIC to improve performance.
After reading all the suggestions, I ended up doing some old trick and it worked miracles!
I had this cursor which was taking almost 3 mins to run, while the enclosing query was instant. I have other databases with more complex cursors that were only taking 1 second or less, so I ruled out the global issue on using cursors. My solution:
Detach the database in question, but ensure you tick Update Statistics.
Attach the database and check performance
This seems to help optimize all the performance parameters without the detailed effort. I am using SQL Express 2008 R2.
Would like to know your experience.

Resources