I am using quite a long dynamic SQL Statement (a bit more than 13000 characters) but when I am trying to execute it, I am noticing that the exec isn't reading the statement completly and cuts the last part off.
I am using:
DECLARE #Statement nvarchar(max)
SET #Statement = N'[LONG STATEMENT]'
EXEC (#Statement)
I did notice, that it could read even less characters, if I am not using the the brackets in EXEC (#Statement)
I also tried using EXEC sp_executesql #Statement
It just stops reading the statement after 12482 characters...
I have the problems with SQL-Server 2008 R2 and SQL Server 2014
EDIT: OK, now I noticed something different. It seems, that the lenght itself of the statement is not exactly the problem. As I mentioned in a comment below, I am using this long dynamic sql statement, because I am creating an update script, which adds a new stored procedure and within this procedure I am using table names, which can differ. So I created variables, which contain the table names and used these variables with the dynamic sql statement, so I don't need to change the table names within the procedures and functions I am adding with this update script, but just changing the content of the variables.
However, if I am NOT using these variables and use the table names "hardcoded" in the statement, the statement can be executed successfully...
I guess the answer is here:
So I created variables, which contain the table names and used these
variables with the dynamic sql statement, so I don't need to change
the table names within the procedures and functions I am adding with
this update script, but just changing the content of the variables.
I guess, your dynamic T-SQL statement is built using string concatenation. So, let's say we have something like this:
DECLARE #DynamicSQLSTatement NVARCHAR(MAX);
DECLARE #TableName01 NVARCHAR(128) = 'T01';
DECLARE #TableName02 NVARCHAR(128) = 'T02';
DECLARE #TSQL NVARCHAR(4000) = REPLICATE(N'X', 4000);
SET #DynamicSQLSTatement = #TableName01 + #TSQL + #TableName02;
We have three short strings (length < max) and when they are concatenated, we expect that the new NVARCHAR(MAX) value will be capable of storing the whole new string (it is with max length, after all).
So, the following statement will give as T02, right?
SELECT RIGHT(#DynamicSQLSTatement, 3);
But no, the output is XXX. So, the question is why the whole concatenation text is not preserved?
When you are concatenating nvarchar(1-4000) strings they output string is not converted to max if it is not possible to store all the data.
In order to fix this, we can cast the first part of the string to nvarchar(max):
SET #DynamicSQLSTatement = CAST(#TableName01 AS NVARCHAR(MAX)) + #TSQL + #TableName02
SELECT RIGHT(#DynamicSQLSTatement, 3);
I'd imagine your hitting a limit of about 16K (8K used to be the most data you could hold in a variable before nvarchar(max) etc was invented. You could try using varchar(max) instead of nvarchar as that will be half the size (unicode is 16-bit, ascii is 8-bit). I have a feeling that won't help much though.
Really though, you're hitting this issue because whatever you're trying to do shouldn't really be done like that, super long SQL statements are a sign that you've gone down the wrong path. Find a way to break down your query or operations. If this is a query, consider if you could restrict your data set over a few steps rather than in one query.
Related
I have the following example of script. It works fine for this example as it is small and basic. I have a far bigger script with around 50 steps that works fine on its own. As in, hit F5 and does everything, checks if table exists, deletes, writes to tables, drops temps and all results are written to the necessary places. I can't seem to place this large script into this small example:
alter procedure james_tester
#tablename nvarchar(200)
as
BEGIN
declare #sql nvarchar(max)
set #sql =
---->
'select * from'
---->
+ #tablename
EXECUTE sp_executesql #sql
END
---When executing:
execute james_tester 'dbo.calendar_delete'
In my case the section in between the arrows is large and will have numerous variables. I just want to know if there is a function or possibly another way to place a large piece of script at a time in that single quote part. I hope I am describing this sufficiently. What affects the entire script currently from just putting a single quote before and after, is that there are already many comments, and single quotes used in the script that seem to stop the entire script from being highlighted red as text and working fine.
James
It is in fact only the presence of single quotes that will cause problems. Take the following illustration:
declare #bigText varchar(max);
SET #bigText = '"The time has come," the Walrus said,
"To talk of many things:
Of shoes and ships and sealing-wax --SQL Comment has no impact
Of cabbages and kings /* c/c++ comment has no impact */
And why the sea is boiling hot //c# comment has no impact
And whether pigs have wings."';
SELECT #bigText;
Single quotes can be doubled using regex or some other string replace function, so that should not be too hard either.
BUT (and there always is a but of course) whether the remaining text is a legal SQL string, which you can execute is an entirely different question. The presence of extraneous comments etc. will almost certainly bite you.
In case anyone comes to this post and needs to know how this ended. I have no real security risks here of SQL injections etc on the database. The procedure takes a few minutes to run now and just takes this execute in the first two rows to run, with the variables being the periods of SAP backup I need to extract:
execute jc_tester '01' , '02' , '03'
go
alter procedure jc_tester
#Period1 varchar (3) ,#Period2 varchar (3) , #Period3 varchar (3)
as
begin
declare #sql nvarchar(max)
set #sql =
replace('...... | | | | ','|','''') + ......... #period1 +
replace('.....','|','''')
execute sp_executesql #sql
end
Each section of text contained ' that I could not seem to avoid and caused havoc when handling as a string. These had to be replace with pipes, then included in a replace statement.
Some other things I learnt along the way that threw my progress out. Don't use Go's in the script to be used in #sql. I did not know this. Rather use ;. Also remove any comments that use the leading '---'. Instead wrap all comments in the script with /...../ . I could swear this made a difference as so many of the comments in the script in #sql were all trailing ---'s.
This process has saved time and now that it is complete I can explore other options other than dynamic sql in a stored procedure and learn how to do it in probably a more appropriate manner. But I'm glad this ran eventually.... Thanks for the guidance.
I am trying to create a generic update procedure. The point of this procedure is that we want to be able to track everything that happens in a table. If a recordis updated, we need to be able to know who changed that record, what it was originally, what it is after the change and when the change occurred. We only do this on our most important tables where accountability is a must.
Right now, we do this through a combination of web server programming and SQL Server commands.
I need to take what we currently have, and make a SQL only version.
So, here are the requirements of what I need:
The original sp is called UpdateWithHistory. Right now, it takes 4 parameters all varchar (or it can be nvarchar, doesn't matter). They are the table name, the primary key field, primary key value and a comma delimited list of fields and values in the format field='value',field1='value1'...etc.
In the background, we have a mapping table that we use to map the string table names to actual tables.
In the stored procedure, I have tried various combinations of OPENROWSET, exec(), select into, xml, and other methods. None seem to work.
So basically, I have to be able to dynamically generate a simple select statement (no joins or other complicated select stuff) from the 4 supplied parameters, then store the results of that query in a table. Since it is dynamic, I don't know the number of fields being queried, or what data types they will be.
I tried select into since that will automatically create a table with the appropriate fields and data types, but it doesn't work in conjunction with the exec command. I have also tried
exec sp_executeSQL #SQL, N'#output xml output', #resultXML output
Where #resultXML is XML datatype and #SQL is the sql command. #resultXML always ends up as null, no matter what I do. I also tried the xml route because I know that "FOR XML Path" always returns one column, but I can't use that in an insert into statement....
That statement output will be used to determine the original values before the update.
I figure once I get past this hurdle the rest will be a piece of cake. Anyone got any ideas?
So here is code for something that I finally got to work, although I don't want to use global tables, so I would gladly accept a different answer...
DECLARE #curRecordString varchar(max) = 'SELECT * into ##TEMP_TABLE FROM SOMEDB.dbo.' + #tbl + ' WHERE ' + #prikey + ' = ''' + #prival + ''' '
exec(#curRecordString)
Basically, as stated before, I need to dynamically build a sql query, then store the result of running the query so that I can access it later. I would prefer to store it as XML datatype, since I will later be using XQuery to parse and compare nodes. In the code above, I am using a global temp table (not ideal, I know) to store the result of the query so that the rest of my procedure can access the data.
Like I said, I don't like this approach but hopefully someone else can come up with something better that will allow me to dynamically build a SQL query, run it, store the results so that I can access the results later in the stored procedure.
This is most definitely a hack, but...
DECLARE #s VARCHAR(MAX)
SET #s = 'SELECT (SELECT 1 as splat FOR XML PATH) a'
CREATE TABLE #save (x XML)
INSERT INTO #save
( x )
EXEC (#s)
SELECT * FROM #save s
DROP TABLE #save
I am new to Transact SQL programming.
I have created a stored procedure that would drop and create an existing synonym so that it will point to another table. The stored procedure takes in 2 parameters:
synonymName - an existing synonym
nextTable - the table to be point at
This is the code snippet:
...
BEGIN TRAN SwitchTran
SET #SqlCommand='drop synonym ' + #synonymName
EXEC sp_executesql #SqlCommand
SET #SqlCommand='create synonym ' + #synonymName + ' for ' + #nextTable
EXEC sp_executesql #SqlCommand
COMMIT SwitchTran
...
We have an application that would write data using the synonym regularly.
My question is would I run into a race condition where the synonym is dropped, while the application try to write to the synonym?
If the above is a problem, could someone give me suggestion to the solution.
Thanks
Yes, you'd have a race condition.
One way to manage this is to have sp_getapplock after BEGIN TRAN in Transaction mode and trap/handle the return status as required. This will literally serialise (in the execution sense, not isolation) callers so only one SPID executes at any one time.
I'm fairly certain you will indeed get race conditions. Synonym Names are intended to be used for shortening the name of an object and aren't supposed to change any more often than other objects. I'm guessing by your description that you are using it for code reuse. You are probably better off using Dynamic SQL instead, which incidentally you already are.
For more information on Dynamic SQL you might want to consider a look at this article on by Erland Sommarskog that OMG Poinies references in a lot of his answers. Particularly the section on Dealing with Dynamic Table and Column Names which I've quotes below
Dealing with Dynamic Table and Column
Names
Passing table and column names as
parameters to a procedure with dynamic
SQL is rarely a good idea for
application code. (It can make
perfectly sense for admin tasks). As
I've said, you cannot pass a table or
a column name as a parameter to
sp_executesql, but you must
interpolate it into the SQL string.
Still you should protect it against
SQL injection, as a matter of routine.
It could be that bad it comes from
user input.
To this end, you should use the
built-in function quotename() (added
in SQL 7). quotename() takes two
parameters: the first is a string, and
the second is a pair of delimiters to
wrap the string in. The default for
the second parameter is []. Thus,
quotename('Orders') returns [Orders].
quotename() takes care of nested
delimiters, so if you have a really
crazy table name like Left]Bracket,
quotename() will return
[Left]]Bracket].
Note that when you work with names
with several components, each
component should be quoted separately.
quotename('dbo.Orders') returns
[dbo.Orders], but that is a table in
an unknown schema of which the first
four characters are d, b, o and a dot.
As long as you only work with the dbo
schema, best practice is to add dbo in
the dynamic SQL and only pass the
table name. If you work with different
schemas, pass the schema as a separate
parameter. (Although you could use the
built-in function parsename() to split
up a #tblname parameter in parts.)
While general_select still is a poor
idea as a stored procedure, here is
nevertheless a version that summarises
some good coding virtues for dynamic
SQL:
CREATE PROCEDURE general_select #tblname nvarchar(128),
#key varchar(10),
#debug bit = 0 AS DECLARE #sql nvarchar(4000)
SET #sql = 'SELECT col1, col2, col3
FROM dbo.' + quotename(#tblname) + '
WHERE keycol = #key' IF #debug = 1
PRINT #sql EXEC sp_executesql #sql, N'#key varchar(10)', #key = #key
- I'm using sp_executesql rather than EXEC().
- I'm prefixing the table name with dbo.
- I'm wrapping #tblname in quotename().
- There is a #debug parameter.
I'm building a stored procedure which is rather stretching my experience. With the help of people who responded to this thread [Nested if statements in SQL Server stored procedure SELECT statement I think I'm most of the way there :)
In short, the SP takes a series of paramterised inputs to dynamically build an SQL statement that creates a temporary table of id values ordered in a specific way. The remainder of the SP, which returns the data according to the requested page from the id values in this temporary table is all sorted.
Reconsider the use of dynamic SQL - you should really know what you are doing if you go that route.
What is the problem you are trying to solve? I am sure people here will be able to find a better solution than the dynamic SQL you are proposing to use.
Take a look at CONVERT() and CAST() for the integers.
To concatenate integer values into the dynamic SQL statement you need to convert to a varchar e.g:
....WHERE
OT.site_id = ' + CAST(#siteid AS VARCHAR)
If the SQL statement is always going to be less than 4000 chars, I'd at least consider using sp_executesql to use parameterised SQL.
e.g.
DECLARE #SQL NVARCHAR(4000)
DECLARE #siteid INTEGER
SET #siteid = 1
SET #SQL = 'SELECT * FROM MyTable WHERE site_id = #siteid'
EXECUTE sp_executesql #SQL, N'#siteid INTEGER', #siteid
All in all, what you're doing is not likely to be very performant/scalable/maintainable and you don't really gain much from having it as a sproc. Plus you need to be very very careful to validate the input as you could open up yourself to SQL injection (hence my point about using sp_executesql with parameterised SQL).
You need to cast the int param to be a char/varchar so that you can add it to the existing string. The fact that you aren't surrounding it with quotes in the final sql means it will be interpreted as a number.
I've got a search screen on which the user can specify any combination of first name, last name, semester, or course. I'm not sure how to optimally code the SQL Server 2005 stored procedure to handle these potentially optional parameters. What's the most efficient way? Separate procedures for each combination? Taking the items in as nullable parms and building dynamic SQL?
I'd set each parameter to optional (default value being null)
and then handle it in the WHERE....
FirstName=ISNULL(#FirstName,FirstName)
AND
LastName=ISNULL(#LastName,LastName)
AND
SemesterID=ISNULL(#SemesterID,SemesterID)
That'll handle only first name, only last name, all three, etc., etc.
It's also a lot more pretty/manageable/robust than building the SQL string dynamically and executing that.
The best solution is to utilize sp_execute_sql. For example:
--BEGIN SQL
declare #sql nvarchar(4000)
set #sql =
'select * from weblogs.dbo.vwlogs
where Log_time between #BeginDate and #EndDate'
+ case when #UserName is null then '' else 'and client_user = #UserName' end
sp_execute_sql
#sql
, #params = '#UserName varchar(50)'
, #UserName = #UserName
--END SQL
As muerte mentioned, this will have a performance benefit versus exec()'ing a similar statement.
I would do it with sp_executesql because the plan will be cached just for the first pattern, or the first set of conditions.
Take a look at this TechNet article:
sp_executesql can be used instead of stored procedures to execute a Transact-SQL statement many times when the change in parameter values to the statement is the only variation. Because the Transact-SQL statement itself remains constant and only the parameter values change, the SQL Server query optimizer is likely to reuse the execution plan it generates for the first execution.
Was just posting the same concept as Kevin Fairchild, that is how we typically handle it.
You could do dynamic sql in your code to create the statement as required but if so you need to watch for sql injection.
As muerte points out, the plan will be cached for the first set of parameters. This can lead to bad performance when its run each subsequent time using alternate parameters. To resolve that use the WITH RECOMPILE option on the procedure.