I am trying to execute a dynamically created query using FORMATMESSAGE SQL function, and when I do that I get an error
Incorrect syntax near '.'
which I am assuming is because of the trimmed SQL query created.
set #sql = FORMATMESSAGE('insert into %s (imported_on, source_filename, md5_hash %s) select GETDATE(), ''%s'', ''%s'' %s from %s',
#target_tablename, #columns, #source_filename, 'TODO', #columns, #source_table);
exec (#sql);
There is a long list of columns that #columns variable holds ~ 300 (columns) and length goes beyond 4000. Variables #columns and #sql both are of type nvarchar(max).
I have seen other posts that recommend splitting to multiple variables, but also mention nvarchar(max) should work. I am clueless on how to proceed if I was not to split the string.
Thanks
Perform the string concatenation yourself.
declare #sql nvarchar(max);
set #sql = convert(nvarchar(max), '')
+ 'insert into ' + #target_tablename + '(imported_on, ....
exec (#sql);
Related
Using SQL Server 2012, I've found that trying to build up a string based on an nvarchar(max) column in a table doesn't seem to work correctly. It seems to overwrite, instead of append. Arbitrary Example:
DECLARE #sql nvarchar(max);
SELECT #sql = N'';
SELECT #sql += [definition] + N'
GO
'
FROM sys.sql_modules
WHERE OBJECT_NAME(object_id) LIKE 'dt%'
ORDER BY OBJECT_NAME(object_id);
PRINT #sql;
This SHOULD print out all the SQL module definitions for all the various dt_ tables in the database, separated by GO, as a script that could then be run. However... this prints out only the LAST module definition, not the sum of all of them. It's behaving as if the "+=" were just an "=".
If you change it just slightly... cast [definition] to an nvarchar(4000) for example, it suddenly works as expected. Also, if you choose any other column that is NOT an nvarchar(max) or varchar(max) type, it works as expected. Example:
DECLARE #sql nvarchar(max);
SELECT #sql = N'';
SELECT #sql += CAST([definition] AS nvarchar(4000)) + '
GO
'
FROM sys.sql_modules
WHERE OBJECT_NAME(object_id) LIKE 'dt%'
ORDER BY OBJECT_NAME(object_id);
PRINT #sql;
Is this a known bug? Or is this working as expected? Am I doing something wrong? Is there any way for me to make this work correctly? I've tried a dozen different things, including ensuring every expression in the concatenation is the same nvarchar(max) type, including the string literal.
NOTE: The example is just an example that shows the problem, and not exactly what I'm trying to do in real life. If your database doesn't have the "dt*" tables defined, you can change the WHERE clause to specify any group of tables or stored procedures in any database you want, you'll get the same result... only the last one shows up in the #sql string, as if you just did "=" instead of "+=". Also, explicitly stating "#sql = #sql + " behaves the same way... works correctly with every string type EXCEPT nvarchar(max) or varchar(max).
I've verified that none of the [definition] values is NULL as well, so there are no NULL shenanigans going on.
The += operator only applies to numeric data types in SQL Server. Microsoft documentation here
For string concatenation, you need to write the assignment and concatenation separately.
DECLARE #sql nvarchar(max);
SELECT #sql = N'';
SELECT #sql = #sql + [definition] + N'
GO
'
FROM sys.sql_modules
WHERE OBJECT_NAME(object_id) LIKE 'dt%'
ORDER BY OBJECT_NAME(object_id);
PRINT #sql;
Also, if you are running this query in Management Studio, keep in mind that there is a limit to the size of the data that it will return (including in a print statement). So if the definitions of your modules exceed this limit, they will be truncated in the output.
To add on to #HABO's comment, the behavior of aggregate string concatenation is undefined and results are plan dependent. Use FOR XML to provide deterministic results and honor the ORDER BY clause:
DECLARE #sql nvarchar(max);
SET #sql =
(SELECT [definition] + N'
GO
'
FROM sys.sql_modules
WHERE OBJECT_NAME(object_id) LIKE 'dt%'
ORDER BY OBJECT_NAME(object_id)
FOR XML PATH(''), TYPE).value('(./text())[1]', 'nvarchar(MAX)');
PRINT #sql; --SSMS will truncate long strings
It works as expected:
DECLARE #sql nvarchar(max);
SELECT #sql =COALESCE(#sql + N' GO ','')+[definition]
FROM sys.sql_modules
Print #sql;
The following code demonstrates taking a column value from a set of ordered rows and, using for xml, creating an aggregated string with separators, including line breaks, between values.
-- Carriage return and linefeed characters.
declare #CRLF as NVarChar(16) = NChar( 13 ) + NChar( 10 );
-- Carriage return and linefeed characters XML encoded.
declare #EncodedCRLF as NVarChar(16) = N'
';
-- Separator to insert between source rows in the accumulated string.
declare #Separator as NVarChar(64) = ';' + #EncodedCRLF + 'go;' + #EncodedCRLF + 'do something with ';
-- Characters in #Separator to remove from the first element in #Result .
declare #SkipCount as Int = 4 + 2 * Len( #EncodedCRLF );
-- The result.
declare #Result as NVarChar( max ) = '';
-- Do it.
select #Result =
Replace(
Stuff(
-- Specify the sorting order of the XML elements in the following select statement.
( select #Separator + name from sys.tables order by name desc for XML path(''), type).value('.[1]', 'VarChar(max)' ),
1, #SkipCount, '' ),
#EncodedCRLF, #CRLF );
-- Output the result.
select #Result as [Result];
-- Output the result with line breaks shown.
print #Result; -- Switch to Messages tab in SSMS to see result.
I´m pretty sure the PRINT command is the reason for the troubles.
When I try:
DECLARE #SQL NVARCHAR(MAX) = N'';
SELECT #sql += SUBSTRING( [definition], 1, 100 ) + CHAR(13) + CHAR(10) + 'GO' + CHAR(13) + CHAR(10)
FROM SYS.SQL_MODULES
PRINT #sql;
I get all my procedures (the first 100 chars ...). So it seems to be a limitation to the PRINT command.
Second try, another database, without "substring":
DECLARE #SQL NVARCHAR(MAX) = N'';
SELECT #sql += [definition] + CHAR(13) + CHAR(10) + 'GO' + CHAR(13) + CHAR(10)
FROM SYS.SQL_MODULES
PRINT #sql;
Now I got 1,5 procedures: The 1st one complete, the second stopped in the middle of on line in the proc. There is a limit in PRINT
Third try, to proof my assumption:
DECLARE #SQL NVARCHAR(MAX) = N'';
SELECT #sql += [definition] + 'GO' FROM SYS.SQL_MODULES
PRINT #sql;
Now I got one line of code more! So for me, this is the proof, that PRINT has somehow a limit.
A last attempt:
DECLARE #SQL NVARCHAR(MAX) = N'';
SELECT #sql += [definition] + N'GO' FROM SYS.SQL_MODULES
PRINT SUBSTRING(#SQL,1,4000)
PRINT SUBSTRING(#SQL,4001,4000)
The result: Now I got more then 2 procedures, although there now is a line break after 4K, this may occur on ... bad places.
This is my first project in Dynamic SQL.
When i run the below query. I'm getting an error:
Must declare scalar variable "
Though i declared the variable #lcrcolumn_total upfront.
EXECUTE (' UPDATE facetswrk.dbo.ODS_SUBSC_PREM_REPORT ' + ' SET ' + #lcrcolumn_name + ' = #lcrcolumn_total')
Thanks in advance!!!
You need to pass variable to dynamic SQL:
DECLARE #sql NVARCHAR(MAX) =
'UPDATE facetswrk.dbo.ODS_SUBSC_PREM_REPORT
SET #lcrcolumn_name = #lcrcolumn_total'
-- WHERE = ?; -- are you sure you want to update all rows
SET #sql = REPLACE(#sql, '#lcrcolumn_name', QUOTENAME(#lcrcolumn_name));
EXEC dbo.sp_executesql
#sql,
N'#lcrcolumn_total INT', -- set type of #lcorumn_total
#lcrcolumn_total;
LiveDemo
Remarks:
Add WHERE condition otherwise you will update all rows
Use sp_executesql instead of EXEC
Pass variable #lcrcolumn_total with correct datatype
Use QUOTENAME to avoid SQL Injection, when replacing column_name
If you are learning dynamic SQL, then just learn to use sp_executesql, because it is the best way to execute such statement. It allows you to pass arguments in and out. This is important because the "exec" statement does not share variables with the outer context.
The code would look more like this:
DECLARE #sql nvarchar(max);
SET #sql = '
UPDATE facetswrk.dbo.ODS_SUBSC_PREM_REPORT
SET #lcrcolumn_name = #lcrcolumn_total
';
SET #sql = REPLACE(#sql, '#lcrcolumn_name', #lcrcolumn_name);
EXEC sp_executesql #sql, N'#lcrcolumn_total int', #lcrcolumn_total = #lcrcolumn_total;
Note that you cannot pass column and table names as parameters, so these are handled using REPLACE().
Can you assist me in understanding this odd NVARCHAR variable behavior?
I'm building a dSQL query that ranges in LEN() from about 6,700 to 6,900 characters, and stored it in a variable:
DECLARE
#SQL NVARCHAR(MAX),
#WhereClause NVARCHAR(MAX) = 'WHERE 1 = 1
',
#OrderByClaose NVARCHAR(MAX)
SET #SQL =
'SELECT
<column list>
FROM
<very complicated FROM clause>
'
Unfortunately, I'm not allowed to post the actual code.
Initially, the dSQL was a long constant SELECT and FROM string of about 6,600 chars. The variable length dSQL WHERE and ORDER BY clauses were determined
IF <condition p>
SET #WhereClause = #WhereClause + 'AND <condition p clause>
'
IF <condition q>
SET #WhereClause = #WhereClause + 'AND <condition q clause>
'
etc.
and then tacked on to the end of #SQL.
The whole of #SQL would PRINT nicely and then execute with sp_executesql without a hitch.
So naturally the customer requires some changes, causing the nice constant string to be broken into several pieces. This required IF statements that add the dynamic column parts into the middle of the SELECT column list.
SET #SQL = '
SELECT <code through first few columns>
...
'
IF <condition>
SET #SQL = #SQL +
' <new dynamic column spec stuff A>
'
ELSE
SET #SQL = #SQL +
' <new dynamic column spec stuff B>
'
SET #SQL = #SQL + ' <next block of constant chars>'
...
When I added the new IF/ELSE blocks, for some reason #SQL stopped accepting data after the first 4,000 characters, about halfway through the FROM clause.
Remove the IF/ELSE blocks and returning the SELECT and FROM to a constant string (just adding one of the new requirements cases for the test) and #SQL again happily accepted all 6,700 or so characters.
As a workaround, I changed the declaration of #SQL to VARCHAR(MAX)
DECLARE #SQL varchar(max)
SET #SQL = <The built-up query string with the new IF/ELSE blocks>
DECLARE #NSQL nvarchar(max) = cast(#SQL as nvarchar(max))
exec sp_executesql #NSQL
and it all ran fine.
Though I haven't tested it, I'm wondering if the only reason this workaround works for this particular task is because the character count in my largest possible dSQL statement is less than 8,000 chars.
I've taken a look at several articles, including
For Nvarchar(Max) I am only getting 4000 characters in TSQL? and
SQL NVARCHAR and VARCHAR Limits
but nothing I've found seems to explain the behavior I've observed with this particular dSQL construction.
Any idea what's at work here?
First of all, since you have a NVARCHAR datatype, you should always use the N'...' format (with the leading N prefix) to clearly denote Unicode string literals.
DECLARE #WhereClause NVARCHAR(MAX) = N'WHERE 1 = 1'
Secondly, if that alone doesn't help, try casting the string literals to NVARCHAR(MAX) before concatenation
IF <condition>
SET #SQL = #SQL + CAST(...... AS NVARCHAR(MAX))
How to trim a large table with more than 300 columns dynamically. when i tried with this code i was getting an error because the variable #sql is of nvarchar(max) and i have like 300 columns so all the column names were not coming when i run this query can any of the guys help me in this ?
DECLARE #SQL nVARCHAR(MAX)
DECLARE #TableName NVARCHAR(128)
SET #TableName = 'MYTAbleName'
SELECT #SQL = COALESCE(#SQL + ',[', '[') +
COLUMN_NAME + ']=LTRIM(RTRIM([' + COLUMN_NAME + ']))'
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #TableName
SET #SQL = 'UPDATE [' + #TableName + '] SET ' + #SQL
PRINT #SQL
EXECUTE #SQL
While you have defined #SQL nvarchar(max), it all boils down to how a query is sent to the SQL server engine.
Quote from MSDN maximum capacity specifications.
"Network Packet Size is the size of the tabular data stream (TDS) packets used to communicate between applications and the relational Database Engine. The default packet size is 4 KB, and is controlled by the network packet size configuration option."
The Max batch size is 64 * 4 or 250 MB for a query to be submitted to the engine. This is alot smaller than the (max) = 2GB.
Add a debugging line to see if you are coming close to this limitation. Comment out printing the t-sql and executing the query.
-- Show how long the dynamic t-sql is
-- PRINT #SQL
PRINT 'LEN OF TSQL'
PRINT LEN(#SQL)
-- EXECUTE #SQL
Please report back if this is the issue!
I do not have a test case like yours.
-- Use msdb
USE MSDB
GO
-- Largest column count = 62, [sysutility_ucp_instances]
SELECT TABLE_NAME, MAX(ORDINAL_POSITION) as MaxCols
FROM INFORMATION_SCHEMA.COLUMNS
GROUP BY TABLE_NAME
order by MAX(ORDINAL_POSITION) desc
Since your TSQL is getting cut off, it must be casting to a varchar() which has a max of 8000 bytes.
Try casting each piece of the update to varchar(max).
SET #SQL = cast('UPDATE [' as varchar(max)) + cast(#TableName as varchar(max)) + cast('] SET ' as varchar(max)) + #SQL;
Use varchar() instead of nvarchar to get one power of 2 back.
I hope this fixes your issue.
I am trying to run a query in SQL 2008 by doing:
#query varchar(max)
SET #query = 'SELECT * FROM Table WHERE [Name] = ' 'Karl' ' '
EXEC(#query)
The problem is that for some reason the apostrophes around 'Karl' don't get escaped, i.e. the query executes as ...WHERE [Name] = Karl and fails.
Anyone has a suggestion?
There are several ways that you can escape character data in SQL Server, some people even advocate the use of the QUOTENAME() functions.
If you really want to develop of solid understanding of this subject area then may I recommend that you take a look at what experienced SQL Server Developers consider to be essential reading with regard to the different methods you can use to incorporate Dynamic T-SQL into your coding.
The Curse and Blessings of Dynamic SQL
Try:
DECLARE #query varchar(max)
SET #query = 'SELECT * FROM Table WHERE [Name] = ''Karl'''
PRINT 'when in doubt, print the query out: '+ISNULL(#query,'')
EXEC(#query)
To have a single quote appear, you need to have two adjacent single quotes. You escape a single quote with a single quote, for example:
PRINT '''' --will print a one single quote
PRINT '''''' --will print two single quotes
PRINT 'can''t' --will print can't
This works on my machine from SQL Server Management Studio:
#query varchar(max)
SET #query = 'SELECT * FROM Table WHERE [Name] = ''''''Karl'''''''
EXEC(#query)
A double single ('') quote will act like a single single quote when inside a string literal.
Have you tried using a variable?
declare #karl_name varchar(10);
set #karl_name = '''Karl''';
SELECT * FROM Table WHERE [Name] = #karl_name
do like this
SET #query = 'SELECT * FROM Table WHERE [Name] = ''''Karl'''''
This works:
create table #demo([Name] varchar(max))
insert into #demo([Name]) values('''Karl''')
insert into #demo([Name]) values('Karl')
declare #query varchar(max)
set #query = 'SELECT * FROM #demo WHERE [Name] = ''''''Karl'''''''
EXEC(#query)
Output:
'Karl'
But if 'Karl' is variable text, it's highly recommended to use something like this instead:
declare #query nvarchar(max)
declare #param varchar(max)
set #param = N'''Karl'''
set #query = N'SELECT * FROM #demo WHERE [Name] = #param'
exec sp_executesql #query, N'#param varchar(max)', #param
Simply escape the apostrophes by using the escaping bar \ like this 'SELECT * FROM Table WHERE [Name] = ' \'Karl\' ' '
Hope it helps