I have:
a table with an xml type column (list of IDs)
an xml type parameter (also list of IDs)
What is the best way to remove nodes from the column that match the nodes in the parameter, while leaving any unmatched nodes untouched?
e.g.
declare #table table (
[column] xml
)
insert #table ([column]) values ('<r><i>1</i><i>2</i><i>3</i></r>')
declare #parameter xml
set #parameter = '<r><i>1</i><i>2</i></r>'
-- this is the problem
update #table set [column].modify('delete (//i *where text() matches #parameter*)')
The MSDN documentation indicates it should be possible (in Introduction to XQuery in SQL Server 2005):
This stored procedure can easily be
modified to accept an XML fragment
which contains one or more skill
elements thereby allowing the user to
delete multiple skill nodes with a
single invocation of stored procedure.
While the delete is a little awkward to do this way, you can instead do an update to change the data, provided your data is simple (such as the example you gave). The following query will basically split the two XML strings into tables, join them, exclude the non-null (matching) values, and convert it back to XML:
UPDATE #table
SET [column] = (
SELECT p.i.value('.','int') AS c
FROM [column].nodes('//i') AS p(i)
OUTER APPLY (
SELECT x.i.value('.','bigint') AS i
FROM #parameter.nodes('//i') AS x(i)
WHERE p.i.value('.','bigint') = x.i.value('.','int')
) a
WHERE a.i IS NULL
FOR XML PATH(''), TYPE
)
You need something in the form:
[column].modify('delete (//i[.=(1, 2)])') -- like SQL IN
-- or
[column].modify('delete (//i[.=1 or .=2])')
-- or
[column].modify('delete (//i[.=1], //i[.=2])')
-- or
[column].modify('delete (//i[contains("|1|2|",concat("|",.,"|"))])')
XQuery doesn't support xml SQL types in SQL2005, and the modify method only accepts string literals (no variables allowed).
Here's an ugly hack w/ the contains function:
declare #table table ([column] xml)
insert #table ([column]) values ('<r><i>1</i><i>2</i><i>3</i></r>')
declare #parameter xml
set #parameter = '<r><i>1</i><i>2</i></r>'
-- build a pipe-delimited string
declare #in nvarchar(max)
set #in = convert(nvarchar(max),
#parameter.query('for $i in (/r/i) return concat(string($i),"|")')
)
set #in = '|'+replace(#in,'| ','|')
update #table set [column].modify ('
delete (//i[contains(sql:variable("#in"),concat("|",.,"|"))])
')
select * from #table
Here's another w/ dynamic SQL:
-- replace table variable with temp table to get around variable scoping
if object_id('tempdb..#table') is not null drop table #table
create table #table ([column] xml)
insert #table ([column]) values ('<r><i>1</i><i>2</i><i>3</i></r>')
declare #parameter xml
set #parameter = '<r><i>1</i><i>2</i></r>'
-- we need dymamic SQL because the XML modify method only permits string literals
declare #sql nvarchar(max)
set #sql = convert(nvarchar(max),
#parameter.query('for $i in (/r/i) return concat(string($i),",")')
)
set #sql = substring(#sql,1,len(#sql)-1)
set #sql = 'update #table set [column].modify(''delete (//i[.=('+#sql+')])'')'
print #sql
exec (#sql)
select * from #table
if you are updating an xml variable rather than a column, use sp_executesql and output parameters:
declare #xml xml
set #xml = '<r><i>1</i><i>2</i><i>3</i></r>'
declare #parameter xml
set #parameter = '<r><i>1</i><i>2</i></r>'
declare #sql nvarchar(max)
set #sql = convert(nvarchar(max),
#parameter.query('for $i in (/r/i) return concat(string($i),",")')
)
set #sql = substring(#sql,1,len(#sql)-1)
set #sql = 'set #xml.modify(''delete (//i[.=('+#sql+')])'')'
exec sp_executesql #sql, N'#xml xml output', #xml output
select #xml
Alternate method using a cursor to iterate through delete values, probably less efficient due to multiple updates:
declare #table table ([column] xml)
insert #table ([column]) values ('<r><i>1</i><i>2</i><i>3</i></r>')
declare #parameter xml
set #parameter = '<r><i>1</i><i>2</i></r>'
/*
-- unfortunately, this doesn't work:
update t set [column].modify('delete (//i[.=sql:column("p.i")])')
from #table t, (
select i.value('.', 'nvarchar')
from #parameter.nodes('//i') a (i)
) p (i)
select * from #table
*/
-- so we have to use a cursor
declare #cursor cursor
set #cursor = cursor for
select i.value('.', 'varchar') as i
from #parameter.nodes('//i') a (i)
declare #i int
open #cursor
while 1=1 begin
fetch next from #cursor into #i
if ##fetch_status <> 0 break
update #table set [column].modify('delete (//i[.=sql:variable("#i")])')
end
select * from #table
More information on the limitations of XML variables in SQL2005 here:
http://blogs.msdn.com/denisruc/archive/2006/05/17/600250.aspx
Does anyone have a better way?
Related
I'm trying to build a stored procedure that will query multiple database depending on the databases required.
For example:
SP_Users takes a list of #DATABASES as parameters.
For each database it needs to run the same query and union the results together.
I believe a CTE could be my best bet so I have something like this at the moment.
SET #DATABASES = 'DB_1, DB_2' -- Two databases in a string listed
-- I have a split string function that will extract each database
SET #CURRENT_DB = 'DB_1'
WITH UsersCTE (Name, Email)
AS (SELECT Name, Email
FROM [#CURRENT_DB].[dbo].Users),
SELECT #DATABASE as DB, Name, Email
FROM UsersCTE
What I don't want to do is hard code the databases in the query. The steps I image are:
Split the parameter #DATABASES to extract and set the #CURRENT_DB Variable
Iterate through the query with a Recursive CTE until all the #DATABASES have been processed
Union all results together and return the data.
Not sure if this is the right approach to tackling this problem.
Using #databases:
As mentioned in the comments to your question, variables cant be used to dynamically select a database. Dynamic sql is indicated. You can start by building your template sql statement:
declare #sql nvarchar(max) =
'union all ' +
'select ''#db'' as db, name, email ' +
'from [#db].dbo.users ';
Since you have sql server 2016, you can split using the string_split function, with your #databases variable as input. This will result in a table with 'value' as the column name, which holds the database names.
Use the replace function to replace #db in the template with value. This will result in one sql statement for each database you passed into #databases. Then, concatenate the statements back together. Unfortunately, in version 2016, there's no built in function to do that. So we have to use the famous for xml trick to join the statements, then we use .value to convert it to a string, and finally we use stuff to get rid of the leading union all statement.
Take the results of the concatenated output, and overwrite the #sql variable. It is ready to go at this point, so execute it.
I do all that is described in this code:
declare #databases nvarchar(max) = 'db_1,db_2';
set #sql = stuff(
(
select replace(#sql, '#db', value)
from string_split(#databases, ',')
for xml path(''), type
).value('.[1]', 'nvarchar(max)')
, 1, 9, '');
exec(#sql);
Untested, of course, but if you print instead of execute, it seems to give the proper sql statement for your needs.
Using msForEachDB:
Now, if you didn't want to have to know which databases had 'users', such as if you're in an environment where you have a different database for every client, you can use sp_msForEachDb and check the structure first to make sure it has a 'users' table with 'name' and 'email' columns. If so, execute the appropriate statement. If not, execute a dummy statement. I won't describe this one, I'll just give the code:
declare #aggregator table (
db sysname,
name int,
email nvarchar(255)
);
insert #aggregator
exec sp_msforeachdb '
declare #sql nvarchar(max) = ''select db = '''''''', name = '''''''', email = '''''''' where 1 = 2'';
select #sql = ''select db = ''''?'''', name, email from ['' + table_catalog + ''].dbo.users''
from [?].information_schema.columns
where table_schema = ''dbo''
and table_name = ''users''
and column_name in (''name'', ''email'')
group by table_catalog
having count(*) = 2
exec (#sql);
';
select *
from #aggregator
I took the valid advice from others here and went with this which works great for what I need:
I decided to use a loop to build the query up. Hope this helps someone else looking to do something similar.
CREATE PROCEDURE [dbo].[SP_Users](
#DATABASES VARCHAR(MAX) = NULL,
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)
)
BEGIN
SET NOCOUNT ON;
--Local variables
DECLARE
#COUNTER INT = 0,
#SQL NVARCHAR(MAX) = '',
#CURRENTDB VARCHAR(50) = NULL,
#MAX INT = 0,
#ERRORMSG VARCHAR(MAX)
--Check we have databases entered
IF #DATABASES IS NULL
BEGIN
RAISERROR('ERROR: No Databases Provided,
Please Provide a list of databases to execute procedure. See stored procedure:
[SP_Users]', 16, 1)
RETURN
END
-- SET Number of iterations based on number of returned databases
SET #MAX = (SELECT COUNT(*) FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i)X)
-- Build SQL Statement
WHILE #COUNTER < #MAX
BEGIN
--Set the current database
SET #CURRENTDB = (SELECT X.Value FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i
ORDER BY RowNumber OFFSET #COUNTER
ROWS FETCH NEXT 1 ROWS ONLY) X);
SET #SQL = #SQL + N'
(
SELECT Name, Email
FROM [' + #CURRENTDB + '].[dbo].Users
WHERE
(Name = #PARAM1 OR #PARAM1 IS NULL)
(Email = #PARAM2 OR #PARAM2 IS NULL)
) '
+ N' UNION ALL '
END
PRINT #CURRENTDB
PRINT #SQL
SET #COUNTER = #COUNTER + 1
END
-- remove last N' UNION ALL '
IF LEN(#SQL) > 11
SET #SQL = LEFT(#SQL, LEN(#SQL) - 11)
EXEC sp_executesql #SQL, N'#CURRENTDB VARCHAR(50),
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)',
#CURRENTDB,
#PARAM1 ,
#PARAM2
END
Split Variable Function
CREATE FUNCTION [dbo].[udf_SplitVariable]
(
#List varchar(8000),
#SplitOn varchar(5) = ','
)
RETURNS #RtnValue TABLE
(
Id INT IDENTITY(1,1),
Value VARCHAR(8000)
)
AS
BEGIN
--Account for ticks
SET #List = (REPLACE(#List, '''', ''))
--Account for 'emptynull'
IF LTRIM(RTRIM(#List)) = 'emptynull'
BEGIN
SET #List = ''
END
--Loop through all of the items in the string and add records for each item
WHILE (CHARINDEX(#SplitOn,#List)>0)
BEGIN
INSERT INTO #RtnValue (value)
SELECT Value = LTRIM(RTRIM(SUBSTRING(#List, 1, CHARINDEX(#SplitOn, #List)-1)))
SET #List = SUBSTRING(#List, CHARINDEX(#SplitOn,#List) + LEN(#SplitOn), LEN(#List))
END
INSERT INTO #RtnValue (Value)
SELECT Value = LTRIM(RTRIM(#List))
RETURN
END
I have the following code to create a SQL function that will parse an XML string and create a table of key value pairs that represent the nodes and values. This works fine for me in my use cases.
CREATE FUNCTION XmlToKeyValue
(
#rootName AS varchar(256),
#xml AS Xml
)
RETURNS #keyval TABLE ([key] varchar(max), [value] varchar(max))
AS
BEGIN
DECLARE #input TABLE (XmlData XML NOT NULL)
INSERT INTO #input VALUES(#xml)
INSERT #keyval ([key], [value])
SELECT
XC.value('local-name(.)', 'varchar(max)') AS [key],
XC.value('(.)[1]', 'varchar(max)') AS [value]
FROM
#input
CROSS APPLY
XmlData.nodes('/*[local-name()=sql:variable("#rootName")]/*') AS XT(XC)
RETURN
END
What I am trying to do is have a stored procedure in my main database that will create another database with all the appropriate functions/procedures/etc. So in that stored procedure I am trying to do something like this:
SET #cmd = '
CREATE FUNCTION XmlToKeyValue
(
#rootName AS varchar(256),
#xml AS Xml
)
RETURNS #keyval TABLE ([key] varchar(max), [value] varchar(max))
AS
BEGIN
DECLARE #input TABLE (XmlData XML NOT NULL)
INSERT INTO #input VALUES(#xml)
INSERT #keyval ([key], [value])
SELECT
XC.value(''local-name(.)'', ''varchar(max)'') AS [key],
XC.value(''(.)[1]'', ''varchar(max)'') AS [value]
FROM
#input
CROSS APPLY
XmlData.nodes(''/*[local-name()=sql:variable("#rootName")]/*'') AS XT(XC)
RETURN
END
'
BEGIN TRY
EXEC(N'USE '+#dbName+'; EXEC sp_executesql N''' + #cmd + '''; USE master')
END TRY
BEGIN CATCH
PRINT 'Error creating XmlToKeyValue'
Print Error_Message();
RETURN
END CATCH
However, I am getting the following error that I can't figure out how to resolve.
Error creating XmlToKeyValue
Incorrect syntax near 'local'.
Can I use local-name in a dynamic sql statement? If not, how can I achieve my goal? Thank you.
The problem is not the local-name function. It is entirely the fact that you are concatenating in the #cmd variable into your Dynamic SQL without properly escaping the embedded single-quotes.
This line:
EXEC(N'USE '+#dbName+'; EXEC sp_executesql N''' + #cmd + '''; USE master')
should be:
SET #cmd = REPLACE(#cmd, N'''', N'''''');
EXEC(N'USE ' + #dbName + N'; EXEC sp_executesql N''' + #cmd + N''';');
Else you are embedding:
XC.value(''local-name(
into the string, but using the same number of escape sequences, hence the XC.value( now becomes the end of the string and the local-name(.) is technically unescaped SQL and not part of a string.
Also:
You don't need the USE master at the end of the Dynamic SQL (so I removed it).
You prefixed the first string literal with N but none of the others (I added the N in for the others so that they all have that prefix).
I have a form with 3 check box dropdown lists enabling multiple selection from each control.
Lets say for talking sake its an accommodation table I am querying and the check box dropdown lists are 'AccommodationName', 'Company', and 'Nights'.
So potentially I could be passing in multiple values from each control and I want to return an aggregated query relevant to all data input.
How should I be going about this query?
Is the query going to have to be dynamic sql?
Please note, I am using sql server 2005.
You will need to create a split function inside you database,
Definition Of Split Function
CREATE FUNCTION [dbo].[split]
(
#delimited NVARCHAR(MAX),
#delimiter NVARCHAR(100)
)
RETURNS #t TABLE (id INT IDENTITY(1,1), val NVARCHAR(MAX))
AS
BEGIN
DECLARE #xml XML
SET #xml = N'<t>' + REPLACE(#delimited,#delimiter,'</t><t>') + '</t>'
INSERT INTO #t(val)
SELECT r.value('.','varchar(MAX)') as item
FROM #xml.nodes('/t') as records(r)
RETURN
END
Stored Procedure
Then you need to create a stored procedure which will build sql query dynamically and use this split function to handle multiple values passed as a comma deliminated list.
CREATE PROCEDURE GetData
#AccommodationName VARCHAR(1000) = NULL,
#Company VARCHAR(1000) = NULL,
#Nights VARCHAR(1000) = NULL
AS
BEGIN
SET NOCOUNT ON;
DECLARE #SQL NVARCHAR(MAX);
SET #SQL = N' SELECT * FROM TableName WHERE 1 = 1 '
+ CASE WHEN #AccommodationName IS NOT NULL
THEN N' AND AccommodationName IN (SELECT Val FROM dbo.split(#AccommodationName )) '
ELSE N'' END
+ CASE WHEN #Company IS NOT NULL
THEN N' AND Company IN (SELECT Val FROM dbo.split(#Company)) '
ELSE N'' END
+ CASE WHEN #Nights IS NOT NULL
THEN N' AND Nights IN (SELECT Val FROM dbo.split(#Nights)) '
ELSE N'' END
EXECUTE sp_executesql #SQL
,N'#AccommodationName VARCHAR(1000), #Company VARCHAR(1000), #Nights VARCHAR(1000)'
,#AccommodationName
,#Company
,#Nights
END
I've been trying for a while to use SQL Server pivot but I just don't seem to be getting it right. I've read a bunch of SO answers, but don't understand how pivot works.
I'm writing a stored procedure. I have Table 1 (received as a TVP), and need to make it look like Table 2 (see this image for tables).
Important: the values in Table1.valueTypeID cannot be hard coded into the logic because they can always change. Therefore, the logic must be super dynamic.
Please see the code below. The pivot is at the end of the stored procedure.
-- Create date: 12/10/2013
-- Description: select all the contacts associated with received accountPassport
-- =============================================
ALTER PROCEDURE [dbo].[selectContactsPropsByAccountPassport]
-- Add the parameters for the stored procedure here
#accountPassport int,
#valueTypeFiltersTVP valueTypeFiltersTVP READONLY
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
DECLARE #accountID int;
DECLARE #contactsAppAccountPassport int;
DECLARE #searchResults TABLE
(
resultContactID int
);
DECLARE #resultContactID int;
DECLARE #contactsPropsForReturn TABLE
(
contactID int,
valueTypeID int,
value varchar(max)
);
create table #contactsPropsForReturnFiltered(contactID int,valueTypeID int, value varchar(max))
/*
DECLARE #contactsPropsForReturnFiltered TABLE
(
contactID int,
valueTypeID int,
value varchar(max)
);
*/
--2. get #contactsAppAccountPassport associated with recieved #accountPassport
-- go into dbo.accounts and get the #accountID associated with this #accountPassport
SELECT
#accountID = ID
FROM
dbo.accounts
WHERE
passport = #accountPassport
-- go into dbo.accountsProps and get the value (#contactsAppAccountPassport) where valueType=42 and accountID = #accountID
SELECT
#contactsAppAccountPassport = value
FROM
dbo.accountsProps
WHERE
(valueTypeID=42) AND (accountID = #accountID)
--3. get all the contact ID's from dbo.contacts associated with #contactsAppAccountPassport
INSERT INTO
#searchResults
SELECT
ID
FROM
dbo.contacts
WHERE
contactsAppAccountPassport = #contactsAppAccountPassport
--4. Get the props of all contact ID's from 3.
--start for each loop....our looping object is #resultContactID row. if there are more rows, we keep looping.
DECLARE searchCursor CURSOR FOR
SELECT
resultContactID
FROM
#searchResults
OPEN searchCursor
FETCH NEXT FROM searchCursor INTO #resultContactID
WHILE (##FETCH_STATUS=0)
BEGIN
INSERT INTO
#contactsPropsForReturn
SELECT
contactID,
valueTypeID,
value
FROM
dbo.contactsProps
WHERE
contactID = #resultContactID
FETCH NEXT FROM searchCursor INTO #resultContactID
END --end of WHILE loop
--end of cursor (both CLOSE and DEALLOCATE necessary)
CLOSE searchCursor
DEALLOCATE searchCursor
-- select and return only the props that match with the requested props
-- (we don't want to return all the props, only the ones requested)
INSERT INTO
#contactsPropsForReturnFiltered
SELECT
p.contactID,
p.valueTypeID,
p.value
FROM
#contactsPropsForReturn as p
INNER JOIN
#valueTypeFiltersTVP as f
ON
p.valueTypeID = f.valueTypeID
DECLARE #cols AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX);
SET #cols = STUFF((SELECT distinct ',' + QUOTENAME(ValueTypeId)
FROM #contactsPropsForReturnFiltered
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,1,'');
set #query = 'SELECT contactid, ' + #cols + ' from
(
select contactid
, Value
,ValueTypeId
from #contactsPropsForReturnFiltered
) x
pivot
(
min(Value)
for ValueTypeId in (' + #cols + ')
) p ';
execute(#query);
END
You need to use dynamic pivot in your case. Try the following
create table table1
(
contactid int,
ValueTypeId int,
Value varchar(100)
);
insert into table1 values (56064, 40, 'Issac');
insert into table1 values (56064, 34, '(123)456-7890');
insert into table1 values (56065, 40, 'Lola');
insert into table1 values (56065, 34, '(123)456-7832');
insert into table1 values (56068, 40, 'Mike');
insert into table1 values (56068, 41, 'Gonzalez');
insert into table1 values (56068, 34, '(123)456-7891');
DECLARE #cols AS NVARCHAR(MAX),
#query AS NVARCHAR(MAX);
SET #cols = STUFF((SELECT distinct ',' + QUOTENAME(ValueTypeId)
FROM table1
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
,1,1,'');
set #query = 'SELECT contactid, ' + #cols + ' from
(
select contactid
, Value
,ValueTypeId
from table1
) x
pivot
(
min(Value)
for ValueTypeId in (' + #cols + ')
) p ';
execute(#query);
drop table table1
Why do you need to present the data in this way?
In many cases, clients are better at pivoting than the database engine. For example, SQL Server Reporting Services easily does this with the matrix control. Similarly, if you are coding a web page in, say, Asp.Net, you can run through the recordset quickly to pass your data into a new data representation (meanwhile collecting unique values) and then in a single pass through the new data object spit out the HTML to render the result.
If at all possible, have your client do the pivoting instead of the server.
UPDATE:
If you really want to use table variables in dynamic SQL, you can just fine in SQL Server 2008 and up. Here's an example script:
USE tempdb
GO
CREATE TYPE IDList AS TABLE (
ID int
);
GO
DECLARE #SQL nvarchar(max);
SET #SQL = 'SELECT * FROM #TransactionIDs WHERE ID >= 4;'
DECLARE #TransactionIDs IDLIst;
INSERT #TransactionIDs VALUES (1), (2), (4), (8), (16);
EXEC sp_executesql #SQL, N'#TransactionIDs IDList READONLY', #TransactionIDs;
GO
DROP TYPE IDList;
I have some tables for storing different file information, like thumbs, images, datasheets, ...
I'm writing a stored procedure to retrieve filename of a specific ID. something like:
CREATE PROCEDURE get_file_name(
#id int,
#table nvarchar(50)
)as
if #table='images'
select [filename] from images
where id = #id
if #table='icons'
select [filename] from icons
where id = #id
....
How can I rewrite this procedure using case when statement or should I just use table name as variable?
You can't use case .. when to switch between a table in the FROM clause (like you can in a conditional ORDER BY). i.e. so the following:
select * from
case when 1=1
then t1
else t2
end;
won't work.
So you'll need to use dynamic SQL. It's best to parameterize the query as far as possible, for example the #id value can be parameterized:
-- Validate #table is E ['images', 'icons', ... other valid names here]
DECLARE #sql NVARCHAR(MAX)
SET #sql = 'select [filename] from **TABLE** where id = #id';
SET #sql = REPLACE(#sql, '**TABLE**', #table);
sp_executesql #sql, N'#id INT', #id = #id;
As with all dynamic Sql, note that unparameterized values which are substituted into the query (like #table), make the query vulnerable to Sql Injection attacks. As a result, I would suggest that you ensure that #table comes from a trusted source, or better still, the value of #table is compared to a white list of permissable tables prior to execution of the query.
Just build SQL string in another variable and EXECUTE it
DECLARE #sql AS NCHAR(500)
SET #sql=
'SELECT [filename] '+
' FROM '+#table+
' WHERE id = #id'
EXECUTE(#sql)
CREATE PROCEDURE get_file_name(
#id int,
#table nvarchar(50)
)as
DECLARE #SQL nvarchar(max);
SET #SQL = 'select [filename] from ' + #table + ' where id = ' + #id
EXECUTE (#SQL)