SQL-SERVER Select UPDATE OUTPUT as XML - sql-server

Running on SQL Server 2016.
I have a routine that updates information across servers. I want to hold a list of any changes that I have been required to make. I am trying to output the changed columns as XML for basic storage, and would like to do this directly from the OUTPUT generated by the insert/update/delete if possible.
As an example:
DROP TABLE IF EXISTS Test
CREATE TABLE Test (myKey INT, myValue INT)
INSERT INTO dbo.Test (myKey, myValue)
VALUES (1, 1), (2, 2), (3, 3)
UPDATE dbo.Test
SET myValue = myValue + 10
OUTPUT Deleted.*
, Inserted.*
WHERE myKey < 3
SELECT *
FROM dbo.Test
FOR XML AUTO
DROP TABLE dbo.Test
I know I can set up a TVP to receive the output and then convert to XML from there, but it seems like I'm taking extra steps to do something that should be quite straight forward.
DROP TABLE IF EXISTS Test
CREATE TABLE Test (myKey INT, myValue INT)
INSERT INTO dbo.Test (myKey, myValue)
VALUES (1, 1), (2, 2), (3, 3)
DECLARE #OutputValues AS TABLE(dMyKey INT, dMyValue INT, iMyKey INT, iMyValue INT)
UPDATE dbo.Test
SET myValue = myValue + 10
OUTPUT Deleted.myKey
, Deleted.myValue
, Inserted.myKey
, Inserted.myValue
INTO #OutputValues
WHERE myKey < 3
SELECT *
FROM #OutputValues
FOR XML AUTO
DROP TABLE dbo.Test
While this second piece of code does achieve the sort of output I am looking for, going via a TVP seems to be a bit wasteful.
If I can format the output from the original code directly as XML I feel that would be a better solution. However, I can't see any way of doing so.
Many Thanks.

What are you trying to achieve? Such monitoring / auditing tasks are most likely better solved within a trigger...
The output clause does not allow sub-selects... The only way - not generic and rather ugly - which came into my mind was this (the clue is the implicit cast from nvarchar to xml):
CREATE TABLE Test (myKey INT, myValue INT)
INSERT INTO dbo.Test (myKey, myValue)
VALUES (1, 1), (2, 2), (3, 3)
DECLARE #OutputValues AS TABLE(dMyKey INT, dMyValue INT, iMyKey INT, iMyValue INT,Changed XML)
UPDATE dbo.Test
SET myValue = myValue + 10
OUTPUT Deleted.myKey
, Deleted.myValue
, Inserted.myKey
, Inserted.myValue
, N'<root><deletedKey>' + CAST(deleted.myKey AS NVARCHAR(MAX)) + N'</deletedKey>' +
N'<deletedValue>' + CAST(deleted.myValue AS NVARCHAR(MAX)) + N'</deletedValue>' +
N'<insertedKey>' + CAST(inserted.myValue AS NVARCHAR(MAX)) + N'</insertedKey>' +
N'<insertedValue>' + CAST(inserted.myValue AS NVARCHAR(MAX)) + N'</insertedValue>' +
N'</root>'
INTO #OutputValues
WHERE myKey < 3
SELECT Changed
FROM #OutputValues
DROP TABLE dbo.Test
attention: If your values include forbidden characters (such as <, > or &) this will fail!

Related

How to fix a SQL Server function that is used millions of times and regenerating in the query plan

I have a SQL Server function (see fn_TestBinaryBitwiseOr below) with millions of executions. While this is anticipated, SQL Server has an expensive query report that says:
There have been multiple executions of this query pattern (query
hash), but all of the executions are generating the same query plan
(query plan hash). This wastes CPU time to compile the query plans
and memory to cache them. To improve overall SQL Server performance,
consider parameterizing this query (via the application, template plan
guide, or forced parameterization). Refer to Books Online for more
information on parameterization.
I am pretty sure I am hitting RedFlag #1 from https://eitanblumin.com/2022/08/25/too-many-plans-for-the-same-query-hash/ but still not sure what I can do in my case. I cannot use a stored procedure because it is used in another function in a select and has the output. I haven't gone to forced parameterization due to some of the downsides but may look into that next.
Is it possible to change fn_TestBinaryBitwiseOr below to a parameterized procedure? Is there a different solution to test.
CREATE OR ALTER FUNCTION[dbo].[fn_TestBinaryBitwiseOr]
(#TestFlags1 binary(16),
#TestFlags2 binary(16))
RETURNS binary(16)
AS
BEGIN
RETURN (SELECT TOP(1)
CASE
WHEN #TestFlags1 IS NULL AND #TestFlags2 IS NULL
THEN 0x0
WHEN #TestFlags1 IS NULL
THEN #TestFlags2
WHEN #TestFlags2 IS NULL
THEN #TestFlags1
ELSE
CONVERT(binary(16), CONVERT(binary(4), (SUBSTRING(#TestFlags2, 1, 4) | CONVERT(bigint, SUBSTRING(#TestFlags1, 1, 4)))) +
CONVERT(binary(4), (SUBSTRING(#TestFlags2, 5, 4) | CONVERT(bigint, SUBSTRING(#TestFlags1, 5, 4)))) +
CONVERT(binary(4), (SUBSTRING(#TestFlags2, 9, 4) | CONVERT(bigint, SUBSTRING(#TestFlags1,9,4)))) +
CONVERT(binary(4), (SUBSTRING(#TestFlags2, 13, 4) | CONVERT(bigint, SUBSTRING(#TestFlags1, 13, 4))))
)
END
)
END
GO
-- another function to call my first function,
CREATE OR ALTER FUNCTION[dbo].[DoAThing]()
RETURNS #T TABLE(ColName binary(16))
AS
BEGIN
-- sample data
DECLARE #Table1 TABLE (id int, binary1 binary(16))
DECLARE #Table2 TABLE(table1Id int, binary2 binary(16))
INSERT INTO #Table1
VALUES (1, 0x10000000000000000000000000000A01),
(2, 0x00000000000000000000000000000101)
INSERT INTO #Table2
VALUES (1, 0x00000000000000000000000000000100),
(2, 0x00000000000000000000000000000100)
-- using a function from a function
INSERT INTO #T(ColName)
(SELECT [dbo].[fn_TestBinaryBitwiseOr](t1.binary1, t2.binary2) AS result
FROM #Table1 t1
JOIN #Table2 t2 ON t1.id = t2.table1Id)
RETURN
END
GO
--executing
SELECT * FROM [dbo].[DoAThing]()

Setting the value of a column in SQL Server based on the scope_identity() of the insert

I have a stored procedure in a program that is not performing well. Its truncated version follows. The MyQuotes table has an IDENTITY column called QuoteId.
CREATE PROCEDURE InsertQuote
(#BinderNumber VARCHAR(50) = NULL,
#OtherValue VARCHAR(50))
AS
INSERT INTO MyQuotes (BinderNumber, OtherValue)
VALUES (#BinderNumber, #OtherValue);
DECLARE #QuoteId INT
SELECT #QuoteId = CONVERT(INT, SCOPE_IDENTITY());
IF #BinderNumber IS NULL
UPDATE MyQuotes
SET BinderNumber = 'ABC' + CONVERT(VARCHAR(10),#QuoteId)
WHERE QuoteId = #QuoteId;
SELECT #QuoteId AS QuoteId;
I feel like the section where we derive the binder number from the scope_identity() can be done much, much, cleaner. And I kind of think we should have been doing this in the C# code rather than the SQL, but since that die is cast, I wanted to fish for more learned opinions than my own on how you would change this query to populate that value.
The following update avoids needing the id:
UPDATE MyQuotes SET
BinderNumber = 'ABC' + CONVERT(VARCHAR(10), QuoteId)
WHERE BinderNumber is null;
If selecting QuoteId as a return query is required then using scope_identity() is as good a way as any.
Dale's answer is better, however this can be useful way too:
DECLARE #Output TABLE (ID INT);
INSERT INTO MyQuotes (BinderNumber, OtherValue) VALUES (#BinderNumber, #OtherValue) OUTPUT inserted.ID INTO #Output (ID);
UPDATE q SET q.BinderNumber = 'ABC' + CONVERT(VARCHAR(10),o.ID)
FROM MyQuotes q
INNER JOIN #Output o ON o.ID = q.ID
;
Also, if BinderNumber is always linked to ID, it would be better to just create computed column
AS 'ABC' + CONVERT(VARCHAR(10),ID)

Stored Procedure SELECT UPDATE Incorrect Values

I am creating a stored procedure, which I intend on running via a job every 24 hours. I am able to successfully run the procedure query but for some reason the values dont seem to make sense. See below.
This is my table and what it looks like prior to the running of the procedure, using the following statement:
SELECT HardwareAssetDailyAccumulatedDepreciationValue,
HardwareAssetAccumulatedDepreciationValue FROM HardwareAsset
I then run the following procedure (with the intention of basically copying the value in DailyDepreciationValue to DepreciationValue):
BEGIN
SELECT HardwareAssetID, HardwareAssetDailyAccumulatedDepreciationValue,
HardwareAssetAccumulatedDepreciationValue FROM HardwareAsset
WHERE HardwareAssetDailyAccumulatedDepreciationValue IS NOT NULL
UPDATE HardwareAsset SET HardwareAssetAccumulatedDepreciationValue = CASE WHEN
(HardwareAssetAccumulatedDepreciationValue IS NULL) THEN
CONVERT(DECIMAL(7,2),HardwareAssetDailyAccumulatedDepreciationValue) ELSE
CONVERT(DECIMAL(7,2),(HardwareAssetAccumulatedDepreciationValue + HardwareAssetDailyAccumulatedDepreciationValue))
END
END
But when i re-run the select statement the results are as follows:
It really doesnt make any sense to me at all any ideas?
I am not able to replicate. We need more detail on the table structure and data. This is what I used to attempt to replicate. Feel free to modify as needed:
create table #t (
AccD1 decimal(7,2)
, AccD2 decimal(7,2)
, AccDaily as AccD1 + AccD2
, AccTotal decimal(7,2)
)
insert #t values
(100, 7.87, null)
, (300, 36.99, null)
, (400, 49.32, null)
, (100, 50.00, 100)
select * from #t
update #t set
AccTotal = isnull(AccTotal, 0) + AccDaily
, AccD1 = 0
, AccD2 = 0
select * from #t
drop table #t

Detecting changes in SQL Server 2000 table data

I have a periodic check of a certain query (which by the way includes multiple tables) to add informational messages to the user if something has changed since the last check (once a day).
I tried to make it work with checksum_agg(binary_checksum(*)), but it does not help, so this question doesn't help much, because I have a following case (oversimplified):
select checksum_agg(binary_checksum(*))
from
(
select 1 as id,
1 as status
union all
select 2 as id,
0 as status
) data
and
select checksum_agg(binary_checksum(*))
from
(
select 1 as id,
0 as status
union all
select 2 as id,
1 as status
) data
Both of the above cases result in the same check-sum, 49, and it is clear that the data has been changed.
This doesn't have to be a simple function or a simple solution, but I need some way to uniquely identify the difference like these in SQL server 2000.
checksum_agg appears to simply add the results of binary_checksum together for all rows. Although each row has changed, the sum of the two checksums has not (i.e. 17+32 = 16+33). This is not really the norm for checking for updates, but the recommendations I can come up with are as follows:
Instead of using checksum_agg, concatenate the checksums into a delimited string, and compare strings, along the lines of SELECT binary_checksum(*) + ',' FROM MyTable FOR XML PATH(''). Much longer string to check and to store, but there will be much less chance of a false positive comparison.
Instead of using the built-in checksum routine, use HASHBYTES to calculate MD5 checksums in 8000 byte blocks, and xor the results together. This will give you a much more resilient checksum, although still not bullet-proof (i.e. it is still possible to get false matches, but very much less likely). I'll paste the HASHBYTES demo code that I wrote below.
The last option, and absolute last resort, is to actually store the table table in XML format, and compare that. This is really the only way you can be absolutely certain of no false matches, but is not scalable and involves storing and comparing large amounts of data.
Every approach, including the one you started with, has pros and cons, with varying degrees of data size and processing requirements against accuracy. Depending on what level of accuracy you require, use the appropriate option. The only way to get 100% accuracy is to store all of the table data.
Alternatively, you can add a date_modified field to each table, which is set to GetDate() using after insert and update triggers. You can do SELECT COUNT(*) FROM #test WHERE date_modified > #date_last_checked. This is a more common way of checking for updates. The downside of this one is that deletions cannot be tracked.
Another approach is to create a modified table, with table_name (VARCHAR) and is_modified (BIT) fields, containing one row for each table you wish to track. Using insert, update and delete triggers, the flag against the relevant table is set to True. When you run your schedule, you check and reset the is_modified flag (in the same transaction) - along the lines of SELECT #is_modified = is_modified, is_modified = 0 FROM tblModified
The following script generates three result sets, each corresponding with the numbered list earlier in this response. I have commented which output correspond with which option, just before the SELECT statement. To see how the output was derived, you can work backwards through the code.
-- Create the test table and populate it
CREATE TABLE #Test (
f1 INT,
f2 INT
)
INSERT INTO #Test VALUES(1, 1)
INSERT INTO #Test VALUES(2, 0)
INSERT INTO #Test VALUES(2, 1)
/*******************
OPTION 1
*******************/
SELECT CAST(binary_checksum(*) AS VARCHAR) + ',' FROM #test FOR XML PATH('')
-- Declaration: Input and output MD5 checksums (#in and #out), input string (#input), and counter (#i)
DECLARE #in VARBINARY(16), #out VARBINARY(16), #input VARCHAR(MAX), #i INT
-- Initialize #input string as the XML dump of the table
-- Use this as your comparison string if you choose to not use the MD5 checksum
SET #input = (SELECT * FROM #Test FOR XML RAW)
/*******************
OPTION 3
*******************/
SELECT #input
-- Initialise counter and output MD5.
SET #i = 1
SET #out = 0x00000000000000000000000000000000
WHILE #i <= LEN(#input)
BEGIN
-- calculate MD5 for this batch
SET #in = HASHBYTES('MD5', SUBSTRING(#input, #i, CASE WHEN LEN(#input) - #i > 8000 THEN 8000 ELSE LEN(#input) - #i END))
-- xor the results with the output
SET #out = CAST(CAST(SUBSTRING(#in, 1, 4) AS INT) ^ CAST(SUBSTRING(#out, 1, 4) AS INT) AS VARBINARY(4)) +
CAST(CAST(SUBSTRING(#in, 5, 4) AS INT) ^ CAST(SUBSTRING(#out, 5, 4) AS INT) AS VARBINARY(4)) +
CAST(CAST(SUBSTRING(#in, 9, 4) AS INT) ^ CAST(SUBSTRING(#out, 9, 4) AS INT) AS VARBINARY(4)) +
CAST(CAST(SUBSTRING(#in, 13, 4) AS INT) ^ CAST(SUBSTRING(#out, 13, 4) AS INT) AS VARBINARY(4))
SET #i = #i + 8000
END
/*******************
OPTION 2
*******************/
SELECT #out

XQuery adding or replacing attribute in single SQL update command

I have a Table with an XML column,
I want to update the xml to insert attribute or to change the attribute value if the attribute already exists.
Let's say the starting xml is: < d />
Inserting:
UPDATE Table
set XmlCol.modify('insert attribute att {"1"} into /d[1]')
Changing:
UPDATE Table
set XmlCol.modify('replace value of /d[1]/#att with "1"')
insert will fail if the attribute already exists, replace will fail if the attribute doesn't exists.
I have tried to use 'if' but I don't think it can work, there error I get: "XQuery [modify()]: Syntax error near 'attribute', expected 'else'."
IF attempt
UPDATE Table
set XmlCol.modify('if empty(/d[1]/#att)
then insert attribute att {"1"} into /d[1]
else replace value of /d[1]/#att with "1"')
Currently I select the xml into a variable and then modify it using T-SQL and then updating the column with new xml, this requires me to lock the row in a transaction and is probably more expensive for the DB.
From what I can tell, you can't do this with single statement. You can use the exist() method to accomplish that with two update statements.
DECLARE #TestTable TABLE
(
Id int,
XmlCol xml
);
INSERT INTO #TestTable (Id, XmlCol)
VALUES
(1, '<d att="1" />'),
(2, '<d />'),
(3, '<d att="3" />');
SELECT * FROM #TestTable;
UPDATE #TestTable
SET XmlCol.modify('replace value of /d[1]/#att with "1"')
WHERE XmlCol.exist('(/d[1])[not(empty(#att))]') = 1;
UPDATE #TestTable
SET XmlCol.modify('insert attribute att {"1"} into /d[1]')
WHERE XmlCol.exist('(/d[1])[empty(#att)]') = 1;
SELECT * FROM #TestTable;
The output from the final select is:
Id XmlCol
----------- -------------------
1 <d att="1" />
2 <d att="1" />
3 <d att="1" />
There is a slightly better way than Tommys:
DECLARE #TestTable TABLE
(
Id int,
XmlCol xml
);
INSERT INTO #TestTable (Id, XmlCol)
VALUES
(1, '<UserSettings> </UserSettings>'),
(2, '<UserSettings><timeout>3</timeout> </UserSettings>'),
(3, '<UserSettings> </UserSettings>');
UPDATE #TestTable
SET XmlCol.modify('replace value of (/UserSettings/timeout/text())[1] with "1"')
WHERE Id = 3 and XmlCol.exist('/UserSettings/timeout') = 1;
IF ##ROWCOUNT=0
UPDATE #TestTable
SET XmlCol.modify('insert <timeout>5</timeout> into (/UserSettings)[1] ')
WHERE Id = 3;
SELECT * FROM #TestTable;
The solution is a combination of Tommys and simple SQL and required only 1 SQL UPDATE if the column exist. Tommys always recuire two updates.

Resources