I am working with merge into statement
I have a table that looks like this (first I insert like this) using below query:
5
2
5
3
5
5
5
6
table type :
CREATE TYPE [dbo].[userid] AS TABLE(
[userid] [bigint] NULL
)
GO
Now I want the below output :
5
2
5
3
5
6
I write the below query like this:
--use test
declare #sid varchar(100) = '5'
declare #uid as userid
insert into #uid(userid) values(2)
insert into #uid(userid) values(3)
--insert into #uid(userid) values(5) // I remove this line
insert into #uid(userid) values(6)
MERGE INTO dbo.test_master AS dest
USING #uid AS sou ON
dest.sid = #sid
AND
sou.userid = dest.testid
WHEN MATCHED THEN UPDATE SET
dest.testid = sou.userid
WHEN NOT MATCHED THEN
INSERT( sid, testid )
VALUES( #sid, sou.userid )
--WHEN NOT MATCHED BY SOURCE
-- THEN
-- DELETE
;
I am trying to achieve this output
5
2
5
3
5
6
I am using delete keyword, see my SQL query, but it is deleting the all records from the table. I try, but can't work it out.
You need to pre-filter the destination table, otherwise all rows, even ones that have a different sid will be deleted. You can pre-filter with a CTE or a view.
I note that the WHEN MATCHED clause makes no sense, as the only column being updated is the join column, which obviously matches anyway.
declare #sid varchar(100) = '5';
declare #uid as userid;
insert into #uid (userid) values
(2),
(3),
-- (5), -- I remove this line
(6);
WITH dest AS (
SELECT *
FROM dbo.test_master dest
WHERE dest.sid = #sid
)
MERGE INTO dest
USING #uid AS sou ON
sou.userid = dest.testid
-- WHEN MATCHED THEN UPDATE SET
-- dest.testid = sou.userid -- doesn't make sense
WHEN NOT MATCHED THEN
INSERT( sid, testid )
VALUES( #sid, sou.userid )
WHEN NOT MATCHED BY SOURCE THEN
DELETE
;
db<>fiddle
Related
I know that the SCOPE_IDENTITY() will get the last inserted row from insert statement. However, for the following case, I am not too sure is SCOPE_IDENTITY() is safe. As SELECT MAX(ID) FROM TableA will have go through scan the table to get the max id and it will have performance issue, even slightly, I believe.
Here is the case:
DECLARE #DaysInMonth INT
DECLARE #FirstID INT
DECLARE #SecondID INT
DECLARE #ThirdID INT
DECLARE #FourthID INT
SET #DaysInMonth = DAY(EOMONTH('2016-09-01'))
BEGIN TRY
BEGIN TRANSACTION
WHILE #DaysInMonth > 0
BEGIN
-- First Insert -- Begin
INSERT INTO tableA ('first insert - ' + #DaysInMonth)
-- First Insert -- End
SET #FirstID = SCOPE_IDENTITY()
-- Second Insert -- Begin
INSERT INTO tableB ('second insert - ' + #DaysInMonth)
-- Second Insert -- End
SET #SecondID = SCOPE_IDENTITY()
-- Third Insert -- Begin
INSERT INTO tableC ('third insert - ' + #DaysInMonth)
-- Third Insert -- End
SET #ThirdID = SCOPE_IDENTITY()
-- Fourth Insert -- Begin
INSERT INTO tableD ('fourth insert - ' + #DaysInMonth)
-- Fourth Insert -- End
SET #FourthID = SCOPE_IDENTITY()
SET #DaysInMonth = #DaysInMonth - 1
END
COMMIT TRANSACTION
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
ROLLBACK TRANSACTION
THROW
END CATCH
As from the case above, I have to insert the records every loop for fourth times for how many days in the month that I have declared.
From what I know, there are 4 to get the last inserted ID:
SCOPE_IDENTITY
##IDENTITY
SELECT MAX(ID) FROM tableA
IDENT_CURRENT
From the following post:
Post
Is mentioned that SCOPE_IDENTITY() is what generally that you want to use.
What I mean with 'Safe' is, do the ID will be unique during the loop?
Thank you.
You can use OUTPUT column in the last insert statement, Ofcourse this is another option where you will get what exactly input statement executed.. Below is just an example
CREATE TABLE #tablea (
id int IDENTITY (1, 1),
val char(10)
)
DECLARE #outputtbl TABLE (
id int,
val char(10)
)
INSERT INTO #tablea (val)
OUTPUT INSERTED.* INTO #outputtbl
VALUES ('test')
SELECT id
FROM #outputtbl
I've read several posts about the TSQL Identity Bug and have been playing around with using SEQUENCE. However, I'm curious about resetting the SEQUENCE on the ID value in a table. For an example:
CREATE SEQUENCE Inc
AS INT
START WITH 1
INCREMENT BY 1
CYCLE
CACHE
-- Quick ability to redo everything if needed:
-- DROP SEQUENCE Inc
-- Our table grabs the next sequence for our ID field:
CREATE TABLE SequenceID(
NewIDField INT DEFAULT NEXT VALUE FOR Inc,
Name VARCHAR(100)
)
INSERT INTO SequenceID (Name)
VALUES ('John')
, ('Tiffany')
, ('Bob')
, ('Jessica')
SELECT *
FROM SequenceID
-- We remove Bob:
DELETE FROM SequenceID
WHERE NewIDField = 3
-- ID value 3 is gone; it moves from 1 to 2 to 4
SELECT *
FROM SequenceID
INSERT INTO SequenceID (Name)
VALUES ('David')
, ('Rosa')
, ('Samuel')
-- ID 3 doesn't exist because the SEQUENCE grabs the next value from 4
SELECT *
FROM SequenceID
-- Let's just reset our ID
;WITH ResetIt AS(
SELECT ROW_NUMBER() OVER (ORDER BY NewIDField) AS ID
, NewIDField AS ExistingID
, Name
FROM SequenceID
)
UPDATE SequenceID
SET NewIDField = ResetIt.ID
FROM ResetIt
WHERE SequenceID.NewIDField = ResetIt.ExistingID
-- Yay!
SELECT *
FROM SequenceID
INSERT INTO SequenceID (Name)
VALUES ('Sarah')
-- Oh Sarah, tsk tsk.
SELECT *
FROM SequenceID
DROP TABLE SequenceID
Is there a way to automatically perform this with SEQUENCE where we can determine the last value and begin there (similar to a RESEED), as even with IDENTITY, if we remove a value, we still must RESEED, see:
CREATE TABLE IDID(
ID INT IDENTITY(1,1),
I INT
)
INSERT INTO IDID (I)
VALUES (1),(2),(3),(4)
SELECT *
FROM IDID
DELETE FROM IDID
WHERE ID = 3
INSERT INTO IDID (I)
VALUES (5),(6),(7)
SELECT *
FROM IDID
DROP TABLE IDID
After you perform your update, you'll have to run some dynamic SQL, as ALTER SEQUENCE only accepts a constant for the RESTART WITH clause:
DECLARE #resetSQL nvarchar(255) = 'ALTER SEQUENCE Inc RESTART WITH ' + (SELECT CAST(MAX(NewIDField)+1 as nvarchar(10)) FROM SequenceID);
exec sp_executesql #resetSQL;
I need to write an insert query to insert some rows into a table using data from different tables. I have:
A variable #ID which contains ID of the new inserted row in a table. (1)
A table (2) contains some IDs
A table (3) define the relation between the two above tables.
Now I need to insert for each ID in (2) a new row in the table (3).
So if #ID=2 and IDs = 1, 2, 3, 4, 5, 6, I want to insert the following rows in Table (3):
table1_ID table2_ID
--------- ---------
1 1
1 2
1 3
1 4
1 5
Assuming that the value 6 was not meant to be discarded (if it was, please explain the logic). Also assuming that you just want every row in table 2 inserted into table 2.
INSERT dbo.Table3(table1_ID, table2_ID)
SELECT #ID, ID
FROM dbo.Table2
-- WHERE ID <> 6???
;
If the list of IDs is really a CSV, then:
First create a Split function (several alternatives described here, except they output strings instead of integers), e.g.
CREATE FUNCTION dbo.SplitInts
(
#List VARCHAR(MAX),
#Delimiter VARCHAR(255)
)
RETURNS #t TABLE(Item INT)
AS
BEGIN
INSERT #t(Item) SELECT CONVERT(INT, SUBSTRING(#List, Number,
CHARINDEX(#Delimiter, #List + #Delimiter, Number) - Number))
FROM (SELECT ROW_NUMBER() OVER (ORDER BY [object_id])
FROM sys.all_objects) AS n(Number)
WHERE Number <= CONVERT(INT, LEN(#List))
AND SUBSTRING(#Delimiter + #List, Number, 1) = #Delimiter;
RETURN;
END
GO
Now your stored procedure can simply say:
CREATE PROCEDURE dbo.whatever
#ID INT,
#IDs VARCHAR(MAX)
AS
BEGIN
SET NOCOUNT ON;
INSERT dbo.Table3(table1_ID, table2_ID)
SELECT #ID, Item
FROM dbo.SplitInts(#IDs, ',');
END
GO
However if you are on SQL Server 2008 or above, and this list of comma-separated IDs is coming from your application (e.g. from a DataTable or other set), you could use a Table-Valued Parameter and avoid the need for splitting.
CREATE TYPE dbo.SetOfIntegers
( Number INT );
GO
CREATE PROCEDURE dbo.whatever
#ID INT,
#IDs dbo.SetOfIntegers READONLY
AS
BEGIN
SET NOCOUNT ON;
INSERT dbo.Table3(table1_ID, table2_ID)
SELECT #ID, Item
FROM #IDs;
END
GO
Then you have to change the C# code to pass your DataTable in as SqlDbType.Structured instead of passing your CSV in as a string. More info here:
http://www.sqlperformance.com/2012/08/t-sql-queries/splitting-strings-now-with-less-t-sql
I am a little stuck with why I can not seem to get the 'new identity' of the inserted row with the statement below. SCOPE_IDENTITY() just returns null.
declare #WorkRequestQueueID int
declare #LastException nvarchar(MAX)
set #WorkRequestQueueID = 1
set #LastException = 'test'
set nocount off
DELETE dbo.WorkRequestQueue
OUTPUT
DELETED.MessageEnvelope,
DELETED.Attempts,
#LastException,
GetUtcdate(), -- WorkItemPoisened datetime
DELETED.WorkItemReceived_UTC
INTO dbo.FaildMessages
FROM dbo.WorkRequestQueue
WHERE
WorkRequestQueue.ID = #WorkRequestQueueID
IF ##ROWCOUNT = 0
RAISERROR ('Record not found', 16, 1)
SELECT Cast(SCOPE_IDENTITY() as int)
Any assistance would be most appreciated.
For now I use a workaround this like so.
declare #WorkRequestQueueID int
declare #LastException nvarchar(MAX)
set #WorkRequestQueueID = 7
set #LastException = 'test'
set nocount on
set xact_abort on
DECLARE #Failed TABLE
(
MessageEnvelope xml,
Attempts smallint,
LastException nvarchar(max),
WorkItemPoisened_UTC datetime,
WorkItemReceived_UTC datetime
)
BEGIN TRAN
DELETE dbo.WorkRequestQueue
OUTPUT
DELETED.MessageEnvelope,
DELETED.Attempts,
#LastException,
GetUtcdate(), -- WorkItemPoisened datetime
DELETED.WorkItemReceived_UTC
INTO
#Failed
FROM
dbo.WorkRequestQueue
WHERE
WorkRequestQueue.ID = #WorkRequestQueueID
IF ##ROWCOUNT = 0 BEGIN
RAISERROR ('Record not found', 16, 1)
Rollback
END ELSE BEGIN
insert into dbo.FaildMessages select * from #Failed
COMMIT TRAN
SELECT Cast(SCOPE_IDENTITY() as int)
END
EDITED FEB'2013
#MartinSmith alerts us that this bug don't want be fixed by Microsoft.
"Posted by Microsoft on 2/27/2013 at 2:18 PM Hello Martin, We
investigated the issue and found that changing the behavior is not an
easy thing to do. It would basically require redefining some of the
behavior when both INSERT & OUTPUT INTO target has identity columns.
Given the nature of the problem & the uncommon scenario, we have
decided not to fix the issue. -- Umachandar, SQL Programmability
Team"
EDITED OCT'2012
This is caused by a bug:
Testing bug:
Quoting OUTPUT Clause doc:
##IDENTITY, SCOPE_IDENTITY, and IDENT_CURRENT return identity values
generated only by the nested DML statement, and not those generated by
the outer INSERT statement.
After test it It seems that scope_identity() only works if outer operation is an insert in a table with identity columns:
Test 1: Delete
create table #t ( a char(1) );
create table #d ( a char(1), i int identity );
insert into #t
values ('a'),('b'),('c');
delete #t
output deleted.a into #d;
select SCOPE_IDENTITY(), * from #d;
a i
---- - -
null a 1
null b 2
null c 3
Test 2: Inserting in outer table with identity
create table #t ( a char(1), i int identity );
create table #d ( a char(1), i int identity );
insert into #t
values ('x'),('x'),('x');
insert into #t
output inserted.a into #d
values ('a'),('b');
select scope_identity(), * from #d;
a i
- - -
2 a 1
2 b 2
Test 3: Inserting in outer table without identity
create table #t ( a char(1) );
create table #d ( a char(1), i int identity );
insert into #t
values ('x'),('x'),('x');
insert into #t
output inserted.a into #d
values ('a'),('b');
select scope_identity(), * from #d;
a i
---- - -
null a 1
null b 2
You might try to use a table variable for your output clause, thus allowing you to explicitly insert into FaildMessages:
declare #WorkRequestQueueID int
declare #LastException nvarchar(MAX)
set #WorkRequestQueueID = 1
set #LastException = 'test'
set nocount off
-- Declare a table variable to capture output
DECLARE #output TABLE (
MessageEnvelope VARCHAR(50), -- Guessing at datatypes
Attempts INT, -- Guessing at datatypes
WorkItemReceived_UTC DATETIME -- Guessing at datatypes
)
-- Run the deletion with output
DELETE dbo.WorkRequestQueue
OUTPUT
DELETED.MessageEnvelope,
DELETED.Attempts,
DELETED.WorkItemReceived_UTC
-- Use the table var
INTO #output
FROM dbo.WorkRequestQueue
WHERE
WorkRequestQueue.ID = #WorkRequestQueueID
-- Explicitly insert
INSERT
INTO dbo.FaildMessages
SELECT
MessageEnvelope,
Attempts,
#LastException,
GetUtcdate(), -- WorkItemPoisened datetime
WorkItemReceived_UTC
FROM #output
IF ##ROWCOUNT = 0
RAISERROR ('Record not found', 16, 1)
SELECT Cast(SCOPE_IDENTITY() as int)
Very simplified, I have two tables Source and Target.
declare #Source table (SourceID int identity(1,2), SourceName varchar(50))
declare #Target table (TargetID int identity(2,2), TargetName varchar(50))
insert into #Source values ('Row 1'), ('Row 2')
I would like to move all rows from #Source to #Target and know the TargetID for each SourceID because there are also the tables SourceChild and TargetChild that needs to be copied as well and I need to add the new TargetID into TargetChild.TargetID FK column.
There are a couple of solutions to this.
Use a while loop or cursors to insert one row (RBAR) to Target at a time and use scope_identity() to fill the FK of TargetChild.
Add a temp column to #Target and insert SourceID. You can then join that column to fetch the TargetID for the FK in TargetChild.
SET IDENTITY_INSERT OFF for #Target and handle assigning new values yourself. You get a range that you then use in TargetChild.TargetID.
I'm not all that fond of any of them. The one I used so far is cursors.
What I would really like to do is to use the output clause of the insert statement.
insert into #Target(TargetName)
output inserted.TargetID, S.SourceID
select SourceName
from #Source as S
But it is not possible
The multi-part identifier "S.SourceID" could not be bound.
But it is possible with a merge.
merge #Target as T
using #Source as S
on 0=1
when not matched then
insert (TargetName) values (SourceName)
output inserted.TargetID, S.SourceID;
Result
TargetID SourceID
----------- -----------
2 1
4 3
I want to know if you have used this? If you have any thoughts about the solution or see any problems with it? It works fine in simple scenarios but perhaps something ugly could happen when the query plan get really complicated due to a complicated source query. Worst scenario would be that the TargetID/SourceID pairs actually isn't a match.
MSDN has this to say about the from_table_name of the output clause.
Is a column prefix that specifies a table included in the FROM clause of a DELETE, UPDATE, or MERGE statement that is used to specify the rows to update or delete.
For some reason they don't say "rows to insert, update or delete" only "rows to update or delete".
Any thoughts are welcome and totally different solutions to the original problem is much appreciated.
In my opinion this is a great use of MERGE and output. I've used in several scenarios and haven't experienced any oddities to date.
For example, here is test setup that clones a Folder and all Files (identity) within it into a newly created Folder (guid).
DECLARE #FolderIndex TABLE (FolderId UNIQUEIDENTIFIER PRIMARY KEY, FolderName varchar(25));
INSERT INTO #FolderIndex
(FolderId, FolderName)
VALUES(newid(), 'OriginalFolder');
DECLARE #FileIndex TABLE (FileId int identity(1,1) PRIMARY KEY, FileName varchar(10));
INSERT INTO #FileIndex
(FileName)
VALUES('test.txt');
DECLARE #FileFolder TABLE (FolderId UNIQUEIDENTIFIER, FileId int, PRIMARY KEY(FolderId, FileId));
INSERT INTO #FileFolder
(FolderId, FileId)
SELECT FolderId,
FileId
FROM #FolderIndex
CROSS JOIN #FileIndex; -- just to illustrate
DECLARE #sFolder TABLE (FromFolderId UNIQUEIDENTIFIER, ToFolderId UNIQUEIDENTIFIER);
DECLARE #sFile TABLE (FromFileId int, ToFileId int);
-- copy Folder Structure
MERGE #FolderIndex fi
USING ( SELECT 1 [Dummy],
FolderId,
FolderName
FROM #FolderIndex [fi]
WHERE FolderName = 'OriginalFolder'
) d ON d.Dummy = 0
WHEN NOT MATCHED
THEN INSERT
(FolderId, FolderName)
VALUES (newid(), 'copy_'+FolderName)
OUTPUT d.FolderId,
INSERTED.FolderId
INTO #sFolder (FromFolderId, toFolderId);
-- copy File structure
MERGE #FileIndex fi
USING ( SELECT 1 [Dummy],
fi.FileId,
fi.[FileName]
FROM #FileIndex fi
INNER
JOIN #FileFolder fm ON
fi.FileId = fm.FileId
INNER
JOIN #FolderIndex fo ON
fm.FolderId = fo.FolderId
WHERE fo.FolderName = 'OriginalFolder'
) d ON d.Dummy = 0
WHEN NOT MATCHED
THEN INSERT ([FileName])
VALUES ([FileName])
OUTPUT d.FileId,
INSERTED.FileId
INTO #sFile (FromFileId, toFileId);
-- link new files to Folders
INSERT INTO #FileFolder (FileId, FolderId)
SELECT sfi.toFileId, sfo.toFolderId
FROM #FileFolder fm
INNER
JOIN #sFile sfi ON
fm.FileId = sfi.FromFileId
INNER
JOIN #sFolder sfo ON
fm.FolderId = sfo.FromFolderId
-- return
SELECT *
FROM #FileIndex fi
JOIN #FileFolder ff ON
fi.FileId = ff.FileId
JOIN #FolderIndex fo ON
ff.FolderId = fo.FolderId
I would like to add another example to add to #Nathan's example, as I found it somewhat confusing.
Mine uses real tables for the most part, and not temp tables.
I also got my inspiration from here: another example
-- Copy the FormSectionInstance
DECLARE #FormSectionInstanceTable TABLE(OldFormSectionInstanceId INT, NewFormSectionInstanceId INT)
;MERGE INTO [dbo].[FormSectionInstance]
USING
(
SELECT
fsi.FormSectionInstanceId [OldFormSectionInstanceId]
, #NewFormHeaderId [NewFormHeaderId]
, fsi.FormSectionId
, fsi.IsClone
, #UserId [NewCreatedByUserId]
, GETDATE() NewCreatedDate
, #UserId [NewUpdatedByUserId]
, GETDATE() NewUpdatedDate
FROM [dbo].[FormSectionInstance] fsi
WHERE fsi.[FormHeaderId] = #FormHeaderId
) tblSource ON 1=0 -- use always false condition
WHEN NOT MATCHED
THEN INSERT
( [FormHeaderId], FormSectionId, IsClone, CreatedByUserId, CreatedDate, UpdatedByUserId, UpdatedDate)
VALUES( [NewFormHeaderId], FormSectionId, IsClone, NewCreatedByUserId, NewCreatedDate, NewUpdatedByUserId, NewUpdatedDate)
OUTPUT tblSource.[OldFormSectionInstanceId], INSERTED.FormSectionInstanceId
INTO #FormSectionInstanceTable(OldFormSectionInstanceId, NewFormSectionInstanceId);
-- Copy the FormDetail
INSERT INTO [dbo].[FormDetail]
(FormHeaderId, FormFieldId, FormSectionInstanceId, IsOther, Value, CreatedByUserId, CreatedDate, UpdatedByUserId, UpdatedDate)
SELECT
#NewFormHeaderId, FormFieldId, fsit.NewFormSectionInstanceId, IsOther, Value, #UserId, CreatedDate, #UserId, UpdatedDate
FROM [dbo].[FormDetail] fd
INNER JOIN #FormSectionInstanceTable fsit ON fsit.OldFormSectionInstanceId = fd.FormSectionInstanceId
WHERE [FormHeaderId] = #FormHeaderId
Here's a solution that doesn't use MERGE (which I've had problems with many times I try to avoid if possible). It relies on two memory tables (you could use temp tables if you want) with IDENTITY columns that get matched, and importantly, using ORDER BY when doing the INSERT, and WHERE conditions that match between the two INSERTs... the first one holds the source IDs and the second one holds the target IDs.
-- Setup... We have a table that we need to know the old IDs and new IDs after copying.
-- We want to copy all of DocID=1
DECLARE #newDocID int = 99;
DECLARE #tbl table (RuleID int PRIMARY KEY NOT NULL IDENTITY(1, 1), DocID int, Val varchar(100));
INSERT INTO #tbl (DocID, Val) VALUES (1, 'RuleA-2'), (1, 'RuleA-1'), (2, 'RuleB-1'), (2, 'RuleB-2'), (3, 'RuleC-1'), (1, 'RuleA-3')
-- Create a break in IDENTITY values.. just to simulate more realistic data
INSERT INTO #tbl (Val) VALUES ('DeleteMe'), ('DeleteMe');
DELETE FROM #tbl WHERE Val = 'DeleteMe';
INSERT INTO #tbl (DocID, Val) VALUES (6, 'RuleE'), (7, 'RuleF');
SELECT * FROM #tbl t;
-- Declare TWO temp tables each with an IDENTITY - one will hold the RuleID of the items we are copying, other will hold the RuleID that we create
DECLARE #input table (RID int IDENTITY(1, 1), SourceRuleID int NOT NULL, Val varchar(100));
DECLARE #output table (RID int IDENTITY(1,1), TargetRuleID int NOT NULL, Val varchar(100));
-- Capture the IDs of the rows we will be copying by inserting them into the #input table
-- Important - we must specify the sort order - best thing is to use the IDENTITY of the source table (t.RuleID) that we are copying
INSERT INTO #input (SourceRuleID, Val) SELECT t.RuleID, t.Val FROM #tbl t WHERE t.DocID = 1 ORDER BY t.RuleID;
-- Copy the rows, and use the OUTPUT clause to capture the IDs of the inserted rows.
-- Important - we must use the same WHERE and ORDER BY clauses as above
INSERT INTO #tbl (DocID, Val)
OUTPUT Inserted.RuleID, Inserted.Val INTO #output(TargetRuleID, Val)
SELECT #newDocID, t.Val FROM #tbl t
WHERE t.DocID = 1
ORDER BY t.RuleID;
-- Now #input and #output should have the same # of rows, and the order of both inserts was the same, so the IDENTITY columns (RID) can be matched
-- Use this as the map from old-to-new when you are copying sub-table rows
-- Technically, #input and #output don't even need the 'Val' columns, just RID and RuleID - they were included here to prove that the rules matched
SELECT i.*, o.* FROM #output o
INNER JOIN #input i ON i.RID = o.RID
-- Confirm the matching worked
SELECT * FROM #tbl t