I having the scenario of loading the data from source table to target table. If the data from source is not present in target, then i need to insert. If it is present in the target table already, then update the status of the row to 'expire' and insert the column as new row. I used Merge query to do this. I can do insert if not exists and i can do update also. But while trying to insert when matched, it says insert not allowed in 'when matched' clause.
Please help me.. Thanks in advance
If you want to perform multiple actions for a single row of source data, you need to duplicate that row somehow.
Something like the following (making up table names, etc):
;WITH Source as (
SELECT Col1,Col2,Col3,t.Dupl
FROM SourceTable,(select 0 union all select 1) t(Dupl)
)
MERGE INTO Target t
USING Source s ON t.Col1 = s.Col1 and s.Dupl=0 /* Key columns here */
WHEN MATCHED THEN UPDATE SET Expired = 1
WHEN NOT MATCHED AND s.Dupl=1 THEN INSERT (Col1,Col2,Col3) VALUES (s.Col1,s.Col2,s.Col3);
You always want the s.Dupl condition in the not matched branch, because otherwise source rows which don't match any target rows would be inserted twice.
From the example you posted as a comment, I'd change:
MERGE target AS tar USING source AS src ON src.id = tar.id
WHEN MATCHED THEN UPDATE SET D_VALID_TO=#nowdate-1, C_IS_ACTIVE='N', D_LAST_UPDATED_DATE=#nowdate
WHEN NOT MATCHED THEN INSERT (col1,col2,col3) VALUES (tar.col1,tar.col2,tar.col3);
into:
;WITH SourceDupl AS (
SELECT id,col1,col2,col3,t.Dupl
FROM source,(select 0 union all select 1) t(Dupl)
)
MERGE target AS tar USING SourceDupl as src on src.id = tar.id AND Dupl=0
WHEN MATCHED THEN UPDATE SET D_VALID_TO=#nowdate-1, C_IS_ACTIVE='N', D_LAST_UPDATED_DATE=#nowdate
WHEN NOT MATCHED AND Dupl=1 THEN INSERT (col1,col2,col3) VALUES (src.col1,src.col2,src.col3);
I've changed the values in the VALUES clause, since in a NOT MATCHED branch, the tar table doesn't have a row to select values from.
Check out one of those many links:
Using SQL Server 2008's MERGE Statement
MERGE on Technet
Introduction to MERGE statement
SQL Server 2008 MERGE
Without actually knowing what your database tables look like, we cannot be of more help - you need to read those articles and figure out yourself how to apply this to your concrete situation.
Related
I have data flowing into one table from multiple other tables lets say: Table_A
Then I have a Merge stored proc that takes data from table A merges it with Table B.
However, something doesn't seem to be right. If i truncate and load the data it works fine, but if i dont truncate and load, and just fetch the query by eachh hour I get the error message saying
Msg 8672, Level 16, State 1, Procedure Merge_Table_A, Line 4 [Batch Start Line 0]
The MERGE statement attempted to UPDATE or DELETE the same row more than once. This happens when a target row matches more than one source row. A MERGE statement cannot UPDATE/DELETE the same row of the target table multiple times. Refine the ON clause to ensure a target row matches at most one source row, or use the GROUP BY clause to group the source rows.
How can I overcome this?
I want to be able to incrementally load the data and not do truncate loads, but at the same time have a stored proc that updates or inserts or doesnt care if the row already exists.
Seems you have duplicate rows in your target table which are loaded from your previous runs.
Note: Matching in a Merge does not consider the rows inserted (even duplicate) while running the Merge itself.
Below is my repro example with a sample data:
Table1: Initial data
Table2: Taget table
Merge Statement:
MERGE tb2 AS Target
USING tb1 AS Source
ON Source.firstname = Target.firstname and
Source.lastname = Target.lastname
-- For Inserts
WHEN NOT MATCHED BY Target THEN
INSERT (firstname, lastname, updated_date)
VALUES (Source.firstname, Source.lastname, source.updated_date)
-- For Updates
WHEN MATCHED THEN UPDATE SET
Target.updated_date = Source.updated_date
-- For Deletes
WHEN NOT MATCHED BY Source THEN
DELETE;
When Merge is executed, it inserts all data without any errors.
New data in tb1:
When I run the Merge statement, it gives me the same error as yours.
As a workaround using one of the below options,
Add additional conditions if possible in the ON clause to uniquely identify the data.
Remove the duplicates from the source and merge the data into tb2 as below.
--temp table
drop table if exists #tb1;
select * into #tb1 from (
select *, row_number() over(partition by firstname, lastname order by firstname, lastname, updated_date desc) as rn from tb1) a
where rn = 1
MERGE tb2 AS Target
USING #tb1 AS Source
ON Source.firstname = Target.firstname and
Source.lastname = Target.lastname
-- For Inserts
WHEN NOT MATCHED BY Target THEN
INSERT (firstname, lastname, updated_date)
VALUES (Source.firstname, Source.lastname, source.updated_date)
-- For Updates
WHEN MATCHED THEN UPDATE SET
Target.updated_date = Source.updated_date
-- For Deletes
WHEN NOT MATCHED BY Source THEN
DELETE;
Data merged into tb2 successfully.
How to insert multiple rows in a merge in SQL?
I'm using a MERGE INSERT and I'm wondering is it possible to add two rows at the same time? Under is the query I have written, but as you can see, I want to insert both boolean for IsNew, also when it is not matched, I want to add a row for IsNew = 1 and one IsNew = 0.
How can I achieve this?
MERGE ITEMS AS TARGET
USING #table AS SOURCE
ON T.[ID]=S.ID
WHEN MATCHED THEN
UPDATE SET
T.[Content] = S.[Content],
WHEN NOT MATCHED THEN
INSERT (ID, Content, TIME, IsNew)
VALUES (ID, TEXT, GETDATE(), 1),
You can't do this directly with a merge statement, but there is a simple solution.
The merge statement <merge_not_matched> clause (which is the insert...values|default values) clause can only insert one row on the target table for each row in the source table.
This means that for you to enter two rows for each match, you simply need to change the source table - in this case, it's as simple as a cross join query.
However the <merge_matched> clause requires that only a single row from the source can match any single row from the target - or you will get the following error:
The MERGE statement attempted to UPDATE or DELETE the same row more than once. This happens when a target row matches more than one source row. A MERGE statement cannot UPDATE/DELETE the same row of the target table multiple times. Refine the ON clause to ensure a target row matches at most one source row, or use the GROUP BY clause to group the source rows.
To solve this problem you will have to add a condition to the when match to make sure only one row from the source table updates the target table:
MERGE Items AS T
USING (
SELECT Id, Text, GetDate() As Date, IsNew
FROM #table
-- adding one row for each row in source
CROSS JOIN (SELECT 0 As IsNew UNION SELECT 1) AS isNewMultiplier
) AS S
ON T.[ID]=S.ID
WHEN MATCHED AND S.IsNew = 1 THEN -- Note the added condition here
UPDATE SET
T.[Content] = S.[Text]
WHEN NOT MATCHED THEN
INSERT (Id, Content, Time, IsNew) VALUES
(Id, Text, Date, IsNew);
You can see a live demo on rextester.
With all that being said, I would like to refer you to another stackoverflow post that offers a better alternative then using the merge statement.
The author of the answer is a Microsoft MVP and an SQL Server expert DBA, you should at least read what he has to say.
It seems you can't achieve this using a merge statement. It may be better for you to split the two into separate queries for update and insert.
For example:
UPDATE ITEMS SET ITEMS.ID = #table.ID FROM ITEMS INNER JOIN #table ON ITEMS.ID = #table.ID
INSERT INTO ITEMS (ID, Content, TIME, IsNew) SELECT (ID, TEXT, GETDATE(), 1) FROM #table
INSERT INTO ITEMS (ID, Content, TIME, IsNew) SELECT (ID, TEXT, GETDATE(), 0) FROM #table
This will insert both rows as desired, mimicking your merge statement. However, your update statement won't do much - if you're matching based on ID, then it's impossible for you to have any IDs to update. If you wanted to update other fields, then you could change it like this:
UPDATE ITEMS SET ITEMS.Content = #table.TEXT FROM ITEMS INNER JOIN #table ON ITEMS.ID = #table.ID
I have a large data feed each morning that is dumped into a SQL Server table with a truncate query.
I need to update parts of my master table with any changes from this dump
and append any new rows (basically the rest) to my Master table also from this dump.
To reduce the run time this truncate query is about to be modified to cover only the last 2 years.
How should I go about do this
Both tables have identical structure so hopefully this should be fairly straight forward.
The apt way of doing should be a merge statement.
See here for more
https://learn.microsoft.com/en-us/sql/t-sql/statements/merge-transact-sql?view=sql-server-2017
The scenario mentioned in Example C would be the case you should follow.
-- Create a temporary table variable to hold the output actions.
DECLARE #SummaryOfChanges TABLE(Change VARCHAR(20));
MERGE INTO Sales.SalesReason AS Target
USING (VALUES ('Recommendation','Other'), ('Review', 'Marketing'),
('Internet', 'Promotion'))
AS Source (NewName, NewReasonType)
ON Target.Name = Source.NewName
WHEN MATCHED THEN
UPDATE SET ReasonType = Source.NewReasonType
WHEN NOT MATCHED BY TARGET THEN
INSERT (Name, ReasonType) VALUES (NewName, NewReasonType)
OUTPUT $action INTO #SummaryOfChanges;
-- Query the results of the table variable.
SELECT Change, COUNT(*) AS CountPerChange
FROM #SummaryOfChanges
GROUP BY Change;
Here I'm trying to migrate some data from a table column to a brand new table in which destination column have an unique constraint. Basically I'm trying to:
INSERT INTO FooTable VALUES (SELECT BarTable.Code FROM BarTable)
FooTable have only 2 columns: ID and Code (column with unique constraint).
But on BarTable.Code, maybe there are some duplicate values that I need to treat and fit them in the new constraint (maybe: Code = Code + 1 or else).
Any ideas on how to do that?
I am using MS SQL Server 2008 R2.
Thank you in advance.
You can use the MERGE command and insert a different record when the codes match.
Here is an example based on your scenario:
MERGE FooTable AS T
USING BarTable AS S
ON (T.Code = S.Code)
WHEN NOT MATCHED BY TARGET
THEN INSERT(Code) VALUES(S.Code)
WHEN MATCHED
THEN INSERT(Code) VALUES(S.Code+1)
You can use Not Exists
INSERT INTO FooTable VALUES (SELECT distinct br.Code FROM BarTable br where NOT EXISTS(SELECT * FROM FooTable bs where br.code=bs.code ) )
I am writing an SSIS package to run on SQL Server 2008. How do you do an UPSERT in SSIS?
IF KEY NOT EXISTS
INSERT
ELSE
IF DATA CHANGED
UPDATE
ENDIF
ENDIF
See SQL Server 2008 - Using Merge From SSIS. I've implemented something like this, and it was very easy. Just using the BOL page Inserting, Updating, and Deleting Data using MERGE was enough to get me going.
Apart from T-SQL based solutions (and this is not even tagged as sql/tsql), you can use an SSIS Data Flow Task with a Merge Join as described here (and elsewhere).
The crucial part is the Full Outer Join in the Merger Join (if you only want to insert/update and not delete a Left Outer Join works as well) of your sorted sources.
followed by a Conditional Split to know what to do next: Insert into the destination (which is also my source here), update it (via SQL Command), or delete from it (again via SQL Command).
INSERT: If the gid is found only on the source (left)
UPDATE If the gid exists on both the source and destination
DELETE: If the gid is not found in the source but exists in the destination (right)
I would suggest you to have a look at Mat Stephen's weblog on SQL Server's upsert.
SQL 2005 - UPSERT: In nature but not by name; but at last!
Another way to create an upsert in sql (if you have pre-stage or stage tables):
--Insert Portion
INSERT INTO FinalTable
( Colums )
SELECT T.TempColumns
FROM TempTable T
WHERE
(
SELECT 'Bam'
FROM FinalTable F
WHERE F.Key(s) = T.Key(s)
) IS NULL
--Update Portion
UPDATE FinalTable
SET NonKeyColumn(s) = T.TempNonKeyColumn(s)
FROM TempTable T
WHERE FinalTable.Key(s) = T.Key(s)
AND CHECKSUM(FinalTable.NonKeyColumn(s)) <> CHECKSUM(T.NonKeyColumn(s))
The basic Data Manipulation Language (DML) commands that have been in use over the years are Update, Insert and Delete. They do exactly what you expect: Insert adds new records, Update modifies existing records and Delete removes records.
UPSERT statement modifies existing records, if a records is not present it INSERTS new records.
The functionality of UPSERT statment can be acheived by two new set of TSQL operators. These are the two new ones
EXCEPT
INTERSECT
Except:-
Returns any distinct values from the query to the left of the EXCEPT operand that are not also returned from the right query
Intersect:-
Returns any distinct values that are returned by both the query on the left and right sides of the INTERSECT operand.
Example:- Lets say we have two tables Table 1 and Table 2
Table_1 column name(Number, datatype int)
----------
1
2
3
4
5
Table_2 column name(Number, datatype int)
----------
1
2
5
SELECT * FROM TABLE_1 EXCEPT SELECT * FROM TABLE_2
will return 3,4 as it is present in Table_1 not in Table_2
SELECT * FROM TABLE_1 INTERSECT SELECT * FROM TABLE_2
will return 1,2,5 as they are present in both tables Table_1 and Table_2.
All the pains of Complex joins are now eliminated :-)
To use this functionality in SSIS, all you need to do add an "Execute SQL" task and put the code in there.
I usually prefer to let SSIS engine to manage delta merge. Only new items are inserted and changed are updated.
If your destination Server does not have enough resources to manage heavy query, this method allow to use resources of your SSIS server.
We can use slowly changing dimension component in SSIS to upsert.
https://learn.microsoft.com/en-us/sql/integration-services/data-flow/transformations/configure-outputs-using-the-slowly-changing-dimension-wizard?view=sql-server-ver15
I would use the 'slow changing dimension' task