I have a query with a simple merge statement to update or insert data in a table:
MERGE INTO table_name AS TARGET
USING (
VALUES (
:a0
,:b0
,:c0
)...
) AS SOURCE(A, B, C)
ON SOURCE.B = TARGET.B
AND SOURCE.C = TARGET.C
WHEN NOT MATCHED
THEN
INSERT (
A
,B
,C
)
VALUES (
SOURCE.A
,SOURCE.B
,SOURCE.C
);
This table has a non clustered index on two columns for uniqueness constraint.
This query works fine on a "Business" database in Azure. After the migration to SQL V12 on an "S2" database, this error happens when i try to merge a huge amount of entries:
Uncaught exception 'PDOException' with message 'SQLSTATE[42000]: [Microsoft][ODBC Driver 11 for SQL Server][SQL Server]Cannot bulk load. The bulk data stream was incorrectly specified as sorted or the data violates a uniqueness constraint imposed by the target table. Sort order incorrect for the following two rows: primary key of first row: (A, B, C), primary key of second row: (A, D, E).
It appears that Microsoft knows this issue : https://support.microsoft.com/en-us/kb/3055799.
But in Azure, i can't update the SQL Server. how can i get it to work ?
After some research and tests, I found a workaround by adding at the end of my query "OPTION (LOOP JOIN, FORCE ORDER);" to bypass the sorting error by modifing the default execution plan.
It also works with "OPTION (MERGE JOIN, FORCE ORDER);" depending on the rows count of the source and the target table.
More infos about the options : https://technet.microsoft.com/en-us/library/ms181714.aspx
we found issue with execution plan generated by the query optimizer.
this will be fixes very soon.
meanwhile another option to workaround this issue is to disable page_lock on the index.
ALTER INDEX [<index name>] ON [<schema>].[<table name>]
REBUILD WITH (ALLOW_PAGE_LOCKS = OFF)
with this workaround you do not need to modify your queries.
Hello #Yochanan Rachamim,
Thanks for your answer.
I tried to apply your fix but it has no effect.
After more research, I found a known bug with concurrency : https://www.mssqltips.com/sqlservertip/3074/use-caution-with-sql-servers-merge-statement/
I finally solved my problem by adding "WITH (HOLDLOCK)" to the MERGE statement.
Related
I try to add a column to a table with GUI Tableplus, but no response for long time.
So I turn to the db server, but got these error:
Maybe some inconsistent data generated during the operation through the Tableplus.
I am new to postgresql , and don't know what to do next.
-----updated------
I did some operation as #Dri372 told, and got some progress.
The failed reason for table sys_role and s2 is that the tables are not empty, they have some records.
If I run sql like this create table s3 AS SELECT * FROM sys_role; alter table s3 add column project_code varchar(50);, I successed.
Now how could I still work on the table sys_role?
I am trying to execute below query and during that one constraint violation exception is generated and due to that insert statement is terminated.
I want suppose from 10 records 9 records are clean then insertion will done for 9.right now statement is terminated and no insertion is performed.
I am using SQL Server 2012 and i do not want to rollback transaction and Insert ignore command is not there in SQL server and i do not want to insert data which contains error.i just want to insert clean data.
Query :
INSERT INTO rcmschargepostingmastertable
(clinicid,
clinicsiteid,
appointmentid,
patientid
)
SELECT clinicid,
clinicsiteid,
appointmentid,
patientid,
FROM #tempautopostbulkchargepostingmastertable
It is not possible to do what you stated in your comment:
i want to ignore any sql error and want to continue insertion for
clean records
SQL Server doesn't have any pure SQL mechanism for doing this. Your only choice is to use one of the proposed work-arounds (SSIS, WHERE clause).
One work-around that hasn't been mentioned because it's the worst performance-wise, but at least it's one that you haven't shot down, is to replace your set-based insert with a cursor that does the inserts one row at a time.
Then you could put the single-row insert in a TRY block, and if it errors, the cursor will skip it and move on to the next one.
I do not want to insert data which contains error.i just want to insert clean data.
Then you need to identify and filter out the bad data/constraint violating records before inserting into target table which will make your life easier.
........
modifiedbyid
FROM #tempautopostbulkchargepostingmastertable
Where some_column <> 'bad data'
Since you are using SQL Server 2012 you can use TRY_CONVERT to identify and filter out the bad data
I am trying to execute stored proc through SSIS and it gives me following error:
[Execute SQL Task] Error: Executing
the query "Exec sp1 ?" failed with
the following error: "Procedure: sp1
Line: 66 Message: Cannot insert
duplicate key row in object
'table.sq1' with unique index
'UIX_sp1_Key'.". Possible failure
reasons: Problems with the query,
"ResultSet" property not set
correctly, parameters not set
correctly, or connection not
established correctly.
Actually the stored Proc sp1 is truncating & reloading a data into a table.
I am not able to figure out where exactly its trying to insert duplicate record.
Thanks in advance.
Your data must have duplicate values across the key column(s). You could remove the primary key temporarily (or create another table with the same structure but without the definition of the primary key, but that would mean changing the stored procedure), then load the data into the table. Then use this SQL statement:
select keycol1 {,keycol2 {,keycol3} ...}, count(*)
from tablename
group by keycol1 {,keycol2 {,keycol3} ...}
having count(*) > 1
That will show you the data values that are duplicates across the key column(s).
If you are truncating the table before load, then you must have duplicate data in the source.
Examine this to see what there is. use Excel or Access or some such if needed to assist. Or drop the unique constraint then query the staging table with an aggregate query.
I want to update a static table on my local development database with current values from our server (accessed on a different network/domain via VPN). Using the Data Import/Export wizard would be my method of choice, however I typically run into one of two issues:
I get primary key violation errors and the whole thing quits. This is because it's trying to insert rows that I already have.
If I set the "delete from target" option in the wizard, I get foreign key violation errors because there are rows in other tables that are referencing the values.
What I want is the correct set of options that means the Import/Export wizard will update rows that exist and insert rows that do not (based on primary key or by asking me which columns to use as the key).
How can I make this work? This is on SQL Server 2005 and 2008 (I'm sure it used to work okay on the SQL Server 2000 DTS wizard, too).
I'm not sure you can do this in management studio. I have had some good experiences with
RedGate SQL Data Compare in synchronising databases, but you do have to pay for it.
The SQL Server Database Publishing Wizard can export a set of sql insert scripts for the table that you are interested in. Just tell it to export just data and not schema. It'll also create the necessary drop statements.
One option is to download the data to a new table, then use commands similar to the following to update the target:
update target set
col1 = d.col1,
col2 = d.col2
from downloaded d
inner join target t on d.pk = t.pk
insert into target (col1, col2, ...)
select (d.col1, d.col2, ...) from downloaded d
where d.pk not in (select pk from target)
If you disable the FK constrains during the 2nd option - and resume them after finsih - it will work.
But if you are using identity to create pk that are involves in the FK - it will cause a problem, so it works only if the pk values remains the same.
I have replication set up between a sql-server 2005 instance and multiple sql-server 2000 instances. The replication will successfully for a while before I get the following error message:
Violation of UNIQUE KEY constraint 'unique_pubsrvdb'. Cannot insert duplicate key in object 'dbo.sysmergesubscriptions'. (Source: MSSQLSERVER, Error number: 2627)
When I checked sysmergesubscriptions there were extra entries that appear to be coming from the 2000 instances.
My question is has anyone encountered this issue and how did you deal with it (without rebuilding the entire thing)
In my case handling multiple subscriptions and just had to adapt to delete subscriptions that had problems with:
delete
from sysmergesubscriptions
where pubid not in (select pubid from sysmergepublications)
and subscriber_server = 'SUBSCRIPTIONSERVER'
The problem was that one of the subscribers had old publications and subscriptions in the system tables that were replicated through out the entire system. Which caused the violation of UNIQUE KEY constraint.
Once we removed these old entires we were able to restart replication.
We were able to identify the valid records in sysmergepublication because we knew the state of this table before the invalid entries were replicated. This forum post shows you how to location invalid publications if you need to.
We used the follow sql to check for additional subscription entries:
select *
from sysmergepublications
select *
from sysmergesubscriptions
where pubid in ( select pubid from sysmergepublications)
select *
from sysmergesubscriptions
where pubid not in ( select pubid from sysmergepublications)
Here is the sql that we used to delete the invalid subscriptions:
delete from sysmergesubscriptions
where pubid not in ( select pubid from sysmergepublications)
Note: the code sample above assumes that the sysmergepublication contains only valid publications
Alternatively: You can use the EXEC sp_removedbreplication #dbname='<dbname>' to remove replication from the database completely. This command appears to remove all replication triggers from the database.