Bulk insert from a table to another - sql-server

We have 2 tables in SQL Server 2008 R2. Periodically, we have to insert a batch of records from Table A to Table B. While the inserting, Table B still able to SELECT & UPDATE. Currently, we use INSERT..SELECT to copy from Table A to Table B. But the problem is while inserting, sometimes will cause UPDATE statement to TABLE B timeout.
Is there a better bulk insert solution from a table to another that won't cause blocking?

They most obvious solution is to use smaller batches as Stanley suggested. If this is really not an option you could explore '(transaction level) snapshot isolation.

1 Set the transaction timeout to a large enough value, so that the statement no longer goes in timeout.
2 Use CURSOR and do it row by row
3 Try this way of doing things. Requires a row identifier (IDENTITY for instance), best to have a PK or INDEX on that field:
SET NOCOUNT ON;
CREATE TABLE #A(
row_id INT IDENTITY(1,1) NOT NULL PRIMARY KEY,
data INT NOT NULL
);
CREATE TABLE #B(
row_id INT NOT NULL PRIMARY KEY,
data INT NOT NULL
);
-- TRUNCATE TABLE #B; -- no truncate needed since you just want to add rows, not copy the whole table
DECLARE #batch_size INT;
SET #batch_size = 10000;
DECLARE #from_row_id INT;
DECLARE #to_row_id INT;
-- You would use this to establish the first #from_row_id if you wanted to copy the whole table
-- SELECT
-- #from_row_id=ISNULL(MIN(row_id),-1)
-- FROM
-- #A AS a;
SELECT
#from_row_id=ISNULL(MAX(row_id),-1)
FROM
#B AS b;
IF #from_row_id=-1
SELECT
#from_row_id=ISNULL(MIN(row_id),-1)
FROM
#A AS a;
ELSE
SELECT
#from_row_id=ISNULL(MIN(row_id),-1)
FROM
#A AS a
WHERE
row_id>#from_row_id;
WHILE #from_row_id>=0
BEGIN
SELECT
#to_row_id=ISNULL(MAX(row_id),-1)
FROM
(
SELECT TOP(#batch_size)
row_id
FROM
#A AS a
WHERE
row_id>=#from_row_id
) AS row_ids
IF #to_row_id=-1
BEGIN
INSERT
#B
SELECT
*
FROM
#A AS a
WHERE
row_id>=#from_row_id;
BREAK;
END
ELSE
INSERT
#B
SELECT
*
FROM
#A AS a
WHERE
row_id BETWEEN #from_row_id AND #to_row_id;
SELECT
#from_row_id=ISNULL(MIN(row_id),-1)
FROM
#A AS a
WHERE
row_id>#to_row_id;
END
DROP TABLE #B;
DROP TABLE #A;

Related

How to drop and recreate the temp table in loop in procedure if problem occurs what should we do?

I am tring a lot about the problem that was given in one of my session
that " create one temp table after that create a loop
in that loop drop the temp table which we created and recreate that one"
if we don't create temp table before loop we can get output as
"Command(s) completed successfully." but if we create that one it is showing error.
please say me the answer to "how to drop and recreate the temp table in loop which already created outside of loop"
create proc newp
as
begin
declare #a int
set #a=5
create table #temp(a int)
while #a >=0
begin
drop table if exists #temp
create table #temp(a int)
set #a = #a-1
end
end
Command(s) completed successfully.
The below way of checking the temp table and drop will work in SQL Server.
IF OBJECT_ID('tempdb..#temp') IS NOT NULL
DROP TABLE #temp
So your stored procedure will be:
CREATE PROCEDURE newp
AS
BEGIN
DECLARE #a INT
SET #a = 5
IF OBJECT_ID('tempdb..#temp') IS NOT NULL DROP TABLE #temp
CREATE TABLE #temp (a INT)
WHILE #a >= 0
BEGIN
-- IF OBJECT_ID('tempdb..#temp') IS NOT NULL DROP TABLE #temp
-- CREATE TABLE #temp (a INT)
INSERT INTO #temp (a) VALUES (#a)
SET #a = #a - 1
TRUNCATE TABLE #temp
END
END

Adding constraints to list items in SQL Server database

I have table holding items for a given list id in my Ms Sql server database (2008R2).
I would like to add constraints so that no two list ids have same item list. Below illustrate my schema.
ListID , ItemID
1 a
1 b
2 a
3 a
3 b
In above example ListID 3 should fail. I guess you can't put constarint/check within the database itself (Triggers,check) and the logic constaint can only be done from the frontend?
Thanks in advance for any help.
Create a function that performs the logic you want and then create a check constraint or index that leverages that function.
Here is a functional example, the final insert fails. The function is evaluated row by row, so if you need to insert as a set and evaluate after, you'd need to do an "instead of" trigger:
CREATE TABLE dbo.Test(ListID INT, ItemID CHAR(1))
GO
CREATE FUNCTION dbo.TestConstraintPassed(#ListID INT, #ItemID CHAR(1))
RETURNS TINYINT
AS
BEGIN
DECLARE #retVal TINYINT = 0;
DECLARE #data TABLE (ListID INT, ItemID CHAR(1),[Match] INT)
INSERT INTO #data(ListID,ItemID,[Match]) SELECT ListID,ItemID,-1 AS [Match] FROM dbo.Test
UPDATE #data
SET [Match]=1
WHERE ItemID IN (SELECT ItemID FROM #data WHERE ListID=#ListID)
DECLARE #MatchCount INT
SELECT #MatchCount=SUM([Match]) FROM #data WHERE ListID=#ListID
IF NOT EXISTS(
SELECT *
FROM (
SELECT ListID,SUM([Match]) AS [MatchCount]
FROM #data
WHERE ListID<>#ListID
GROUP BY ListID
) dat
WHERE #MatchCount=[MatchCount]
)
BEGIN
SET #retVal=1;
END
RETURN #retVal;
END
GO
ALTER TABLE dbo.Test
ADD CONSTRAINT chkTest
CHECK (dbo.TestConstraintPassed(ListID, ItemID) = 1);
GO
INSERT INTO dbo.Test(ListID,ItemID) SELECT 1,'a'
INSERT INTO dbo.Test(ListID,ItemID) SELECT 1,'b'
INSERT INTO dbo.Test(ListID,ItemID) SELECT 2,'a'
INSERT INTO dbo.Test(ListID,ItemID) SELECT 2,'b'
Related

Creating a temporary table within a TRIGGER with GROUP BY

I need to create and use a temporary table with GROUP BY clause within a trigger, but I'm having difficulties doing so.
My attempt:
Here I'm trying to use two temporary tables which are dropped after the trigger reach an end.
First I create a #Temptable and the trigger.
CREATE TABLE #TempTable (admID smallint, diagID smallint);
CREATE TRIGGER tr_newTest
ON Adm_Diag
FOR INSERT
AS
BEGIN
...
END
Since the table inserted only contains rows for a current INSERT and UPDATE statements I'm passing several INSERT and UPDATE statements to #TempTable.
DECLARE #admID smallint
SELECT #admID = Adm_ID
FROM inserted
DECLARE #diagID smallint
SELECT #diagID=Diag_ID
FROM inserted
INSERT INTO #TempTable VALUES (#admID, #diagID)
Now with this data I want to create a temporary table that groups the rows of #TempTable:
SELECT *
INTO #TempGroupTable
FROM
(
SELECT admID, COUNT(*) as Diag
FROM #TempTable
GROUP BY admID
) t1
WHERE Diag > 2
The whole script
CREATE TABLE #TempTable (admID smallint, diagID smallint);
CREATE TRIGGER tr_newTest
ON Adm_Diag
FOR INSERT
AS
BEGIN
DECLARE #admID smallint
SELECT #admID = Adm_ID
FROM inserted
DECLARE #diagID smallint
SELECT #diagID=Diag_ID
FROM inserted
INSERT INTO #TempTable VALUES (#admID, #diagID)
-- Below I'm tring to create #TempGroupTable
SELECT *
INTO #TempGroupTable
FROM
(
SELECT admID, COUNT(*) as Diag
FROM #TempTable
GROUP BY admID
) t1
WHERE Diag > 2
END
After executing the trigger I get an error:
Msg 208, Level 16, State 0, Line 41 Invalid object name
'#TempGroupTable'.
How can I create #TempGroupTable?
Not quote sure what you are trying to do but would a global temporary tables which starts with ## work for you? So make the #TempGroupTable into ##TempGroupTable?
Why not dispense with all the overhead of temp tables and variables? Try:
CREATE TRIGGER tr_newTest
ON Adm_Diag
INSTEAD OF INSERT
AS
BEGIN
INSERT INTO Adm_Diag (adminID, Diag)
SELECT admID, COUNT(*) as Diag
FROM inserted
GROUP BY admID
END

SQL Server 2012 Insert a Set and Get all PK Values?

I have a sproc that takes a TVP, the input has 6,000 rows. The current code copies the TVP into a temp table, iterates RBAR over the temp table to insert a row on a real table, gets the Identity/PK value from the insert, updates the temp table with the PK (used later in the stored procedure), and repeats.
Is there a cool and/or quick way to insert the whole temp table and then update it after that with the PK values? Nothing straightforward is coming to mind, all ideas appreciated.
I think I can add an additional column to the temp table and put a sequence number on it so the rows are all unique and follow this idea: http://www.sqlteam.com/article/using-the-output-clause-to-capture-identity-values-on-multi-row-inserts but I'm open to other set-based suggestions....
Thanks.
I would suggest that you bulk insert from your TVP into your table, using the OUTPUT clause to populate a temporary table with your results.
Here's an example:
-- Setup
-- Create the data type
CREATE TYPE dbo.TVPtest AS TABLE
(
c1 int NOT NULL,
c2 int NULL
)
GO
CREATE TABLE tableA
(
id int primary key identity,
c1 int not null,
c2 int null
)
-- This is the example
declare #t TVPTest
declare #i int = 1000
-- get a TVP with 1000 rows
WHILE #i >= 0
BEGIN
INSERT INTO #t
VALUES (#i, ABS(CHECKSUM(NewId())) % 99)
SET #i= #i -1
END
-- logic from here is what you would put in your stored procedure
CREATE TABLE #new
(
id int,
c1 int not null,
c2 int null
)
INSERT INTO tableA (c1, c2)
OUTPUT inserted.id, inserted.c1, inserted.c2 into #new
SELECT c1, c2
FROM #t
SELECT *
FROM #new

Do Inserted Records Always Receive Contiguous Identity Values

Consider the following SQL:
CREATE TABLE Foo
(
ID int IDENTITY(1,1),
Data nvarchar(max)
)
INSERT INTO Foo (Data)
SELECT TOP 1000 Data
FROM SomeOtherTable
WHERE SomeColumn = #SomeParameter
DECLARE #LastID int
SET #LastID = SCOPE_IDENTITY()
I would like to know if I can depend on the 1000 rows that I inserted into table Foo having contiguous identity values. In order words, if this SQL block produces a #LastID of 2000, can I know for certain that the ID of the first record I inserted was 1001? I am mainly curious about multiple statements inserting records into table Foo concurrently.
I know that I could add a serializable transaction around my insert statement to ensure the behavior that I want, but do I really need to? I'm worried that introducing a serializable transaction will degrade performance, but if SQL Server won't allow other statements to insert into table Foo while this statement is running, then I don't have to worry about it.
I disagree with the accepted answer. This can easily be tested and disproved by running the following.
Setup
USE tempdb
CREATE TABLE Foo
(
ID int IDENTITY(1,1),
Data nvarchar(max)
)
Connection 1
USE tempdb
SET NOCOUNT ON
WHILE NOT EXISTS(SELECT * FROM master..sysprocesses WHERE context_info = CAST('stop' AS VARBINARY(128) ))
BEGIN
INSERT INTO Foo (Data)
VALUES ('blah')
END
Connection 2
USE tempdb
SET NOCOUNT ON
SET CONTEXT_INFO 0x
DECLARE #Output TABLE(ID INT)
WHILE 1 = 1
BEGIN
/*Clear out table variable from previous loop*/
DELETE FROM #Output
/*Insert 1000 records*/
INSERT INTO Foo (Data)
OUTPUT inserted.ID INTO #Output
SELECT TOP 1000 NEWID()
FROM sys.all_columns
IF EXISTS(SELECT * FROM #Output HAVING MAX(ID) - MIN(ID) <> 999 )
BEGIN
/*Set Context Info so other connection inserting
a single record in a loop terminates itself*/
DECLARE #stop VARBINARY(128)
SET #stop = CAST('stop' AS VARBINARY(128))
SET CONTEXT_INFO #stop
/*Return results for inspection*/
SELECT ID, DENSE_RANK() OVER (ORDER BY Grp) AS ContigSection
FROM
(SELECT ID, ID - ROW_NUMBER() OVER (ORDER BY [ID]) AS Grp
FROM #Output) O
ORDER BY ID
RETURN
END
END
Yes, they will be contiguous because the INSERT is atomic: complete success or full rollback. It is also performed as a single unit of work: you wont get any "interleaving" with other processes
However (or to put your mind at rest!), consider the OUTPUT clause
DECLARE #KeyStore TABLE (ID int NOT NULL)
INSERT INTO Foo (Data)
OUTPUT INSERTED.ID INTO #KeyStore (ID) --this line
SELECT TOP 1000 Data
FROM SomeOtherTable
WHERE SomeColumn = #SomeParameter
If you want the Identity values for multiple rows use OUTPUT:
DECLARE #NewIDs table (PKColumn int)
INSERT INTO Foo (Data)
OUTPUT INSERTED.PKColumn
INTO #NewIDs
SELECT TOP 1000 Data
FROM SomeOtherTable
WHERE SomeColumn = #SomeParameter
you now have the entire set of values in the #NewIDs table. You can add any columns from the Foo table into the #NewIDs table and insert those columns as well.
It is not good practice to attach any sort of meaning whatsoever to identity values. You should assume that they are nothing more than integers guaranteed to be unique within the scope of your table.
Try adding the following:
option(maxdop 1)

Resources