How to migrate Azure SQL Database to another Azure SQL Database - sql-server

I have one SQLServer in Azure portal and in that server has 2 SQL Databases TestDB1 and TestDB2 is copy of TestDB1. But we used TestDB2 for testing and now it has more data compared to TestDB1. I want to migrate only unavailable data from TestDB2 to TestDB1 as both are having same DB schema. How to do it?

Something like this might work, I've tested it locally and it does merge in test data from a sperate database, full example shown
USE [Playground2] -- Swap for your database name
CREATE TABLE MyTable (
Id BIGINT NOT NULL IDENTITY(1,1) CONSTRAINT PK_MyTable PRIMARY KEY,
[Name] NVARCHAR(50),
[Age] INT,
)
INSERT INTO MyTable([Name], [Age])
VALUES('Andrew', '28'),
('Robert', '38'),
('James', '40'),
('Robin', '40'),
('Peter', '56'), -- second database has this extra data
('Steve', '22') -- second database has this extra data
GO
USE [Playground] -- Swap for your database name
CREATE TABLE MyTable (
Id BIGINT NOT NULL IDENTITY(1,1) CONSTRAINT PK_MyTable PRIMARY KEY,
[Name] NVARCHAR(50),
[Age] INT,
)
INSERT INTO MyTable([Name], [Age])
VALUES('Andrew', '28'),
('Robert', '38'),
('James', '40'),
('Robin', '40')
GO
-- Check that the tables have slightly different data
SELECT * FROM Playground.dbo.MyTable
SELECT * FROM Playground2.dbo.MyTable
BEGIN TRANSACTION
BEGIN TRY
SET IDENTITY_INSERT dbo.MyTable ON
MERGE INTO dbo.MyTable AS TGT
USING [Playground2].dbo.MyTable AS SRC -- Note that we point to the other database here seeing as it is on the same SQL instance
ON TGT.Id = SRC.Id
WHEN MATCHED THEN
UPDATE SET
TGT.[Name] = SRC.[Name],
TGT.[Age] = SRC.[Age]
WHEN NOT MATCHED THEN
INSERT(Id, [Name], [Age])
VALUES(SRC.Id, SRC.[Name], SRC.[Age])
OUTPUT $action AS [Action],
deleted.[Name] AS OldName,
inserted.[Name] AS [NewName],
deleted.[Age] AS OldCountry,
inserted.[Age] AS NewCountry;
SET IDENTITY_INSERT dbo.MyTable OFF
SELECT * FROM dbo.MyTable
ROLLBACK TRANSACTION -- Change to COMMIT TRANSACTION when you are happy with the results
END TRY
BEGIN CATCH
PRINT 'Rolling back changes, there was an error!!'
ROLLBACK TRANSACTION
DECLARE #Msg NVARCHAR(MAX)
SELECT #Msg=ERROR_MESSAGE()
RAISERROR('Error Occured: %s', 20, 101,#msg) WITH LOG
END CATCH
But there also will be tools to do this, but this could be one answer, cheers

Related

Inserting to table having issue - Explicit value must be specified for identity column in table

I'm getting ready to release a stored procedure that gets info from other tables, does a pre-check, then inserts the good data into a (new) table. I'm not used to working with keys and new tables as much, and my insert into this new table I'm creating is having this error message having to do with the insert/key:
Msg 545, Level 16, State 1, Line 131
Explicit value must be specified for identity column in table 'T_1321_PNAnnotationCommitReport' either when IDENTITY_INSERT is set to ON or when a replication user is inserting into a NOT FOR REPLICATION identity column.
BEGIN
...
BEGIN
IF NOT EXISTS (SELECT * FROM sys.tables where name = N'T_1321_PNAnnotationCommitReport')
BEGIN
CREATE TABLE T_1321_PNAnnotationCommitReport (
[id] [INT] IDENTITY(1,1) PRIMARY KEY NOT NULL, --key
[progressnote_id] [INT] NOT NULL,
[form_id] [INT] NOT NULL,
[question_id] [INT],
[question_value] [VARCHAR](max),
[associatedconcept_id] [INT],
[crte_date] [DATETIME] DEFAULT CURRENT_TIMESTAMP,
[create_date] [DATETIME] --SCHED_RPT_DATE
);
print 'test';
END
END --if not exists main table
SET IDENTITY_INSERT T_1321_PNAnnotationCommitReport ON;
...
INSERT INTO dbo.T_1321_PNAnnotationCommitReport--(progressnote_id,form_id,question_id,question_value,associatedconcept_id,crte_date, create_date) **I tried with and without this commented out part and it's the same.
SELECT progressnote_id,
a.form_id,
question_id,
questionvalue,
fq.concept_id,
getdate(),
a.create_date
FROM (
SELECT form_id,
progressnote_id,
R.Q.value('#id', 'varchar(max)') AS questionid,
R.Q.value('#value', 'varchar(max)') AS questionvalue,
create_date
FROM
#tableNotes t
OUTER APPLY t.form_questions.nodes('/RESULT/QUESTIONS/QUESTION') AS R(Q)
WHERE ISNUMERIC(R.Q.value('#id', 'varchar(max)')) <> 0
) a
INNER JOIN [CKOLTP_DEV]..FORM_QUESTION fq ON
fq.form_id = a.form_id AND
fq.question_id = a.questionid
--select * from T_1321_PNAnnotationCommitReport
SET IDENTITY_INSERT T_1321_PNAnnotationCommitReport OFF;
END
Any ideas?
I looked at some comparable inserts we do at work, insert into select and error message, and insert key auto-incremented, and I think I'm doing what they do. Does anyone else see my mistake? Thanks a lot.
To repeat my comment under the question:
The error is literally telling you the problem. You turn change the IDENTITY_INSERT property to ON for the table T_1321_PNAnnotationCommitReport and then omit the column id in your INSERT. If you have enabled IDENTITY_INSERT you need to supply a value to that IDENTITY, just like the error says.
We can easily replicate this problem with the following batches:
CREATE TABLE dbo.MyTable (ID int IDENTITY(1,1),
SomeValue varchar(20));
GO
SET IDENTITY_INSERT dbo.MyTable ON;
--fails
INSERT INTO dbo.MyTable (SomeValue)
VALUES('abc');
GO
If you want the IDENTITY value to be autogenerated, then leave IDENTITY_INSERT set to OFF and omit the column from the INSERT (like above):
SET IDENTITY_INSERT dbo.MyTable OFF; --Shouldn't be needed normally, but we manually changed it before
--works, as IDENTITY_INSERT IS OFF
INSERT INTO dbo.MyTable (SomeValue)
VALUES('abc');
If you do specifically want to define the value for the IDENTITY, then you need to both set IDENTITY_INSERT to ON and provide a value in the INSERT statement:
SET IDENTITY_INSERT dbo.MyTable ON;
--works
INSERT INTO dbo.MyTable (ID,SomeValue)
VALUES(10,'def');
GO
SELECT *
FROM dbo.MyTable;
IDENTITY_INSERT doesn't mean "Get the RDBMS to 'insert' the value" it means that you want to want to tell the RDBMS what value to INSERT. This is covered in the opening sentence of the documentation SET IDENTITY_INSERT (Transact-SQL):
Allows explicit values to be inserted into the identity column of a table.
(Emphasis mine)

Using Identity Insert on a linked server

I have created a linked server object in SQL management studio on our on premise SQL box and I can insert into it as following syntax:
insert into [Azure].[budget].dbo.Bill
I want to set identity insert and have tried the following:
SET IDENTITY_INSERT [Azure].[budget].dbo.Bill ON
insert into [Azure].[budget].dbo.Bill
This is yielding the error that there are too many prefixes. Why can I insert into it without the identity insert and is it possible to do this any other way?
I have then changed the identity insert part to be SP as follows:
EXEC [Azure].[budget].dbo.sp_executesql N'SET IDENTITY_INSERT Bill ON'
insert into [Azure].[budget].dbo.Bill
But I am getting a warning about not having permission on the BillID field
You can't use SET IDENTITY INSERT directly in linked server.
You need to use dynamic SQL to SET IDENTITY_INSERT ON
sp_executesql N'SET IDENTITY_INSERT [Azure].[budgetenergy].dbo.Bill ON;insert into [Azure].[budget].dbo.Bill ....';
You can INSERT an identity value into a table with an identity column on a linked server with the "SWITCH TO" trick.
If you haven't used the "SWITCH TO" trick to add and remove identity on a column, it's very quick, even on large tables!
Conceptually you simply create a new SCHEMA exactly like the table you are wanting to INSERT to without the identity defined. Then switch the table to that SCHEMA and do your INSERT. Then switch back to the SCHEMA with the identity defined.
The sample below has been tested on a linked server in AZURE.
All the caveats of using "SWITCH TO" apply (indexes must be the same, drop and recreate foreign keys, etc)
To test, you can run the full script below on an Linked Azure SQL Server database. You'll need to do a find/replace with [LINKED_SERVER_NAME] and [DATABASE_NAME], replacing with your values. On a non-Azure DB you may need to add "ON PRIMARY" to the table creations.
--Let's setup the example by creating a table with an IDENTITY column on the Linked Server
EXEC('
CREATE TABLE [DATABASE_NAME].[dbo].[Example_Table](
[ID] [int] IDENTITY(1,1) NOT NULL,
[Name] [nchar](10) NULL
)
'
) AT [LINKED_SERVER_NAME]
--INSERT some data into the table
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] ([Name]) VALUES ('Travis')
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] ([Name]) VALUES ('Jay')
-- Looks good
SELECT * FROM [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table]
GO
-- Create a TABLE with an identical schema, without the identity defined
EXEC('
CREATE TABLE [DATABASE_NAME].[dbo].[Example_Table_temp](
[ID] [int] NOT NULL,
[Name] [nchar](10) NULL
)
'
) AT [LINKED_SERVER_NAME]
--Now Use the "SWITCH TO" to move the data to the new table
EXEC('
ALTER TABLE [DATABASE_NAME].[dbo].[Example_Table] SWITCH TO [DATABASE_NAME].[dbo].[Example_Table_temp]
'
) AT [LINKED_SERVER_NAME]
--Drop the old table (It should now be empty, but you may want to verify that if you are unsure here)
EXEC('
DROP TABLE [DATABASE_NAME].[dbo].[Example_Table]
'
) AT [LINKED_SERVER_NAME]
--Rename the new table back to the old table name
-- NOTE the lack of database and owner identifiers in the new name
-- NOTE the use of double single qoutes (ESCAPED single quotes)
EXEC('
EXEC sp_rename ''[DATABASE_NAME].[dbo].Example_Table_temp'',''Example_Table''
'
) AT [LINKED_SERVER_NAME]
-- Now do your IDENTITY INSERTs !!!!
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] (ID,[Name]) VALUES (888,'Travis')
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] (ID,[Name]) VALUES (999,'Jay')
--Verify they got put in
SELECT * FROM [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table]
--Now let's switch it back to our SCHEMA with an IDENTITY
EXEC('
CREATE TABLE [DATABASE_NAME].[dbo].[Example_Table_temp](
[ID] [int] IDENTITY(1,1) NOT NULL,
[Name] [nchar](10) NULL
)
ALTER TABLE [DATABASE_NAME].[dbo].[Example_Table] SWITCH TO [DATABASE_NAME].[dbo].[Example_Table_temp]
DROP TABLE [DATABASE_NAME].[dbo].[Example_Table]
EXEC sp_rename ''[DATABASE_NAME].[dbo].Example_Table_temp'',''Example_Table''
'
) AT [LINKED_SERVER_NAME]
--Data is still there
SELECT * FROM [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table]
GO
-- And note you can no longer INSERT the IDENTITY
INSERT INTO [LINKED_SERVER_NAME].[DATABASE_NAME].[dbo].[Example_Table] (ID,[Name]) VALUES (45,'Travis')
GO

Dropping Tables within an AFTER INSERT Trigger

I am looking for suggestions on how to handle incoming data from a C# client app that sends via System.Data.SqlClient.SqlBulkCopy. The original intent was to maintain all data sent via the app into our sql server where the data would be 'massaged'. To keep things secure, data is sent to write only tables where an AFTER INSERT trigger will copy to another table and delete incoming.
Simple Example:
ALTER TRIGGER [dbo].[FromWeb_To_MyTable] ON [dbo].[_FromWeb_MyTable]
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
INSERT INTO [dbo].[MyTable]([MyTableID],[MemberID],[LocationID],[ODBCID],[InsertDateTime])
SELECT i.[MyTableID],i.[MemberID],i.[LocationID],i.[ODBCID],i.[InsertDateTime]
FROM Inserted i;
DELETE FROM _FromWeb_MyTable
WHERE [MyTableID] IN (SELECT i.[MyTableID] FROM Inserted i);
END
Now I am going to be using a bit differently and need to delete everything in the 'go to' table. My largest table will be around 350,000 records. I intend to DROP and re-CREATE said table like so:
(this method IS working fine see my questions below)
ALTER TRIGGER [dbo].[FromWeb_To_MyTable] ON [dbo].[_FromWeb_MyTable]
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
IF EXISTS(SELECT [name] FROM sys.TABLES WHERE [name] = 'MyTable') DROP TABLE [dbo].[MyTable];
CREATE TABLE [dbo].[MyTable] (
[MyTableID] [int] NOT NULL,
[MemberID] [varchar](6) NOT NULL,
[LocationID] [varchar](50) NOT NULL,
[ODBCID] [varchar](50) NOT NULL,
[InsertDateTime] [datetime] NOT NULL
) on [PRIMARY];
INSERT INTO [dbo].[MyTable] ( [MyTableID], [MemberID],[LocationID],[ODBCID],[InsertDateTime] )
SELECT i.[MyTableID],i.[MemberID],i.[LocationID],i.[ODBCID],i.[InsertDateTime]
FROM Inserted i;
DELETE FROM _FromWeb_MyTable
WHERE [MyTableID] IN (SELECT [MyTableID] FROM Inserted);
END
Does anyone see any problems with this method? Any different suggestions? Can someone explain when and how to index the newly recreated table?
Since you reuse your table instead of dropping and re-creating it just use TRUNCATE.
ALTER TRIGGER dbo.FromWeb_To_MyTable ON dbo._FromWeb_MyTable
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
TRUNCATE TABLE dbo.MyTable;
INSERT INTO dbo.MyTable (MyTableID, MemberID,LocationID,ODBCID,InsertDateTime)
SELECT i.MyTableID,i.MemberID,i.LocationID,i.ODBCID,i.InsertDateTime
FROM Inserted i;
DELETE FROM _FromWeb_MyTable
WHERE MyTableID IN (SELECT MyTableID FROM Inserted);
END

"Invalid column name" error when calling insert after table created

I'm developing SQL script, using SSMS, which makes some changes in database:
USE MyDatabase;
BEGIN TRANSACTION;
-- some statements
PRINT(N'#1');
IF (EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = N'dbo' AND TABLE_NAME = N'Table1' AND COLUMN_NAME = 'Table2_Id'))
BEGIN
ALTER TABLE [dbo].[Table1] DROP CONSTRAINT [FK_Table1_Table2_Table2_Id];
ALTER TABLE [dbo].[Table1] DROP COLUMN [Table2_Id];
DROP TABLE [dbo].[Table2];
PRINT(N'Table2 was dropped.');
END
PRINT(N'#2');
IF (NOT EXISTS (SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = N'dbo' AND TABLE_NAME = N'Table2'))
BEGIN
CREATE TABLE [dbo].[Table2]
(
[Id] INT NOT NULL PRIMARY KEY IDENTITY,
[Number] INT NOT NULL UNIQUE,
[Name] NVARCHAR(200) NOT NULL,
[RowVersion] TIMESTAMP NOT NULL
);
PRINT(N'Table2 was re-created.');
INSERT INTO [dbo].[Table2]([Number], [Name]) VALUES(-1, N'Default value');
PRINT(N'Default value was inserted in Table2.');
END
-- some statements
COMMIT TRANSACTION;
If Table1 has a column, named Table2_Id, then database has two tables (Table1 and Table2) and a foreign key relationship between them. In that case, I need to:
drop foreign key relationship FK_Table1_Table2_Table2_Id;
drop foreign key column Table1.Table2_Id;
drop Table2;
re-create Table2, using new table schema;
insert some default value in Table2.
When I'm trying to execute this script, I'm getting these errors:
Msg 207, Level 16, State 1, Line 262 Invalid column name 'Number'.
Msg 207, Level 16, State 1, Line 262 Invalid column name 'Name'.
Looks like SQL Server uses old schema for Table2 (which indeed hasn't these columns), but how is this possible, if the table has just created with new schema?
What am I doing wrong?
Server version is SQL Server 2012 (SP1) - 11.0.3128.0 (X64).
UPDATE.
I've added PRINT calls (see script above). There's nothing in message window, except error messages. So, the script isn't being executed... What's going on??
SQL Server tries to compile the whole batch. If the table already exists then it will compile according to the pre-existing definition. The statement referencing the new columns doesn't compile and so the batch never executes.
You need to group the statements using the new definition into a new batch. If you are running this in SSMS just insert a GO
USE MyDatabase;
BEGIN TRANSACTION;
-- some statements
PRINT(N'#1');
IF (EXISTS (SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = N'dbo' AND TABLE_NAME = N'Table1' AND COLUMN_NAME = 'Table2_Id'))
BEGIN
ALTER TABLE [dbo].[Table1] DROP CONSTRAINT [FK_Table1_Table2_Table2_Id];
ALTER TABLE [dbo].[Table1] DROP COLUMN [Table2_Id];
DROP TABLE [dbo].[Table2];
PRINT(N'Table2 was dropped.');
END
GO
PRINT(N'#2');
IF (NOT EXISTS (SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = N'dbo' AND TABLE_NAME = N'Table2'))
BEGIN
CREATE TABLE [dbo].[Table2]
(
[Id] INT NOT NULL PRIMARY KEY IDENTITY,
[Number] INT NOT NULL UNIQUE,
[Name] NVARCHAR(200) NOT NULL,
[RowVersion] TIMESTAMP NOT NULL
);
PRINT(N'Table2 was re-created.');
INSERT INTO [dbo].[Table2]([Number], [Name]) VALUES(-1, N'Default value');
PRINT(N'Default value was inserted in Table2.');
END
COMMIT
Otherwise you could run the offending line in a child batch
EXEC(N'INSERT INTO [dbo].[Table2]([Number], [Name]) VALUES(-1, N''Default value'');')

DROP TABLE fails for temp table

I have a client application that creates a temp table, the performs a bulk insert into the temp table, then executes some SQL using the table before deleting it.
Pseudo-code:
open connection
begin transaction
CREATE TABLE #Temp ([Id] int NOT NULL)
bulk insert 500 rows into #Temp
UPDATE [OtherTable] SET [Status]=0 WHERE [Id] IN (SELECT [Id] FROM #Temp) AND [Group]=1
DELETE FROM #Temp WHERE [Id] IN (SELECT [Id] FROM [OtherTable] WHERE [Group]=1)
INSERT INTO [OtherTable] ([Group], [Id]) SELECT 1 as [Group], [DocIden] FROM #Temp
DROP TABLE #Temp
COMMIT TRANSACTION
CLOSE CONNECTION
This is failing with an error on the DROP statement:
Cannot drop the table '#Temp', because it does not exist or you do not have permission.
I can't imagine how this failure could occur without something else going on first, but I don't see any other failures occurring before this.
Is there anything that I'm missing that could be causing this to happen?
possibly something is happening in the session in between?
Try checking for the existence of the table before it's dropped:
IF object_id('tempdb..#Temp') is not null
BEGIN
DROP TABLE #Temp
END
I've tested this on SQL Server 2005, and you can drop a temporary table in the transaction that created it:
begin transaction
create table #temp (id int)
drop table #temp
commit transaction
Which version of SQL Server are you using?
You might reconsider why you are dropping the temp table at all. A local temporary table is automatically deleted when the connection ends. There's usually no need to drop it explicitly.
A global temporary table starts with a double hash (f.e. ##MyTable.) But even a global temp table is automatically deleted when no connection refers to it.
I think you aren't creating the table at all, because the statement
CREATE TABLE #Temp ([Id] AS int)
is incorrect. Please, write it as
CREATE TABLE #Temp ([Id] int)
and see if it works.
BEGIN TRAN
IF object_id('DATABASE_NAME..#TABLE_NAME') is not null
BEGIN
DROP TABLE #TABLE_NAME
END
COMMIT TRAN
Note:Please enter your table name where TABLE_NAME and database name where it says DATABASE_NAME

Resources