A new column has been added to a table, but the new column was not added to the end of the table definition (rightmost column), but the middle of the table.
When I try to commit this in Redgate SQL Source Control, I get the warning "These changes may result in data loss"
Will data loss really occurr?
Is there a way preview the change script to confirm that no data will be lost?
Can I copy the script and easily turn it into a Migrations V2 script?
Will I just have to
Edit the table in SSMS and move the new column to the end
or write a migration script?
If so, are there any handy tools to do the repetitive stuff?
Up front disclosure that I work for Red Gate on SQL Source Control.
That change will need to re-create a table. By default SSMS won't let you save that change. However that option must have been disabled in SSMS. It's under Tools->Options->Designers->Table and Database Designers->Prevent saving changes that require a table re-creating.
Given that feature is disabled SQL Source Control has then picked that up as a potential data loss situation, and prompted to see if you want to add a migration script.
If other developers within your team pull this change in through a get latest, then SQL Source Control will let them about any potential data loss with more details, depending on the current state of their local database. If the only change is adding columns to an existing table then this will not drop the data in columns that are unchanged.
If you are deploying to another DB (e.g. staging/UAT/prod) and you have SQL Compare you can use that to see exactly what will be applied to a DB if you try and run this against another non-local database. Choose the create deployment script option and you can sanity check the SQL before running.
As you say adding the column to the end of the table will avoid the need for the rebuild, so is probably the simplest way to avoid this if you don't need to worry about where the column is.
Alternatively you can add a migration script to:
Create a new table with the new structure using a temp name
Copy the existing data to the temp table
Drop the existing table
Rename the new temp table to the original name
You mention Migrations v2, the beta feature that changes how migrations work in order to better support branching and merging and DVCS systems. See http://www.red-gate.com/migrations
Version 1 migration scripts will need some modifications in order to be converted to a v2 migration script. It's a fairly trivial change. We're working on documenting this at the moment, and please reach out to us on the Google Group if you'd like more information on this change. https://groups.google.com/forum/#!forum/red-gate-migrations
I moved the column to the end of the table using SSMS to negate the need for a migration script.
In a similar scenario, where it was not convenient to move the column, this is what I did to convert an SSMS script to a Migrations V2 script.
Undo the change in SSMS (deleted the column)
Redo the change in SSMS, but instead of saving the change direct to the database, I saved the change script
Modified the change script
Trimmed the SSMS transaction & environment wrapper
Added a guard clause: IF COL_LENGTH('MyTable','MyColumn') IS NULL
Wrapped the script in BEGIN TRAN - ROLLBACK TRAN to test the script without dirtying the database
Replaced GO with END BEGIN
Tested within rolled-back transaction
Removed BEGIN TRAN - ROLLBACK TRAN development wrapper
Here is the simple sql query that will help to insert column in database table without data loss.
Lets say CCDetails is the table in which we want to insert column GlobaleNote just before column Sys_CreatedBy:
declare #str1 nvarchar(1000)
declare #tableName nvarchar(1000)
set #tableName='CCDetails'
set #str1 = ''
SELECT #str1 = #str1 + ', ' + COLUMN_NAME
FROM Information_Schema.Columns
WHERE Table_Name = #tableName
ORDER BY Ordinal_Position
set #str1 = right(#str1, len(#str1) - 2)
set #str1 = 'select ' + #str1 +' into '+#tableName+'Temp from '+#tableName+' ; Drop Table '+ #tableName + ' ; EXEC sp_rename '+#tableName+'Temp, '+#tableName
set #str1 = REPLACE(#str1,'Sys_CreatedBy','CAST('''' as nvarchar(max)) As GlobaleNote , Sys_CreatedBy' )
exec sp_executesql #str1
Related
I know this may seem like a commonly asked question, but I have a unique situation which I can't find an answer to.
I have a simple SSIS package which I want to use to update a table from an Excel spreadsheet. I'll do this using an OLE DB command task which executes a stored procedure. However, it's not retrieving any column names to map. The error is similar to this:
The metadata could not be determined because statement 'insert into #TempTable ... ' uses a temp table.'
I understand why it's returning this error, and know the typical workarounds. However, the error is not coming from the stored procedure that the package calls. That proc doesn't use temp tables. It's a result of the update firing off a database trigger that calls another proc, which DOES use temp tables.
Unfortunately, the triggered proc causing the error is from the third party application I'm developing for, and cannot be modified.
Does anyone know of a solution that doesn't involve adding code to the offending proc?
Thanks!
Option 1, load Excel spreadsheet to a staging table, then use 'Execute SQL Task' to update your final table.
Option 2, create Script task, read from spreadsheet and update your final table, thus avoid going through data flow for column mapping.
Your best bet would be to stage the updates to a dedicated table (OLE DB Destination instead) and then have an Execute SQL Task as a successor event.
Within the Execute SQL Task, set up a cursor to shred the staging table and then call your stored procedure.
DECLARE CSR CURSOR
READ_ONLY
FOR SELECT Col1, Col2
FROM staging;
DECLARE #Col1 nvarchar(100)
, #Col2 nvarchar(100);
OPEN CSR;
FETCH NEXT FROM CSR INTO #Col1, #Col2;
WHILE (##fetch_status <> -1)
BEGIN
IF (##fetch_status <> -2)
BEGIN
EXECUTE dbo.MyProcedure #Col1, #Col2;
END
FETCH NEXT FROM CSR INTO #Col1, #Col2;
END
CLOSE CSR;
DEALLOCATE CSR;
The staging table approach occurred to me shortly after I posted this, and that's the route I went with. While not ideal for such a simple package, it does work. A cursor won't even be necessary...for this package, I can simply update the appropriate table from the staging table, no need for a stored proc. Thanks to you both!
I'm creating some views with a lot of references to tables in another database.
At some point the other database needs to change.
I want to make it easy for the next developer to change the scripts to use another database.
This obviously work like it should:
CREATE VIEW ViewName
AS
SELECT *
FROM AnotherDatabase.SchemaName.TableName;
But when I do:
DECLARE #DB CHAR(100)
SET #DB = 'AnotherDatabase'
GO
CREATE VIEW ViewName
AS
SELECT *
FROM #DB.SchemaName.TableName;
I get the error:
Msg 137, Level 15, State 2, Procedure ViewName, Line 3
Must declare the scalar variable "#DB".
I could do something like:
DECLARE #SQL ...
SET #SQL = ' ... FROM ' + #DB + ' ... '
EXEC (#SQL)
But that goes against the purpose of making it easier for the next developer - because this dynamic SQL approach removed the formatting in SSMS.
So my question is: how do I make it easy for the next developer to maintain T-SQL code where he needs to swap out the database reference?
Notes:
I'm using SQL Server 2008 R2
The other database is on the same server.
Consider using SQLCMD variables. This will allow you to specify the actual database name at deployment time. SQL Server tools (SSMS, SQLCMD, SSDT) will replace the SQLCMD variable names with the assigned string values when the script is run. SQLCMD mode can be turned on for the current query windows from the menu option Query-->SQLCMD mode option.
:SETVAR OtherDatabaseName "AnotherDatabaseName"
CREATE VIEW ViewName AS
SELECT *
FROM $(OtherDatabaseName).SchemaName.TableName;
GO
This approach works best when SQL objects are kept under source control.
When you declare variables, they only live during the execution of the statement. You can not have a variable as part of your DDL. You could create a bunch of synonyms, but I consider that over doing it a bit.
The idea that your database names are going to change over time seems a bit out of the ordinary and conceivably one-time events. However, if you do still require to have the ability to quickly change over to point to a new database, you could consider creating a light utility directly in SQL to automatically generate the views to point to the new database.
An implementation may look something like this.
Assumptions
Assuming we have the below databases.
Assuming that you prefer to have the utility in SQL instead of building an application to manage it.
Code:
create database This;
create database That;
go
Configuration
Here I'm setting up some configuration tables. They will do two simple things:
Allow you to indicate the target database name for a particular configuration.
Allow you to define the DDL of the view. The idea is similar to Dan Guzman's idea, where the DDL is dynamically resolved using variables. However, this approach does not use the native SQLCMD mode and instead relies on dynamic SQL.
Here are the configuration tables.
use This;
create table dbo.SomeToolConfig (
ConfigId int identity(1, 1) primary key clustered,
TargetDatabaseName varchar(128) not null);
create table dbo.SomeToolConfigView (
ConfigId int not null
references SomeToolConfig(ConfigId),
ViewName varchar(128) not null,
Sql varchar(max) not null,
unique(ConfigId, ViewName));
Setting the Configuration
Next you set the configuration. In this case I'm setting the TargetDatabaseName to be That. The SQL that is being inserted into SomeToolConfigView is the DDL for the view. I'm using two variables, one {{ViewName}} and {{TargetDatabaseName}}. These variables are replaced with the configuration values.
insert SomeToolConfig (TargetDatabaseName)
values ('That');
insert SomeToolConfigView (ConfigId, ViewName, Sql)
values
(scope_identity(), 'dbo.my_objects', '
create view {{ViewName}}
as
select *
from {{TargetDatabaseName}}.sys.objects;'),
(scope_identity(), 'dbo.my_columns', '
create view {{ViewName}}
as
select *
from {{TargetDatabaseName}}.sys.columns;');
go
The tool
The tool is a stored procedure that takes a configuration identifier. Then based on that identifier if drops and recreates the views in the configuration.
The signature for the stored procedure may look something like this:
exec SomeTool #ConfigId;
Sorry -- I left out the implementation, because I have to scoot, but figured I would respond sooner than later.
Hope this helps.
I use SQL Server 2012 and SQL Server 2008 R2.
I create a script from all object (tables / trigger / stored procedure / function ...) in my database.
I generated this script from SQL Server Management Studio. I can recreate my database with this scrips on the other server. But I miss all diagrams of my database after run my script for create another database.
Therefore, I need create backup script from all diagrams that exist in my database.
I need execute this script on the destination database for recreating all my diagrams.
I found this Link. but i need some thinks that create all script (Insert Command) automatically.
I have found a reasonable solution. The problem is that Management Studio cannot display more that 65535 characters for Non-XML data, and cannot be set to display more than 65535.
See code for documentation :)
Backup script:
-- 1. Read from DB, using XML to workaround the 65535 character limit
declare #definition varbinary(max)
select #definition = definition from dbo.sysdiagrams where name = 'ReportingDBDiagram'
select
'0x' + cast('' as xml).value('xs:hexBinary(sql:variable("#definition") )', 'varchar(max)')
for xml path('')
-- 2. Open the result XML in Management Studio
-- 3. Copy the result
-- 4. Paste this in backup script for #definition variable
Restore script:
declare #definition varbinary(max)
set #definition = 0xD0CF -- Paste 0x0 value from Backup script
-- Create diagram using 'official' Stored Procedure
exec dbo.sp_creatediagram
#diagramname = 'ReportingDBDiagramCopy',
#owner_id = null,
#version = 1,
#definition = #definition
Scripting your database does not include diagrams as they are not server objects in the same way as a table or stored procedure; they exist as data in the sysdiagrams table.
A similar question on SO asked How do you migrate SQL Server Database Diagrams to another Database?
The accepted answer is to copy the contents of the sysdiagrams table to the new database, so you could include the table contents in your script. The answer with the most up-votes has a link to a way of scripting diagrams.
I've tried backing up and then restoring a database to the same server, deleting the diagram I had created (I only had one) and then running the following query:
INSERT INTO database2.dbo.sysdiagrams
(
NAME
,principal_id
,version
,DEFINITION
)
SELECT NAME
,principal_id
,version
,DEFINITION
FROM database1.dbo.sysdiagrams
The diagram was successfully restored, however I did do this on a restored backup, I should really test it with a new database generated from a script.
UPDATE:
I scripted a database and then created a new database from it. When trying to rebuild the diagrams using an INSERT statement I got the error
So although it seems possible it's not trivial to create diagrams in a new database created from a script. Go with the answer given regarding scripting diagrams and modify it for your own needs.
Perhaps you can investigate further and post your own answer :)
Here's a quick & dirty method I use. Since the query window won't display the full varbinary(max) value of the definition field, but the XML editor will, I output the rows to XML as follows:
Run the following query on the server/database that contains the diagrams:
SELECT 'INSERT sysdiagrams(name,principal_id,diagram_id,version,definition) VALUES('''+name+''','
+CONVERT(varchar(2),principal_id)+','+CONVERT(varchar(2),diagram_id)+','+CONVERT(varchar(2),version)+','
+'0x' + CAST('' as xml).value('xs:hexBinary(sql:column("definition"))','varchar(max)') +')'
FROM RCSQL_ClaimStatus.dbo.sysdiagrams
FOR XML PATH
Click on the generated link to open the XML result, and ctrl-a & ctrl-c to copy all rows generated.
Paste that output back into your query window. I usually paste it between a pair of IDENTITY_INSERT's like this:
--TRUNCATE TABLE sysdiagrams
SET IDENTITY_INSERT sysdiagrams ON;
<row>INSERT sysdiagrams(name,principal_id,diagram_id,version,definition) VALUES('ERD1',1,1,1,0xD0CF11E0A1B11AE100000...)</row>
<row>INSERT sysdiagrams(name,principal_id,diagram_id,version,definition) VALUES('ERD2',1,2,1,0xD0CF11E0A1B11AE100000...)</row>
<row>INSERT sysdiagrams(name,principal_id,diagram_id,version,definition) VALUES('ERD3',1,3,1,0xD0CF11E0A1B11AE100000...)</row>
SET IDENTITY_INSERT sysdiagrams OFF;
Remove the row & /row XML tags from your inserts, and run them on the target server. You can truncate the sysdiagrams table if you're replacing all values with new values.
In SQL Server data Tools you have the deployment option "Block incremental deployment if data loss might occur", which I'd wager is a best practice to keep checked.
Lets say we have a table foo, and a column bar which is now redundant - has no dependencies, foreign keys etc etc, and we have already removed references to this column in our data layer and stored procedures as it's simply not used. In other words, we are satisfied that dropping this column will have no adverse effects.
There are a couple of flies in the ointment:
The column has data in it
The database is published to
hundreds of distributed clients, and it could take months for the
change to ripple out to all clients
As the column is populated, publishing will fail unless we change the "Block incremental deployment if data loss might occur" option. This option is at the database level, not table level however, and so due to the distributed nature of the clients, we'd have to turn off the "data loss" option for months before all databases were updated, and turn it back on once all clients have updated (our databases have version numbers set by our build).
You may think we could solve this with a pre-deployment script such as
if exists (select * from information_schema.columns where table_name = 'foo' and column_name = 'bar') BEGIN
alter table foo drop constraint DF_foo_bar
alter table foo drop column bar
END
But again this fails unless we turn the "data loss could occur" option off.
I'm simply interested as to what others have done in this scenario as I'd like to have granularity which doesn't currently seem possible.
So I've been accomplishing this task via the following steps:
1) Since we are going to make table #Foo, make sure to drop that table before moving forward if it exists.
2) In a pre-deployment script: If the column exists, create a temporary table #Foo and select all rows from Foo into #Foo.
3) Remove the column from #Foo
4) Delete all rows in Foo (now there will be no data loss since no data exists)
5) In a post-deployment script: If #Foo exists, select all rows from #Foo into Foo
6) Drop table #Foo
And code:
pre-deployment script
if(Object_ID('TempDB..#Foo') is not null)
begin
drop table #Foo
end
if exists (
select *
from sys.columns
where Name = 'Bar'
and Object_ID = Object_ID('Foo')
)
begin
select * into #Foo
from Foo
alter table #Foo drop column Bar
-- Now that we've made a complete backup of Foo, we can delete all its data
delete Foo
end
post-deployment script
if(Object_ID('TempDB..#Foo') is not null)
begin
insert into Foo
select * from #Foo
drop table #Foo
end
Caveat: Depending on your environment, it might be wiser to depend on versions rather than column & temp table existence in your conditionals
The PreDeployment script doesn't work the way you are hoping to use it because of the order of operations for SSDT:
Schema Comparison
Script generation for schema difference
Execute PreDeployment
Excecute generated script
Execute PostDeployment.
So of course, the schema difference is identified as part of #2 and appropriate SQL is generated to drop the column (including the check to block on data loss), before your manual pre-deployment script can 'get rid of it'.
If you take a look at the script generated behind the scenes to detect (and therefore block) on possible data loss, it checks to see if there are any rows by running something along the lines of this:
IF EXISTS (select top 1 1 from [dbo].[Table]) RAISERROR ('Rows were detected. The schema update is terminating because data loss might occur.', 16, 127)
This means the simple existence of rows will stop the column being dropped. We haven't found any way around this other than manually dealing with the problem outside (and before) the SSDT deployment, using conditional deployment steps based on version numbers.
You mention distributed clients, which implies you have some sort of automated publication/update mechanism. You also mention version numbers as part of the database - could you include in your deploy (before the sqlpackage.exe command I assume you are running) a manual SQL script? This is akin to what we do (ours is in Powershell, but you get the gist):
IF VersionNumber < 2.8
BEGIN
ALTER TABLE X DROP COLUMN Y
END
Disclaimer: in no way is that valid SQL, it's simply pseudo code to imply an idea!
I have three websites which uses an abstract database structure with tables like: Items, Places, Categories, etc... and stored procedures like GetItemsByCategory, GetRelatedItems, etc... Actually im using exactly the same database structure for these 3 different websites.
From a code perspective im using the same code for all websites (except the HTML which is specific foreach one), and all the common code is in few projects used by all websites, so everytime that i detect a bug (which is in all websites) i just fix it on one place (the common part used by all) and automatically all websites get the fix.
Actually im using Asp.net MVC3 and Sql server.
Everytime i want to extend some funcionality, and i need a new table, stored procedure or something related with database, i have to do the modification in each database.
Do you know any approach that i could use to be able to have the same flexibility and do database modifications only one time for all websites?
Do you think I'm using a good approach or i should use something different in your opinion?
If the databases are on a single server, you could generate the script for the procedure from Management Studio, and make sure to use the option to "check for object existence" (Tools > Options > SQL Server Object Explorer > Scripting). This will yield something like this (most importantly it produces your stored procedure code as something you can execute using dynamic SQL):
USE DBName;
GO
SET ANSI_NULLS ON;
GO
SET QUOTED_IDENTIFIER ON;
GO
IF NOT EXISTS (...)
BEGIN
EXEC dbo.sp_executesql #statement = N'CREATE PROCEDURE dbo.whatever ...
'
END
GO
Now that you have this script, you can modify it to work across multiple databases - you just need to swipe the #statement = portion and re-use it. First you need to stuff the databases where you want this to work into a #table variable (or you can put this in a permanent table, if you want). Then you can build a command to execute in each database, e.g.
DECLARE #dbs TABLE (name SYSNAME);
INSERT #dbs(name) SELECT N'db1';
INSERT #dbs(name) SELECT N'db2';
INSERT #dbs(name) SELECT N'db3';
-- now here is where we re-use the create / alter procedure command from above:
DECLARE #statement NVARCHAR(MAX) = N'CREATE PROCEDURE dbo.whatever ...
';
-- now let's build some dynamic SQL and run it!
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'';
SELECT #sql = #sql + '
EXEC ' + QUOTENAME(name) + '.dbo.sp_executesql N''' + #statement + ''';'
FROM #dbs;
EXEC sys.sp_executesql #sql;
Alternatively, you could create a custom version of my sp_msforeachdb or sp_ineachdb replacements:
Making a more reliable and flexible sp_MSforeachdb
Execute a Command in the Context of Each Database in SQL Server
I used to use a tool called SQLFarms Combine for this, but the tool doesn't seem to exist anymore, or perhaps it has been swallowed up / re-branded by another company. Red Gate has since produced SQL Multi Script that has similar functionality.
If you added a column to all your tables called websiteId you could just have one database. Store the unique websiteId in each site's web.config and just pass it with each request for data. Obviously each site's data is stored with their websiteId so data can be queried per website.
It means a bit of refactoring in your db and any calls to your your db, but once done, you only have one database to maintain.
Of course this is assuming your databases are on the same server...