I am working on a pre-existing MS SQL Server database that was not designed very well. Indexes and primary keys were missing in most cases. There were NO foreign key constraints.
I made a local copy of the DB (so as not to work on the live version), and fixed most of the problems. I now want to syncronize my changes to the production server,
but...
Because there were no previous foreign key constraints there are key mismatches in the tables, thousands of them. I have tried a few synchronization tools, but none of them will allow to ignore or resolve FK mismatches. (in SQL Server Management Studio it's called "Check Existing Data On Creation Or Re-Enabling: No")
Is there a way to auto-resolve the discrepancies of the old database?
Try to use SQL DATA COMPARE from red-gate to syncrhonize data.
https://www.red-gate.com/dynamic/downloads/downloadform.aspx?download=sqldatacompare
You can also try SQL compare to syncrhonize structure, before synchronize data if SQL data compare don't work.
SQL compare
What do you mean by "auto-resolve"?
Existing data is "bad" - that is, the constraints you are trying to impose are violated. There is no way to make your data "good" without modifying it. And there is obviously no way to decide automatically how to fix the data.
So, the best thing you could do is to analyze the data, find out how to correct it, do the corrections manually, and then add the constraints.
You could also just delete all the inconsistent rows (probably a bad idea, if you need the data), or force the server to ignore the constraints for existing data (definitely a bad idea).
If you just want to drop the inconsistent data, I'd suggest you to write (or generate, if there're lots of foreing keys) SQL scripts like this:
DELETE a FROM a LEFT JOIN b ON a.b_id = b.id WHERE b.id IS NULL
ALTER TABLE a ADD CONSTRAINT FK_a_b_id FOREIGN KEY (b_id) REFERENCES b (id)
Related
I have a project that run some integration tests. The project creates a new database each time and run the tests against this new db. Recently I moved this project to a new server and I'm having some issues when delete data from the db.
I execute the following query:
DELETE FROM TABLE1;
DELETE FROM TABLE2;
DELETE FROM TABLE3;
On the server A everything works as expected, however using server B I'm getting the following error:
The DELETE statement conflicted with the REFERENCE constraint
"FK_....". The conflict occurred in database
"TestDB", table "Table1", column
'...'.
Both servers have the same version of SQL server
Microsoft SQL Server 2012 (SP1) - 11.0.3401.0 (X64)
Jan 9 2014 13:22:15
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows NT 6.2 <X64> (Build 9200: ) (Hypervisor)
Both Schema and data are the same and DB are created using the same process.
Any Ideas?
Actually, it is possible to have the same schema in both places, and even the same data, and experience different behavior. How? Because constraints (CHECK and FOREIGN KEY) can be disabled. Yup.
To see if the Foreign Key in question is enabled or disabled, just run the the following, making sure to put the name of the FK in question in the WHERE clause:
SELECT *
FROM sys.foreign_keys
WHERE [name] = N'{name_of_FK_in_question}';
Look at the column named is_disabled. I suspect that it is set to 1 on Server A (where everything "works") and is 0 on Server B (where things "don't work"). I put "works" and "don't work" in quotes, because if this is truly the case, then reality is the opposite of what you are experiencing. Meaning, the system with the FK enabled and getting the error is actually working as that is what FKs should do. The system not getting an error is possibly allowing orphaned (i.e. bad) data in.
To enable the FK, run the following:
ALTER TABLE {table_name}
WITH CHECK -- verifies the data currently in the table
CHECK CONSTRAINT [{name_of_FK_in_question}];
Of course, if the bad data is there, you either need to:
Delete the bad data first, or
Specify WITH NOCHECK on the ALTER so that it will accept the bad data:
ALTER TABLE {table_name}
WITH NOCHECK -- accept the bad data aleady there
CHECK CONSTRAINT [{name_of_FK_in_question}];
However, this does not solve the issue 100%. If you run that SELECT query (noted above) again, you should see that the is_disabled field is now set to 0. BUT, if you look at the is_not_trusted field, it will be set to 1. If a constraint is enabled yet not trusted, it will enforce its rule as expected, but the Query Optimizer (Q.O.) will ignore it, which is generally not a great thing as the constraints are actually used not just to enforce data integrity rules, but also as clues for the Q.O. to logically reduce certain steps in some queries (i.e. they sometimes help increase performance). In order to get the constraint to be "trusted", you will have to delete the bad data and verify all constraints on the table via:
ALTER TABLE {table_name}
WITH CHECK -- verifies the data currently in the table
CHECK CONSTRAINT [{name_of_FK_in_question}];
However, if for some reason you need the "bad" data, then you will just have a Foreign Key that enforces data integrity but has no potential for improving performance (which is still much better than having no FK defined :).
For more info, please see the MSDN page for ALTER TABLE.
For the sake of completeness I will also mention that it is possible that on Server A (where there is no error), that the FK is defined with the option ON DELETE CASCADE which auto-deletes and related data prior to removing the row(s) from the table that is being deleted from, while Server B (where there is an error) does not have the ON DELETE action specfied (or is specified as NO ACTION). This, however, is something that should show up when doing a schema compare (unless specifying to ignore the ON DELETE and ON UPDATE actions), whereas whether the constraint is enabled or disabled is more likely to be ignored in a schema compare.
There is a foreign key (named in the error message) which is preventing you from deleting an item from Table1 because it would violate the foreign key (i.e. a there would be a row in the other table which referenced the row that you were about to delete)
The difference between the two databases is going to either be:
The first database didn't have the foreign key
The first database didn't have any rows in the table constrained by the foreign key
To find which table the foreign key is on see the question How can I list all foreign keys referencing a given table in SQL Server?
EXEC sp_fkeys 'TableName'
Like Justin says, either the keys or data isn't the same, if it works for your case, then setting the keys to cascade of delete will remedy this, but you must identify the keys first.
I have a tool which uses SQL scripts to apply changes to a customer database. Often this invloves changing a column definition (datatype etc). The problem is that often there are primary keys applied by the user that we don't know about (and they don't remember), which trips up the process (eg when changing columns belonging to the indexes or primary keys).
The requirement given to me is that this update process should be 'seamless', with no human involvement to prepare the ground. I have also researched this on this forum, and as far as I can see my particular question has not yet been asked.
I know how to disable and then later rebuild all indexes on a database, and even those only in certain tables, but if the index is on a primary key I still can't change any column that is part of the primary key unless I explicitly drop the PK by name, and later recreate it explicitly, which means I have to know about it at code-time. I can probably write a query to find the name of the primary key on a table if one is there, but how to know how to recreate it?
How can I, using Transact-SQL (or PL/SQL), detect, drop and then recreate the primary keys on given tables, without knowing at code time what they are or what columns belong to them? The key is that the tool cannot know in advance what the primary keys are are on any given table, nor what they comprise. The SQL code must handle this itself.
Better still would be to detect if a known column belongs to a primary key, then drop and later recreate that after I have changed the column.
This needs to be done in both Oracle and Sql Server, ideally purely with SQL code.
TIA
I really don't understand why would a customer define his own primary keys for the tables? Moreover, I don't understand why would you let them? In my world, if customer changes schema in any way, this automatically means end of support for them.
I will strongly advise against dropping and recreating primary keys on production database. Any number of bad things can happen, leading to data loss.
And it's not just the PKs, you will have to drop the foreign key constraints first. And FKs may reference not only the PKs but the unique constraints as well, so yao have to deal with those as well.
Your best bet would be to create a new table with the required schema, copy the data, drop original table and rename the new one. Of course, you will have to handle the FKs, but it's easier. Check this link an example:
http://sqlblog.com/blogs/john_paul_cook/archive/2009/09/17/script-to-create-all-foreign-keys.aspx
Recently I've been trying to restructure an old database that was not designed with filegroups (just the default PRIMARY) and, among other things, move a bunch of tables to a new Data filegroup residing on a SAN. I know how to migrate the data:
ALTER TABLE MyTable
DROP CONSTRAINT PK_MyTable WITH (MOVE TO [MyDB_Data])
ALTER TABLE MyTable
ADD CONSTRAINT PK_MyTable
PRIMARY KEY CLUSTERED (MyID)
ON [MyDB_Data]
But damned if this isn't the most tedious work I've ever had to. And it's error-prone. At one point I was halfway (I assume, since there's no progress indicator) through moving a 30 GB table before I realized that I had accidentally included one of the value columns in the PK. So I had to start all over again.
It's even worse when the table has a lot of dependencies. Then I can't just drop the primary key; I have to drop and recreate every foreign key that references it. This leads to hundreds of lines of boilerplate; multiply by 100 tables and it becomes downright asinine. My wrists hurt.
Has anybody come up with a shortcut for this? Are there maybe any tools out there (priced with the notion of one-time-use in mind) that can do it? Perhaps somebody here has had to go through this process before and wrote their own tool/script that they wouldn't mind sharing?
SSMS won't do it, obviously - it can only generate migration scripts for non-clustered indexes (and they have to be indexes, not UNIQUE constraints - on at least a few tables, for better or for worse, the clustered index is not actually the primary key, it's a different UNIQUE constraint).
It's not that the syntax is so complicated that I can't write a code gen for it. At least for the basic drop-and-recreate-the-primary-key part. But add in the overhead of figuring out all the dependencies and generating drop/recreate scripts for all the foreign keys and this starts to feel like it's just over that threshold where it's more work to automate and fully test than it is to just do every table manually as with the example above.
So, the question is: Can this process be automated in any reasonably straightforward way? Are there any alternatives to what I've written above?
Thanks!
The simplest way to do it, IMO, would be to use one of the schema comparison tools (My tool, red gate's SQL Compare, Apex SQL Diff as a couple of examples) to create a script of your schema. Then, edit that script to create all the objects, empty, in the right file groups. Having done that, you can then use the same tools to compare your new DB with correct filegroups, and they will generate the scripts to migrate the data for you. It's worth testing with multiple ones to find which is the most appropriate for you.
I am trying to sync up the schemas between to different databases. Basically, I ran tasks->Generate Scripts with SQL Server Management Studio (2005) on both databases and am comparing the output with a diff tool.
For some reason, one script adds the constraint WITH CHECK and one WITH NO CHECK, followed by both constraints being re-enabled.
I for the first database I get:
ALTER TABLE [dbo].[Profile] WITH CHECK ADD CONSTRAINT [FK_Profile_OrganizationID] FOREIGN KEY([OrganizationID])
REFERENCES [dbo].[Organization] ([OrganizationID])
GO
ALTER TABLE [dbo].[Profile] CHECK CONSTRAINT [FK_Profile_OrganizationID]
GO
The second database generates as
ALTER TABLE [dbo].[Profile] WITH NOCHECK ADD CONSTRAINT [FK_Profile_OrganizationID] FOREIGN KEY([OrganizationID])
REFERENCES [dbo].[Organization] ([OrganizationID])
GO
ALTER TABLE [dbo].[Profile] CHECK CONSTRAINT [FK_Profile_OrganizationID]
GO
So I have two questions:
Is the end result the same?
(Edit:
It seems that a lot of people are picking up on only the first statement of the two scripts. I am interested in the end result of the entirety of both scripts.)
If the end result is the same, why does Management Studio generate them differently for different databases?
The end result is not the same!
SQL Server will not trust the uniqueness of the FK is it is not checked. This means additional processing is required if you use the column in a query.
Long story short is that you should get SQL Server to check the column so it's considered trusted.
As for why they're different from different servers, check the isnottrusted column in sys.foreign_keys. This may affect what SSMS is generating?
For more of a rant on this, check my other answer that relates to FK & NO CHECK/ CHECK options.
Yes the two scripts are different
WITH CHECK will check existing data against the new constraint.
WITH NOCHECK will not check existing data against the new constraint. This will allow you to have child records without a corresponding parent.
EDIT:
As for why SSMS is doing this I have no idea
Both are SQL Server 2005 servers? As the result is the same, the code generation tool maybe use different routines based in different versions of the product
I have the same database running on two different machines. The DB's make extensive use of Identity columns, and the tables have clashed pretty horribly. I now want to merge these two together before sorting out the undelying issue which I may do by
A) Using GUIDs (unweildy but works everywhere)
B) Assigning Identity ranges, kind of naff, but means you can still access records in order, knock up basic Sql and select records easily, and it identifies which machine originated the data.
My question is, what's the best way of re-keying (ie changing the primary keys) on one of the databases so the data no longer clashes. We're only looking at 6 tables total, but lots of rows ~2M in the 3 tables.
Update - is there any real sql code out there that does this, I know about Identity Insert etc. I've solved this issue in a number of in-elegant ways before, and I was looking for the elegant solution, preferable with a nice TSQL SP to do the donkey work - if that doesn't exist I'll code it up and place on wiki.
A simplistic way is to change all keys on the one of the databases by a fixed increment, say 10,000,000, and they will line up. In order to do this, you will have to bring the applications down so the database is quiet and drop all FK references affected by this, recreating them when finished. You will also have to reset the seed value on all affected identity columns to an appropriate value.
Some of the tables will be reference data, which will be more complicated to merge if it is not in sync. You could possibly have issues with conflicting codes meaning the same thing on different instances or the same code having different meanings. This may or may not be an issue with your application but if the instances have been run without having this coordinated between them you might want to check carefully for this.
Also, data like names and addresses are very likely to be out of sync if there wasn't a canonical source for these. You may need to get these out, run a matching query and get the business to tidy up any exceptions.
I would add another column to the table first, populate that with the new Primary key.
Then I'd use update statements to update the new foreign key fields in all related tables.
Then you can drop the old Primary key and old foreign key fields.