We have Oracle databases for different environment such as Development, System Integration Test, User Acceptance Test and Production. We want to make sure the DB objects such as tables, index, stored procedure etc. are synchronized after each deployment, so we are carrying out some evaluation on DB comparison tools.
But as our budget is really tight, we may eventually need to find a solution that is free.
We don't need to know which line of s.p. is different from the other. We just need to know what table / column / stored. proc. is missing,
Does Oracle come with some tools that help us perform this task? Can we write some scripts that can help us find out the difference between Oracle databases?
You can use comparision tool comparedb from ems which i have used in previous project.
There are other ways like script and a tool TOAD which can also be used. Additional info.
https://community.oracle.com/thread/361526
Related
There are many questions like this on Stack Overflow, but it seems to me that they already have the migration scripts in place. For example, insert and update statements are available, as such, Flyway can just use those scripts to create the tables in the target database and its data.
However, my question is that, what if we don't have those scripts? For example, tables are being created manually or with some other tools and the data are being inserted over the years with the bound application, now we want to switch to a different SQL database. Can Flyway be used as a tool to transfer all the tables and databases only with providing connections?
If the answer is no, how this sort of migration can be done and what are the best practices.
I did a search and went through Flyway documentation but they are all vague and doesn't give you a clear example of that. Some of these tools I found are used for Salesforce but I need a tool/library that possibly can be used in Java using JDBC connection, or other languages such as Python etc, as our databases - for security reasons - cannot be accessed directly and are cloud based.
For your information, we are using a range of databases PostgreSQL, Aurora MySQL, SQL Server.
No, Flyway can't do this sort of thing.
Flyway is a deployment tool. While it certainly can include data movement, as with the deployment of database objects, the scripts supporting data movement have to be completely idempotent or completely isolated in their deployment. Neither of these is lending itself to what you're talking about.
What you're talking about is something like Redgate SQL Compare along with SQL Data Compare. These two would allow you to compare two databases, identify the differences, then generate the necessary scripts. I'm aware of no open source tools that do all that, especially that do all that across multiple data platforms. And that tool only supports SQL Server (there is a second one for Oracle, but no others).
The thing is, if you're allowing deployments to occur using manual processes or 3rd party mechanisms, without going through source control as centralized management of your code, you can't use Flyway anyway. Flyway requires a consistent and stable process wherein it is the thing running deployments. Allowing, or even encouraging, drift through out-of-band deployments will break your Flyway deployments.
DISCLOSURE: I work for Redgate, but we're not the solution you're looking for.
I am developing a multi-tenant app. I chose the "Shared Database/Separate Schemas" approach.
My idea is to have a default schema (dbo) and when deploying this schema, to do an update on the tenants' schemas (tenantA, tenantB, tenantC); in other words, to make synchronized schemas.
How can I synchronize the schemas of tenants with the default schema?
I am using SQL Server 2008.
First thing you will need is a table or other mechanism to store the version information of the schema. If nothing else so that you can bind your application and schema together. There is nothing more painful than a version of the application against the wrong schema—failing, corrupting data, etc.
The application should reject or shutdown if its not the right version—you might get some blowback when its not right, but protects you from the really bad day when the database corrupts the valuable data.
You'll need a way to track changes such as Subversion or something else—from SQL you can export the initial schema. From here you will need a mechanism to track changes using a nice tool like SQL compare and then track the schema changes and match to an update in version number in the target database.
We keep each delta in a separate folder beneath the upgrade utility we built. This utility signs onto the server, reads the version info and then applies the transform scripts from the next version in the database until it can find no more upgrade scripts in its sub folder. This gives us the ability upgrade a database no matter how old it is to the current version. If there are data transforms unique the tenant, these are going to get tricky.
Of course you should always make a backup of the database that writes to an external file preferable with an human identifiable version number so you can find it and restore it when the script(s) go bad. And eventually it will so just plan on figuring out how to recover and restore.
I saw there is some sort of schema upgrader tool in the new VS 2010 but I haven't used it. That might also be useful to you.
There is no magic command to synchronize the schemas as far as I know. You would need to use a tool - either built in house or bought (Check out Red Gate's SQL Compare and SQL Examiner - you need to tweak them to compare different schemas).
Just synchronizing can often be tricky business though. If you added a column, do you need to also fill that column with data? If you split a column into two new columns there has to be conversion code for something like that.
My suggestion would be to very carefully track any scripts that you run against the dbo schema and make sure that they also get run against the other schemas when appropriate. You can then use a tool like SQL Compare as an occasional sanity check to look for any unexpected differences.
I am developing a data driven website and quite a lot of programming logic resides in database stored procedures and database functions. I found myself changing the stored proc/functions quite a lot in order to fix bugs or add new functionality. The data (tables) have remained mostly untouched.
The issue I am having is keeping track of versions of stored proc/functions. Currently I am incrementing version of whole database when I do a set of changes. As data is huge (10 Gb) I get issues having to run development version and release versions of databases in parallel.
I wish to put all the stored procs and functions in one database and keep data in one database, so that I can better manage the changes.
I am sure others would have encountered similar suggest and request suggestions on how to best handle this situation.
I would also recommend using source control keyword expansion in your stored procedures ($Version:$)
That way you can eyeball, grep, search syscomments, etc to see what version you have on your deployed database.
You can version just the schema dumps. In combination with source control keword expansion (as suggested by Rawheiser), you just take a look at what version you have in the database, generate a diff and apply it.
Also, there are several excellent tools to compare databases and their schemas, generate DDL scripts etc.: SQL Workbench, Power Architect, DDLUtils and Redgate SQL Compare, to name a few. SQL Compare is likely to work best with SQL Server, although all the others are FOSS and provide a higher ROI (in terms of time spent learning and what you can do with them) as they are platoform and RDBMS independent.
Finally, I have to say...I understand that the immediate results you get with logic in the DB are tempting, but if you've gone beyond more than a couple of procedures in the database, you're setting your self up for quite a lot of pain, sifting through what easily turns into spaghetti code and locking your application to a single database vendor. You might have your reasons, but I've been there and didn't like it very much. Logic can live very nicely in a different layer.
For source control you have several options:
Use a Visual Studio Database project.
Use SQL Server 2005's built-in support for source control
Use a third part tool such as SQL Compare
IMO Option 1. is preferable.
Working on a team where people are prone to amending dev SQL Server tables and forgetting about it, or preparing a change for deployment and having to wait for that deployment. This leaves our dev and live tables inconsistent, causing problems when SPROCs are pushed live.
Is there a tool whereby I can enter a SPROC name and have it check all tables referenced in it in the dev and live DBs, and notify of any differences?
I know two excellent tools for diffing SQL database structures - they don't specifically look inside stored procedures at their text, but they'll show you structural differences in your databases:
RedGate SQL Compare
ApexSQL's SQL Diff
Redgate also has a SQL Dependency Tracker which visualizes object dependencies and could be quite useful here.
Marc
For SQL Server 2005/2008, Open DBDiff works pretty well. The great part about this is that it's free. Also note that I am writing this answer for version 0.9 which currently works for SQL 2005/2008.
It'll show you the differences between the database schema between a source database you specify and the destination database you specify. There are also buttons you can click that can update or create the table that is in question.
I would recommend SQL compare and SQL Data Compare from Redgate Software. I worked with these tools for several projects and they did a great job. Documenting changes is also a good thing to do, but some changes are to complex to write your own SQL code for (including juggling data around between tables).
The redgate tools create scripts in a matter of seconds and those scripts are almost always correct (some older versions had a hard time with table dependencies in big databases, but when playing around with the statements (in a begin transaction / rollback) I was able to quickly fix those problems).
Another strong point in the redgate suites is that you can save your comparison project. This is especially useful when you don't want to convert a certain table (or data), you can exclude them. When loading the project the next time the software will automatically ignore those tables.
One disadvantage is the cost of the software (smaller companies I worked with did not want to buy the software). SQL compare and SQL data compare together will cost you about 800 dollars, but if you look at the time you will save when releasing you will save a lot of money. There is also a trial you can play around with (30 days I believe).
SQLDBDiff is a nice and user-friendly and lite tool.
SQLDBDiff supports SQL Server 2000 to 2016 and also SQL Azure.
SQLDBDiff available with both free with limited use and full with a trial.
More Screen
Try Microsoft Visual Studio Database Edition aka Data Dude (formerly for Database Professionals). It'll do a complete schema comparison and generate the necessary scripts to upgrade the target schema.
Of course, this shouldn't replace a proper build process ;-)
If you need a quick schema comparison tool for SQL Server, you should take a look at dbForge Schema Compare for SQL Server.
I've made a MssqlMerge utility that allows to compare (and merge) MSSQL database data and programming objects. It also allows to search for particular word or phrase across table definitions and programming objects.
Our clients use SQLServer/Oracle databases. Over the years, we've sent them many update scripts which they had to run manually. Most of the time, everything went smooth, but every now and then a script did not run completely to the end or had some errors in it (which weren't detected at the time of the upgrade). Also, sometimes even "smart users" added indexes/tables into those databases themselves, for whatever reason. Later on, those irregularities lead to problems.
Now I have been tasked to figure out a way to verify/validate our clients databases against our own database schema (tables, datatypes, indexes, views, ...). The output should be some kind of difference file indicating what is missing/what should not be in the database. I could do this in code (C++) from inside our application or I can create an external tool for just this one purpose.
Now before I start coding, I wanted to ask if there is already a tool out there that would produce the necessary results, or that at least could help me produce a decent xml file from our master-databases (Oracle and SQLServer)? Or is there a library which could help me write my own tool?
I've used this technique before and it doesn't require buying any tools.
Enterprise Manager has a "Create Script" feature. Perform this on your reference database and the comparison database. Select the appropriate options to generate scripts for the objects you care about. Next, just compare the two generated files with your favorite diff tool.
You can do a similar procedure with Oracle tools that let you export the DDL scripts.
There are three options using Red Gate's tools:
Have your client run the comparison.
You would need to convince your
clients to purchase a license of SQL
Compare and send them a schema
snapshot of your database.
Write an application of your own using Red
Gate's SQL Comparison SDK ($595 for
10 distributions) which can be run
at the client site.
Ask your client to send you a schema snapshot and
run the comparison yourself using your own
copy of SQL Compare. Red Gate
supplies a free schema snapshot tool
called SQL Snapper that will create
snapshots that can then be emailed to you
by your client. As this doesn't include any data, it may be something your client is willing to consider.
The SQL Snapper tool and SQL Comparison SDK sample code can be downloaded from our labs.red-gate.com website.
Oracle compatibility is now available in the form of an Early Access Build. If you're interested or would like to try out the tool visit the product page. You can use this for free until the full release of the tool.
David Atkinson, Product Manager, Red Gate Software.
We use Redgate SQL Compare for this and it's served us well over the years.
We also use Redgate SQL Data Compare for comparing the content of lookup tables.
The folks at redgate have a great tool called SQL Compare.
Can you create a schema dump like MySQL's SHOW CREATE TABLES?
If you're on Windoze, I have used Advanced Query Tool for years, and can attest that, for the money, it does more than anything else. In particular, it will generate a diff report between databases. It is ODBC/VB6, and can run against dozens of databases. Check it out. (No, I am not of QueryTool nor do I own any part of it, just a happy client.)