I maintain a Windows based application backed by SQL Server DB so there is a set of SQL entities, like tables, views. With time I add new features and fix bugs so schema of the tables and views changes. Once I need to deploy a new version of the application I deploy the DB part by relying on DacPac/DacFx which automatically generates a difference between already deployed DB and the supplied DacPac so the already deployed DB is altered to match the DacPac's content. This way I don't have to write a code which compares 2 schemas and then generates a difference - DacFx does that for me.
That works well but now I need to expand the application so it also supports SQLite DB, I will for sure have to create a new application layer working with SQLite which is doable but one place I need help with is being able to create and maintain SQLite DB schema in the same way I do for SQL Server with DacPac/DacFx so a difference in schemas is computed and applied. While doing that I ideally want do write the SQL schema once so it could be applied to SQL Server as well as SQLite. Ideally, I need to generate SQLite schema based on the schema specific to SQL Server.
I looked into sqldiff which is capable of generating difference between 2 SQLIte DBs and thought I could:
use a technique from here to migrate SQL Server schema to SQLite
generate a temporary SQLite DB based on the generated above schema
compare the above deployed temporary DB to an existing SQLite DB by using the sqldiff and finally apply the difference to the target SQLite DB
but the sqldiff, as stated in the Limitations section:
The sqldiff utility is not designed to support schema migrations
In addition it has limitations around views:
The sqldiff.exe utility does not (currently) display differences in
TRIGGERs or VIEWs.
So I interpret that like that tool could probably be used for some migration cases but it is not really recommended.
How do you suggest generating and applying the schema differences?
I'm also interested to know how others solve the task of incrementally updating schema of their SQLite DB even if I take SQL Server completely out of equation and would instead maintain SQLite schema, in the source code, only. Does everyone create their own schema comparing tools instead of using something similar like DacFx in SQL Server world?
Related
I recently moved a SQL Server 2012 database from an old web server to Azure SQL. I also keep a copy of the database on my personal machine which is now running SQL Server 2019. During development, I frequently make changes to tables on my local machine and then need to transfer those tables to the server. I used to do this using a Visual Studio SSIS package. It was very easy. I used the "Transfer SQL Server Objects" task to select one or more tables, specify whether the existing tables should be dropped first, and replaced the tables on the server. The "Transfer SQL Server Objects" task does not work when trying to transfer objects to Azure SQL because it uses the "USE" statement. There must be an easy way to transfer tables to an Azure SQL database. I've used the "Microsoft Data Migration Assistant" and it works great for the initial migration, but does not allow you to replace tables. I feel like I am missing something very obvious because transferring tables is a routine task and there must be an easy way to do this with Azure SQL.
Manually managing and synchronizing different database versions can be time-consuming. The Schema Compare addon facilitates database comparison and provides you complete control when syncing them — you may filter particular differences and categories of differences before making modifications. The Schema Compare addon is a trustworthy tool that will save you time and code.
Hence, the Schema Compare extension provides an easy-to-use experience
to compare two database definitions and apply the differences from the
source to the target. MSFT Ddocument which could be usefull : Schema
Compare extension- here and How to: Use Schema Compare to Compare
Different Database Definitions - Here.
I'm modeling a new microservice architecture migrating some part of a monolithic software to microservices.
I'm adding a new PostgreSQL database and the idea is in the future use that database but for now I still need to keep updated the old SQL Server database and also synchronize the PostgreSQL database if something new appears in the old database.
I've searched for ETL tools but are meant to move data to a datawarehouse (that's not what I need). I just can't replicate the information because the DB model is not the same.
Basically I need a way to detect new rows inserted in the SQL Server database, transform that information and insert it in my PostgreSQL.
Any suggestions?
PostgreSQL's foreign data wrappers might be useful. My approach would be, to change the frontend to use PostgreSQL and let postgreSQL handle the split via it's various features (triggers, rules, ...)
Take a look at StreamSets Data Collector. It can detect changes in SQL Server and insert/update/delete to any DB that has a JDBC driver including Postgres. It is open source but you can buy support. You can also make field changes/additions/removals/renaming to the data stream so that the fields match the target table.
Does Visual Studio SQL Server Data Tools (any version) support changing the default comparison mappings, for objects with different names, and/or in different schemas?
For example, I want to compare tables in two SQL Server databases, but the tables are in different schemas in each database:
database1.SchemaA.MyTable
database2.SchemaB.MyTable
Can these tables be compared with each other using SSDT, and if so how can I do it? I cannot find any mapping options or documentation.
As far as I know, no, you cannot do the comparison using SSDT if they are in different schema. The database doesn't matter though, you could specify the connection and database when running schema compare, but not the schema.
One workaround to get it working in less than ideal way would be to:
deploy your SchemaA.MyTable locally
after the schema has been deployed successfully, replace the table definition of SchemaA.MyTable with that of SchemaB.MyTable
Run sqlcompare with your project as the source and the local db (where you deployed your project in step 1) as the target.
What kind of database the Sql Server Migration Assistant uses as an internal data repository and stores it in the source-metabase.mb file?
I guess that this it is one of standard tool that I could use to open and edit some entries (I need to autamatically add some custom scripts for tables with BLOBs data migration )
You could also just suggest the way how to check most popular database formats: SqlServer Compact, MySQL, Access..
it is one of standard tool that I could use to open and edit some entries
I would not count on it :) It was a proprietary metadata format that has nothing to do with DB products that SSMA supports. It can store metadata for representing Oracle and also SQL Server among others, obviously formats are not connected with file structure that actual DBs use. SSMA format has no open docs, also it may fail to synchronize your changes after manual intervention if you reverse engineer it (due to the fact it was designed just as the migration tool to target SQL Server and was supposed to mostly create new objects there based on source database counterparts).
Can you just write some stored procedures or triggers in your database? For most DBs metadata is exposed as special tables/views anyway. Probably you need to do it only for SQL Server as it's your target db after migration, right? Looking into ways to directly parse or manipulate files managed by "big" DB (like SQL Server or Oracle) doesn't seem to be a good idea for most scenarios. (except digital forensics, for example)
SQL Server metadata related views are here and functions are here. You may profile your SQL Server instance while connecting to it with SSMA just to get some feel what it does to extract metadata (object names, columns of tables, source of SPs etc.)
Data manipulation is pretty much clear from the DB side if you need it too.
We are trying to build a data access layer for using both Oracle and SQL Server (not at the same time).
We use EF Model-first for creating the model and the create the SQL scripts for building the database. Our first thought was to create 2 EDMX files, one for each type, and use the appropriate one depending on the client's need. We are using the Oracle and SQL Server database generation workflow and DDL generation template to create the scripts for each database.
Our main problem is when the database schema changes we do not want to drop and recreate the DB but only create the migration scripts to update the DB base on our model (clients have many data that will be lost).
We use EF power pack for extracting the migration scripts for SQL Server but there is nothing like it for Oracle.
We want help to find a good data layer (1 EDMX for both Oracle and SQL Server if it's possible and not complicated) and a good way to generate database changes from our model to update existing client DBs in case of a new application release
We found this as a starting point
http://msdn.microsoft.com/en-us/data/ff830362 but there is not mention for Oracle support.
We have tried code-first and EF Migrations but Oracle failed us again on the DB creation and migration.
Any recommendation on how we can accomplish this?
Thank You
There is no way to have single EDMX for both SQL Server and Oracle. EDMX consists of three parts CSDL (entity definition), SSDL (database definition), MSL (mapping between those definitions). SSDL must always target concrete database so you need at least separate SSDL for Oracle and SQL Server and if you are lucky you will not need separate MSL as well (mapping must be exactly same which will probably not happen if you are using any tool to generate the database).
So you always need at least part of EDMX file for second DB and manually maintain it.
If you need DB migration supporting Oracle you must look for tool from Oracle (or third party). For example RedGate offers tools supporting schema migration for both SQL Server and Oracle.
Visual Studion Premium and Ultimate edition also offers tools for comparing database schemas. It by default supports only SQL Server but Toad Extensions should add support for Oracle as well.
Once you have any of these tools you just need to compare schema deployed on customer server with your new schema and the tool should create migration script for you.
The best article I found on this topic is from Paul Reynolds Blog.
Try to go through from part 5 to part 9.
There are so many gotchas about Oracle mentioned there... is very helpful!