I have few tables in production environment with data in it. I want to add new tables retaining the existing tables with data in it.
I tried using flyway but for that initial schema (schema with which existing production tables and data were created) also need to be set and drop the existing tables. However, in my use case I do not want my existing tables to be deleted.
Related
I've created a Data Fusion replication job to replicate some tables on a test database.
It works well at the beginning if I don't change the tables schema. But I've added a new column and that column is ignored from the replication job. I guess that if I create a new table, even that table would be ignored.
Is there a way to include schema updates (new table, update column field, new column etc...) inside an already running Data Fusion replication job?
I guess a possible solution would be to stop the currently running job and create a new one including new tables, new columns etc... but I'd like to avoid that a new job would replicate all the database again.
Any possible solution?
Unfortunately, Data Fusion Replication for SQL server currently does not support DDL propagation during runtime; you will need to delete and recreate the replicator pipeline in order to propagate any changes to schema to the new BigQuery table.
One way to avoid replicating existing data with DDL change is that , you can manually modify the BigQuery table schema (But BigQuery also has limited support for schema changes) and create a new replication job and disable replicating existing data(there is an option that let you choose whether to replicate existing data, default is true)
when generating a database in PowerDesigner I have the option to drop all Tables/Tablespaces and so on before generating the Database: Option to Select
But this Option isnt there when applying changes to a database. I need this, because some local databases could already have some of the Changes I made to the model and I cant drop all tables because of the data.
I'm not sure I understand the question, when you are ready to drop some tables instead of modifying them, but you don't want to drop all tables.
It seems to me that you could use Database > Apply Model Changes to Database with the Connect to a Data Source to compare the model with an existing database, and just generate the necessary modification scripts.
Maybe you could use a saved Selection to generate an Alter Script for most tables, and another to generate a Drop+Create script for some "transient" tables.
I have a scheduled SSIS package where it loads the data overnight into the data warehouse. Before loading, it drops the entire database and drops all the tables. But now I had a situation where I don't want to drop one table and want to do an incremental load using merge SQL statement. Because it is dropping the entire database, I won't be able to do that in the current scenario. If I change drop database to delete database, I think, I should be able to do incremental load on the table I want. Are there any possible complications of doing that. Can you foresee any problems if I change drop database to delete database, will I be missing something. Any thoughts highly appreciated. Thanks for your time.
As far as I know with delete database you only delete the rows whereas with drop database you delete all tables incl. the rows. If your logic works, you could do a delete database, then drop all tables except the one you want to keep.
A drop/delete of the database will remove all of the contents of the database. If the requirement is to retain a single table, you'll need to retain the schema and database that holds it as well.
If I'm understanding correctly, you're dropping the target database. Is this a STAGE database for the data warehouse? If so, you'll also have a TARGET (the main tables of the warehouse) that are loaded from STAGE. If this is the case, you should be able to run a MERGE statement from the newly STAGED table to the TARGET table.
We recently moved our database to a new server. However, at the time, we did not allow the Code First migrations to create the database. We used another tool to migrate the tables and data. The __MigrationHistory table was not moved during this time. The __MigrationHistory is a system table in our original DB.
I cannot seem to find a way to import or export the __MigrationHistory table so we can allow future migrations to take place.
The only other thought we had, is to have the application recreate the database and migrate the copied data to the new version of the DB. The only issue is we have millions of records to move and it is quite a long process.
I use the following script to move the EF MigrationHistory table from the system tables to the user tables (of a database tree structure):
SELECT * INTO [dbo].[TempMigrationHistory] FROM [dbo].[__MigrationHistory];
DROP TABLE [dbo].[__MigrationHistory];
EXEC sp_rename 'TempMigrationHistory', '__MigrationHistory';
This way I can export the table by choosing standard script/export option under SSMS.
(The complete description of handling this issue is here)
I want to create a simple versioning system and I was wondering about some simple yet elegant design patterns.
Since these tables are being used by lots of code so I need to leave them intact or all the existing queries will break. I was thinking:
-Duplicate the schema for existing tables but append Versioning to the names and a version col
-Deploy a script to copy all existing data into the Version tables as version 1.0
-Allow users to save as a new version or rollback to a previous one
-If they choose to rollback to an existing one I just update the original Tables data from the versioning tables for that version
-If they choose to save a new version, I create new records in the Version tables and update the original afterwards
I know this is not an efficient way so anyone have a more elegant solution?
If your database supports updatedable views then you can:
add versioning columns to the existing tables
rename the existing tables by appending some suffix to them (e.g. _NONVER)
create updatable views for every table with the original table names that will return all the columns expect the newly added versioning columns