I'm aware of the various options in place for migrating a single database up to Azure. My problem is that these all only seem to cater for a single database at a time. However, I have a db per tenant model with over 2000 databases to migrate and not a lot of time to play with.
Can anyone point me in towards the best (ie fastest) way of doint this?
In the end we accomplished this with Powershell and the Azure API. Essentially batch creating bacpacs on the source server, uploading them to blob storage then importing them into Azure SQL server pools.
If I was facing the same challenge now I'd take a look at the Azure Database Migration Service - https://azure.microsoft.com/en-gb/services/database-migration/
I am also facing this problem and am going down the route of using the Visual Studio data compare tool.
All my tenant databases have the same schema so I made an empty template database in Azure, and just use the CREATE AS COPY command to make a new one each time ready for receiving the migration.
Then I ask Visual Studio to compare the empty database with the live database and automatically insert the data for me.
Seems to be working well so far, there's very little manual steps needed and it doesn't involve using the Azure Portal, or blob storage or creating databases outside of the elastic pool which is great. But the overall time will be slow to migrate data for all the databases.
Related
I want to move all data from one Azure SQL Server to different Azure SQL Server which more than 90 days old, and after moving need to delete moved data from first Azure SQL Server.
I want to run these steps on daily basis.
I am new to Azure and able to do same with Azure Data Factory. Can you please suggest any other best suited approach?
You are already using the best approach.
Azure Data Factory is an easy to use when it comes to extract and copy the data between the services. It also provide scheduling the triggers, i.e., triggering the copy pipeline after specific interval of time or any event. Refer Create a trigger that runs a pipeline on a schedule.
If the volume of data is large, you can re-configure the Integration Runtime (IR) resources (Compute type and Core count) to overcome the performance issue, if required. Refer below image.
I need to regularly (but incrementally) sync (one way) the contents of a set of SQL Server Azure tables to a PostgreSQL Azure instance.
Here are some of the avenues I've considered:
Linked server from SQL Server. No go. Apparently Azure doesn't support linked servers.
Foreign Data Wrapper from PostgreSQL. No go. PostgreSQL on Azure only supports the postgres_fdw, not the needed tds_fdw.
Azure Data Factory. No go. The data copy process doesn't work incrementally, and the sink pipeline component doesn't support PosgreSQL.
Commercial replication solutions. Too expensive for a startup and most aren't hosted.
SymmetricDS or ReplicaDB. These might work, but aren't hosted so we may or may not save time over building a custom solution after all the time and effort of configuration and debugging.
Am I missing an obvious solution?
Congratulations, you solved your problem. It will be better that if you could share us more detail about your simple replication system.
I'm relatively new to Azure and am having trouble finding what options are out there for connecting to an existing SQL database to push data into it.
The situation is that we have an external client who needs to connect to our Azure SQL database to push data into it, on an on-going basis. We can't give them permission to get into our database, so we're looking at what we can do allow data in. At this point the best option seems to be to create a web service deployed in Azure that will validate the data and then push it into our database.
The question I have is, are there other options to do this in an easier way? Are there Azure services or processes that can be set up to automatically process a file and pull the data into a database? Any other go-between options when each side has their own database and for security reasons can't just open up access to it?
Azure Data Factory works great for basic ETL. If neither party can grant direct access, you can use an intermediate repository like Blob Storage to drop csv/xml/json files for ingestion. If they'll grant you access to pull, you can setup a linked service that more or less functions the same as a linked server in MSSQL. As of the last release ADF now supports Azure hosted SSIS packages too.
I would do this via SSIS using SQL Studio Managemenet Studio (if it's a one time operation). If you plan to do this repeatedly, you could schedule the SSIS job to execute on schedule. SSIS will do bulk inserts using small batches so you shouldn't have transaction log issues and it should be efficient (because of bulk inserting). Before you do this insert though, you will probably want to consider your performance tier so you don't get major throttling by Azure and possible timeouts.
Since I initially created an Azure SQL Server database while building an app for a client, they have changed Azure subscriptions. The client wants the database migrated to a new account and eventually deprecate use on the old account when we are done with the new version of the app. In the meantime, they want any updates from the old database duplicated into the new database.
My question is, is there a way to update two databases, one a copy of the other, at the same time on separate Azure accounts?
I've read into active geo-replication but that looks like it can only be done between two databases in the same Azure subscription.
Thanks for the help.
Active geo-replication indeed is only possible inside a subscription, if you are using Azure SQL that wouldn't be possible with it. You could possibly use some kind of replication using on Premise SQL, but that would be pretty hacky. You would want to work with Azure SQL Data Sync.
If you are using a VM with the SQL Server you could create AlwaysOn databases for that purpose.
But if the question is: My question is, is there a way to update two databases, one a copy of the other, at the same time on separate Azure accounts?. You could probably code your application in such a way to write to 2 databases, but again, you would also need to keep track of operations
I'm in the process of migration from dedicated servers to Azure. In my existing SQL Server, I have a few jobs that move data from live database to archives.
From what I have read so far, in Azure you cannot use cross database scritps. The other options I have seen include Azure SQL Data Sync, Azure Factory and maybe SSIS. I have to note that there's some logic on what data is archived and I need the ability to specify this in the query.
Has anyone some experience and what would you recommend?
Thanx
You can use the copy feature inside of data factory to do this now directly in Azure.
Azure Data Factory