Could anyone advise me how can sync data between SQL Server on premise to AWS using CDC or change tracking. I'm not sure,can I use SSIS for integration?
Thank you
The canonical AWS method for SQL Server-to-cloud CDC is Database Migration Services, it works with both self-managed SQL Server instances and RDS instances of SQL Server. Be aware it has some limitations and prerequisites for your SQL Server, so do an assessment up front that your scenario is applicable.
If you can't meet the prerequisites or constraints above, you can also consider using the popular open source tool Debezium, especially if you have to have a lot of custom hooks before or after the CDC, since it posts all the changes to a Kafka topic.
SSIS is an acceptable tool if you're only doing batch-level updates and don't need near-realtime replication and you want to fully control the synchronization, but there is a lot of overhead to developing and maintaining those types of packages. I wouldn't recommend it.
Related
We have 2 SQL Server databases. One of them is to be moved to AWS cloud.
Is Amazon RDS the ONLY approach to move the SQL Server to AWS?
What is the best way to automate data transfer of table data from the SQL Server that IS in the cloud to the SQL Server that is NOT in the cloud?
The volume and complexity of data is not high in our case. We need to migrate only ONE table and we'd like to automate it to run every night.
Would AWS Glue be useful for this? Would it help in loading data from INSIDE the cloud to OUTSIDE?
I also just found DMS - https://aws.amazon.com/dms/
But the homepage says "Migrate your databases **to** AWS with minimal downtime"
Can I do the opposite with DMS? Can I migrate data **FROM** AWS to outside AWS?
Amazon RDS is not the only approach to host SQLServer in the cloud. You can use an EC2 instance and install SQLServer manually. This could make sense in case you want to have sys_admin privileges. But be careful, as you would have to do patching all by yourself. I would not recommend doing that.
DMS is a good way to migrate your data every night. The service you are looking for is continuous data replication from DMS. Glue can also be used for this task. Glue's purpose was never to do database replication. Hence, it could be a pain monitoring the replication & debug in case of errors. That is why I would recommend DMS.
I wondered if I could ask for fellow Stack Overflow users for their advice? I have a scenario where we have multiple SQL Server databases with the same schema and we'd like to create a centralised database containing the data from each of the databases but with a stripe to differentiate the source of the data.
The centralised database would also be on SQL Server and would need updating in real or near real-time.
There are plenty of options available such as SQL Server Integration Services, SQL Server's Service Broker or possibly Microsoft Sync Framework and others too.
What's technology would you suggest to achieve this? If you're doing this successfully, would you mind sharing your experience?
Thank you in advance.
Reading some conflicting answers from Google searches, not sure if the answer is Yes, No, or maybe.
I thought it was pretty clear when reading this:
Amazon RDS currently does not support the following SQL Server features:
The ability to run Reporting, Analysis, Integration, or Master Data Services on the same server as the DB instance. If you need to do this, we recommend that you either install SQL Server on an EC2 instance or use an on-premise SQL Server instance to act as the Reporting, Analysis, Integration, or Master Data Services server.
Amazon now supports SSIS on RDS as of May 2020.
More info here
My company's product has a big relational database storage(around 20 TB) and we want to move all or some part of our database. Now I am a bit confused regarding which cloud SQL service will be suitable for me. I have read that some cloud storage only provides specific SQLs like MySQL ... which I cannot afford, the reason is that here we have lots of Transact-SQL scripts on our MSSQL server, which periodically updates the data.
So what should be my basic approach to achieve that, How can I compare between various cloud-based relational database storage like Google Cloud SQL, Azure SQL and Amazon RDS?
If your existing SQL server is Microsoft SQL Server then it probably makes the most sense to use a Microsoft SQL server offering in a cloud environment. That will help minimize any sorts of compatibility issues and also minimize the need to learn a new SQL server environment.
Amazon's RDS service lets you choose between various versions of MySQL, Postgres, Oracle, and Microsoft SQL Server. With regards to Microsoft SQL Server, Amazon offers an express edition, as well as SE and EE versions. As far as licensing goes, they offer both "bring your own" as well as pay-by-usage licenses. More information on the license details can be found in their RDS documentation. They also have a usage calculator that lets you estimate what your monthly costs will be based on the license you use, the size of the instance, etc.
Since Azure SQL is a Microsoft product I would hope that it's compatible with Microsoft SQL Server but I have never used it so I honestly have no idea if there are any compatibility issues. Microsoft also has some basic pricing information available for Azure SQL so you can work out an estimate of your costs if you go with them.
If you're concerned about any scripts working properly on one of the cloud platforms then I'd suggest creating a small test database that you can thoroughly test those scripts with. Upload that test database to the different clouds and test the scripts there to ensure they function properly. Amazon offers a very light-weight RDS instance for free so you can do basic testing without it costing you anything. Perhaps Azure offers something similar. Even if they don't or the free RDS instance isn't powerful enough for your testing needs, spinning up an instance for a day or two to run these sorts of tests shouldn't cost you very much.
I have OLAP analysis services related with relational database.when the data updated or changed in relational database I must make process operation manually in my project(project to build the OLAP from relational database) to push the new data in OLAP.
I want every change in relational database to affect the OLAP automatically.
Thanks in advance
This is called: "Proactive Caching feature in SQL Server Analysis Services"
Assuming that you are using Sql Server 2005 or later version this article help you:
Configure Proactive Caching feature in SQL Server Analysis Services 2005
To know more: What is and how does it work the Proactive Caching
In case you still have doubts, here another article:
Implement Proactive Caching in SQL Server Analysis Services SSAS
Hope it help
We use a SQL CLR proc call to the ADOMD to issue the XMLA command.