I wondered if I could ask for fellow Stack Overflow users for their advice? I have a scenario where we have multiple SQL Server databases with the same schema and we'd like to create a centralised database containing the data from each of the databases but with a stripe to differentiate the source of the data.
The centralised database would also be on SQL Server and would need updating in real or near real-time.
There are plenty of options available such as SQL Server Integration Services, SQL Server's Service Broker or possibly Microsoft Sync Framework and others too.
What's technology would you suggest to achieve this? If you're doing this successfully, would you mind sharing your experience?
Thank you in advance.
Related
Could anyone advise me how can sync data between SQL Server on premise to AWS using CDC or change tracking. I'm not sure,can I use SSIS for integration?
Thank you
The canonical AWS method for SQL Server-to-cloud CDC is Database Migration Services, it works with both self-managed SQL Server instances and RDS instances of SQL Server. Be aware it has some limitations and prerequisites for your SQL Server, so do an assessment up front that your scenario is applicable.
If you can't meet the prerequisites or constraints above, you can also consider using the popular open source tool Debezium, especially if you have to have a lot of custom hooks before or after the CDC, since it posts all the changes to a Kafka topic.
SSIS is an acceptable tool if you're only doing batch-level updates and don't need near-realtime replication and you want to fully control the synchronization, but there is a lot of overhead to developing and maintaining those types of packages. I wouldn't recommend it.
Following is the exact scenario in my application:
SQL Server database is hosted on-premise locally in US office for
development purpose.
Developers are distributed in 3 different regions (US, India and
Australia).
Developers from India and Australia faces a lot of delay while
trying to access the database from application.
In order to resolve the above issue, we identified a following approach -
Create a VM and install SQL Server there.
Restore database over there.
US developers would continue using the database deployed in their
environment.
India and Australia developers would use the SQL database instance
hosted in Azure VM.
In order to enable synchronization of data and schema between these two databases (Azure VM SQL, and On-prem SQL instance), we are planning to use Azure Data Sync.
I believe most of the things in above scenario are subject of research. But guidance of someone who has already worked on similar things would be very much helpful. Also, we are not using Azure SQL because that would require changes in database schema, as its very old and legacy database
Could you please suggest if the above approach is ideal or not? Note, this is only for the ease of development, and we are not moving our production database outside on-prem setup.
I would not attempt to use Azure Data Sync here, first because you are not otherwise using Azure SQL Database, and second because it's not intended to sync schema changes like this.
Instead pick a primary replica for data changes, and periodically ship and restore backups to refresh the secondary instance. For schema changes, use SQL Server Data Tools and your Source Code Repository (Azure DevOps) to manage the changes.
I have about 20 reference tables in my application which is running in an Microsoft hosted SQL database on Azure. I think that this is currently called a SQL Azure database but I'm having difficulty coming up with Microsoft naming.
The data in my reference tables changes very rarely. Is there any way that I can identify these as tables that should be stored in memory. If not mistaken I think this feature is offered on SQL Server 2014 but I think that's not available as a hosted cloud database yet.
if you're referring to Azure SQL Database, this is still not supported today. you might want to explore using Azure Redis Cache instead to cache the datasets.
I have OLAP analysis services related with relational database.when the data updated or changed in relational database I must make process operation manually in my project(project to build the OLAP from relational database) to push the new data in OLAP.
I want every change in relational database to affect the OLAP automatically.
Thanks in advance
This is called: "Proactive Caching feature in SQL Server Analysis Services"
Assuming that you are using Sql Server 2005 or later version this article help you:
Configure Proactive Caching feature in SQL Server Analysis Services 2005
To know more: What is and how does it work the Proactive Caching
In case you still have doubts, here another article:
Implement Proactive Caching in SQL Server Analysis Services SSAS
Hope it help
We use a SQL CLR proc call to the ADOMD to issue the XMLA command.
In an installation with 2 SQL Server 2008 SP2 Standard servers we use mirror to have failover partner for the databases.
One of the purposes of the databases are to supply reports through SQL Server Reporting Services.
My question is how I set this up to have failover of the reports AND keep cost at a minimum.
I can see one solution where the reporting databases are mirrored as well as the user databases they use. But will the reporting services failover?
Another way could be to install 2 separate instances of reporting services on each database server where the reporting databases are not mirrored and both are running. But then how do I redirect the users browsers from one entry address to a running reporting service?
There's also a witness server running SQL Server 2008 Express. This could be upgraded to standard and run the reporting services (should be able to failover through native client connection), but then the reporting service it self has no failover.
I don't know if I'm missing something here but is it impossible to do this in a good way without using SQL Server Enterprise and a failover cluster solution?
Any ideas, thoughts and links would be appreciated, but be aware that I have read a lot of guides and best practices around the internet and haven't found any that answers this question so please only post relevant links.
Any info I gather from you post, links and my research and testing will of course go back to Stackoverflow for future reference for others as I can't imagine I'm the only one who would like such a solution.
Smint
You will need to mirror all data and then manually repoint the report server when the data moves to the mirrored servers.
Moving interrelated databases with mirroring complicates things. If one goes offline for some reason and failes to the mirror, will all other databases fail as well?
You may want to have a human make these decisions and use a manual failover on the mirroring as well.