I have 2 servers. I need to copy some columns from 4 different tables from server 1 into the corresponding (empty) tables in server 2.
So basically, it's about replicating data from one table to another. How is this done best (and easiest)? Also, how do I make sure that the copied/replicated data is updated at the same frequency as the source (which runs completely fine and automatically)?
I want to avoid using Linked Server.
How is this done best (and easiest)?
For a one time replication consider a SQL Server Import and Export Wizard. This approach can also be scheduled by saving a final package and schedule it by SQL Agent
Example: Simple way to import data into SQL Server
For a continuous, low latency data syncronization - SQL Server Transactional Replication.
Further read: Tutorial: Configure replication between two fully connected servers (transactional)
Worth to mention, that transactional replication is not the easiest topic, however, it fits quite good to a requirement.
Related
I have nearly a hundred services sending data to our message queue. This data is processed by Java service and loaded into import tables in our SQL Server. After data is loaded, a few procedures are executed that load this data into proper tables. Recently we had to add new instances of service reading and loading messages. It was suggested that we should change database isolation model to snapshot (I'm not very accustomed with databases so I simply did what was proposed). Unfortunately we had a lot of problems with it, so we had to duplicate import tables and aforementioned procedures - this of course resulted in a huge mess that I'm currently trying to clean up.
My current understanding is such that snapshot isolation was suggested so that services could work using the same table without problems and errors that we encountered stem from some misunderstanding or improper implementation on our (developers) side.
My question is: is it possible, and if yes then how, to bulk load data into single table, transform it and load into target table (everything in parallel, so lets say that there are 3 or 4 services doing it) in a way that causes no deadlocks or data loss.
Our SQL Server is: Microsoft SQL Server 2014 (SP2-GDR) (KB4019093) - 12.0.5207.0 (X64)
I don't know much more, but I know that for example we don't have support for partitioning or online index creation - maybe this will help somehow.
I ended modifying services loading data and import tables in such a way that each record loaded has its own identifier. Also services no longer execute import procedures, they are scheduled using SQL Agent and run once per minute. Solution is really simple and while on average data is stored in destination tables 30 seconds after they are received by the services, this is something that we can live with - we can load much, much, MUCH more data that way.
Can anyone explain the differences from a replication db vs a mirroring db server?
I have huge reports to run. I want to use a secondary database server to run my report so I can off load resources from the primary server.
Should I setup a replication server or a mirrored server and why?
For your requirements the replication is the way to go. (asumming you're talking about transactional replication) As stated before mirroring will "mirror" the whole database but you won't be able to query unless you create snapshots from it.
The good point of the replication is that you can select which objects will you use and you can also filter it, and since the DB will be open you can delete info if it's not required( just be careful as this can lead to problems maintaining the replication itself), or create specific indexes for the report which are not needed in "production". I used to maintain this kind of solutions for a long time with no issues.
(Assuming you are referring to Transactional Replication)
The biggest differences are: 1) Replication operates on an object-by-object basis whereas mirroring operates on an entire database. 2) You can't query a mirrored database directly - you have to create snapshots based on the mirrored copy.
In my opinion, mirroring is easier to maintain, but the constant creation of snapshots may prove to be a hassle.
As mentioned here
Database mirroring and database replication are two high data
availability techniques for database servers. In replication, data and
database objects are copied and distributed from one database to
another. It reduces the load from the original database server, and
all the servers on which the database was copied are as active as the
master server. On the other hand, database mirroring creates copies of
a database in two different server instances (principal and mirror).
These mirror copies work as standby copies and are not always active
like in the case of data replication.
This question can also be helpful or have a look at MS Documentation
I have a local SQL Server database that I copy large amounts of data from and into a remote SQL Server database. Local version is 2008 and remote version is 2012.
The remote DB has transactional replication set-up to one local DB and another remote DB. This all works perfectly.
I have created an SSIS package that empties the destination tables (the remote DB) and then uses a Data Flow object to add the data from the source. For flexibility, I have each table in it's own Sequence Container (this allows me to run one or many tables at a time). The data flow settings are set to Keep Identity.
Currently, prior to running the SSIS package, I drop the replication settings and then run the package. Once the package completes, I then re-create the replication settings and reinitialise the subscribers.
I do it this way (deleting the replication and then re-creating) for fear of overloading the server with replication commands. Although most tables are between 10s and 1000s of rows, a couple of them are in excess of 35 million.
Is there a recommended way of emptying and re-loading the data of a large replicated database?
I don't want to replicate my local DB to the remote DB as that would not always be appropriate and doing a back and restore of the local DB would also not work due to the nature of the more complex permissions, etc. on the remote DB.
It's not the end of the world to drop and re-create the replication settings each time as I have it all scripted. I'm just sure that there must be a recommended way of managing this...
Not doing it. Empty / Reload is bad. Try to update the table via merge - this way you can avoid the drop and recreate, which also will result in 2 replicated operations. Load the new data into temp tables on the other server (not replicated), then merge them into the replicated tables. If a lot of data is unchanged, this will seriously reduce the replication load.
I have two database servers running SQL Server 2005 Enterprise that I want to make one of them as mirror database server.
What I need is; to create an exact copy database from primary server on mirror server, so when the primary server was down, we could switch database IP on application to use mirror server.
I have examined "mirror" feature on SQL Server 2005, and based on this article:
http://aspalliance.com/1388_Database_Mirroring_in_Microsoft_SQL_Server_2005.all
The mirror database cannot be accessed directly; however snapshots of the mirror database can be taken for read only purposes. (Prerequisites no. 4)
So how it can be useful when I can't access it when primary server was down?
I've been thinking about creating a regular backup on primary server and restore it on mirror server on hourly basis, but that's quite inefficient (slow) especially if I want an exact copy (since hundreds data's are added once in minute).
Any other suggestion?
EDIT:
Maybe what I mean was a replication thing, not a mirror (thanks JP for commenting)
They are referring to the fact that you can't perform queries on the mirrored copy, but you can get around that limitation by creating a snapshot of the mirrored database. This is often done to create a read-only database copy for reporting uses. You would have full access of the mirror if the primary were to fail, but it will not failover automatically.
Log shipping is another option, which allows you to query (read-only) the standby database without having to create a snapshot.
If I understand your question correctly, you shouldn't have to do that. There are several role switching forms you can use to have your mirror take over as primary. You don't change the IP address at the application level, the cluster itself has a virtual IP address that allows access to the data at any given time (given a reasonable amount of time for the switch over to the mirror from a primary failure). The mirror stays in synch by itself. :) There are good articles here and here on clustering.
Edit: Okay, based on the comments, check out the various options for replication.
Your confusion is common - there's a lot of ways to do disaster recovery planning with SQL Server. I've recorded a 10-minute video tutorial of SQL Server disaster recovery options including log shipping, mirroring, replication and more. If you like that one, we've got a longer one at Quest called Disaster Recovery Techniques but that one requires registration.
Instead of investigating a specific technology here, what you might want to do is tell us what your needs are, and then we can help you find out what option is right for you. The videos will give you an idea of what kinds of information you need to know before selecting a particular solution.
When using only two SQL Servers, you need to do the fail-over manually. The 'backup' database will be usable after you do two things;
Disable mirroring on it
Restore the database with RECOVERY (but without a backup file, this will make the database usable).
Therefore mirroring in this manner does make scense, however it is hard to maintain;
Moving back from the backup database to the primary is a 'pain' as you have to set-up the complete mirroring again using a backup of the redundant server. This is needed to get the primary back up to speed.
My recommendation would be to get a thrid SQL Server into the picture that can act as a witness. The witness will monitor the status of the mirroring databases. Your bonus; you will get automatic failover, and will not have the fail-over (and after fail-over) issues.
If I remeber correct, the witness server can be running SQL Express so no need for the Enterprise version on all three - just the two where the actual mirroring will take place.
Let me know if you need Transact SQL for the commands to fail-over and 'anti-fail-over' in a two server scenario, and I can dig them up.
I am currently working with 2 sql 2000 servers one I can query but not add any databases to which lead to the second one having lots of queries that use the first as a linked server. I would like to improve performance while still querying live data.
Is it possible to mirror the live data to the second server, that way the queries would be local to that server? What are my options?
If an upgrade from sql 2000 is needed that is a possibility.
The data has low update, insert, delete use and high select use so I think this would be ideal if possible.
You have three options.
use replication to move the data from the first server to the second. This has the drawback of making changes extremely difficult due to the need to tear down and set the replication up for each change.
Use log shipping in SQL 2000. The idea is that immediately after changes occur on your primary server, data is transferred to your standby server. I haven't done this so I dont personally know the drawbacks. http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/logship1.mspx
Upgrade to SQL 2005 or SQL 2008 and use the built in mirroring features of it. Be aware that you should do some fairly in depth testing to make sure everything works correctly after a major upgrade like this. Although compatible I have had to make tweaks to certain application queries after an upgrade to keep everything functioning normally.