I have two SQL 2016 servers configured in a mirror with a witness. We've been running some failover tests and every test has succeeded except the following two scenarios. Does anyone know why these scenarios wouldn't result in a failover?
Dismounting the storage that the database files reside on the primary server will not fail the server over. I thought the witness would notice that the files do not exist anymore and fail over?
Throttling the network down to 1kbps on the primary server disconnects the mirror on both the primary and secondary. I would think that the witness would lose connectivity to the primary because the network is so slow and fail over but instead both servers go to disconnected status.
Has anyone ran into any of these issues?
Partial answer:
(1) Dismounting disk does not cause failover because mirroring doesn't actively check to confirm that SQL Server components are up and running - rather it listens to errors and uses a timeout mechanism. Per BOL, disk failures are unlikely to be detected.
Related
We have an SQL Server Mirroring established between a database on two SQL Server 2012 Standard instances on two different servers. The witness is on a third server running the express edition. The IP addresses (not hostnames) of primary, mirror and witness servers are mentioned in the mirroring configurations while creating the mirror using SSMS. Now problem is that a change in the IP address of the mirror is required. There is a proper reasoning behind that and it can't be avoided. The system is live and outage should be avoided to the max. The new IP is accessible from both primary and witness.
When we take the mirror server out, the mirroring is affected, but as soon the server is back with the same old IP, mirroring resumes appropriately. However, how to change the endpoints IP address without having the need to remove and then recreate the mirror. Is it possible or there is NO way this can be achieved without remove/recreate?
If there is an absolute need here to do this remove/recreate, how to ensure that we don't copy over the complete database and its logs and redo the process from scratch. If all client's access to the primary is blocked during the time to ensure that no transaction is taking place, would this suffice?
A solution without remove/recreate would be the preferred one.
Thanks.
I've got SQL2012 running on 2 different servers with public, static IP addresses. I want to implement replication in a way that will keep both servers in sync at all times, regardless of which server is actually receiving the data. I've been reading about the subscriber/publisher model but I'm not exactly sure which should be which. A few facts about our setup:
I'm trying to achieve failover. If server A goes down, I need server B to be operational and have all latest data, or as close as possible. And vice versa. When the server comes back online, I need the replication to get caught up quickly and start working again. I need failures to be graceful, in other words I can't have server A get weird just because server B went offline.
I don't need realtime replication, but close would be nice. If server A was 10 seconds behind server B with data updates, nobody would care. But if it were an hour behind, that would be bad. Fast DB performance is more important that realtime replication, but again, close would be nice.
My database is just shy of 900Mb, and grows by 3Mb per day.
I am looking for advice on the best way to set this up given my setup and needs. Much appreciated.
Since one server will be Primary and the other Failover, use Log Shipping. It will keep two databases the same for all transactions completed on Primary server upto the failure moment. All transactions that have not completed at the moment of failure, will not appear on Failover server, so they should resubmitted by the application and hit Failover server.
Also there should be a Recovery procedure, to ensure than Primary server is up to date.
Useful articles:
Database Mirroring and Log Shipping.
Configure Log Shipping
In principle an SQL Server failover cluster presents itself as a virtual machine that applications can connect to oblivious to the fact that the SQL Server is actually a cluster of servers, hence, in principle no additional logic is required within the database access tier of the application.
My question is whether the above is true and whether there are best practice modifications to how the DB access tier operates when using a failover cluster. E.g. presumably when failover occurs there will be a delay that may cause a time-out error at the DB access tier, we are considering putting logic in that tier to re-try [some] DB calls upon a timeout occurring (we already have retry logic for DB deadlocks). This provides another level of protection from errors affecting the application.
If a failover switch occurs and results in the higher application level receiving a timeout error on a service call then that is not seamless switch over. Should we simply be setting our timeouts at a duration that allows for failover?
Thanks.
In principle an SQL Server failover cluster presents itself as a virtual machine that
applications can connect to oblivious to the fact that the SQL Server
is actually a cluster of servers
Ah? Really? That contradicts documentation. A cluster is basically nothing more than a moving IP address with different installation on different servers, hardly a virtual machine.
in principle no additional logic is required within the database access tier of the
application.
Yes and no - a failing node DOES kill all ongoing transactions and connections, obviously, so the CLIENT must be able to react to that and retry. If the client crashes because a connection is down an does not retry, it does not help you that server is reachable again after a second or two.
Should we simply be setting our timeouts at a duration that allows for failover?
No, a connection is broken by failover as the ongoing transaction state is lost. You need to reestablish the connection and then start all Sql commands again that were issued in the transaction.
Note from a security point, clustering is bad and you should use mirroring - you have a specific risk that a failing cluster node turns the database files corrupt in which case the fail-over fails. Mirroring is more robust.
Service dependency is not enough to guarantee that database clients will find their SQL Server up and running after a system reboot. They may be able to create a connection to the master database, but a specific database may still be in the process of opening (in the recovery mode) and connections to it will be refused for some initial period of time.
The exact duration of the delay is highly variable and depends on factors such as number of databases on the system, recovery mode, LDF file size, or traffic prior to the reboot.
The well known approach to dealing with the dependency on the client is sleeping and retrying. However, this approach does not seem exactly clean or reliable, especially when the clients are diverse in terms of technology and ownership and when this race condition appears only very rarely.
Is there any better way to synchronize the clients with the server start-up, ideally through keeping the SQL Server service in the "Starting" state until all databases have been either opened or marked suspect?
Create a windows service that tries to access the database which does not report itself as having started unless successful in establishing a reliable connection. Basically that service would return an error and the service manager would try to run it according to the service settings.
And now you can depend on that new service in your other services that would guarantee that the database is responsive.
Related questions
In C# what is the best way to determine if a database is up and running?
Update
Also, for a much cleaner way to know if the database is up and running is to check the following value:
SELECT state_desc
FROM sys.databases
WHERE name = 'YourDatabase'
If it says anything but 'ONLINE', then the database is not yet ready. (if it exists at all)
http://msdn.microsoft.com/en-us/library/ms190442.aspx
I'm not a DBA so this may be a stupid question but I'll ask it anyway. We're upgrading our SQL Servers from 2000 to 2005 and we will probably use either database replication or database mirroring. Our DBA would like to "multipurpose" the standby server meaning that he'd like to increase our capabilities and capacity by running other database applications on the standby server since "it's just going to be sitting there anyway" (his words, not mine). Is this such a good idea? Right now, our main application server uses only one instance that contains 50+ databases. As I understand it, what we're doing now and what our DBA is proposing for a failover server is a bad idea because all of these databases are sharing memory, CPUs, and working areas. If one applications starts behaving badly, the other DBs could be affected.
Any thoughts?
It's really a business question that needs to be answered?? is a slow app better then no app if you can't afford the expense of extra hardware?
Standby and mirrored db's can be used for reporting. Using it as the failover db can work if you have enough headroom (i.e. both databases will comfortably run on the server)
Will you depend on these extra applications? Where do they run in the failover case?
You really need to understand your failure modes.
If you look at it as basic resource math, that doesn't often make sense unless the resources you have running in the failure scenarios can handle the entire expected load. Sometimes this is the case, but not always. In this case, to handle the actual load you may need yet another server to come in (like RAID - perhaps your load needs a minimum of 5 servers, but you have a farm of 6, then you need 1 standby server for ever server to fail above 1). Sometimes a farm can run degraded, but sometimes they just puke and die.
And in the case of out of normal operation, you often have accident cascading where a legitimate incident causes a cascade of issues - e.g. your backup tape is busy restoring a server from a backup (to a test environment, even - there are no real "failures"), now your sql server or exhcange server (or both) is not backed up and your log gets full.
Database Mirroring would not be the way to go here in my opinion as it provides redundancy at the database level only. So you would need to configure database mirroring for up to 50 databases based on the information you provided. The chances are that if one DB where to fail all, 50 would probably follow, as failures typically occur at the hardware level rather than a specific database.
It sounds to me like you should be using SQL Server Clustering technology. You could create an Active/Active cluster to support your requirements.
What is an Active/Active Cluster?
An Active/Active SQL Server cluster means that SQL Server is running on both nodes of a two-way cluster. Each copy of SQL Server acts independently, and users see two different SQL Servers. If one of the SQL Servers in the cluster should fail, then the failed instance of SQL Server will failover to the remaining server. This means that then both instances of SQL Server will be running on one physical server, instead of two.
Applying this to your scenario
You could then split the databases between two instances of SQL server, one active instance on each node. Should one node fail, the other node will pick up the slack and vice versa.
Further Reading
An introduction to SQL Server Clustering
I suspect that you will find the following MSDN thread useful reading also
"it's just going to be sitting there anyway"
It will be sitting there applying transactions...
Take note of John Sansom's recommendation. Keep in mind that a Active/Active cluster requires two sql server licenses and a failover cluster/mirror only needs one.
Setting up mirroring for a large number of db's could turn into a big pain. You need any jobs/maintenance to move over as well - which can be achieved with alerts on WMI failover events. There's probably more to think about that could complicate things.