I am a beginner and struggling with figuring out increasing storage size on MySQL RDS.
The issue is that I want to increase RDS storage size without any downtime. Let's assume that I have not enabled autoscaling on RDS by default. I researched this and found that there are fundamentally two options:
Option 1) Enable Auto Scaling. AWS documentation is at https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIOPS.StorageTypes.html. However, the official documentation doesn't specify whether there will be no downtime. It says that I can turn on/off autoscaling without talking about downtime.
Option 2) Provision more space directly The official documentation also says that I can provision more storage size with the caveat that
In most cases, scaling storage doesn't require any outage and doesn't
degrade performance of the server.
From SO thread (https://serverfault.com/questions/613182/downtime-for-increasing-aws-rds-storage), it seems this process doesn't seem to be seamless.
Option 3) Back-up and restore: I would back-up RDS. Then, I'd increase the storage capacity. Then, I'd restore RDS and stop the previous instance.
Question: Can someone please guide me which option will have no downtime and will create relatively low amount of headache for us?
My hypothesis is that Option 1) will the best, but I just want to confirm my hypothesis with experts.
From reading that documentation, Auto-scaling of storage triggers a "scale-up" of storage that is the same as "Increasing DB instance storage capacity" directly. It also has the benefit that it will automatically scale in future.
There is a special-case mentioned for SQL Server instances that have been running since 2017, but it would appear from reading the documentation that all other db instances would not have an outage.
Option #3 would definitely have an outage, so that would not be a wise option.
I would recommend taking a manual backup, then activating Auto Scaling during a quiet period (eg at night).
You can always experiment by creating another database and triggering a scale-up of storage, to see what happens.
Related
I have a master db in one region.. and I want to create a read replica of it in another region just for disaster recovery practices.
I do not want it to be that costly, but I want the replication to work.
My current master db has db.t2.medium.
My question is:
What type should I keep for my read replica? Is db.t2.small fine for my replica?
It should not have much effect as read replica (RR) replication is asynchronous:
Amazon RDS then uses the asynchronous replication method for the DB engine to update the read replica whenever there is a change to the primary DB instance.
This means that your RR will be always lagging behind the master. For exactly how much, it depends on your setup. Thus you should monitor the lag as shown in Monitoring read replication. This is needed, because you may find that the lag is unacceptably large for the RR to be useful for DR purposes (i.e. large RPO).
According to this article and several others I've found:
Performance best practices for SQL Server
It is considered a best practice to disable caching on premium storage disks for the SQL Server Log disks. However, I can't find anywhere that explains why.
Does anyone have some insight?
Let me add that the reason I see disabling read-only cache on the log drive as an issue is because it makes you have to set up two separate Storage Pools inside the VM, which makes upgrading/downgrading VM's inside Azure more problematic and considerably less performant.
For example, say you start with a DS V13 which has a 16 drive limit, but about 6 of those drives can be maxed before you're throttled (25000 IOPs). Since best practices says read-only cache for data and no cache for logs, you give 8 of those drives to the data and 8 to the log.
Now, the server needs to be upgraded, so you upgrade it to a DS V14. Now you can max out 12 drives before being throttled (50000 IOPs). However, your data drive's Storage Spaces column size is only 8, which is throttled to 40000 IOPs. So you're not using the IO's full potential.
However, if you can start with a DS V13 and assign all 16 of those drives to a single Storage Pool then put both the log and data on it. You can upgrade/downgrade all the way up to a DS V15 without any concern for not using up your full IOP's potential.
Another way to put it is: If you create a single Storage Pool for all 16 drives, you have considerably more flexibility in upgrading/downgrading the VM. If you have to create two Storage Pools, you do not.
We recommend configuring “None” cache on premium storage disks hosting the log files.. Log files have primarily write-heavy operations and they do not benefit from the ReadOnly cache. Two reasons:
Legitimate Cache contents will get evicted by useless data from the log.
Log writes will also consume Cache BW/IOPS. If the cache is enabled on a disk (ReadOnly or ReadWrite), every write on that disk will also write that data into the cache. Every read will also access/put the data in the cache. Thus every IO will hit cache if the cache is ON.
Thanks,
Aung
Logs files are used as part of the recovery, and can help restore a database to a point in time. Having corrupt data on a log file from a power outage or hard reboot is not good with MSSQL. See the below article from MS, they relate to older versions of SQL but the purpose of the log file has not changed.
Information about using disk drive caches with SQL Server that every database administrator should know
https://support.microsoft.com/en-us/kb/234656
Description of caching disk controllers in SQL Server
https://support.microsoft.com/en-us/kb/86903
We're migrating our environment over to AWS from a colo facility. As part of that we are upgrading our 2 SQL Server 2005s to 2014s. The two are currently mirrored and we'd like to keep it that way or find other ways to make the servers redundant. # of transactions/server-use is light for our app - but it's in production, requires high availability, and, as a result, requires some kind of fail over.
We have already setup one EC2 instance and put SQL server 2014 on it (as opposed to using RDBMS for licensing reasons and are now exploring what to do next to achieve this.
What suggestions do people have to achieve the redundancy we need?
I've seen two options thus far from here and googling around. I list them below - we're very open to other options!
First, use RDBMS mirroring service, but I can't tell if that only applies if the principal server is also RDBMS - it also doesn't help with licensing.
Second, use multiple availability zones. What are the pros/cons of this versus using different regions altogether (e.g., bandwidth issues) etc? And does multi-AZ actually give redundancy (if AWS goes down in Oregon, for example, then doesn't everything go down)?
Thanks for the help!
The Multi-AZ capability of Amazon RDS (Relational Database Service) is designed to offer high-availability for a database.
From Amazon RDS Multi-AZ Deployments:
When you provision a Multi-AZ DB Instance, Amazon RDS automatically creates a primary DB Instance and synchronously replicates the data to a standby instance in a different Availability Zone (AZ). Each AZ runs on its own physically distinct, independent infrastructure, and is engineered to be highly reliable. In case of an infrastructure failure (for example, instance hardware failure, storage failure, or network disruption), Amazon RDS performs an automatic failover to the standby, so that you can resume database operations as soon as the failover is complete. Since the endpoint for your DB Instance remains the same after a failover, your application can resume database operation without the need for manual administrative intervention.
Multiple Availability Zones are recommended to improve availability of systems. Each AZ is a separate physical facility such that any disaster that should befall one AZ should not impact another AZ. This is normally considered sufficient redundancy rather than having to run across multiple Regions. It also has the benefit that data can be synchronously replicated between AZs due to low-latency connections, while this might not be possible between Regions since they are located further apart.
One final benefit... The Multi-AZ capability of Amazon RDS can be activated by simply selecting "Yes" when the database is launched. Running your own database and using mirroring services requires you to do considerably more work on an on-going basis.
As far as I can tell, scaling an Azure DB from the retired tiers to the new tiers is simply a matter of using the scale function in the Azure portal.
What I cannot seem to find anywhere is a definitive answer as to whether there are any connection string changes required (or any other issues that could cause unavailability) when scaling from the retired to new tiers.
I have a production database that needs to be upgraded, service interruption would be very bad.
The scale operation will not change the connection string. You could face some (very small, but) finite amount of downtime while the switchover happens.
Please refer to the documentation for details. Note that you will be have to suspend geo-replication (if already enabled) for the duration of the upgrade.
Techincaly it will be the same server, same connection string, same everything, but version and features.
But I would be concerned about the following statement from docu:
The duration of upgrade depends on the size, edition and number of
databases in the server. The upgrade process can run for hours to days
for servers especially for servers that has databases:
Larger than 50 GB, or
At a non-premium service tier
Which is kind of concerning.
What I would do, if possible is:
Put my service into read-only mode (put on hold any writes to the DB)
Create new db in same server from the existing one with the command - CREATE DATABASE AS COPY OF ...
When creation of DB is ready, export the new db to backpac and delete the DB when export is ready.
Perform upgrade.
In theory you could do the process without putting your system into Read-Only mode, but I am just taking more precautions measures.
And yes, you also have to aware that you are upgrading your Azure SQL DB Server not just a single Database.
I have a bunch of SQL servers which I periodically performs maintainance on (Windows Update patches etc.). Now I want to the database online 24/7 and need to implement one of the high-availability solutions for SQL server.
The solutions needs to be cheap and simple to use.
I have no problems tweaking the connection strings for the clients of the database, so currently I'm looking into database mirroring with manual failovers when taking down a partner instance for patching etc.
Is this the best thing to do or are there other options which doesn't involve setting up a failover cluster?
The servers are virtualized with a fully redundant storage solution.
Any tips are appreciated, thanks in advance!
Mirroring with a PARTNER-server would probably be the cheapest solution (you can skip the PARTNER-server if you plan to switch manually).
Failover requires shared disks (NAS) aswell as cluster-capable Windows-licenses (very expensive).
I'm not sure about replication, or how it differs from mirroring, but my research I did gave the conclusion that mirroring was the one for me. However I don't mind some downtime when doing upgrades, I just keep mirrored instances of the database in case of severe hardware failure.
It might be that replication is for a complete instance of an SQL-server, whilst mirroring is done per database. In my case, I have 2 production servers, that both replicates it's databases to a third, backup-server for disaster-recovery. I think that wouldn't have been possible with replication.
The four high availability solutions I'm aware of are:
Failover cluster
Log shipping
Mirroring
Replication
Log shipping is probably not 24/7, so that leaves three. Serverfault is definitely a better place to ask about their relative merits.
For auto failover I would choose mirroring. You can build a 2nd database connection string into your app and whenever the preferred isnt available it will default to the backup - therefore giving your app 24/7. This has its downsides though, once 'flipped' to the mirror you have to either accept that this is the way it is until another maintenance job requires the mirror to shift back again or you have to manually swap the mirror over.
In order for this to be truly 24/7 you will need to enable auto rather than manual, maybe you will need a witness server to make the decision... There are lots of factors to include in the choice - are you working with servers on different sites, clustering, multiple web/app servers ... ?
As previous answers have suggested, https://serverfault.com/search?q=sql+mirroring will have people who have made just this choice, ready to help you in much more detail
A big benefit of mirroring is that providing the mirror server has no other activity it is license free, the live server license transfers over if the mirror takes over. Full details on SQL licensing pages at microsoft.com