Can RDS automated backup be done in another region? I am seeing conflicting data from AWS:
https://aws.amazon.com/blogs/database/implementing-a-disaster-recovery-strategy-with-amazon-rds/ says that automated backups are limited to single region.
https://aws.amazon.com/about-aws/whats-new/2020/01/aws-backup-supports-cross-region-backup/ talks about AWS Backup and says that RDS has cross-region support.
I am now confused. Can someone please help? My goal is disaster recovery, as highlighted in the first link.
For the first link you shared it is about the RDS snapshots (automatic snapshots) that are taken by the RDS service. These snapshots are only accessible within the single AWS region in which the snapshot was taken, however you can copy an automatic snapshot which will then make it a manual snapshot. From here you would then be able to copy the snapshot to another AWS region.
The second link is actually referring to the AWS Backup service which is a service for centrally managing your backups. When this snapshot is taken the Backup service can actually handle the process of copying the snapshot to another region for you.
Its worth noting that AWS Backup Service is a more recent addition to AWS, whereas the automated snapshots for RDS have existed since the service launched.
Related
I have an RDS Database serving a production application.
The RDS DB is a db.m3.large and is Multi-AZ Instance Deployment. (Role is "Instance" in the console).
I've been informed by AWS that I have to migrate my Database from an M3 instance to one of the newer types this time next month. This is the first time I have had to touch this Database on the hardware side for quite a while so I would like some validation on my understanding on how best to approach this.
I am aware there is documentation but its a bit vague and I would like a statement in laymans terms.
My current approach/understanding on how to handle this would be:
I modify the instance type
While the instance type is modifed, failover occurs and the traffic is redirected to a standby instance to handle traffic while the changes are modified
The failover will occur from between 60-120 seconds, so that would be the expected amount of downtime.
Once the job is done, the traffic is redirected to original primary instance
I want to migrate my SQL Server Databases to AWS EC2 instances or AWS RDS using DMS Service of AWS?
How can I do that? What will be the architecture for the same? How to secure Databases in AWS?
What is the difference between both approaches? Can anyone provide architecture for both the approaches?
We're in the middle of moving our on-prem SQL Servers to AWS EC2 right now. For us, we're adding replicas to existing Availability Groups. When the time comes, we'll fail over the AG and tada! we'll be in AWS. But even if we didn't have AGs, I'd still do this with backup/restore over DMS. From the little bit that I looked at DMS, you need either CDC or a rowversion column on every table you want DMS to work with. Which seemed like a lot to me.
In case you're not acquainted with moving a database with backup/restore, here's the bones of it.
Copy a full backup to the target system
Restore that backup, being sure to specify with norecovery to allow it to accept more backups
Copy/restore a differential backup, again specifying with norecovery
Copy/restore transaction log backups. You'll keep doing this on a periodic basis until you're ready to do your cut over.
When you're finally ready to turn off the source database, take a log backup of the source, specifying with norecovery. This will create what is called a tail of the log backup and will stop anything from being able to write to the source database. Copy/restore that last log backup over and you're migrated.
So I recently started using AWS and Elastic Beanstalk with RDS.
I wonder whats the best practices for creating database backups?
So far my setup is this.
Enable automatic backups.
bash script that creates manual snapshots everyday and removes manual snapshots older than 8 days.
bash script that creates a sql dump of the database and uploads it to S3.
Reason why I am creating the manual snapshots is if I were to delete the database by misstake, then I still have snapshots.
The bash scripts is on an EC2 instance launched with a IAM role which is allowed to execute these scripts.
Am I on the right track here?
I really appreciate answers, thanks.
A bit of context...
That automated backups are not saved after DB deletion is a very important technical gotcha. I've seen it catch devs on the team unawares, so thanks for bringing this up.
After the DB instance is deleted, RDS retains this final DB snapshot and all other manual DB snapshots indefinitely. However, all automated backups are deleted and cannot be recovered when you delete a DB instance. source
I suspect for most folks, the final snapshot is sufficient.
Onto the question at hand...
Yes. 110%. Absolutely.
I wouldn't create manual snapshots; rather, copy the automated ones.
Option 1: You already have the automated snapshots available. Why not just copy the automated snapshot (less unnecessary DB load; though, admittedly less of an issue if you're multi-AZ since you'll be snapshoting from the replica), which created a manual snapshot. I'd automate this using the aws sdk and a cron job.
Option 2: Requires manual adherence. Simply copy your automated snapshots (to create manual snapshots) before terminating a DB.
Unclear why you'd need the s3 dump if you have the snapshots.
For schema changes?: If you're doing it for schema changes, these should really be handled with migrations (we use knex.js, but pick your poison). If that's a bridge too far, remember that there's an option for schema-only dumps (pg_dump --schema-only). Much more manageable.
Getting at the data?: Your snapshots are already on s3, see faq. You can always load a snapshot, and sql dump it if you choose. I don't see an immediately obvious reason for purposely duplicating your data.
We have an asp.net mvc web application with IIS and SQL Server residing in the same Amazon EC2 instance. We plan to move the database from the EC2 instance to Amazon RDS (to improve performance, the application is a bit slow at the moment - high cpu usage on the server).
I can't figure how this could be potentially helpful. On the flip side.
Migrating the database to Amazon RDS is not straightforward
Amazon RDS does not allow change in allocated storage for SQL Server
You can not use Backup / Restore to move DBs in and out of RDS
What are the benefits of migrating a SQL Server DB from EC2 to RDS? Rather than migrating to RDS should we look to resize the EC2 instance itself?
The biggest benefit to migrating to RDS, imo, is that your database becomes a database-as-a-service: you no longer need to manage it like you do when its running on your own instance. Automated backups, ability to take a snaphot and spin up an identical instance etc - all great things.
A while back I migrated all the SQL databases I supported to RDS and slept very well at night no longer needing to worry about them - but then, little by little, all the problems you mentioned became clearer and harder to ignore - inability to take backups of a single db and use locally, no longer have true administrator privileges to the DB or server and so ultimately, in a furry of activity migrated everything back to my own EC2 instances over a weekend.
IMO, until RDS allows you to take a local single-database backup and restore it to RDS, (and the opposite allow you to take a single database in RDS and make a backup that can be restored locally), I don't consider RDS to be a viable solution for me for any solution that is likely to need changes.
If I had a system that was 100% 'done', (if there is such a thing), and no need to enhance, modify, customize the database, than RDS has some benefits - but not having the ability to import/export backups is a show stopper for me as long as there is any chance I may need to make changes down the road - for the life of me can't figure out why AWS has not allowed/enabled this feature.
As to what you should do, first and foremost, you need to make sure that the database is really the bottleneck - it might not be.
If it is, then:
If you are not already, start using a cache in front of your database
Upgrade the size of your ec2 instance.
Split the database onto its on EC2 instance.
Moving your database to RDS has some benefits, but an immediate boost in performance is not necessarily one of them.
One approach organisations sometimes use for backing up Cloudant is to run a standalone instance of CouchDB in their private network or a public network and replicate data from Cloudant to that CouchDB instance. The CouchDB data can then be exported to mass storage such as Amazon Glacier.
Questions:
What are the steps required to implement this?
Are there any gotchas to be aware of?
Here are the approximate steps:
a server running CouchDB (e.g. in EC2)
continuous replication from Cloudant --> CouchDB
periodic (e.g. nightly) cron job to
copy the relevant .couch file over somewhere
zip it up
use AWS command-line tools to put the zipped file on S3
use AWS command-line tools to send that S3 file to Glacier
Things to remember:
Glacier keeps everything unless you say "kill that backup from 30 days ago", so you keep paying for old backups. Best to delete really old stuff
with continuous replication: if you delete a doc on Cloudant it immediately deletes on your backup (oops)
restoring from Glacier is a pain, then you can restore it to CouchDB, then you can replicate it to Cloudant.
Cloudant will not be able to support your CouchDB installation - you will need to support it yourself.