is it possible to create a live replica of my PostgreSQL database which is inside Google Cloud SQL?
Currently I found the way to export the database via dump in this guide:
https://cloud.google.com/sql/docs/postgres/import-export/import-export-sql
Or is it possible to schedule this dump automatically?
Related
We have the Microstrategy reports pointing to Teradata as backend. The plan is to migrate the Microstrategy from On Prem to Microstrategy Cloud with Snowflake as backend.
I wanted to know the steps and various ways/processes to do this migration.
For the data, there are two basic approaches:
Export the data to files on a cloud storage platform and then use Snowflake’s COPY INTO command to load the files into Snowflake
Use a tool that can read from Teradata and write to Snowflake. This could be a dedicated ETL tool or a generic coding tool like Python
For Metadata backup, you can use "mstr bak" to do the migration. This is officially supported to migration from on-prem to cloud. Refer to https://www2.microstrategy.com/producthelp/Current/Cloud/en-us/Content/upgrade_mstr_back_up.htm
For warehouse migration(Teradata->Snowflake), I believe there are other tools that support this.
Is it possible to load data from S3 to SQL Server using Matillion ETL tool?
This is not a good use case for Matillion. The Matillion tool works by ingesting source data into a cloud database, and then running the transforms inside the database. There are 4 cloud databases supported: Snowflake, BigQuery, Redshift, and Synapse.
There are output components that can then output data to a target DB, but MSSQL is not currently supported (Azure SQL is).
I suggest looking at SSIS which has native support for MSSQL
Matillion ETL can help you to
Load data into its supported Cloud Data Warehouse (currently there are five options which are: Snowflake, Amazon Redshift, Google BigQuery, Microsoft Azure Synapse, and Delta Lake on Databricks)
Transform and integrate that data inside the Cloud Data Warehouse
MSSQL is not among the list of Matillion ETL supported Cloud Data Warehouses, so you can't use it to directly load data from S3 to MSSQL.
However you could use a Microsoft SQL Server Output component to transfer data into MSSQL, provided it went via one of the above listed supported Cloud Data Warehouses first.
I have a requirement to move around 30 DB's from On-prem SQL Server to google Cloud SQL instance with very minimal downtime.
I thought of doing Logshipping/High avaialbility but Gcloud SQL is not supporting sysadmin role to add secondary instance in On-prem SQL Server.
Please let me know if there are any other ways to do this
The easiest way to do this is with a backup and a restore, but you're going to have downtime in that regard. Quite a lot of it really.
The one way you can do this is through replication. No other facility is likely to help you move to Google Cloud SQL without downtime. Here's an example of how to set it up.
This document applies exactly to your case and it's the best way to do it, basically the steps are:
Enable the Cloud Storage and Cloud SQL APIs
Create the SQL instance and Cloud Storage bucket (where you will deposit the backup created from your On-prem DB)
Create a backup from your On-prem DB, this steps depends on your version but I think that the step detailed gives you and idea, and then upload it into the just created bucket
Import the backup created into Google Cloud SQL from your Storage bucket
The last step performed in the guide is just to validate recent imported data by connecting to the instance and execute a query.
For SQL Server 2017 this is the exact same guide but for that specific version.
We have 2 SQL Server databases. One of them is to be moved to AWS cloud.
Is Amazon RDS the ONLY approach to move the SQL Server to AWS?
What is the best way to automate data transfer of table data from the SQL Server that IS in the cloud to the SQL Server that is NOT in the cloud?
The volume and complexity of data is not high in our case. We need to migrate only ONE table and we'd like to automate it to run every night.
Would AWS Glue be useful for this? Would it help in loading data from INSIDE the cloud to OUTSIDE?
I also just found DMS - https://aws.amazon.com/dms/
But the homepage says "Migrate your databases **to** AWS with minimal downtime"
Can I do the opposite with DMS? Can I migrate data **FROM** AWS to outside AWS?
Amazon RDS is not the only approach to host SQLServer in the cloud. You can use an EC2 instance and install SQLServer manually. This could make sense in case you want to have sys_admin privileges. But be careful, as you would have to do patching all by yourself. I would not recommend doing that.
DMS is a good way to migrate your data every night. The service you are looking for is continuous data replication from DMS. Glue can also be used for this task. Glue's purpose was never to do database replication. Hence, it could be a pain monitoring the replication & debug in case of errors. That is why I would recommend DMS.
AWS DMS has schema conversion tool and other advanced features in database migration.So is it possible to do SQL server database inside a vm to some other cloud's mysql(due to business reasons) migration by using AWS DMS?or the target database should be inside AWS RDS?The source data size is close to 60GB and tables are close to 300.Please advice
the target database should be inside AWS RDS.