We are using SQL DB for our application and these both are hosted on AWS. We have already setup CI/CD pipeline for application as per environment.
So, my question is how can we automate SQL schema changes using AWS? I know there are lot of third party tools are available in the marked, but is it possible using AWS managed services?
Thanks for the help !
Related
We are migrating an application from Azure to AWS and we have migrated the SQL Server database to AWS RDS SQL Server database.
In AWS we use codepipeline / codebuild to deploy our application and which is pretty much automated when a push happens to github we have an application deployed. Similarly we also want any DB changes to also be deployed via pipeline.
We will have the dacpac scripts available when we build our project and all we need is to deploy these dacpac scripts from codepipeline to AWS RDS, so I researched to see if this is possible and how to achieve, but I got little help. If any of you have implemented this or are aware, can please help to share your experience, would be highly appreciated.
Thanks in advance.
I am a 1 man team, more dev's coming on board, I have created a few on Premise SQL DBs; I have a Azure SQL DB managed instance and Azure Devops account setup
Currently:
I use SSDT -->Tools--> SQL Server: to manage the data and Schema comparisons , make sure it is in sync, On-Premise and Azure managed instance (as a backup at present, managed instance is not used for anything else)
I want to do
To manage the code for multiple devs,
I want to setup Azure SQL DB managed instance as production
SSDT or Git hub for version control(manage check-in / check out of SQL objects, we are all in sync)
Local on premise version as the development area, sync in with prod, use TFVC (SSDT) as tool to manage
the code conflicts , make sure all devs are working off the same DB
Q: Can I link my on premise SQL DB to DevOps source control , then links to SQL Azure managed DB? Can I do this with SSDT / GIT hub or do I need both?
Any step by step tutorials to set this up? In SSDT I am connected to my on premise SL db and to DevOps Repo , finding it difficult to figure out how to establish the link between the two
Getting overwhelmed with all the acronyms and 101 different products you can use (state based approach, DACPAC files, YAML , arm template , octopus deploy, redgate, dacfx,)
Cannot find anything online so far to show me how to setup from the ground up
Q: Can I link my on premise SQL DB to DevOps source control , then links to SQL Azure managed DB? Can I do this with SSDT / GIT hub or do I need both?
You have to install a self-hosted agent to access your on-premises instances
Then the pipeline flow can be built this way:
Build DacPac -> Deploy to Test (On-Premises) -> Deploy to Managed Instance (Prod)
Other points:
Since you are starting a new project, consider to use YAML pipelines.
Also, SSDT, starting VS 2019, is not suitable anymore for free use in a commercial environment. You have to get Professional or Enterprise license.
As the alternative, cross-platform Azure Data Studio + Database Project add-on
And some of the concepts blogged by me earlier:
https://www.alexvolok.com/2020/azure-devops-sql-server-dba-creating-database-project/
https://www.alexvolok.com/2020/azure-devops-for-sql-server-dba-automating-deployments/
We have 2 SQL Server databases. One of them is to be moved to AWS cloud.
Is Amazon RDS the ONLY approach to move the SQL Server to AWS?
What is the best way to automate data transfer of table data from the SQL Server that IS in the cloud to the SQL Server that is NOT in the cloud?
The volume and complexity of data is not high in our case. We need to migrate only ONE table and we'd like to automate it to run every night.
Would AWS Glue be useful for this? Would it help in loading data from INSIDE the cloud to OUTSIDE?
I also just found DMS - https://aws.amazon.com/dms/
But the homepage says "Migrate your databases **to** AWS with minimal downtime"
Can I do the opposite with DMS? Can I migrate data **FROM** AWS to outside AWS?
Amazon RDS is not the only approach to host SQLServer in the cloud. You can use an EC2 instance and install SQLServer manually. This could make sense in case you want to have sys_admin privileges. But be careful, as you would have to do patching all by yourself. I would not recommend doing that.
DMS is a good way to migrate your data every night. The service you are looking for is continuous data replication from DMS. Glue can also be used for this task. Glue's purpose was never to do database replication. Hence, it could be a pain monitoring the replication & debug in case of errors. That is why I would recommend DMS.
I am looking for a means to use Git deployment on Windows Azure together with either their MySQL or other database solutions.
I need a means of migrating database schema changes as part of deployment - does Azure provide support for this without using Visual Studio or .NET.
You basically want to use Azure as a Continuous Integration (CI) platform and my view is that Azure by itself just isn't there yet.
A basic git based CI scenario would involve the following steps.
Your src code and any database schema/data updates are pushed to a central git repository like GitHub. [Feature available]
GitHub would then push the updates to Azure. [Not supported in Azure until they support GitHub hooks]
Azure compiles and deploys the code. [Supported]
Azure updates the database from sql files pulled from the repo (remember, this needs to be automated). [Not supported]
Azure reports any error in running the database or web application. [Supported]
Azure runs user provided integration tests to check more thoroughly specific functionality and reports the status of those tests to the user [Not supported].
Azure allows you to roll back to a previous deployment snapshot [Partly supported. Snapshots are not based on git commits for instance].
I might be wrong on some of those points or new features might be added since I've written this. Corrections are very welcomed and I'll try to update the list accordingly.
I am not sure how much experiences you have with Windows Azure Websites, however when you are creating a Windows Azure Websites, you have ability to use SQL Azure Database or MySQL cloud database directly integrated with your Windows Azure Websites.
So if you will use SQL Azure Database then there are several ways you can migrate your DB following the link here:
Migrating Databases to Windows Azure SQL Database (formerly SQL Azure)
To manage MySQL you can use Local Web Server to do it. The way I have done is to use PHP, the MySQL Command-Line Tool (part of MySQL), and a web server set up on my local machine, and that I have enabled the PDO extension for MySQL. This way I can manager my MySQL directly from my own local machine and the details are explain at the bottom of this article:
Create a PHP-MySQL Windows Azure web site and deploy using Git.
Also the SQL part you can write in Workbench if you use MySQL and Oracle SQL Developer if you use Oracle SQL.
Finally Git Deployment allows you to deploy your any kind of application directly to Windows Azure without using VSx.
Recently, we have migrated a business application database to Amazon AWS SQL Server RDS. We have loved the flexibility of backup and scaling with the cloudified database, but we have need for writing reports for our gathered analytics data. Our in house data analyst is most comfortable using SQL Reporting Services. Normally, this is installed as a component with the SQL Server install, but we obviously do not have that option on RDS.
My question is, has anyone been able to successfully deploy and use SQL Reporting services with Amazon RDS?
What I have tried:
I attempted to spin up another EC2 instance, install "SQL Server Express with Advanced Services", and point SSRS to RDS, but it said that I had incompatible license types.
I plan on contacting the AWS team directly for this as well, but I thought that I would reach out to SO first to see if anyone has run into this. Thanks.
You might need to use a certain Microsoft-created "Amazon Machine Image", specifically the "SQL Server Optimized" AMI: Microsoft AMIs.
Another option would be to apply SQL Server SSRS containers, hosted on an EC2 VM, connected to RDS. There's a blog on Windocks.com that outlines steps to deliver configured containers for improved scalability (multiple SSRS containers), with simplified management.
I have successfully ran SSRS in Docker containers on Windows ECS/2 VMs for a while and just today came across a fresh out of the oven announcement from Amazon on running SSRS natively on RDS: https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-rds-for-sql-server-now-supports-sql-server-reporting-services/?nc1=h_ls