I have a Spring Boot app running for filling a MySQL database with data I want to analyze with OLAP. I chose Pentaho Data Integration, Schema Workbench and Business Intelligence Server (CE versions) to create my data warehouse. In the ETL process I just fill my data warehouse with the data I generated with my Spring Boot app. In the BI-Server I use the Saiku plugin to make all the OLAP operations and it all works quite nicely.
However I want to make all of that public.
Where is it possible to deploy my app with a database (preferably MySQL) and the BI-Server?
I've looked into Heroku and it seems to use PostgreSQL databases and I haven't found out about a possibility to deploy my BI-Server... Or should I move away from Pentaho and use other solutions to make OLAP on my data warehouse?
Any recommendations?
You`d better use some cloud service like https://www.digitalocean.com/ or amazon. For your own linux server. At this server you can install BI server and Mysql.
Related
We have the Microstrategy reports pointing to Teradata as backend. The plan is to migrate the Microstrategy from On Prem to Microstrategy Cloud with Snowflake as backend.
I wanted to know the steps and various ways/processes to do this migration.
For the data, there are two basic approaches:
Export the data to files on a cloud storage platform and then use Snowflake’s COPY INTO command to load the files into Snowflake
Use a tool that can read from Teradata and write to Snowflake. This could be a dedicated ETL tool or a generic coding tool like Python
For Metadata backup, you can use "mstr bak" to do the migration. This is officially supported to migration from on-prem to cloud. Refer to https://www2.microstrategy.com/producthelp/Current/Cloud/en-us/Content/upgrade_mstr_back_up.htm
For warehouse migration(Teradata->Snowflake), I believe there are other tools that support this.
We have 2 SQL Server databases. One of them is to be moved to AWS cloud.
Is Amazon RDS the ONLY approach to move the SQL Server to AWS?
What is the best way to automate data transfer of table data from the SQL Server that IS in the cloud to the SQL Server that is NOT in the cloud?
The volume and complexity of data is not high in our case. We need to migrate only ONE table and we'd like to automate it to run every night.
Would AWS Glue be useful for this? Would it help in loading data from INSIDE the cloud to OUTSIDE?
I also just found DMS - https://aws.amazon.com/dms/
But the homepage says "Migrate your databases **to** AWS with minimal downtime"
Can I do the opposite with DMS? Can I migrate data **FROM** AWS to outside AWS?
Amazon RDS is not the only approach to host SQLServer in the cloud. You can use an EC2 instance and install SQLServer manually. This could make sense in case you want to have sys_admin privileges. But be careful, as you would have to do patching all by yourself. I would not recommend doing that.
DMS is a good way to migrate your data every night. The service you are looking for is continuous data replication from DMS. Glue can also be used for this task. Glue's purpose was never to do database replication. Hence, it could be a pain monitoring the replication & debug in case of errors. That is why I would recommend DMS.
I am planning to use azure mobile services for one of our products. I was confused about how can I use any of my existing databases (like MySQL, PostgreSQL, or Oracle instead of SQL database offered by Azure) which is available via TCP?
Kindly confirm.
Are you using the JavaScript/Node backend or the .NET backend?
If you're using JS, you'll need to create API scripts and import the Node module for your database. You can't use the default data scripts since they are only hooked to SQL Azure.
If you're using a .NET backend, there's no restriction whatsoever. You can use any database available on Azure or install your own in a VM.
I am developing an application that will use SQL Azure for the production database. I will be developing locally against SQL Express on my machine. If I make changes to the schema, how do I migrate those changes to SQL Azure but still retain all of my production data? For the initial migration I used SQL Azure Migration Wizard, but that was before I had any data. Should I make a backup of the data, recreate the database with the new schema, and then restore the data?
I think that its a general schema versioning question. One possible solution is having migration scripts that you write and test during development and run on production during deployment phase. Entitiy Framework offers migrations as one of it features. My current prefered tool is FluentMigrator. I works for Azure Sql as well.
In order to migrate changes to SQL Server but still retain all of your production data, it is highly recommended that you version your database schema. This also applies to SQL Azure and all other RDBMSs.
This is very important when there are several developers working on a project but I believe it is also advantageous when you are working on a project by yourself.
RoundhousE is one of several simple tools you can use to achieve database versioning. It served us well for this purpose for the past 4 years: https://code.google.com/p/roundhouse/
I am looking for a means to use Git deployment on Windows Azure together with either their MySQL or other database solutions.
I need a means of migrating database schema changes as part of deployment - does Azure provide support for this without using Visual Studio or .NET.
You basically want to use Azure as a Continuous Integration (CI) platform and my view is that Azure by itself just isn't there yet.
A basic git based CI scenario would involve the following steps.
Your src code and any database schema/data updates are pushed to a central git repository like GitHub. [Feature available]
GitHub would then push the updates to Azure. [Not supported in Azure until they support GitHub hooks]
Azure compiles and deploys the code. [Supported]
Azure updates the database from sql files pulled from the repo (remember, this needs to be automated). [Not supported]
Azure reports any error in running the database or web application. [Supported]
Azure runs user provided integration tests to check more thoroughly specific functionality and reports the status of those tests to the user [Not supported].
Azure allows you to roll back to a previous deployment snapshot [Partly supported. Snapshots are not based on git commits for instance].
I might be wrong on some of those points or new features might be added since I've written this. Corrections are very welcomed and I'll try to update the list accordingly.
I am not sure how much experiences you have with Windows Azure Websites, however when you are creating a Windows Azure Websites, you have ability to use SQL Azure Database or MySQL cloud database directly integrated with your Windows Azure Websites.
So if you will use SQL Azure Database then there are several ways you can migrate your DB following the link here:
Migrating Databases to Windows Azure SQL Database (formerly SQL Azure)
To manage MySQL you can use Local Web Server to do it. The way I have done is to use PHP, the MySQL Command-Line Tool (part of MySQL), and a web server set up on my local machine, and that I have enabled the PDO extension for MySQL. This way I can manager my MySQL directly from my own local machine and the details are explain at the bottom of this article:
Create a PHP-MySQL Windows Azure web site and deploy using Git.
Also the SQL part you can write in Workbench if you use MySQL and Oracle SQL Developer if you use Oracle SQL.
Finally Git Deployment allows you to deploy your any kind of application directly to Windows Azure without using VSx.