Given a database on premise of size 20GB and another database of size 120TB - what would be faster? MYSQLDUMP or MYSQLIMPORT?
Related
So for my Azure cloud-hosted application, I am using two databases, primary and secondary, respectively. The old data in the primary db is set up to be moved to the secondary db after it is not needed anymore. As a result, the secondary db with the bulk amount of data over a period of time takes a long enough time to be accessed through the SQL queries which is undesired.
So I have stored the secondary db's older data in azure cold storage and now I am trying to figure out a way to access it through SQL queries from the cold storage itself. But even though, through SSMS, connection to Azure cold storage is possible, I am not getting any option to run SQL queries since I am connected to Azure storage account, not a database engine.
Is there any solution to this? Or ANY workaround?
My aim is to store that unnecessary secondary db data in the Azure cold storage and access it from there through SQL queries whenever needed.
Any suggestion is welcome. Thanks in advance.
I have recently completed two migrations from Hadoop to Snowflake on AWS. I am now moving on to work with another customer where snowflake is on Azure. I wanted to know if there are any feature difference between Snowflake on AWS and snowflake on Azure.
Thanks
not in terms of Snowflake - most customers pick depending on the other systems they use and the deployment (to avoid the extra costs related to data transfer - Snowflake doesn't charge for data load, but AWS and Azure do charge to get the data into the correct deployment if it's not there already)
Metadaten Operations in Azure are slightly slower.
And the maximum filesize ist limited to 256 MB while its 5 GB on AWS and Google.
I'm using RDS SQL Server database for my live environment.
I'd need to fully replicate the database into another RDS database on a fixed frecuency (maybe daily). The goal is to have test environment always provisioned with latest real data (cualitative and cuantitative) in order to make tests meaninfull.
How can I get this done in AWS?
NOTE: db footprint about 20Gb
You can create a SQL Server Integration Services project that will copy the entire database from one environment to the other. You can schedule it as a Job. You should investigate using the Transfer Database Task.
Your tests should not use production data. Not only is it a waste of processing/money, it also doesn't have any justification for it and is bad practice. You should know what your production data looks like and have a test table with that data, and only update that data if there is a change to how you process the data.
What is the fastest way to backup/restore Azure SQL database?
The background: We have the database with size ~40 GB and restoring it from the .bacbac file (~4GB of compressed data) in the native way by Azure SQL Database Import/Export Service takes up to 6-8 hours. Creating .bacpac is also very long and takes ~2 hours.
UPD:
UPD.
Creating the database (by the way transactional consistent) copy using CREATE DATABASE [DBBackup] AS COPY OF [DB] takes only 15 minutes with 40 GB database and the restore is simple database rename.
UPD. Dec, 2014. Let me share with you our experience about the fastest way of DB migration schema we ended up with.
First of all, the approach with data-tier application (.bacpac) turned out to be not viable for us after DB became slightly bigger and it also will not work for you if you have at least one non-clustered index with total size > 2 GB until you disable non-clustered indexes before export - it's due to Azure SQL transaction log limit.
We stick to Azure Migration Wizard that for data transfer just runs BCP for each table (parameters of BCP are configurable) and it's ~20% faster than approach with .bacpac.
Here are some pitfalls we encountered with the Migration Wizard:
We run into encoding troubles for non-Unicode strings. Make sure
that BCP import and export runs with same collation. It's -C ... configuration switch, you can find parameters with which BCP calling
in .config file for MW application.
Take into account that MW (at least the version that is actual at the moment of this writing) runs BCP with parameters that will leave the constraints in non-trusted state, so do not forget to check all non-trusted constraints after BCP import.
If your database is 40GB it's long past time to consider having a redundant Database server that's ready to go as soon as the main becomes faulty.
You should have a second server running alongside the main DB server that has no actual routines except to sync with the main server on an hourly/daily basis (depending on how often your data changes, and how long it takes to run this process). You can also consider creating backups from this database server, instead of the main one.
If your main DB server goes down - for whatever reason - you can change the host address in your application to the backup database, and spend the 8 hours debugging your other server, instead of twiddling your thumbs waiting for the Azure Portal to do its thing while your clients complain.
Your database shouldn't be taking 6-8 hours to restore from backup though. If you are including upload/download time in that estimate, then you should consider storing your data in the Azure datacenter, as well as locally.
For more info see this article on Business Continuity on MSDN:
http://msdn.microsoft.com/en-us/library/windowsazure/hh852669.aspx
You'll want to specifically look at the Database Copies section, but the article is worth reading in full if your DB is so large.
Azure now supports Point in time restore / Geo restore and GeoDR features. You can use the combination of these to have quick backup / restore. PiTR and Geo restore comes with no additional cost while you have to pay for
Geo replica
There are multiple ways to do backup, restore and copy jobs on Azure.
Point in time restore.
Azure Service takes full backups, multiple differential backups and t-log backups every 5 minutes.
Geo Restore
same as Point in time restore. Only difference is that it picks up a redundant copy from a different blob storage stored in a different region.
Geo-Replication
Same as SQL Availability Groups. 4 Replicas Async with read capabilities. Select a region to become a hot standby.
More on Microsoft Site here. Blog here.
Azure SQL Database already has these local replicas that Liam is referring to. You can find more details on these three local replicas here http://social.technet.microsoft.com/wiki/contents/articles/1695.inside-windows-azure-sql-database.aspx#High_Availability_with_SQL_Azure
Also, SQL Database recently introduced new service tiers that include new point-in-time-restore. Full details at http://msdn.microsoft.com/en-us/library/azure/hh852669.aspx
Key is to use right data management strategy as well that helps solve your objective. Wrong architecture and approach to put everything on cloud can prove disastrous... here's more to it to read - http://archdipesh.blogspot.com/2014/03/windows-azure-data-strategies-and.html
I have a database with 468 MB.
I deleted all tables and the size of this database is the same.
Why size of this database is the same?
You need to Shrink or Compact the database to reclaim the empty space in the database file.
See this blog post: SqlCeEngine/ISSCEEngine: Shrink Vs Compact