My company's product has a big relational database storage(around 20 TB) and we want to move all or some part of our database. Now I am a bit confused regarding which cloud SQL service will be suitable for me. I have read that some cloud storage only provides specific SQLs like MySQL ... which I cannot afford, the reason is that here we have lots of Transact-SQL scripts on our MSSQL server, which periodically updates the data.
So what should be my basic approach to achieve that, How can I compare between various cloud-based relational database storage like Google Cloud SQL, Azure SQL and Amazon RDS?
If your existing SQL server is Microsoft SQL Server then it probably makes the most sense to use a Microsoft SQL server offering in a cloud environment. That will help minimize any sorts of compatibility issues and also minimize the need to learn a new SQL server environment.
Amazon's RDS service lets you choose between various versions of MySQL, Postgres, Oracle, and Microsoft SQL Server. With regards to Microsoft SQL Server, Amazon offers an express edition, as well as SE and EE versions. As far as licensing goes, they offer both "bring your own" as well as pay-by-usage licenses. More information on the license details can be found in their RDS documentation. They also have a usage calculator that lets you estimate what your monthly costs will be based on the license you use, the size of the instance, etc.
Since Azure SQL is a Microsoft product I would hope that it's compatible with Microsoft SQL Server but I have never used it so I honestly have no idea if there are any compatibility issues. Microsoft also has some basic pricing information available for Azure SQL so you can work out an estimate of your costs if you go with them.
If you're concerned about any scripts working properly on one of the cloud platforms then I'd suggest creating a small test database that you can thoroughly test those scripts with. Upload that test database to the different clouds and test the scripts there to ensure they function properly. Amazon offers a very light-weight RDS instance for free so you can do basic testing without it costing you anything. Perhaps Azure offers something similar. Even if they don't or the free RDS instance isn't powerful enough for your testing needs, spinning up an instance for a day or two to run these sorts of tests shouldn't cost you very much.
Related
As the graph shows above. Which type of sql server edition is for the production? I know the "Developer" version is for the development. Also I already got the Azure VM and have the "Developer SQL Server" on my VM, how can I install the production SQL Server in my VM? Is this free?
My purpose is to make a production SQL Server database. Is there two options for me? One is install a production SQL Server in my VM, another is to create a new Azure SQL database. Which one would be the best way to do this?
As the graph shows above. Which type of sql server edition is for the
Production? I know the "Developer" version is for the development.
Also I already got the Azure VM and have the "Developer SQl server" on
my VM, how can I install the production sql server on my VM? Is this
free?
Developer is not licenced for production - you aren't allowed to use it for production purposes.
Express is licenced for production but if your database gets bigger that 10Gb then it is unsuitable for your use
My purpose is to make a production sql server database. Is there two
options for me? One is install a production sql server on my VM,
another is to create a new Azure SQL database. Which one would be the
best way to do this?
"Best" doesn't mean anything. What are your constraints? Are you creating a brand new database? What tools will be connecting to and using the database? Does it need to be accessible from the internet?
If this is a brand new application / database, and you have limited capability for maintaining a VM then I definitely recommend using SQL Azure instead of a VM
Most importantly, and based on you other question, make sure you understand the term "Production"
Can Azure SQL Server on VM be the production database?
For example, you usually don't provision a 'production' environment without also provisioning at least a dev environment.
It would also help to give us some background on "My purpose is to make a production sql server database". It sounds like this is a request someone has given you but maybe you don't fully understand the term 'production'
None of them are sql azure. They are all sql server on premise on a virtual machine on azure.
If you want to add a sql server azure, you go on your main azure portal page, then on the left panel at the top "create a resource", then chose "SQL Database" on the popular column. Then follow instructions.
If you want as less pain as possible, clearly choose sql azure instead of a sql server on premise on a virtual server on azure. It is by very far the only reasonable choice if you work on azure: cheap, strong, backup automatically done, disaster recovery extremely easily applicable without any prior setup, extremely easy to up size its capacity in case of overload, perfectly secured without any prior setup either.
The only problem of this is the security: it can be accessed only by recognized IP addresses that you mention on the azure portal. So typically, you mention the IP address of you development computer. If your website is on your azure subscription as well, you don't need to worry, it will go through the azure firewall with no setup.
If you reeeeeeally want to use a sql server on premise, well, don't use the developer edition. If you want to avoid performance and load issues, don't use express. The entreprise is very complete, but not useful in most of cases for simple application (like web applications).
If you want most of features, go for the Standard edition, if you want to keep focused on the database engine for your web application, go for the Web edition.
Finaly, if you wanna have a licence free edition on a virtual server, the express is free of charges, but not powerful and extremely limited. The developer edition is free of charges as well and contains every possible and impossible features of sql server. The only pb of this edition is that you are not allowed to use it in production. Only for tests and developments.
As you are already having Azure VM with Developer edition installed you can go for either of the below options.
If you are comfortable to manage the Azure VM yourself, go for IAAS(Infrastructure as a Service) approach: Install SQL Server Standard Edition or SQL Server Enterprise Edition (based on your application needs). Read the capability difference between them. If SQL Server Express edition would be suitable for your needs, then install the same. It does have limited features and many constraints. See the scalability support for different editions in the same link above.
If you want to offload the database server management, go for PAAS(Platform as a Service) approach: Create a Azure SQL database and point your application to it. Azure SQL database is more like SQL Server Enterprise Edition with some limitations like CLR not being supported. Read Azure SQL database differences with SQL Server editions
I'm signed up to Amazon Web Service (AWS) free tier, which allows you to run some very limited instances of EC2 and RDS for free.
I've created an RDS instance running SQL Server 2012 Express - this does not allow remote desktop access, so I also have an EC2 instance running Windows Server 2012 base, and I have installed SQL Server Management Studio and I'm able to connect.
I would like to import some kind of sample database - for example, Northwind, AdventureWorks, or anything that has at least a handful of tables and relationships. The reason is I want to just experiment and practice on a few things (for example, I want to try out the EXPLAIN feature to get to learn how the relational algebra works and how SQL plans out various test queries).
My problem is that Amazon's RDS does not allow me to restore a .BAK file, as local disk access is not allowed, and neither is FILESTREAM, so I can't just download a sample database backup and restore it.
Amazon document their workaround here, which involves creating a local instance of SQL Server on the EC2 instance (my machine with SQL Server Management Studio installed, which I have RDP access to), and use the Generate and Publish Scripts Wizard to import data from the EC2 to the RDS. My problem here is that I don't have enough disk space to install a local database instance - as I'm on the free tier I'm limited to 30GB total disk space for the server, so the OS + SQL Server Management Studio leaves just a couple of gigs. So I'm totally stuck.
Does anyone have any genius ideas here? Maybe some pre-made bulk insert scripts I can use (I can't find any) or some other solution? The database doesn't have to be big, just something to work with. Of course I could always pay to increase my disk space, and go through the whole tedious process outlined by Amazon, but I'd rather not!
Thanks
I got a perfect answer over on dba.stackexchange.com - there's a mini version of Northwind (2.6MB) here which has a script for creating the schema / importing the data. Thanks Tripp for suggesting I ask the question over there!
I want to start a rather big project (in the end). My tool of choice is basically ASP.NET MVC/WEB Api.
I would like to develop this in the beginning by using SQLite or SQL Developer edition.
Then when i go live i would plan to go to a shared hosting by using SQL Server 2012/2014 instance.
My target is that if everything is ok to convert to cloud and Azure.
So my plan that i would like to enhance/verify is:
Frontend:
Since my knowledge of AngularJS/Knockout e.t.c is lomited. I will start by using the normal Jquery/web API approach and gradually i will conver tot latest best practices and standards.
Backend:
Program to an Interface. I will have an interface for my DAL
Implement different interface for my different cases (sqlite/sql server/azure e.t.c)
In order to get a fast start to building it i would start with a good domain class design and then head to SQLite in combination of a light ORM (Dapper) in order to have my logic up and running fast.
This post: SQl Azure offline , suggests that i could start developing for sql azure right away, but i don't know if i can afford from the beginning to go to azure.
Sql Azure is basically the same as Sql Server 2014. If you start with the free SQL Server Express edition then you will have very few problems moving to SQL Azure later, especially if you use a tool like the SQL Server Migration Wizard and pehaps setup a free Azure trial account so you can test as you go?
You can just use your local Sql edition as mentioned by #Neil Thompson, If you are using SQL Server Database Project, there is an option to target your deployment to SQL Azure database instead of full version of Sql Server,
I have been using above option without much problem.
Sql Federation in Azure which is I think the only major non-supported feature in on-premise/local Sql edition that you would need Sql Azure for day-to-day development/testing.
I have a client that runs a small business. They need a custom database solution and I'm looking into various options. My experience is limited to .NET using local SQL Servers (no ASP.NET), however, this client is non-technical and would benefit from being able to outsource the DBA tasks. I'm a complete Azure noob, I just scanned the tutorials and they all appear targeted at developing MVC ASP solutions. The client doesn't need a browser based solution. A fat desktop client used from different geographical offices would be the least expensive option I can deliver. I'm just trying to save some time going through all the tutorials and docs only to find out that this isn't what SQL Azure is intended to do. In effect my questions boil down to:
Can I develop a C#/.NET WPF desktop application using Entity Framework 4 and have it hit SQL Azure instead of a local SQL Server?
Are there any known gotchas with EF4 and SQL Azure?
Are there other hidden development costs/complications in using SQL-Azure instead of a local SQL Server.
Is the basic tool support the same? One specific example I can think of; do I get a SQL profiler tool for troubleshooting?
The final question is security related and I'm not sophisticated enough to ask a good question, but is hitting a SQL Azure db this way considered a security no no?
Yes, you can, but a more suitable approach would be to use WCF Data Services or another form of web services (asmx or WCF) as a services layer for your application. I like this approach for line of business applications. I hate web apps for line of business and by using a services connected WPF desktop application, you get the benefits of running in the cloud and having a cloud offering without the necessity to be HTML based.
SQL Azure has full support for EF these days. In the past there were some issues, but I have not encountered any these days.
In terms of development costs and complications - the Azure desktop hosted environment is a bit of a PITA from a development perspective, but I haven't had major problems. You lose the ability to share a local DEV SQL Server unless you use a hosted instance....of course there's a development cost in that because you have to pay for usage.
Good point! SQL Azure does not provide SQL Profiler support at present. I personally use the built in EF tracing support for this functionality.
Exposing a SQL Azure DB directly isn't a good idea from a security perspective. That's why I suggest hitting a WCF Data Services (or other web services) endpoint in point 1.
You can develop a desktop or on-premise application that uses SQL Azure for your database.
You need to take the standard Azure precautions - assume that connection failures will occur and ensure that your application has retry logic to restore operation. Also note that SQL Azure will terminate any operations that take longer than a minute, to preserve the service for other users. If you have lots of data and some nasty queries, that might be relevant.
EF works fine with SQL Azure. There are some limitations to SQL Azure, itself, which you can read about from the documentation on Microsoft's web site. If you design you database for Azure, it'll work fine on SQL Server or SQL Express (but not necessarily the other way around).
In addition to the monthly charge for the database, you will pay for data that leaves the data centre. Design your application carefully to minimise the amount of data that is retrieved from the database. You no longer have to pay for data going into the data centre, which helps.
You can still use SQL Management Studio and Data Connections within Visual Studio. No SQL Profiler, though. There are a few irritating things you can't do with Management Studio, but nothing insurmountable.
You will have to open up firewall rules for access to the database, but hopefully, they'll be limited. Authentication is by SQL Server credentials, not integrated authentication.
I wouldn't tend to do it this way, but it works.
I have a project requirement to choose a light database for the application.
It's required to choose between SQL Server Express Edition or SQLLite. Which one would be efficient and reliable. I am supposed to run it under Windows 7/Windows 2008 R2. I am newbie in the database programming. It would be helpful if you can share some information in terms of
Reliability
Stability
Size Limits
Memory consumption
Performance
SQL Server Express and SQLite aren't really comparable database systems.
SQL Server Express is the free version of Microsofts full SQL Server product, a standalone database server (often run on a dedicated machine) that client applications connect to. It is designed for things like web applications where many users will be using the database at the same time and there is a requirement for high availability.
SQLite is a compact in-process database that is often used in applications that benefit from having access to a SQL based database system however it isn't feasible to install a full standalone database alongside the application. For example Firefox uses SQLite to store bookmarks and Adobe Lightroom uses it to store its photos catalog, there are also several mobile (e.g. iPhone) applications that use SQLite.
The Microsoft equivalent to SQLite would be SQL Server compact edition (CE) which is free to distribute and use. SQL Server CE is very similar to SQLite in most respects.
Similarly the non-Micrososft equivalent of SQL Server (both the Express edition and the full edition) would probably be MySQL.
Although there are crossovers (you might build a small web application that uses SQLite, or a large desktop application that requires users to install SQL Express) typically the choice between the two "types" of database system (in-process vs standalone database server) is down to the type of application being developed.
You might find more details in the following article: http://erikej.blogspot.com/2011/01/comparison-of-sql-server-compact-4-and.html