I'm writing some code that needs to work against an array of different database products.
MySql
Sql Server 2000 to 2008
PostgreSQL
Oracle 9i & 10g
Jet 4.0 (MS Access)
MSDE
Sybase Adaptive Server Anywhere
Sybase Sql Anywhere
Progress OpenEdge
We have a set of generic integration tests that we need to run against each database product, as part of a continuous integration build.
I'm sure other people have had to set-up similar test environments and I would like to tap into some of that wisdom - what strategies do you end up using, what worked well or did not work well?
Some thoughts:
Separate virtual machines for each of the products, each allocated a small amount of memory (easier to manage for certain scenarios, or where we have slightly different setup's for individual products).
A couple of virtual machines or even a single virtual machine for all the products (i.e. perhaps an ubuntu box for postgresql & mysql, and a windows 2008 server machine for the remaining products) - I like one or two vm's because this is a more portable environment for running the tests i.e. when on the road / off-site, as my laptop would probably crawl to a halt running 8 or 10 small VM's.
And finally how have you tackled the prohibitive cost of some of these commercial products i.e. Oracle or Progress OpenEdge, and are the previous versions still available i.e. are there free "single-developer" editions available, or cheaper routes to purchasing these products?
We also use vmware for our servers, one vmware host per database. You should be OK with putting several databases on one vmware.
You shouldn't have much a problem with expensive software licenses. Oracle, for example, allows you to have a development license for all of their products. So long as you are not running a production DB on your laptop vmware, you should be OK. Of course, IANAL.
For Oracle, get the Enterprise Edition if that's what your customers will be using (most likely).
Be sure and take advantage of vmware snapshots. You can get the databases configured perfectly for running your tests, run your tests, and then revert the databases back to the pre-test state ready for running your tests again.
What we do here is (mostly) a VM for each configuration required, we also target different operating systems so have something like 22 configurations we are testing. These VM's are hosted in an ESX server, and scripted to start and stop when required so that they are not all running at once. It was a lot of work to set up.
In terms of cost of the Database servers, the kind of software we are building required top-end versions so the testing costs have been factored into the overall dev costs (we just had to "suck it up")
Oracle does have a free 'Express Edition'. It doesn't have all of Oracle's functionality but then if you are also running against Jet then that shouldn't be an issue.
Architecturally, it may be better off creating an integration testing farm of server(s) that are powerful enough to run the variously configured VMs. It could be fronted with a queue process listening for integration testing requests (which could be triggered using svn commits or other source control check in triggers).
For the most accurate range of testing, you probably want to have a separate VM for each configuration. This will reduce the risk of conflicting configs and also increase the flexibility of running a custom set of VMs (e.g. Oracle + MySQL + PostgreSQL and not the others, etc.). Depending on your build process, it may also allow you to run 10-minute builds.
A benefit of running an integration test farm is that if you're on the road using your laptop, you can trigger off the integration build after code check-in, run all the tests and have it notify you of the results. You can also archive each request and the results to help diagnose failing builds.
Oracle license excerpt:
We grant you a nonexclusive, nontransferable limited license to use the programs only for the purpose of developing, testing, prototyping and demonstrating your application, and not for any other purpose.
So, you can download full Oracle versions 9, 10 and 11 and use them free of charge as stated in license.
Check downloads section at www.oracle.com
It sounds like you have a good idea of what to do already. I'd suggest the following VM layouts:
A Windows Server 2008 box hosting the following:
MSDE
Jet 4.0 (MS Access)
Sybase Adaptive Server Anywhere
Sybase Sql Anywhere
Progress OpenEdge
On an Ubuntu or Red Hat box:
MySql
PostgreSQL
Oracle 9i Download it here
Oracle 10g (Use the free express edition)
I don't see any need to test against the full blown SQL and Oracle editions. The free editions are stripped down, so there is very little risk of that code breaking when you run it against the full version of those servers. If you had tested against the full server and went down to the free stuff, then yes some things might break, but by testing with the least common denominator, you should ensure good quality.
Related
We are in the process of rethinking our development environment. At the moment, we all have Elitebook notebooks which are not as fast as we'd like. We're thinking of virtualizing our development environment to a central VM server.
Our developers work in Visual Studio and use SQL Server as a database. We also have a few SharePoint developers who need a 64bit Win2k8 machine for SharePoint 2010. These are already virtual machines with their own local SQL Server installation.
Every developer's machine or VM has SQL Server installed. This requires resources from all boxes, and is challenging when working with a team on a project. Therefore we're looking into the possibility of centralizing the resources into a single DB server. That box would have to run multiple SQL Server instances (each Sharepoint developer needs a separate one to begin with). We also have the need for an older SQL Server 2005 and SQL Server 2000 installation for backwards compatibility. Besides the SQL Server box, the plan consists of a VM session for each developer with the development tools installed. So a developer could just RDP into the development environment, have his own image and make use of the centralized DB server. Test servers will also be virtualized in the same environment.
I'm looking for some tips and best practices on this matter. For instance:
What's the number of SQL Server instances a normal box can take? And if we upscale virtualized cores / memory; is that enough to add new instances? I don't expect heavy usage in dev.
What's the downside on centralizing the SQL Server instances as opposed to keeping a local instance on every development box?
How should this be integrated in a DTAP strategy?
Just some thoughts:
The number of instances is hardware dependent; I'm not sure if there's a mathematical formula to help calculate how many instances you can run on a VMbox, but it sounds like you're going to need a beast of a machine to run multiple instances per developer PLUS development tools (I can barely get SSMS and Visual Studio to play nice on my laptop). Better hardware = more cost.
Developing on a terminal system may have some advantages, but I can think of a few disadvantages as well:
network latency. If your developers work remotely at all, they'll
need a fast internet connection in order to do anything. If the dev
environments stay on their laptops, they could work disconnected.
instance interdependence. Although SQL Server instances are
completely seperate, sometimes you gotta reboot the server. You'll
need to coordinate that with all of your developers (may not be a big
deal if there's only a handful of you).
redundancy/maintenance. If one developer's machine is down, you
lose a person-day (is that even a word?) of productivity; if the
server goes down, your company is paying for a holiday for everyone
:)
Have you priced out what it would cost to upgrade your individual workstations so that you could compare costs on building out a centralized infrastructure?
Again, these are just some challenges that I think you should consider; there are probably ways to offset them, but they may factor in to your decision.
I would centralize the database servers as best as you can. Having all the dev's work off of centralized database instances should make migrating changes between environments MUCH easier. That alone is worth the effort.
For the SharePoint development environment, I'd highly recommend investing in a few books to make sure you go down the right path. You should be able to have all the dev's working off the same development instances for that too. Here is a good book on the subject, I'm sure there are others: SharePoint 2010 Development
As far as making the developer machines VM's too, I would seek the input of the development staff first and foremost and seriously consider Stuart's pros & cons.
I'm trying to use an old laptop (Win2000 P3 1.1Ghz 256MB ram) as an oracle database server for a small group project of 3/6 people on my college.
I don't need much, just some databases with 2 accounts, 10/15 tables, some views and triggers, however even the simplest database requires 1GB of physical ram (2GB of swap) and the installer refuses to overlook this need.
Is there a simple Oracle server database that I can install on this system without upgrading it?
I would really like to reuse this 9yo laptop as a server since it isn't really doing much besides gathering dust at a corner, but all the software seems to have high requirements for something so simple as a MySQL database.
11g won't install on something this small. Given that a Standard Edition license is a couple of thousand dollars, why try to run it on hardware that is worth a tenth of that cost.
You may be able to use the Express Edition (which has the added advantage of being free to use). The requirements suggest it can barely run on 256 Mb. Personally, I'd spring for a memory upgrade. Stick a note up in the college, as there's probably someone who has done a RAM upgrade on a laptop and has some old chips lying around.
I'd also dump Windows and go for a simple Linux install.
I doubt you can install any current version of Oracle on such a system. Have you considered using Amazon or some other hosting solution? See Need info on Oracle database server
I am considering the migration for 4 reasons:
1) SQLSERVER installation is a nightmare, expecially for 1-user software (Even if typically I have 3-20 users, sometimes I sell my software to single users: it is incredible to have troubles installing the DB, while installing the applicatino means copying an exe...). (note my max installation is 100 users, but there is no an upper limit). Software installs in 10 seconds, SQLServer in 1 hour. Firebird installation is much easier.
2) SQLSERVER runs on windows server only
3) My customers have all the express edition
4) i am not using any advanced feature, I am now starting using filestream, but the main reason for this is that Express edition has 4/10GB db size limit
So these are all Pros of moving to Firebird.
Which are the cons?
I can also plan to support both platforms, but this will backfire I fear.
MSSQL server is faster and better optimized for large databases and complex queries, especially if administered properly, while Firebird alows you to run without any administration and just forget about it. Although this penalty affects very small percentage of people using it, before complete migration I suggest you to first just migrate data and then test speed of most complex query on both systems. If speed satisfies you then you are good to go.
I don't see any besides need to thoroughly test all of your existing code for compatability issues.
Firebird is wonderful for server installations or single user installations.
It has an embedded version that is suitable for single user scenarios and you do not have to install anything.
It uses the same database file for both server and embedded database so you can easy go from single user to multi user and vice versa.
I have embedded Firebird 2.5 today in my freeware Software. It's great, and there had never been connection problems. I used multiple processes to do both insert and read long operations simultaneously and it all gone correct as was expected. I am waiting for Firebird 3.0. I recommend Firebird when you don't want to trust on other commercial database software.
If there is only one user you can use Sqlite which is even easier to manage than Firebird.
I have a VPS with Windows Server 2008 R2, 1 GB RAM with Dual Processor Quad Core 2 CORES.
My sample application is running slower than my current website on shared hosting, when I asked the reason the says following reason:
"SQL Server Express tends to be slower compared to SQL Server Web edition. This is due to the limitations imposed in SQL Server Express which is basically meant for only development environment. We have observed significant amount of improvements with SQL Server Web edition for production web facing applications on VM's. Hence, this can be something that you might consider."
Is this logic just because they want to sale the Web edition database or there is really some performance benefit. I googled and I didn't find anything like that between different versions.
Any help on this from experts?
This is due to the limitations imposed
in SQL Server Express which is
basically meant for only development
environment.
That is absolutely wrong.
SQL Express is used in many, many, many production environments, where it can be a great tool.
SQL Express is not throttled in terms of the number of queries per second it can process. However, as others have said in their answers here, it is limited by RAM (1GB), DB size (4GB) and CPUs (1). If your DB is between 1GB and 4GB, you could easily notice a performance difference vs. SQL Web for scenarios where everything fits into memory on one, but not on the other.
Having said that, there are a bunch of other configuration / setup / design options that can have a much larger impact; the details depend on your data, schema, access patterns, etc.
hope that help:
[link text][1]http://www.microsoft.com/sqlserver/2008/en/us/editions-compare.aspx
I have seen the references to VistaDB over the years and with tools like SQLite, Firebird, MS SQL et. al. I have never had a reason to consider it.
What are the benefits of paying for VistaDB vs using another technology? Things I have thought of:
1. Compact Framework Support. SQLite+MSSQL support the CF.
2. Need migration path to a 'more robust' system. Firebird+MSSQL.
3. Need more advanced features such as triggers. Firebird+MSSQL
The VistaDB client runtime is free. The runtime will never "expire at 3am" as you put it. Only the developer tools are licensed in that manner. You need 1 license per developer, simple. We even offer a really inexpensive Lite version with no Visual Studio tools.
Some other benefits
100% managed code - there are no interop or other unmanaged calls in the engine. This is a big deal to some, and others couldn't care less.
No registry access required - Most other in proc databases require registry access to look for parent controls, or permissions. VistaDB only does what you tell it to do, and will even run in Medium Trust.
XCopy deployment for runtime and your database (single file). You can xcopy you application, the runtime, and your database and run. Nothing to install or configure on the machine, no special privileges needed (we can run in Medium Trust or higher).
Isolated storage - You can put your entire database into Isolated Storage and run it from there directly. This makes it very easy to build secure click once applications that write databases in a domain friendly way for corporate environments. There is no need to store the user data on a shared drive or worry about permission mapping.
CLR Triggers / CLR Procs - You can write CLR Code and use them as Triggers or Stored Procs. We have just recently introduced changes to make it even easier to maintain a single CLR Assembly that can run in both VistaDB and SQL Server 2005/2008.
T-SQL Procs - VistaDB T-SQL Procs are compatible with SQL Server 2005/2008. Any procedure that works in our engine will run in SQL Server. That does not mean anything that runs there will port to us. We are a subset of the functionality in SQL Server. But we are also the only way to run T-SQL Procs without SQL Server (SQL CE can't do it).
I personally think one of the biggest features is the ability to upsize to SQL Server later. All of the VistaDB types, syntax, and CLR Procs, T-SQL procs, etc all will run on SQL Server. (You can't take everything from SQL Server down to VistaDB though, it is a subset)
32/64 bit Deployment - VistaDB is a single assembly deployment that runs both 32 and 64 bit without changes. SQL CE requires two different runtimes depending upon the OS, and cannot run under IIS at all. Access has no 64 bit runtime, and the most recent 32 bit runtime can only be deployed through MSI. The 32 bit version of Windows has the runtime, the 64 bit version does not.
Relational Integrity - VistaDB also actually enforces your constraints and Foreign Keys. You can specific cascade update, and delete operations. The person who commented we are like SQLITE is wrong in this regard. They parse constraints, but do not enforce them.
EDIT: They do have support for FK's now in SQLite. But they are not compiled in by default, and do not use the same syntax as SQL Server.
Medium Trust - The ability to run on a medium trust web server is another feature that many will not care about, but it is a big deal. Many third party controls can't even run in Medium Trust. We can run the complete engine within Medium Trust because of our commitment to 100% managed code and least permission required.
- Full disclosure - I am the owner of VistaDB so I may be biased. :)
Well, the main thing is that it is pure managed code - for what that is worth; it works not only on your typical Windows machines running .NET, but works wherever you run the Compact Framework and even works on Mono. Here are some noteworthy bullet points from their homepage:
Small < 1 MB footprint truly embedded ZeroClick
Microsoft SQL Server 2005 compatible data types and T-SQL syntax
None of the SQL CE limits
Single user, multi user local or using shared network.
Partially trusted shared hosting is no problem.
Royalty-free distribution - single CPU deployment of SQL Server costs more than a site license of VistaDB!
One thing worth noting is that Rob Howard's company, telligent, uses it as the default database for their new CMS software, "Graffiti."
I have played with it here and there but have yet to build anything against it.
For me this most interesting feature of VistaDB is that it can be run in Medium Trust environment. Which makes it perfect solution for creating small to medium .NET websites which can be deployed on server by copying and pasting (x-copy deployment).
And almost all windows shared hosting providers (like GoDaddy) won't let you run your websites in Full Trust mode. And also won't install for you any 3rd party binaries into GAC like System.Data.SQLite.dll if you wish to use SQLite for example.
I hadn't seen VistaDB before, it does look pretty cool.
Update: Received a comment from someone from VistaDB - their update model is only for getting new versions. Your old ones won't stop working if your license expires, which is good to know.
Keeping the original post here as IMHO the warning about expiring software licenses is still worth thinking about, even though VistaDB itself is fine.
It definitely seems 'more featureful' than SQLite, but I don't see anything there to justify the cost. The site seems to indicate that you can buy one license for $279, but it implies this is just a 1 year subscription. Would you have to then pay another $279 next year to stop your site falling over?
If so, remember to factor into the 'cost' how much inconvenience it's going to be when you get a call at 3am (murphy's law, it's always 3am) from your panicking customers because their VistaDB license has expired :-(
I've had this experience personally with some expiring software, and it's never good. You can send your customers emails and messages and flash their entire screen blinking red saying "YOU NEED TO GET A NEW LICENSE BEFORE NEXT WEEK" and they'll still never do it, and you'll still get the pain at 3am when it does expire.