In a corporate development environment writing mostly administrative software, should every developer use their own database instance, or should they use a central database instance during development? What are the advantages and disadvantages of each approach? What about other environments and other products?
If you all share the same database, you might have some issues if someone make a structure change to the database and that the code is not "Synchronized" with it.
I highly recommend one DB per developer for the only reason that you don't want to do "write" test to see someone else override you right after. A simple exemple? You try to display product for a website. Everything works until all the products disappear. Problem? Another developer decided to play with the "Active" flag of the product to test something else. In cases like that, a transaction might not even work. End of the story, you spend time debugging for someone else action.
I highly recommend replicating the staging database to the developer database once in a while to synchronize the structure (or better, have a tool to rebuild a database from scratch).
Of course, we require scripts for changes to the database and EVERYTHING is in a Source Control.
The days when database environments should be scarce are long gone. I'm writing this posting on a XW9300 with 5x15k SCSI disks in it. This machine will run a substantial ETL job in a fairly reasonable length of time and (in mid-2007) cost me about £1,700 on ebay including the disks. From a developer's perspective, especially on database centric projects like data warehousing, the line between a developer and a DBA is quite blurred. As I write this I am building a partition management framework for a SQL Server 2005 data warehouse.
Developers should have one or more development databases of their own for (IMO) these reasons:
Requires people to keep stored procedures, patch scripts and schema definition files in source control. Applying the patches can be automated to a fairly large extent. There are even tools such as Redgate SQL Compare Pro that do much of the grunt work for this.
Encourages an application architecture that facilitates easy configuration management and deployment, as people have to deploy onto their own workstations. Many deployment wrinkles will get sorted out long before they hit production or people even realise they could have gone wrong.
Avoids developers tripping up on each other's work. On something like a data warehouse where people are working with ETL code this is an even bigger win.
It encourages a degree of responsibility as developers have to learn basic database administration. This also eliminates a lot of the requirements for operational support staff and some of the dev-vs. ops friction.
If you have your own database, there are no gatekeepers obstructing experimentation or other work on it. The politics around managing 'servers' disappear as there are no 'servers'.
This is a productivity win in an any environment with significant incumbent bureaucracy.
For small data volumes an ordinary PC is fast enough for this. Developer editions or licencing are available for most if not all database management systems and will run on a desktop O/S. If you're working with Linux or Unix this is even less of an issue. For larger data volumes, up to and including most MIS applications, a workstation like an HP XW9400 or Lenovo D10 can be outfitted with 5 15k disks for less than the cost of a lot of professional development tooling. (Yes, I know it's dual licence, but a commercial all-platform licence for QT is about £4000 a seat).
A machine like this will run an ETL process with 10's to 100's of millions of rows faster than you might think.
It facilitates setting up more than one environment for smoke testing or reconciliation purposes. As you have complete control over the machine, you have quite a lot of scope for mocking up conditions in a production environment. For example, I once made a simple emulator for Control-M by just bodging some of its runtime scripts.
Where you have this level of control and transparency over the environment you can produce a fairly robustly tested deployment process which does quite a lot to eliminate opportunities for finger-pointing in production deployment.
I've seen small teams working with 14 environments, and had 7 active on a workstation at the same time. On database heavy work such as ETL, where you're with with whole tables, working in a single dev environment is a recipe for time wastage or spending your time walking on eggshells.
Also, you can use single user development licences for database platforms, which can save you the cost of the workstations just in database licencing. Most developer licences (Microsoft and OTN are a couple of examples I'm familiar with) will let you use the system on a single workstation for a single developer free or for a nominal price.
Conversely, licencing terms on shared development servers are often somewhat murky and I've seen vendors try to shake customers down for licencing on dev servers on more than one occasion.
Each of our developers has a fully functional database. Changes are scripted and source controlled like any other code.
Ideally, yes, each developer should have a "sandbox" development environment, so they can test their code even before deploying it to a shared testing/staging environment.
Each developer's environment should run scripted tests that reset the database to a known state. This is impossible to do in a shared environment.
The cost of giving each developer their own instance is less than the cost of the chaos resulting from multiple developers trying to test volatile changes together in a shared environment.
On the other hand, in many IT shops the system uses complex infrastructure, involving multiple application servers or multiple physical nodes. Then the economics change; it's less expensive for people to cooperate and avoid stepping on each other's work than it would be to replicate it for each developer. Especially true if you integrate expensive third-party systems that don't give you licenses for multiple development environments.
So the answer is yes and no. :-) Do give each developer their own environment if that environment can be reproduced inexpensively.
My recommendation is to have 2 levels of development environment:
Each developer has their own personal development system, with its own dp, web servers, etc. This allows them to code against a known setup, write automated (system level) tests that initialize their database and systems to a known state, etc.
The development integration environment is shared by all developers and used to make sure everything is working together as expected before handing it off to QA. Code is checked out from source control and installed there, and there's only a single instance of any servers (db or otherwise).
This question hints at what a developer needs to do his/her job. Certainly a private DB instance should be provided. Equally important, I would make sure that the DB is the same product/version as what you intend to deploy to. Don't develop on MySQL 6.x and deploy to MySQL 5.x. (This goes for app servers, and web servers as well!)
Having a developer DB doesn't necessarily ean you need it hosted on your local machine. You could have a central DBMS host machine with all dev dbs located on it. The pros are the garauntee that you develop against the target DB. Less overhead on dev boxes, more space/horsepower for beefy IDEs and app servers. The cons are single point of failure for all devs. (The DBMS server goes down nobody can work.) Lack of dev exposure to setting up and administering the DBMS. Devs cannot experiment as easily with upcoming DB releases or alternate DB choices to solve tough problems.
Some of the pros can be cons and vice-versa depending on your organization and structure. Maybe you don't want devs administering the DBMS. Maybe you do plan to support varying DB platforms. The decision boils down to your organization as well as your target platform choices. If you plan to target a variety of DB/OS/app server combinations then each dev should not only have their own DB but should work in a unique combination. (MySQL/Tomcat/OSX for one DB2/Jetty/Linux for another Postegres/Geronimo/WinXP for a 3rd, etc.) If you setup an ASP (Application Service Provider) type shop on an iSeries on the other hand then of course you'll likely have a central host with all dev dbmses still each dev should have at least a separate db instance to allow structural changes to schema.
I have an instance of SQLServer Development Edition installed locally. We have a QA DB server, as well as multiple production servers. All development and integration testing is done using my local server (or other developers local servers). New releases are staged to the QA server. Each release, after acceptance by the customer, is put into production.
Since I mostly do web development, I use the web server bundled with VS2008 for development and local test, then publish the web app to a QA web server hosted on a VM. Once accepted by the customer, it is published to one of several different production web servers -- some virtual, some not, depending on the application.
My department at my company only has limited development environments, purely because of cost of support and hardware. We have a couple of environments which are based on t-1 nightly refreshes from production, and some static ones.
Ideally, everyone should have their own, but in many cases, this is going to be impractical when the following are true:
you have a large number of developers needing resources (our department has maybe 80)
each developer needs multiple resources (typically i use 4-5 different dbs each day)
up to date data is important (you just cant refresh them fast enough)
In these cases, shared instances and good communication are whats needed.
One advantage to one database per developer, each developer has a snapshot of their own data in a "known" state.
I like the idea of using a local version when a developer must be isolated - developing a schema change, performance testing, setting up specific scenarios, etc...
At other times use the shared version as to insure everything is in sync with each other.
I think there's a terminology problem here. It's been a while since I've worn my DBA hat (golly gee, almost 10 years) - so someone else can chime in and correct me.
I think everyone is in agreement that each developer should have his own sandbox schema set.
In MySQL and Sybase/MS SQLServer, each database engine can support multiple databases. Each database is (normally) fully independent of the other database. So you can have one database engine instance, and give each developer his database space to do as he wish. the only problem is if the developers are using tempdb -- there can be collisions there (I think -- this you will need to look up). Just be careful that cross-database queries with fixed database names are not used.
In Oracle, the database engine instance is tied to a particular schema set. If you have multiple developers on the same engine, they are all pointing to the same tables. In this case, yes, you will need to run multiple instances.
Each of our developers has a local database. We store the create script AND a dump of the "standard data" in our SVN repo. We have an extensive set of tests that must pass against this test data. We also have a "sandbox" database that is available for people to put data in that they want shared into the standard data. This works well for us and allows us to let developers modify their local copies of data to test things, but we control what gets passed to other developers. We also strictly control schema changes, so we don't encounter the problems that someone else mentioned.
It really depends on the nature of your application. If yours is a client-server architecture in a distributed environment, it is best to have a central database that everyone uses. If the product gives users an environment with local database instances, you can use that. It is best if your development mirrors the real world environment as closely as possible.
It is also dependent on what stage of development you are in. Probably in the early stages, you dont want to get bogged down by connectivity, network and distributed environment issues and just want to be up and running. In such a case, you can start with a database instance-per-user model before switching to the central model as the product reaches some level of maturity.
In my company we tend to copy the entire DB when working on non-trivial new features. The reasoning there is disk space is cheap, whereas accidental data loss (even if it's test data) isn't.
I've worked in both types of development environments. Personally, I prefer to have my own DB/app server. However, there may be some advantages to using a shared infrastructure for development.
The main one is that a shared environment more closely resembles a real-world scenario: you are more likely to uncover problems with locking or transactions when all developers share a DB. Giving each developer their own DB may lead to "it works on my DB" syndrome.
However, if you need to apply and test schema changes or optimisations, then I can see problems in this sort of set-up.
Maybe a compromise solution would work best: all developers share infrastructure, and if someone needs to test schema changes, they create their own temporary DB instance (maybe there is one just sitting there for this purpose?) until they are happy to commit the new schema to source control.
You do have your entire schema (and test data) in source control, right? Right???
I like the compromise solution (all developers share infrastructure, and if someone needs to test schema changes, they create their own temporary DB instance (maybe there is one just sitting there for this purpose?) until they are happy to commit the new schema to source control.)
One DB per developer. No question. But the issue really is how to script entire databases, "control data", and version them. My solution is here : http://dbsourcetools.codeplex.com/
Have fun. - Nathan.
The database schemas should be held in source control and developers should own the changesets checked in for code and db together. Prior to checkin the developer should be working on his own database. After checkin, an automated build (eg: on checkin, nightly, etc), should update a central integrated db, along with the apps themselves.
At developer instance level the data loaded should be appropriate for unit testing, at least. At integrated level, the shared db should hold data also appropriate for testing, but should not rely on production replication - this is just a slack substitute for managed test data.
In my experience the only reason that developers opt for a shared db is that they believe that developing and running on recent production data is somehow 'real' and means that they can put less effort into testing. They prefer to tread on each others toes and put up with a shared db that slowly corrupts before the next production refresh than write and manage proper tests. It's this kind of management practice that gives the IT world the poor reputation to deliver that it currently has.
I'd suggest to use one instance of the database. You don't want your database to be a moving target.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Consider the need to develop a lightweight desktop DB application on the Microsoft platforms.
It could be done fairly easily with MS Access but I'd like to be able to distribute it to others and I don't want to pay for a runtime license.
Requirements:
easy distribution to others
no runtime licensing issues
Considerations and Candidates:
Base from the OpenOffice suite. My concerns were around its stability.
MySQL + writing custom DB code in C++ or Python or whatever seems like a rather heavy-handed solution.
Question: What are the low cost or free database alternatives to MS Access?
See Also: Open Source Reporting Engines
#Schnapple
Bruceatk kind of hit on what I'm thinking of; it's not so much the DB engine as I want the other niceties that Access brings to the party. The nice form designer, the nice reporting engine etc. But you do raise a very good point about the installation footprint. I had considered that but I've not made any firm decisions about which way I'm going with this yet anyway. It'll probably be something fairly lightweight anyway and a small installation footprint would definitely be a plus.
#Remou,
No I was unaware that the MS Access 2007 runtime is free; thanks for pointing that out. The last time I'd bothered to investigate it (I don't remember when it was) I think it was a fairly expensive license for the runtime because I think they were trying to sell it to Corporate IT departments.
And thanks to everyone else who responded as well; I was completely unaware of those other options you all pointed out.
When people ask about a replacement for Access, a lot of them only think about the database, but what they are really asking about are all of the other features in Access. They usually don't care what database Access is using.
Some of the functionality provided by Access are: Forms, Query Building, Reports, Macros, Database Management, and some kind of language when you need to go beyond what the wizards provide.
SQLite, MySQL, and FireBird are free database back ends. They do not have those additional Access functions built into them. Any free alternatives to Access require you combining something like SQLite and a development language.
Probably the best free option would be SQLite and Visual Basic 2008 or C# 2008 Express Edition. This would have a heavy runtime dependency, so installing on a bare client could take quite the installer.
There really isn't a non-Access option for free with minimum runtime requirements. I wish there was.
I'll be interested in hearing if anybody knows any good alternatives.
Are you aware that the Access 2007 runtime can be downloaded for free?
Links for newer versions:
2010 Runtime
2013 Runtime
2016 Runtime
Schnapple asks:
Are you referring to the concept of a
free database to distribute with an
application, or an Access-like "single
file, no installation" database?
Er, nobody who has any competence with Access application development would ever distribute a single MDB/ACCDB as application/data store. Any non-trivial Access application needs to be split into a front end with the forms/queries/reports (i.e., UI objects) and a back end (data tables only).
It's clear that what is needed here is a database application development tool like Access. None of the database-only answers are in any way responsive to that.
Please learn about Access before answering Access questions:
Access is a database application
development tool that ships with a
default database engine called Jet.
But an Access application can be
built to work with data in almost
any back end database, as long as
there's an ISAM, or an ODBC or OLEDB
driver for that database engine.
Microsoft itself has done a good job of obfuscating the difference between Access (development tool) and Jet (database engine), so it's not surprising that many people don't recognize the difference. But developers ought to use precise language, and when you mean the database engine, use "Jet", and when you mean the front-end development platform, use "Access".
One thing to keep in mind here is the MS Access product is much more than just the raw database engine. It provides a full application development platform, including form and menu designer, client application language and environment (VBA), and report designer. When you take all those things together, MS Access really has no peer.
But for the scope of this question, we're concerned with the raw database engine. With that in mind:
SQLlite,
Firebird,
VistaDB (not free),
SQL Server Compact Edition (not Express)
and now SQL Server LocalDB
all come to mind.
Another thought: while the original question does ask about desktop databases, its likely some people will land here looking for a database to use with a web site. It's important to remember that these are all in-process databases, and as such are rarely if ever appropriate for use on the web. If you want to build a web site, where it's common to need to support significant concurrent access, you generally want a database server engine, like MS SQL, Postgresql, MySQL, Oracle, or their brethren. At the same time, those server engines are rarely if ever appropriate for a single-user desktop application.
To be honest - there aren't any free alternatives to MS Access. At least if you mean database development tool (forms, reports, queries, VBA support etc.). If you think about MS Access as a database engine (you mean MS Jet or ACE in fact) then yes - you have a lot of possibilities. There are a lot of free database engines - the most popular are MySQL and PostgreSQL. I can recommend both - it depends what you want to do.
For writing database frontends C++ is one of the worst choices. You should consider MS Visual C#, MS Visual Basic .NET or... Even Java/Swing (if we are talking about desktop application). If you think about the web-enabled frontend - consider PHP (with MySQL or PostgreSQL on the backend) or ASP.NET (with MSSQL Server at the backend).
I strongly recommend you not to use C++ for such job. This language is very efficient and flexible, but advanced database frontend development with C++ is not the best idea. C++ is great in system programming, games development, maths and physics simulations, everywhere where efficiency is the key - like real-time applications etc. Frontends don't have to be daemons of speed - they should look nice and have advanced end-user features (like sorting, coloring etc.). If you are looking for free tools - maybe C# Express or Visual Basic.NET Express 2008 would be the proper choice? Or maybe Java/Swing (check the NetBeans IDE)? Maybe SharpDevelop? But not C++... Leave C++ for the things it suits the best.
Check out suneido.
I made a fairly complicated GIS app as an experiment with it some years ago (database, complex gui, reports, client/server). It was a pleasant experience (apart from some documentation issues...) and I became productive with it very fast.
I don't use it anymore mainly because:
it's not really general purpose
it's not cross platform (windows only)
I decided to stop exploring exotic
technologies and specialize in something
more mainstream.
Of the Free Software alternatives these haven't been mentioned yet:
Bond
Rekall (not sure about the status of the Windows version currently though)
Glom (Windows version under development)
I'd also keep an eye on what DB RAD tools the Flex/Air community is coming up with, since with those tools it's possible to get unified desktop and web interfaces.
Oracle XE With Application Express.
Has a nice web based gui,
Is a "Real" database
Will scale beyond a single desktop
Offers a clear scale path beyond a small team
Applications as web based, easily accessible.
Can convert Excel spread sheets into Applications
The issue is finding an alternative to MS Access that includes a visual, drag and drop development environment with a "reasonable" database where the whole kit and caboodle can be deployed free of charge.
My first suggestion would be to look at this very complete list of MS Access alternatives (many of which are free), followed by a gander at this list of open source database development tools on osalt.com.
My second suggestion would be to check out WaveMaker, which is sort of an open source PowerBuilder for the cloud (disclaimer: I work there so should not be considered to be an unbiased source of information ;-)
WaveMaker combines a drag and drop IDE with an open source Java back end. It is licensed under the Apache license and boasts a 15,000-strong developer community.
NuBuilder (www.nubuilder.net) might be right.
NuBuilder is a GPLv3-licensed PHP web application that requires MySQL as backend database. Users and programmers both use the web interface.
They promote it as a free, web based MS Access alternative.
I'm creating my second NuBuilder application these days. The NuBuilder seems to be very actively developed, and I found it stable and well documented (provided you can stand video tutorials.)
You may want to look into SQLite (http://sqlite.org/). All depends on your usage though. Concurrency for example is not its greatest virtue. But for example Firefox uses it to store settings etc..
In the context of a programming forum, we don't usually think of the programmer also needing the application portion of the database. Normally a programmer wants to use their own development environment for the business logic and front end, and just use the store, query, retrieval, and data processing capabilities of the database.
If you really want all those other things, then you're talking about a much larger and more complicated run time environment. You're not going to find anything that's 'lightweight' any more. Even MS Access itself no longer qualifies, because it's hardly light weight. It's just lucky in that a lot of users might already have it, making it appear to be light weight.
This doesn't mean you won't find anything. Just that it's not likely to have the same level of maturity or distribution as Access, especially since the underlying access engine is already baked into Windows.
The Access runtime license has never been all that expensive -- the cost for the developer tools/extensions has been around $300 as long as I can remember (which would be as far back to the Access 2 Developers Toolkit, or ADT), but that gives you the ability to distribute your app with the runtime to an unlimited number of users. As long as your runtime app was used by three or more users, you'd have been saving money (assuming a cost of $100/user to install a full copy of Access).
The runtime for Access 2007 is completely free, but really, the cost before that was not all that great.
Marc Gravell added (in what should have been a comment, in my opinion):
Being free, though, is certainly an encouragement for people to try it out which the $300 price really would have discouraged.
VistaDB has an express version which is free to use and is syntax and driver compatible with SQL Server. VistaDB is a single file and only requires their driver .dll to work in your asp.net or winforms project.
Since it is syntax and datasource compatible you can upgrade to SQL Server if needed.
from their site:
VistaDB is a fully managed and
typesafe ASP.NET and WinForms
applications using C#, VB.NET and
other CLR-compliant languages.
VistaDB.net
You mentioned Python, have you considered Dabo?
http://dabodev.com/
That would avoid much of the grunt work in a custom app.
Are you referring to the concept of a free database to distribute with an application, or an Access-like "single file, no installation" database?
As in, things like SQL Server Express Edition require things like runtimes to be installed, databases to be created and mounted, entries on people's Start menus that they won't recognize (my wife asked why SQL Server was on her laptop the other day) whereas an Access database can be run in a single file.
I guess what I'm asking is do you want to think of the database as a document you write to or as an instance of something on someone else's machine?
What about r:Base? Way back in the day r:Base was a very robust DOS (then Windows) RDMBS and this is pre-Access / pre-Paradox days. Its closest competitor was dBase but that wasnt fully relational, at the time. I developed some very nice r:Base applications AND, like Access today, had a built in report generator, forms facility, queries and table manipulation.. To my surprise, its still alive! http://www.rbase.com/ Its got all that access offers, it seems. Might be something for you to consider.
Kexi 2007.1.1 may be what you are looking for.
Its express version is free but DB size limited. Full version cost $72.
The description from its home page:
Kexi is an easy to use application for visual database design for Linux and MS Windows. Kexi competes with MS Access, FoxPro, Oracle Forms and FileMaker.
Visit http://www.kexi-project.org/about.html for details.
Apache Derby is a nice db alternative.
Gambas
Much in line with Aurelio's answer, I now work in Ruby on Rails on some applications that I might formerly have done in MS Access. The back end database for a Rails App. is usually, MySql (works well enough and is available on most shared Web hosting) or PostgreSQL (the better choice when possible).
What about Microsoft's Visual Studio Express?
http://www.microsoft.com/express/default.aspx
SQL Server Express is also at that link...
I'd the same problem of you. I had a MS access application but I wanted to go to a web application accessible to everybody and without paying money to MS. So I decided to use MySql and Wavemaker (open source) to get the scope..I'm very happy of this decision. and that's the result http://www.mara-database.org/
Also check out http://www.sagekey.com/installation_access.aspx for great installation scripts for Ms Access. Also if you need to integrate images into your application check out DBPix at ammara.com
What you appear to be looking for is not just a database program, but a database with forms, reports, etc (basically an IDE of sorts). I would recommend trying OpenOffice.org Base, which comes with the office suite. It's free and open source. It's nowhere near as polished as access, but it does pretty much the same things.
Plus, if you know access, it will be at least somewhat familiar.
http://www.openoffice.org/
EDIT: Sorry, failed to read that you are considering OpenOffice.org. With regard to stability, I've had it crash and do some "odd" things when I played with it, but Access has done the same thing. The best way to find out is to play with it a bit and see if it suits you.
I think the database included with OpenOffice.org has the form designer in it. I've never tried writing code for it though. A forum post I saw had a link to a tutorial they said had some code in it.
I started to set up a database for my wife and the interface was coming out pretty good as far as I could tell.
oooForum.org tutorial
for sqlite, check out the firefox extension. It offers a serviceable GUI.
VistaDB is the only alternative if you going to run your website at shared hosting (almost all of them won't let you run your websites under Full Trust mode) and also if you need simple x-copy deployment enabled website.