Related
I have to develop a new (desktop) app for a small business. This business currently has an Access database with millions of records. The file size is about 1.5 GB. The boss told me that searching on this DB is very slow. The DB consists of a single table with about 20 fields.
I also think the overall DB design isn't great. I thought to use another DB server with a new design to improve both performance and efficiency.
Considering this is a relatively small business, I don't want to spend much for a DB license, so I want to ask you what would you do.
Continue to use Access, maybe improving and optimizing the DB in some way
Buy a DB server license (in this case, which one?)
? (any idea?)
Things like SQL Server Express, MySQL and PostgreSQL are available for free, no license purchase necessary.
For improving search speeds, you will probably also want to look at things like what indexes are defined for the table, what exactly searches are doing, et cetera.
2nd the recommendation for Firebird. We've been using it for about 5 years and never had an issue. Cross platform, embedded & server deployments... brilliant. Oh, and Free as in Beer. Mozilla Public LIcense.
Put me down as another recommendation for Firebird. We use it with our commercial Point of Sale product. We have it installed at over 1,000 sites, with databases as large as 40+ Gigabytes. It's fast, stable, simple, easy to deploy, and requires no management.
Your could replace the Access Database with a SQL Server database that will scale well moving forward. You can use SQL Server Express which is free and supports databases up to 4Gb I beleive.
SQL Server Express. Free for database size up to 10 GB.
SQL Server Express is a perfect fit for this. http://www.microsoft.com/express/database/
I warmly recommend MySQL. Its sometimes free and is easy to install on both Windows and linux.
There are also a lots of great free tools to manage its content like tables, users, indexes etc...
I would look into if the big table needs to be broken up into smaller ones(rarely needed, but still) and also what indexes are on it. And for Database software I would recommend PostgreSQL. It is free, easy to use(and I consider it easy to setup, though others beg to differ), and it is fast enough for enterprise applications.
You can take a look at Firebird
Firebird is one of the best database for desktop application and will allways be free.
Some tools exist to convert database from access to firebird.
I also recommend Firebird.
Its key advantages for your scenario are (from top of my head):
embedded version. You can ship it with your application - no separate installation kit needed, no .NET dependencies etc.
later on you can scale seamlessly to the full client-server model. No code changes required.
very small footprint
the entire database is stored in a single file. Much easier to deploy compared with other solutions.
you can have your server on any platform you want: Windows, Linux, MacOSX etc. Of course, you can have your client also on the same platforms but since you mentioned Access, I suppose that you have a Windows application.
no need for server administration. It just works.
I recommend PostgreSQL as well (especially as an alternative to MySQL)
No-one ever seems to mention it, but Oracle also do a free-as-in-free-beer version of their database: Oracle Express Edition (aka XE). It is limited to 1 CPU, 1GB RAM and 4GB of user data but that sounds plenty big enough for your application.
As for your database design, just one table sounds more like a spreadsheet than a database application. Probably you have lots of denormalised data. Splitting those out into smaller de-duplicated tables might well speed up certain queries. However if you only have twenty columns there may not be a lot of scope for tuning.
As for recommendation, the question is, which products do you know? If you have a familiarity with Access then I suggest you try to optimize your existing database. I have worked with Access databases which store several million rows and they performed well enough. After all, there is no guarantee that moving the same design to a different product will automatically make things run x times faster. Another advantage of Access as a tool is that it comes with a built-in front-end tool. If you move from Access you may need to think about re-building your application.
Wow, folks have a lot of platform recommendations for you.
Let me just say this.
If you feel like there are design issues as well as platform issues, why not try the design changes first? These are changes you are likely to make anyway.
If they make no performance difference in Access, you are no worse off, since these changes improve maintainability on any platform.
Then you can try other platforms with the knowledge that you have a solid design, and that you have not wasted any time.
SQLite can also be a candidate.
I would recommand not going with express anything - yep your small but what if the busness takes off and need much higher loads in the future - surely thats the way you want it to go ?? do not want to have to switch db layers / run 2 ..
I would look at MySQL or Firebird (used to borland interbase) both very high quality.
Please, give me the most serious arguments against this.
Application directly opens a connection to ms sql server, directly executes queries.
So what I'd like to ask:
1) Why it is wrong when the number of users can be up to 1000 executing huge queries?
2) What serious problems can that cause?
3) What should I do?:)
Arguments, the most serious arguments against this kind of implementation!
One of the things to consider is how the queries are done. 1000 queries against a SQL Server DB might be manageable, but 1000 Access queries in which the table is locked, or which are actually joins or views, could use dramatically more memory. It really depends on how the application is written. Some Access apps open a recordset and page through the records one at a time, or fetch a few dozen and work on those, but sometimes Access grabs the whole recordset, for example to allow users to page through data. And I have seen Access lock a set of tables to allow editing of them. That would be bad in your scenario.
Of course, I wholeheartedly agree with the "10 years out of support" issue. That is a guaranteed problem. Mine is only a possibility. And you should probably update SQL Server to a current version also, for the same reason.
What about:
The version of access 97 is totally outdated and wont get any updates, has a crappy look, crappy functionality and in general - IF it requires rework - should be updated.
Problems? You run on a 10 year old out of support platform. What problem can that cause? Well - what about limited support?
Upgrade at least to 2007, better 2010 (coming in a couple of weeks) when you have a momnent time. I personally dispise access based applications (crappy architecture to start with etc.), but if there is one, the update to access 2010 is possible the most painless way to go.
Access 2003 or 2007 would be just fine for the scenario as long as you had an Access developer who was up to speed on how to develop for client/server with large user populations.
Access 97 is still an awfully nice version of Access. I think it's the best version ever produced.
But it is out of support and predates the alteration of default permissions in Windows implemented with the release of Windows 2000. This means that it has some problems in installing with its default permissions (it expects write access to its application folders and registry keys). An installation script can easily alter these appropriately, but you're still left with problems in certain contexts, like trying to run it in Windows Terminal Server/Citrix, where it very often just completely breaks.
I would like to hear an explanation of exactly why someone would choose A97 for new development. Of course, I may be misinterpreting. You may be asking about an existing app, in which case I'd go with "if it ain't broke, don't fix it," and then ask exactly what it is that is perceived as "broken." Those things can be fixed, though it's unlikely that simply upgrading from A97 to something more recent is going to do the job.
I’m currently nearly finished with a brand new application written in access 97 that stores its data in SQL server 2008. As has been said many times before the access/SQL server combination really works great.
Inline with my other applications it is completely unbound using ADO to get the data from the server. I wont drag up that debate again here but it is something you really want to look into as it can offer some great benefits.
Most of the SQL server guide you will find will ask you to check that you have the correct indexes and try to identify the slowest running parts of the system or the ones that get called a lot and then look at making them faster. That might cause you to make a covering index or to denormalise the data in someway.
Generally what is good practice for JET also works well for SQL server, make a good table schema with a good clustered index choice and good supporting indexes and you are 95% of the way there
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
A potential customer has asked me to look at some promotional flyers for a couple of apps which fall into the contact management / scheduler category. Both use Filemaker as their backend. It looks like these two apps are sold as web apps. At any rate I had not heard of Filemaker in about ten years, so it was surprising to see it pop up twice in the same sitting. I think it started out as a Mac platform db system.
I am more partial to SQL Server, MY SQL, etc, but before make any comments on Filemaker, I'd like to know some of the pros and cons of the system. It must be more than Access for Mac's, but I have never run across it as a player in the client / server or web app arena.
Many thanks
Mike Thomas
Calling Filemaker Pro, Access for the Mac is kind of like saying, Mac OS X is Windows for the Mac. They're both in the same category of software, they're integrated programming environments. It's like you have MySQL, PHP, HTML and your editor put together in a GUI. Comparing the two, they both have pros an cons. Here are the pros and cons of using Filemaker Pro vs PHP/MySQL/HTML in my experience.
Pros:
Easy to get started
Easy to deploy locally, turn on sharing and connect from another client
Cross-platform (Mac OS X, Windows, iOS)
There are many plugins available to extend functionality
Includes starter solutions
Anyone with access can edit the program
For the most part, drag and drop programming
Changing field/database/script names after the fact is free
Has some neat built in tricks like built in graphs, tab controls, web viewers
Built in support for importing exporting excel, cvs, tab-formatted
Cons:
Inflexible: it does what it does well, but if you need more your out of luck for the most part
Expensive compared to the free alternative: It costs about $100 per year for a local user, $150 per developer, if you are using it as a website you need specialized hosting, which tends to cost more. In addition the server part of the software is about $300-$800 a year
The plugins required to extend functionality can be expensive as well
Pretty much only drag and drop programming, you can only use predefined script steps, relationships are made by making a graph
Source control is problem
Lack of scalability
Unable to copy and paste/import or export some items from solutions
Requires the mouse to access functionality
Layout design is fairly static and dated (this is improving with the Filemaker 12 and above)
In general I would say that if you're developing exclusively for the web or a large organization Filemaker Pro probably isn't the best fit. It's difficult to have multiple people developing on the same solution. On the other hand, for a smaller organization in need of a customizable in-house database it could be a great boon. You can build rather complicated applications very quickly with it if your willing to deal with it's deficiencies.
Pros:
It's cheap
Cons:
It's cheap(ly made)
It's non-standard (easy to find
MySQL/Oracle/MSSQL/Access experts
but nobody knows Filemaker)
Using subpar and/or nonstandard technologies only creates technology debt. I've never found a respectable dev that actually enjoyed (or wanted to) using this niche product.
In my opinion this product exists because it is Access for Macs, and it gained enough of a userbase and existing applications that enough people bought each upgrade to keep it in business. There are many products on the market that still exist because it's users are locked in, not because it's a good choice.
I'll admit to bias on this subject -- I work with one of the larger FileMaker development shops out there, and have written the odd book on the subject. We actually employ many respectable developers who love using FMP. I'll try to keep it brief. :-)
FileMaker Pro is a rapid app development tool. It's primarily client-server, though it has some very respectable web publishing capabilities which work well for many applications. It is not SQL-based, but does have ODBC and JDBC interfaces, as well as an XML/HTTP interface.
As far as lock-in, FileMaker Inc has grown sales steadily, with very significant growth in new users who are attracted to the platform's solidity and ease of use.
I think Matt Haughton nailed it -- for the right applications, FMP is simply the best choice going. That said, your customer is looking at apps written in FMP Pro, and you need to evaluate those apps on their own merit. They may be good instances of FMP development, or they may not.
To know more about FMP's fitness for the task, we'd need to hear more about the proposed application and user base. Are these indeed web apps, or client-server? How many users will be using it? Do they work at one or two site, or are they spread across the Internet?
Happy to elaborate further if there's more interest.
FileMaker is designed to integrate very simply with other databases and client applications. If you are looking at building a complicated distributed system, look elsewhere.
FileMaker is NOT good to use as a front-end to another datasource due to the design goals of the External SQL Data Sources (ESS) feature set, and it is NOT good to use as a back-end to anything other that the FM client due to slow and buggy ODBC drivers. The nature of FileMaker's architecture means it doesn't scale very well with complicated solutions regardless of how well it can integrate with other systems.
Here's a developer's perspective on some limitations I've found when teaming FileMaker with other back-ends and ODBC clients:
The ODBC driver is limited, slow, and leaks memory on the client-side. The xdbc_listender.exe has similar memory leaking issues on the server side and will eventually crash when it uses a certain amount of RAM. We have a scheduled script to restart it each night.
FileMaker needs to load all related databases into memory before it can connect to a database. If its a complicated database, opening and closing a connection can be quite slow (1-2 seconds) depending on how it is structured, and more so if the database references tables in other FM databases because they need to be loaded as well. I get around this by creating persistent connections that stay open for the lifetime of the application. Although we try to minimize the number of open connections, we have yet to see a performance hit on the server.
The ODBC driver interprets queries in strange ways. For example I ran a query on 76k rows to UPDATE table_1 SET field_1 = 1 and it took 5 mins to perform the query because I think it split the one query into 46k update queries, one for each row. I know this because I watched it update the rows one-by-one in the FM client. So I don't trust the ODBC driver at all.
Here's another example of 3 different queries and how long they took searching on two date fields:
SELECT id FROM table
WHERE datefield1 = {d '2014-03-26'}
.5 seconds
SELECT id FROM table
WHERE datefield2 = {d '2014-03-26'}
.5 seconds
SELECT id FROM table
WHERE datefield1 = {d '2014-03-26'} OR datefield2 = {d '2014-03-26'}
1 minute 13 seconds!
We had problems with how FileMaker cached data from an SQL Express database. We tried to run the command to clear the cache, but it didn't always work (spent a lot of time investigating this).
FileMaker uses pessimistic locking of records; before editing (from the client or as part of an odbc transaction) FileMaker attempts to lock the row first.
The FileMaker Server service "prefers" being stopped using the Admin Console (though the Admin Console may sometimes be unable to stop it either). If the FileMaker Server service stops any other way (including power loss, via the management console, or even a normal system shutdown) then some of your databases may become corrupt. Same if a client crashes during an operation, or if the network connection is lost suddenly. The solution for a power loss is to write a batch script to try and automate the shutdown, and then buy a UPS and program it to execute your script before the juice runs out. And hope it works. Otherwise backup hourly using the built-in scheduler. Aside: SQL server doesn't have this problem because it can roll back uncommitted transactions.
Performing backups with the built-in scheduler actually suspends operations to the database during backup process. ie, if its a large database, then it might take a minute to backup and users will notice the pause because they wont be able to edit/insert, etc.
If you're using the FileMaker PHP API, take note that you can't use AND and OR together in the same request.
Running an intensive query using the ODBC driver might be fast on its own, but run the same query simultaneously (as in a multi-user environment) and it will slow down by about 300% exponentially. You will run into speed issues if you’re expecting a large volume of intensive queries to hit the database at the same time.
We have found that when the FileMaker ODBC driver says it has finished an update/insert operation, it still does not guarantee the transaction is committed; it appears that FileMaker will continue to hold the changes in the server cache until the auto-enter calculated fields are evaluated/indexed and then it saves to disc, meaning there may be more of a delay until the record is actually committed. So really the ODBC write operations are not always immediate writes, but rather eventual writes. This delay will be especially evident in complicated tables with many calculated fields and triggers.
Calculated fields may slow down execution and reading via the ODBC driver, depending on what is being evaluated. Try to read stored values whenever possible.
Using BLOB containers: Not Recommended. Storing documents such as PDFs in a container field will inflate your database file size, take longer to backup and complicate the retrieval and editing of those files via ODBC. It’s much easier to store files on a network share and write to the file on disk.
If you must use FM as a front-end solution to another database, make sure to carefully read FileMaker's Introduction to External SQL Sources.
Also refer to the the appropriate version FileMaker ODBC Guide found on their website.
Just a few comments on the subject
FileMaker is certainly cheaper than some enterprise solutions in licensing costs. However, the real cost benefit is in development time. The development life-cycle is typically orders of magnitude lower than other enterprise platforms (whatever the licensing costs of those platforms). By this I mean days instead of weeks, or weeks rather than months to develop some feature.
There is a strong argument that FileMaker is Access for the Mac. While this was a valid argument a few years ago, FileMaker has come into its own in recent years. It's worth noting that FileMaker is cross platform and used extensively on Windows as well as Mac. That being said there are still huge similarities and differences between FileMaker and Access, the truth is none of them have any bearing on your situation.
While FileMaker is non-standard it does support live connection to MySQL, MS SQL Server and Oracle.
Also, there are numerous FileMaker developers not as much as more standard platforms, but they are definitely about, if you let me know where you are I can put you in touch with a selection of developers in your area.
The important point I want to make is that in the correct context FileMaker is the best thing in the world at what it does - if you try to do something that it's not meant to do, you'll get stuck. However, it could support offices in 4 locations, it can and is being done.
Before you go and rewrite your system in some other platform you should get in touch with a FileMaker expert and see what they have to say about what you've currently got, writing more details on this site and having non-experts answer positively or negatively won't help you. In the end it has to be a business choice of costs vs. benefits.
No need to list anymore "Cons" - but here is a significant "Pro" - Filemaker Go. Once you have your database setup, download a ipad/iphone app (free for FM12) and run it from a mobile device. The database can be stored locally on the ipad/iphone or synced back to a host PC.
I'm sure this mobile solution is possible elsewhere - but the fundamental point is that an entry-level user (and I mean NO previous database experience) can create an impressive solution within a few weeks.
Personal experience: main database running FM 11 hosted on PC under my desk - 4 researchers scattered across the city collecting data on ipads - all syncing back to my PC. Previous solution was using paper and entering in data by hand.
FileMaker is an interesting app :) It started as an end-user tool and it still is one of very few database apps that a non-programmer can actually use. But somehow FileMaker developers managed to make it very scalable. There's no other platform where one can start with a useful tool and end up with a client-server app that for the whole company. In old days they used to have a splash screen that captured this very idea (I only found an imperfect version):
I.e. something as simple as a file cabinet that can grow quite big.
All FileMaker pros and cons come from its origin. As an end-user tool it's very much unlike other DBMS apps. No SQL. No real programming: scripts are basically macros that repeat user actions in a slightly more general way with variables and some logic. Lots of limitations; e.g. a list view cannot have a sidebar; a dynamic value list is always sorted alphabetically; to open a Save As dialog and read back the file name you'll need a plug-in; and so on. For a programmer this can be very frustrating, because most his assumptions will be wrong. And existing apps written by non-programmers are not exactly paragons of clarity and solid design.
But if you manage to overcome the obstacles you'll find a rather good RAD for client-server, single-user, web, and mobile apps, that stays rather usable over WAN, with such niceties as runtime and kiosk mode.
Having said that, I'm not quite sure about generic contact management and scheduling apps in FileMaker. If this is what they are, then they should be unlocked, so the customer can make changes; or they have to be niche apps that do for the customer what nothing else does.
Filemaker is enormously powerful and versatile. Excellent multi-user support. You can create wonderful solutions in Filemaker with document management, web interface, iphone interface, automated publishing support, scheduled scripts, PDF/Excel/HTML reports, XML support, caller ID record lookup, integration of web data (UPS & Fedex linked to order record for example). Extensible with plugins. It's like being in the Home Depot of data. Don't try to build Amazon; other than that what can't you build with it, and faster app dev than most anywhere else?
It has been more than a year now since I run through FM and use it in developing solutions for various clients. The following are my FM experience:
learning curve is much less than using the hard coded industry standard technology;
it can fit well as to industry standards platforms because of it's ODBC and JDBC connectivity. Your data is not locked in FM and other data format can get in FM;
it fits well as front end and back end solutions.
FM can match enterprise platform having a right database design and deployment i.e. workgroup or department oriented solutions. This is data to it's workgroup owner and make it available for other workgroups or departments;
FM is fits well for rapid application development that employs prototyping;
FM has many more capabilities you therein...
I suggest you try it yourself and I'm sure you'll love the stuff FM can offer!
Happy computing...
A little research has made me think that FileMaker is indeed Access for Mac, but perhaps a little more robust. I worked with Access for years, never really liked it, and am glad to be away from it (I always held a grudge for MSFT killing FoxPro, which I did like).
It is hard for me to imagine it as a good solution for a web based app used by offices in four locations around the country, plus many others logging on from home, etc.
Using it does not make much sense when MySQL, SQL Server, etc are available for the data storage and ASP.NET, PHP, Ruby etc are there for the programming.
Mike Thomas
While the comparisons to "Access for Mac" is inevitable, there are some important distinctions that have to be made.
FileMaker databases can be shared out to more than one person provided 1 of 2 things happen. One, a person on your network opens the DB and shares it from their computer, acting as the host. Two, you buy and install FileMaker server which hosts the DBs.
Also it's been my experience that while FileMaker developers LOVE FM, they're having to learn other technologies because more and more government agencies (my primary employer the past 10 years) are moving off of FM and into SQL Server, Oracle and to some extent Access and open source. FileMaker skills are becoming less and less in demand in the public sector, so getting support for these applications is harder and consequently, more expensive.
That being said, we have a FM server and FM 5.5 clients running an application that has been rock solid for the past 5 years.
i've been using FM for more than a year now. i'm doing and providing solutions for SMBs using the SQL standard for several years. i love those SQL stuff, but just a year a ago i run through FM Pro 9 and have it a try. amazingly, i got all i wanted in just a short time. in my experience as developer, FM Pro impressed me the way it does things.
true enough, FM is not an industry database standard but a good number of its features can compensate to what "standard" is being required of. FM pro has live connectivity to MySQL, MS SQL Server and Oracle. for me, it doesn't make sense to speak about standard if you can move your data around from FM to other platforms and vice-versa.
well, this note can't make that much convincing. it's good to try it for yourself... especially now that FM has its new version 10. believe me... you'll love it...
happy computing.
Two points seem to dominate this discussion and need consideration:
Non-Standard and what Government Agencies are doing.
Let's consider the small business owner or the single user both of whom a creating databases to meet their needs.
Now it doesn't matter what the government is doing, this is your database for your employees. Do what you want (as long as its legal, of course).
Non-Standard, well often this is the best idea since what you want to do works for you. Name your fields and tables as you like and later on rename this as you prefer. Don't try this with dbf or sql... Anyone remember those 'standard' file names bks1999.dbf bks2000.dbf Keep in mind that 'standards' exist because someone else wrote them before you arrived, not because they are the best possible idea.
And yes, there are a lot of 'bad' Filemaker solutions but they are working and supporting hundreds of thousands of people. But try to improve one of these bad solutions and compare that effort to improve a similarly bad dbf solution. A renamed field filters effortlessly through thousands of scripts and scripts in related Filemaker files. In a dbf solution it can become a nightmare as each instance has to be manually retyped.
One real test would be to compare how easily Filemaker can work with SQL, etc. as compared to other applications. That might be interesting. I've never done that but I bet I could create a working file in very little time that works with such data.
I have always said that every developer should use and be familiar with all of the tools.
25 years with Filemaker Pro, 3 years with FoxPro, 2 with 4D, etc.
Lots of comments about FileMaker being non-standard. But what is "standard"? By "standard", many people mean that a database supports Structured Query Language (SQL) (ISO Standard 9075) and FileMaker has and continues to support SQL. How every database engine supports SQL is proprietary to every database. Now it might be open source such as MySQL, but SQL is a standard to support, not the underlying language of how it is accomplished.
When most people talk about databases, they are only talking about the backend tables and schema. The front end user interface is frequently something else. And most of them now render those results as html pages via open standards like PHP. Again, FileMaker fully supports PHP calls and Apache or IIS (depending on which OS platform you are on).
So I would disagree with people saying FileMaker is non-standard.
What is unique about FileMaker is its tight integration between the schema and the User Interface. This is similar to Apple's tight integration between hardware and the Operating system, which has some nice benefits. Interestingly, FileMaker is owned by Apple, but I guess that is another topic.
Generally, FileMaker's User Interface is considerably easier to use than most open standards and most people stick to FileMaker's client User Interface instead of web interfaces. There are still a number of things supported only in FileMaker User Interface that can't be duplicated in a web browser.
FileMaker really makes rapid application development much easier with its close integration of schema and user interface. This makes development cost a whole lot less in most cases.
FileMaker's database services can be spread among up to 3 machines giving it primitive load balancing abilities with web services. While FileMaker easily supports hundreds of users, if you go into thousands of simultaneous users, many SQL only databases (eg Oracle, MS SQL Server, MySQL, Postgres) are designed to better spread out the load across more machines. Basically, if you have high simultaneous transactions, FileMaker is not your solution. For example, a company with many point of sale terminals from all over the county hitting it at the same time.
While FileMaker supports SQL and PHP, using it only that way is a waste of the money spent on the license for the FileMaker User Interface. It would not be a cost effective solution to develop a web front end and pay the full FileMaker license cost for only a backend. So, FileMaker's support of PHP and SQL is best combined with companies that have an in-house solution for staff, but also want to integrate that with their web development team for outside customers.
One last note is that FileMaker's tight integration of schema and User Interface makes security much easier. Obviously you have to set up the groups and users and I usually integrate FileMaker with Active Directory (or Open Directory). But when you use the FileMaker Client and Server connections, turning on encryption security is a single checkbox on the server. FileMaker handles all of the certificates and uses an AES 256bit cipher (at least since version 11, maybe before then too). Currently, the US Government considers that approved for up to and including the first level of Top Secret communications. In typical SQL systems, there is a lot of work to configure security on the database end as well as the user interface end of things and it is much more work than a single checkbox.
FileMaker's target audience has been small to medium sized companies, usually with 5 to 200 users, and it is a well priced product for rapid application development of databases for companies of that size.
And I can't end this comment without commenting on how easy it is to create and deploy a mobile solution on iOS devices like iPads and iPhones. FileMaker Go is a free app for use on these mobile devices and they fully support the same user interface and security. In fact, I am aware of one company that uses FileMaker as a front end interface for their Oracle database simply for access on iPhones. Expect a lot more in the mobile market in the future and FileMaker is clearly targeting mobile users.
Just to add my 2¢ to the already given answers: Everything everyone has written in the voted answers is true about Filemaker. The product is robust enough to warrant both positive and negative opinions.
I'm not a pro enough to speak to your concerns but there are a number of large complex applications written in FMP that you may want to look at. Jungle Software is a good place to start.
The down side to FMP for me as a user of some of those apps is that they come with a stack of files. The runtime of a FMP application isn't packaged as a bundle so it can look a bit complex with a large app. We did some tests a long time back because FMP had a reputation of being slow. At that time (12 years ago) FMP needed to index the db or it was slow but once it was indexed it was as fast as anything else we tested. It's big upside for semi pros is that it is very easy to do basic stuff and end up with working tool. My experience with Access was extremely negative so I wouldn't compare it at all with FMP.
In the end it doesn't really mater what it was written in, if the software does what you want and is stable buy it. If it doesn't don't. It is very easy to get data in and out of FMP so the proprietaryness of the db format doesn't really enter into it.
I am creating a desktop app in Delphi and plan to use an embedded database. I've started the project using SQlite3 with the DISQLite3 library. It works but documentation seems a bit light. I recently found Firebird (yes I've been out of Windows for a while) and it seems to have some compelling features and support.
What are some pros and cons of each embedded db? Size is important as well as support and resources. What have you used and why?
I'm using Firebird 2.1 Embedded and I'm quite happy with it.I like the fact that the database size is practically unlimited (tested with > 4 GB databases and it works) and that the database file is compatible with the Firebird Server so I can use standard tools for database management and inspection. Distribution consists of dropping few files in your exe folder.
Simultaneous access from multiple programs is not supported but simultaneous access from multiple threads is (as long as you ensure that only one 'connect' operation is in progress at any given moment).
I have used SQlite3 for a lot of projects (but from C/C++ and Objective-C). It's extremely small -- no dependencies whatsoever -- database is in a single file.
It's the db of choice for Mac developers because it's directly supported by CoreData and on the iPhone -- so there is a big user base (not to mention all of the other users).
I've been using SQLite (via DISQLite3) in FeedDemon for several months, and I highly recommend it - it has been extremely fast and stable. As Javier said, the docs for the library may be thin, but the docs for SQLite itself are very good.
I've used DBISAM on a number of projects. It is completely embedded without even a need for an external DLL. Unlike the others you listed it is commercial. A lot of great features though and very well documented and supported. The have a successor to it that I haven't tried yet though.
Let's see, quick comparison:
SQLite:
dynamic typing in the database
cross-platform files
runs on Windows, Linux, Mac, etc.
public domain
supports transactions
relies on file system security, does not include own security
Firebird embedded:
strong typing in the database
not all SQL datatypes are supported
cross-platform files
Firebird embedded only runs on Windows
Files from Firebird embedded are in the same format as the full server version
Files from Firebird embedded can be copied to a non-Windows server for use
available under a modified MPL ("what's ours is ours and must remain free, what's yours is yours and you don't have to release it")
supports transactions, triggers, etc.
MySQL embedded:
support for SQL features depends on file format
(IIRC) cross-platform files
GPL unless you pay royalties
runs on Windows, Linux, Mac
incredibly popular with the open source crowd
Even embedded databases have their strengths and weaknesses. You'll need to weigh those strengths and weaknesses against what you're doing to decide.
Firebird embedded is our #1 choice because with no code changes, a single user Delphi app with embedded database can be migrated to a multi-user server based deployment without sacrificing any of the high end features (such as stored procedures, triggers, views, etc.). And its a TRUE free database and doesn't GPL your code in the process.
Strongly recommend to use AnyDAC when working with Databases and Delphi - then you can choose to target FB or SQLite seamlessingly.
My preference would be for FB for embedded apps.
Tom
I use Sybase's Advantage Database Server, but I'm also the R&D Manager, so this post is biased. :)
We have native Delphi TTable and TQuery components for both WIN32 VCL and VCL.NET. Direct table access in addition to SQL support makes Advantage unique among many of the other Delphi offerings. Advantage supports large tables (only limited by the number of records, 2 billion) and has a free local engine, which is nice for development PCs and for small customer sites that don't require client/server functionality. Switch to client/server with a single connection property, no other changes.
We have a ton of clients so accessing the data outside of Delphi is also very easy (.NET data provider, ODBC, OLE DB, PHP, Perl, JDBC, etc).
Main Product Web Site: http://www.advantagedatabase.com
Developer's Web Site: http://devzone.advantagedatabase.com
It really depends what you need. For single-user applications, Firebird Embedded or SQLite are probably best choices (and price is right). On the other end, if you need support for large number of multiple users, you should probably use regular Firebird instead of Embedded version (server is simple to install so you won't have much problems here).
And if you need something in between, for a moderate multi-user application, one of flat databases would be better. I found that ComponentAce's Absolute Database better choice for my needs than DBISAM, NexusDB or VistaDB.
It leaves relatively small footprint (no DLLs), it's a single-file db (a must for me), supports Unicode, BLOB compression, crypting, and technical limits seem impressing for a flat database. Moreover, support was good in few occasions when I needed it.
For cons, I have noticed it doesn't support nested transactions, but other than that, I had no problems.
As for size, nothing beats SQLite.
when you refer about lack of documentation, i guess it's doc for DISQLite3. The SQLite docs are quite complete
Take a look at NexusDB. Have used very successfully in the past.
The problem with (embedded) firebird is, that the database cannot reside on a network drive. Also, it is difficult to have a database on a read only drive (CD/DVD).
For some hacks around these limitations see the Delphi Wiki:
http://delphi.wikia.com/wiki/Firebird_tipps
NexusDB offers the full range from embedded, to full client/server / remote. Also SQL2003 compliant, I believe. I'm using it on a few projects, and am very pleased so far, and the fact that it can work in such a wide range of "scales" is a big plus (not having to learn another DB for scaled-up apps, etc).
Look at this embedded database comparison: http://sql-db.cz.cc/, it can be helpful. Most of abovementioned products are presented there: Advantage, DBISAM, Firebird, MS SQL Server, and much more: Accuracer, Apollo, ElevateDB, NexusDB, TurboDB.
I am partial to Component Ace's Absolute DB. Although a commercial product ($), it is solid, easy to use, small footprint and well documented. If you are looking for a huge multi-user application, this is not the way to go, but if your multi-user needs are light (or non-existent) this is a solid option.
I'm using SQL Server Express and the ADO components. Works great. You can run the SQL Server Express install with commandline to hide the complexities from the users. You can also distribute a database that you load by filename. There are millions of SQL server users so solutions to any problems are easily found in the intertubes :-)
I did a websearch to find a fast database package for my Delphi Application. I wanted it to be completely contained in the executable with no external DLLs or libraries required. I originally found Accuracer by AidAim. They had posted how fast their database was and even gave comparisons with other similar packages to “prove” their point.
I wanted to believe their claims but I thought I’d search the web a bit more to find timings of other packages. I was very surprised to find a post at the Delphi discussion forums where a person asked what database to use, and there were 14 different suggestions. One of the responders had done his own timing comparisons and had found Accuracer to be quite slow compared to several others, which Accuracer had (conveniently) left out of their own comparison page.
The post, plus additional followup web research by me, led me to lean toward DISQLite3, a product based on the Open Source SQLite program, but with enhancements to work in Delphi very quickly, with very small overhead, and with command-based calls - which I like. It is actively under development and will soon have an official Delphi 2009 version, although apparently the current version will work under D2009.
Addenum: DISQLite3 Version 2.0.0, released Nov 17, supports D2009.
I know MS access is a comparatively crap db (and expect to be shot down in flames here), but if only small data is needed it may have advantages if ms office is used anyway. For me it was a way to store program data with more flexibility than csv files which is a common approach for scientific code.
You can create an access db from delphi code without having ms office installed using ado & odbc driver (might be necesary to have an initial .accdb file without tables to copy from then populate, I can't remember this detail. not sure licensing situation doing this.
The .accdb extension can be changed to something else & the file password protected (to a limited degree) so its not immediately obvious to users its access if that's desired.
I know a few commercial developers do this method & copied it myself. Found it easier to setup than sqlite, but maybe because I'd already used ado & access in the past.
I have used ScimoreDB. It has its quirks as they give it royalty free and it has its quirks in data types and with some installation issues. This was on a C# project.
If embedded is an absolute must, look at DBISAM.
kbMemTable is a good candidate. Runs in memory, fast, multi-threadding. Used to be free.
Components4Developers
I have used DBISAM and kbMemTable on different occasions.
What I like about DBISAM is that it has great features, and is usually very reliable. I have used it in large databases, full-text search, read-only mode, CGIs and many other situations.
It is fairly large compared to kbMemTable or SQLite based components, though. And you can't have a single file per database (or even table) - depending on the situation, that is a major disadvantage.
kbMemTable is tiny and it's great for small amounts of data. Since it runs in memory, it has to be a small amount of data, of course.
One other option I've taken on a couple of my desktop apps is dumping the data directly from/to my object hierarchy using TWriter/TReader. This is by far that smallest option, and is absurdly fast compared to using a database. The data files are tiny, too.
It has all kinds of drawbacks, though - you have to code versioning in if you might want to ever add/change fields, unless it's in-memory it is even more complicated, no multi-user support at all, etc.
Firebird embedded is our #1 choice as well. And the suite Unified Interbase v2.0 with it. A great and stable solution!
I have a database that I have to record 5 field data for every 20 sec for 10 days.. 3 field are integer , 1 field is double ( time ) and 1 field is string[5].
I am still using Delphi6 srv2 because of my components. Newer delphi versions are terrible at components that I have to spend thousands of dollars of money to rebuild my component library. Therefor delphi 6 is still best for real commertial applications that never version of delphis give many problems. At many points such as USB or comport readings so on... they release newer ones before previous versions never sit on market.
I have setup a code with Delphi6 what appends 43200 records at a table for test because I will deploy the table in application while it has 43200 records. I will shown all the data on DBChart.
Test result is below databases filled the tables by insert command with 43200 records
Dbisam = 34 sec,
ElevateDb = 11 sec,
AbsoluteDB = 45 sec,
SQLlite = 32 Minute,
Firebird = 12 min,
MSSQL12 localDB = 28 Minute,
Easy table = 8 minute,
BDE = Blocked ,
I havent tested oracle , blackfish , sysbase, nexsusDb etc.. but it seems they will also very slow. I have connected with DBChart and only elevateDb and absoluteDB has loaded 43200 records on DBchart in exceptable time such as 7~10 secs. Other all taken minutes. So slower databases always needs coding tricks to succeed in some real jobs..
I have tested their search speed as well by locate command that unfortunatly the server based databases are always slower in.
MSSQL and SQLLite3 are extremely difficult to manage in to delphi that they made me very tired.
These are my test results
At the end I decided to use AbsoluteDB, Dbisam and Elevate. I have thrown the rest off the PC .
Elevate software doesnt support recno function that requires extra codes at runtime to manage. This makes the database slower Other bug is with Elevate software is autoinc fields. There is no way to reset it . Therefore I have not chosen the Elevat software even it is the fastest database. They say many good functions but how many of them we use it in fact . They just left the most important functions not supported but fixed many many unnecessary functions. and it seems since 8 years there is no any advantage either.
If you want to see with your own eye pls just try and see..
I am thinking between two now absolute DB or DBisam4
Firebird all the way. Does pretty well everything and so far version 2.1 is very solid.
FireBird offers the opportunity to scale up to multi-users sometime down the line, or if you need concurrency (if your application goes multi-threaded).
SQLite is quite unrivaled if you only need single-user access, no other database comes close to it on any aspect, be it performance, convenience, SQL support or stability.
Firebird is really awsome and has a small footprint so you can use embedded
and it can be scaled upward for many users
and does unicode faily well
I use devart components with delphi 2009
and FIB plus for delphi 6/7 (their version for 2009 and unicode is not ready yet too bad)
Hmmm, no one has recommended the BDE - I wonder why that is ;-)
BlackFishSQL is another possibility, although I haven't tested in depth as yet.
when it comes to embedded databases the first question is : is it multiuser ?
Actually,who needs a database that does not allow multiple connections (read&write) to it ?
I have tried (intensly) all mentioned databases and found only one that actually functions the way it should. And that is Accuracer.
The only pity with accuracer is that its a three man band and chronic lack of proper support. It also is mainly static in development as we have seen no real features in years.Not surprising since only one person actually develops it. It seems they are living on old fame. Users praise reflect that (usually 10 years old comments).
For a single user experience I would recommend Absolute Database.
As for major players I would recommend SQL Server from Microsoft. Oracle has become a bloatware and is slowly dying out.
ps
what is nice in accuracer is that their embedded database functions just like full blown server. It locks only current record if its in use while the rest functions normally. Nice database. Pity only it is stagnant.
I would like to build and deploy a database application for Windows based systems, but need to live within the following constraints:
Cannot run as a server (i.e., have open ports);
Must be able to share database files with other instances of the program (running on other machines);
Must not require a DBA for maintenance;
No additional cost for run-time license.
In addition, the following are nice to have "features":
Zero-install (e.g., no registry entries, no need to put files in \Windows\..., etc.);
"Reasonable" performance (yes, that's vague);
"Reasonable" file size limitations (at least 1GB per table/file--just in case).
I've seen this question
Embedded Database for .net that can run off a network
but it doesn't quite answer it for me.
I have seen the VistaDB site, but while it looks promising, I have no personal experience with it.
I have also looked at SQLite, and while it seems good enough for Goggle, I (again) have no personal experience with it.
I would love to use a Java based solution because it's cross-platform (even though my main target is Windows, I'd like to be flexible) and WebStart is a really nice way to distribute software, but the most commonly used DBs (Derby and hsqldb) won't support shared access.
I know that I'm not the only one who's trying/tried to do this, so I'm hoping I could get some advice.
I'd go with SQLite. There are SQLite bindings for everything, and it's very widely used as a embedded database for a large number of applications.
I use SQLite at work and one thing that you should keep in mind is that its file based and uses a file lock for managing concurrent connections. It is not a great solution when you have multiple users trying to use the database at the same time. SQLite is however a great database for one user application, its fast, has a small foot print and has a thriving community built around it.
If you've got VStudio sitting around, how about SQL Server 3.5 Compact edition? MSSQL running in-proc.
http://www.microsoft.com/sql/editions/compact/downloads.mspx