I am relatively new to database programming. I use firebird 2.5 with IBPP. I have at least two applications using the sampe firebird database. I want to connect with the embedded variant (fbembedded.dll, icudt30.dll, icuc30.dll), since it will be a host application on customer PCs. I wrote a simple test application reading data from the database and started this application three times at the same time. Everything worked.
But now I am not shure if this works always and if this works stable without the danger to corrupt data. Because when I have a connection with the database with the viewer ibexpert my test application cannot connect to the database. Additionally, the documantation sais (firebirdEmbedded):
You can have multiple embedded servers running at the same time, and
you can have multiple apps connecting to the same embedded server.
Having a regular server already running isn't a problem either.
However, an embedded server locks a database file for its own
exclusive use after successful connection. This means that you cannot
access the same database from multiple embedded server processes
simultaneously (or from any other servers, once an embedded server has
locked the file).
Is the documantation right? My sample application seems to show the opposite. I had a firebird superserver installed on my pc a while ago but uninstalled it before testing this.
The document you refer to is based on Firebird 2.0 or 2.1. The 'server' architecture of Firebird Embedded on Windows was changed in Firebird 2.5. Before Firebird 2.5, Firebird Embedded on Windows behaved as SuperServer, meaning it required exclusive access to the database file.
Starting with Firebird 2.5, Firebird Embedded on Windows behaves like the SuperClassic server model, which means it uses shared access to the database files, and that the same database can be accessed by multiple Firebird Embedded applications and Firebird servers in the Classic or SuperClassic server model (but not SuperServer) if they are running on the same machine. The downside of this change is that embedded applications need to be able to create, read and write the shared database lockfiles (in C:\ProgramData\Firebird).
You don't need to worry about corruption: if the embedded engine can't access the shared lockfile, the connection will fail. The reason you can't connect with IB Expert, is probably that you attempt to connect through a Firebird server with the SuperServer model (which requires exclusive access).
See also the Firebird 2.5 release notes: Changes in the Firebird Engine:
The embedded server in the Windows library, fbembed.dll, now uses Superclassic, not Superserver as previously, thus unifying its model with that of local connection to Superclassic on POSIX. The database file-lock that previously restricted connections to a single application space is replaced by a global lock table that allows simultaneous access to the same database from different embedded server modules. This facilitates concurrent debugging of applications and use of native utility tools like gbak, gstat and so on.
Related
Given a multi-developer project using some sort of version control, and in which one part of the project would include using a desktop application in which the users would add real production data to a sqlite database file, which will then later be used as input/configuration data in an another application. How this file should be distributed between the developers? In version control? On a shared network drive?
You can also switch the sqlite to something else, the only requirement is that it should be embeddable in the desktop application.
Sounds like you have distributed uses with local desktop applications, and you want to have them enter and save data into a shared database.
The client desktop applications can simply connect to a database server using TCP. You can't use Sqlite for this, as it is a file-based database and not a database server. You could use PostgreSql though.
Other options:
local database server on each client computer that synchronizes with master databases or other client databases using replication. Generally speaking, these should all be the same "brand" of database.
local file database on each client computer (such as sqlite) that synchronizes with master database or other client databases using a tool like SymmetricDS or Daffodil which lets you sync different databases over HTTP.
I've created client-server applications before where the database server is on one machine and the clients all connect remotely to that machine/server.
However, it's occurred to me that applications with a lot of locally stored data (for example a music player with a library), must be using a local database server that installs and runs alongside the application.
So my questions are:
Is it just a standard installation of mySQL/Postgres/whatever that is bundled with the application, or is there something specific that should be used when there's only ever one local connection to the server?
Are there easily locatable instructions out there for how to deploy the server software silently with your application?
My question is intended for the general case, but the application I'll be developing will be in Matlab if that makes any difference.
I suggest using a non-service based database when storing data for local application use. I'm aware of SQLite, however I believe MS SQL Compat is similar.
If the files being stored are large (music files), it is probably more efficient to save the file on the disk and store the file path in the database. (See http://www.sqlite.org/intern-v-extern-blob.html)
You can use ODBC to communicate with databases in MATLAB. (See http://www.mathworks.com/help/toolbox/database/ug/database.html)
A colleague I work with recently told me that SQL Express and MS Access were essentially the same thing; that does not seem to be an accurate statement. I know you can convert Access to a SQL DB and maybe under the covers they are similar, but I would assume that the SQL DB engine and what is used to run access are not the same. Not only that, but the SQL statement syntax, etc. I know are not the same.
I am mainly trying to understand so that I am more informed about the versions.
Um, no, not the same.
First off, I need to clear up some terminology. MS Access is a Rapid Application Development (RAD) tool that allows you to quickly build forms and reports that are bound to relational data. It comes with a file-based database engine (Jet/ACE).
Access the RAD tool can be used with many different backend databases (Jet, SQL Server, any db that supports ODBC, etc). I have to assume your colleague was specifically commenting on Jet/ACE, ie the database engine that MS Access uses.
I think the single biggest difference between the Jet/ACE database engine and MS SQL Server Express is that Jet/ACE is file-based and SQL Server Express uses a client/server model. This means that SQL Server Express requires a running service to provide access to the datastore. This can complicate deployment in some scenarios.
SQL Server Express is really just a throttled-back version of SQL Server: max database size of 4GB (10GB in 2008R2), only uses a single physical CPU, etc. These limitations are imposed to prevent large organizations from using the freely available Express edition in place of a full-blown SQL Server install. The upshot to this is that SQL Server Express offers a truly seamless upgrade path to SQL Server. It is also (generally speaking) a more robust and fully featured database management system then Jet/ACE.
Similarities
relational database management systems
written by Microsoft
Differences
MS Access
File based
free distributable runtime (2007 or later)
RAD tools (form/report designer)
uses Jet SQL
max file size 2GB
SQL Server Express
Client/Server model
free
no RAD tools
uses Transact-SQL
max database size 4GB (10GB for SSE R2), max one physical CPU
I think what your colleague had in mind was SQL Server CE, which is a super-lightweight embedded database, which is still (IMO) far superior to Access in database-management aspect. SQL Express cannot even be compared with Access without offending the former.
Here are the datasheets for both products so you can see some hard facts on the difference between the two databases.
Access:
http://office.microsoft.com/en-us/access-help/access-specifications-HP005186808.aspx
SQL (Express is listed on the far right column):
http://www.microsoft.com/sqlserver/2008/en/us/editions-compare.aspx
The comment I have always read is that Access is great for single user single access database use, the minute you scale beyond a single user look elsewhere. While that may be a "bit" of a stretch, Access really does not do well in a multi-user environment. From experience we've had a client who has ignored and ignored our requests to migrate a backend database from Access to SQL, and there have been numerous occasions where we have had to restore from backups, or take the Access database offline due to corruption.
They are two completely different technologies with two different target markets. The database engines are indeed different, as you mention T-SQL is different than Access SQL.
You can "scale up" an Access database to SQL by creating an SSIS package or other tool to do the import, but this takes the Access schema and data and migrates it to a true SQL database. It does more than just attach the Access database or the like.
Anytime you need a "real" database I'd highly recommend looking at any of the SQL versions that are available over Access.
Just remember that with MS-Access you don't have size limitations if you play your cards right. There is no reason, for example, not to have many 2 to 4 Gig tables each contained singularly in their own database. Your ODBC applications can open a connection to multiple MS-Access databases and query the single table in each. So you can have a database containing trillions of records, stored in multiple MDB files. One company I went to work for was using a single MS-Access database to run a issue tracking system done in MS-Access forms. They could only use it one person at a time because of sharing issues that would lock MS-Access up. I wrote a Win32 Perl native Windows GUI user-interface to the database that was better at field/record validation, and my ODBC code was able to manage the connection for simultaneous user access. I managed the opening and reading and writing and closing of the database for each user through my Perl program. I did not leave the database open. I did not maintain a persistent connection for each user, but instead only maintained a connection long enough to retrieve a record for edit. Then I closed the connection until it was time to write the record back to the database. Also, I wrote my own record locking program logic by maintaining a user login table that contained the record id of the record a user was currently editing, then erased that entry when no longer editing that record. When another user went to edit the same record, the program checked if that record was currently open for edit by another user. The system worked flawlessly. MS-Access never locked up via ODBC and multi-user access. I even embedded the password to the database in my compiled Perl program so that no one could get to the data in the Access database other than through my Perl program.
I have a two-part application, where there is a central database that is edited, and then at certain times, the data is released and distributed as its own application. I would like to use a standalone database for the central database (MySQL, Postgres, Oracle, SQL Server, etc.) and then have a reliable export to an embedded database (probably SQLite) for distribution.
What tools/processes are available for such an export, or is it a practice to be avoided?
EDIT: A couple of additional pieces of information. The distributed application should be able to run without having to connect to another server (ex: your spellchecker still works even you don't have internet), and I don't want to install a full DB server for read-only access to the data.
If you really only want your clients to have read-access to the offline data it should not be that difficult to update your client-data manually.
A good practice would be to use the same product for the server database and the client database. You wouldn't have to write SQL-Statements twice since they use the same SOL-Dialect and same features.
Firebird for example offers a server
and an embedded version.
Also Microsoft offers their MS SQL Server
as a mobile version (compact edition) and there are
also Synchronization services
provided by Microsoft (good blog
describing sync services in visual
studio:
http://keithelder.net/blog/archive/2007/09/23/Sync-Services-for-SQL-Server-Compact-Edition-3.5-in-Visual.aspx)
MySQL has a product which is called "MySQLMobile" but I never actually used it.
I can also recommend SQLite as an embedded database since it is very easy to use.
Depending on your bandwidth and data amount you could even download the whole database and delete the old one. (in Firebird for example only copy the database files and it will also work with the mobile version) Very easy - BUT you have to know if it will work for your scenario. If you have more data you will need something more flexible and sophisticated, only updating the data that really changed.
I like the simplicity of SQLite, but by design, it doesn't support concurrent access. The apps I write don't have heavy needs, so I'd like to avoid heavier solutions like MySQL that are more difficult to deploy (remote customers with usually no computer personnel).
Does someone know of a good solution that would offer the following features?
Client available for VB.Net applications
The server itself doesn't have to be a .Net application. Actually, I'd rather a bare-metal server so that it can run even on embedded Linux hosts with less RAM/CPU than regular PC's
Easy install: the client part should either be statically linked inside the client application or be available as a single DLL, and the server should just be a single EXE listening for queries, à la Fossil (http://www.fossil-scm.org)
clients can locate the server on the LAN by broadcasting data picked up by the server, so users don't have to write down the IP address and paste it into each client
open-source, or moderately priced closed-source
I would recommend Firebird SQL http://www.firebirdsql.org
Can be embedded and can work as a separate server. Very little footprint with high capabilities.
Have you considered SQL Server? It scales well from Compact Edition (free, file based), through Express Edition (again, free, but requires installation) and then through the paid-for editions for when your product needs to scale further.
HSQLDB is also very leightweight, and the upcoming 2.0 release supports concurrency in form of MVCC. It is based on Java though, but there is an ODBC connector available.
You may also consider Derby... Its free and open source.