Related
Which is better (and for what reasons) to use to connect to MS SQL, Oracle or Firebird from a Delphi Win32 application -- ADO or DBX (Database Express)?
Both allow you to connect to the major databases. I like the way ADO does it all with a connection string change and the fact that ADO and the drivers are included with Windows so nothing extra to deploy (it seems, correct me if I'm wrong).
DBX is also flexible and I can compile the drivers into my app, can I not?
I really am keen to have a single source if possible, with the ability to vary databases depending on the customer's IT department/preferences.
But which is easier to program, performs better, uses memory most efficiently? Any other things to differentiate them on?
Thanks, Richard
ADO is simple to use and is there, you only must make sure to install the correponding client driver in the client side.
I found DBX more flexible and it is better integrated within IDE and another technologies like DataSnap.
For the same purpose than you, I have used DBX with Third Party Drivers from DevArt.
You can compile the drivers with your application if you buy the drivers sources.
In the beginning of Delphi, people praised the multi-DBMS support in Delphi. Everyone loved the BDE (because that was the only way to do that).
But when looking at customers over more then the past decade, I have seen a steady decrease of multi-DBMS support in their applications.
The cost of supporting multiple DBMS from one application is high.
Not only because you have to have knowledge of each DBMS, but also because each DBMS has its own set of pecularities, where you have to adapt for in your data access layer. These not only include differences in syntax and underlying data types, but also optimization strategies.
Also, some DBMS work better with ADO, some better with a direct connection (like skipping your Oracle client all together).
Finally testing all the combinations of your software with multiple DBMS systems is very intensive.
I've been involved in a few projects where we had to change the DBMS backend and/or the data access technology (from i.e. BDE to DBX, or from DBX to a direct connection). Changing the backend always was much more painfull than changing the data access technology. Multi-tier approaches made them somewhat easier, but increased the degrees of freedom and therefor the testing efforts.
Some of products that I do see that support multi-DBMS are in vertical market applications where the final customer already has their own DBMS infrastructure and the application needs to adapt to that. For instance in Dutch governmental areas, Oracle has been really strong, but SQL Server has established quite a user base as well.
So you need to think about what combinations of DBMS you want to support, not only in terms of functionality, but also in terms of cost.
If you stick to one DBMS, then it makes no sense to go for a generic data access layer like BDE, DBX or ADO: it pays off doing a connection as direct as possible.
My experience has taught me that these combinations do work well:
Interbase or Firebird with FIBPlus, AnyDAC, IBO or IBX*
Oracle with AnyDAC, DOA or ODAC
Microsoft SQL Server with ADO
IBX does not like Firebird very much.
Hope this gives you some insight in the possibilities and limitations of supporting multiple DBMS from your Delphi applications.
--jeroen
General rule: every layer of components will possibly add an additional layer of bugs. Both ADO and DBX are component wrappers around standard database functionality, thus they're both equally strong.
So the proper choice should be based on other factors, like the databases that you want to use. If you want to connect to MS-Access or SQL Server, ADO would be the better choice since it's more native for these databases. But Firebird and Oracle are more native for the DBX components.
I personally tend to use the raw ADO API's, though. Then again, I don't use data-aware components in my projects. It's less RAD, I know. But I often need to work this way because I generally write client/server applications with several layers between the database and the GUI, thus making things more complicated.
My two cents: DBX is significantly faster (on both oracle and sql), and significantly more finicky and harder to deploy.
If performance is a factor, I'd go with DBX. Otherwise, I'd just use ADO for simplicity's sake.
As others have said, DBX may have the edge in raw performance in certain cases or under specific circumstances, but ADO is the basis for a very larger number of applications in the world so although performance of ADO may be relatively poorer, clearly that does not mean "unacceptably" poor.
For myself, and informed by major projects I have worked on, the biggest "problem" with DBX is that no matter how good it may be, it is a key infrastructure technology provided by a language/tools company.
Anyone that built applications on the previous BDE technology will testify to the disruption caused when that technology is deprecated and no longer supported. Whilst no technology is immune from deprecation by it's provider, ADO clearly has the edge when it comes to industry support beyond the technology provider themselves.
For that reason I myself now always use ADO. Just changing the connection string isn't always the only thing to worry about when changing from one database type to another however. Stored procedure call syntax can vary from one ADO provider to another, and you still have to watch the SQL syntax you use if you intend deploying against multiple different SQL engines, where the SQL support may vary from on to another.
To mitigate these issues I use my own encapsulation of the ADO object model. This encapsulation does not attempt to mutate the object model into something that doesn't resemble ADO, it simply exposes those parts of ADO that I need to use directly in a more ObjectPascal friendly (and "type" safe) form (e.g enum types and sets for constants and flags etc, rather than just scores if not hundreds of integer constants).
My encapsulation also takes care of some of the minor variations in different provider behaviours/requirements, such as the previously mentioned differences in stored procedure call syntax.
I should say also that similar to another poster, I too long ago stopped used "data aware controls", which opens up this approach. If you need or wish to use data aware controls and wish to use ADO, then you cannot use ADO directly and must instead find some encapsulation that exposes ADO thru the VCL dataset model.
ADO is Microsoft world
DBX was created at the beginning (Delphi 6) for cross platform and Kylix
I am developing an application which at the moment queries a (rather large) database via ADO.NET and hard-coded SQL statements. Admittedly this is ugly (i.e. no compile time errors thrown if a mistake is made in the SQL) and potentially dangerous (due to SQL injections, etc although this is unlikely to be a problem for this particular application) but this wasn't considered initially because this application is really only interested in a very small subset of tables in this database (at least for now...).
LinqToSQL seemed interesting but because this application is required to have the ability to connect to Oracle databases as well, that plan was a non-starter.
Is a project like mine suitable for integration with an ORM framework or would that be overkill?
I think an ORM should always at least be considered.
But it doesn't sound like you're even using business objects (Sometimes referred to as a Data Access Layer or DAL) which greatly undermines the usefulness of an object oriented language. I would address this first. If you find it's too time consuming to create all the CRUD for the business objects it's time for an ORM...
My personal favorite is nHibernate. Big learning curve but definitely worth it.
I would recommend a generated DAL instead of an ORM, or Linq.
Look into subsonic http://subsonicproject.com/. It is an open source DAL generator that is very easy to learn and use, and has a very low overhead.
I would definitely say that it is a candidate for an ORM framework. The overhead of setting up the ORM is quite small once you have familiarized yourself with a framework, and the benefits are many.
As you say, LinqToSQL is not appropriate if you might need Oracle support, but most other frameworks support Oracle.
If you only use a small subset of the tables, then you will only have to map a small subset of the tables and hence the setup cost will decrease even further.
Good luck!
Try using something that generates sql (Like Linq, only with Oracle), instead of an orm.
Why? Jeff Atwood explains.
Quote:
"At first you're like "whee! objects!" and then you realize-- hey, this is a lot of tedious, error-prone mapping code I didn't have to write before... "
I come from a java background.
But I would like a cross-platform perspective on what is considered best practice for persisting objects.
The way I see it, there are 3 camps:
ORM camp
direct query camp e.g. JDBC/DAO, iBatis
LINQ camp
Do people still handcode queries (bypassing ORM) ? Why, considering the options available via JPA, Django, Rails.
There is no one best practice for persistence (although the number of people screaming that ORM is best practice might lead you to believe otherwise). The only best practice is to use the method that is most appropriate for your team and your project.
We use ADO.NET and stored procedures for data access (though we do have some helpers that make it very fast to write such as SP class wrapper generators, an IDataRecord to object translator, and some higher order procedures encapsulating common patterns and error handling).
There are a bunch of reasons for this which I won't go into here, but suffice to say that they are decisions that work for our team and that our team agrees with. Which, at the end of the day, is what matters.
I am currently reading up on persisting objects in .net. As such I cannot offer a best practice, but maybe my insights can bring you some benefit. Up until a few months ago I have always used handcoded queries, a bad habit from my ASP.classic days.
Linq2SQL - Very lightweight and easy to get up to speed. I love the strongly typed querying possibilities and the fact that the SQL is not executed at once. Instead it is executed when your query is ready (all the filters applied) thus you can split the data access from the filtering of the data. Also Linq2SQL lets me use domain objects that are separate from the data objects which are dynamically generated. I have not tried Linq2SQL on a larger project but so far it seems promising. Oh it only supports MS SQL which is a shame.
Entity Framework - I played around with it a little bit and did not like it. It seems to want to do everything for me and it does not work well with stored procedures. EF supports Linq2Entities which again allows strongly typed queries. I think it is limited to MS SQL but I could be wrong.
SubSonic 3.0 (Alpha) - This is a newer version of SubSonic which supports Linq. The cool thing about SubSonic is that it is based on template files (T4 templates, written in C#) which you can easily modify. Thus if you want the auto-generated code to look different you just change it :). I have only tried a preview so far but will look at the Alpha today. Take a look here SubSonic 3 Alpha. Supports MS SQL but will support Oracle, MySql etc. soon.
So far my conclusion is to use Linq2SQL until SubSonic is ready and then switch to that since SubSonics templates allows much more customization.
There is at least another one: System Prevalence.
As far as I can tell, what is optimal for you depends a lot on your circumstances. I could see how for very simple systems, using direct queries still could be a good idea. Also, I have seen Hibernate fail to work well with complex, legacy database schemata, so using an ORM might not always be a valid option. System Prevalence is supposed to unbeatingly fast, if you have enough memory to fit all your objects into RAM. Don't know about LINQ, but I suppose it has its uses, too.
So, as so often, the answer is: know a variety of tools for the job, so that you are able to use the one that's most appropriate for your specific situation.
The best practice depends on your situation.
If you need database objects in table structures with some sort of meaningful structure (so one column per field, one row per entity and so on) you need some sort of translation layer inbetween objects and the database. These fall into two camps:
If there's no logic in the database (just storage) and tables map to objects well, then an ORM solution can provide a quick and reliable persistence system. Java systems like Toplink and Hibernate are mature technologies for this.
If there is database logic involved in persistence, or your database schema has drifted from your object model significantly, stored procedures wrapped by Data Access Objects (with further patterns as you like) is a little more involved than ORM but more flexible.
If you don't need structured storage (and you need to be really sure that you don't, as introducing it to existing data is not fun), you can store serialized object graphs directly in the database, bypassing a lot of complexity.
I prefer to write my own SQL, but I apply all my refactoring techniques and other "good stuff" when I do so.
I have written data access layers, ORM code generators, persistence layers, UnitOfWork transaction management, and LOTS of SQL. I've done that in systems of all shapes and sizes, including extremely high-performance data feeds (forty thousand files totaling forty million transactions per day, each loaded within two minutes of real-time).
The most important criteria is destiny, as in control thereof. Don't ever let your ORM tool be an obstacle to getting your work done, or an excuse for not doing it right. Ultimately, all good SQL is hand-written and hand-tuned, but some decent tools can help you get a good first draft quickly.
I treat this issue the same way that I do my UI design. I write all my UIs directly in code, but I might use a visual designer to prototype some essential elements that I have in mind, then I tear apart the code it generates in order to kickstart my own.
So, use an ORM tool in any of its manifestations as a way to get a decent example--look at how it solves many of the issues that arise (key generation, associations, navigation, etc.). Tear apart its output, make it your own, then reuse the heck out of it.
Is LINQ a kind of Object-Relational Mapper?
LINQ in itself is a set of language extensions to aid querying, readability and reduce code. LINQ to SQL is a kind of OR Mapper, but it isn't particularly powerful. The Entity Framework is often referred to as an OR Mapper, but it does quite a lot more.
There are several other LINQ to X implementations around, including LINQ to NHibernate and LINQ to LLBLGenPro that offer OR Mapping and supporting frameworks in a broadly similar fashion to the Entity Framework.
If you are just learning LINQ though, I'd recommend you stick to LINQ to Objects to get a feel for it, rather than diving into one of the more complicated flavours :-)
LINQ is not an ORM at all. LINQ is a way of querying "stuff", and can be more or less seen as a SQL-like language extension for different things (IEnumerables).
There are various types of "stuff" that can be queried, among them SQL Server databases. This is called LINQ-to-SQL. The way it works is that it generates (implicit) classes based on the structure of the DB and your query. In this sense it works much more like a code generator.
LINQ-to-SQL is not an ORM because it doesn't try at all to solve the object-relational impedance mismatch. In an ORM you design the classes and then either map them manually to tables or let the ORM generate the database. If you then change the database for whatever reason (typically refactoring, renormalization, denormalization), many times you are able to keep the classes as they are by changing the mapping.
LINQ-to-SQL does nothing of the sort. Your LINQ queries will be tightly coupled to the database structure. If you change the DB, you will probably have to change the LINQ as well.
LINQ to SQL (part of Visual Studio 2008) is an OR Mapper.
LINQ is a new query language that can be used to query many different types of sources.
LINQ itself is not a ORM. LINQ is the language features and methods that exist in allowing you to query objects like SQL.
"LINQ to SQL" is a provider that allows us to use LINQ against SQL strongly-typed objects.
I think a good test to ascertain whether a platform or code block displays the characteristics of an O/R-M is simply:
With his solution hat on, does the developer(s) (or his/her code generator) have any direct, unabstracted knowledge of what's inside the database?
With this criterion, the answer for differing LINQ implementations can be
Yes, knowledge of the database schema is entirely contained within the roll-your-own, LINQ utilizing O/R-M code layerorNo, knowledge of the database schema is scattered throughout the application
Further, I'd extend this characterization to three simple levels of O/R-M.
1. Abandonment.
It's a small app w/ a couple of developers and the object/data model isn't that complex and doesn't change very often. The small dev team can stay on top of it.
2. Roll your own in the data access layer.
With some managable refactoring in a data access layer, the desired O/R-M functionality can be effected in an intermediate layer by the relatively small dev team. Enough to keep the entire team on the same page.
3. Enterprise-level O/R-M specification defining/overhead introducing tools.
At some level of complexity, the need to keep all devs on the same page just swamps any overhead introduced by the formality. No need to reinvent the wheel at this level of complexity. N-hibernate or the (rough) V1.0 Entity Framework are examples of this scale.
For a richer classification, from which I borrowed and simplified, see Ted Neward's classic post at
http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx
where he classifies O/R-M treatments (or abdications) as
1. Abandonment. Developers simply give up on objects entirely, and return to a programming model that doesn't create the object/relational impedance mismatch. While distasteful, in certain scenarios an object-oriented approach creates more overhead than it saves, and the ROI simply isn't there to justify the cost of creating a rich domain model. ([Fowler] talks about this to some depth.) This eliminates the problem quite neatly, because if there are no objects, there is no impedance mismatch.
2. Wholehearted acceptance. Developers simply give up on relational storage entirely, and use a storage model that fits the way their languages of choice look at the world. Object-storage systems, such as the db4o project, solve the problem neatly by storing objects directly to disk, eliminating many (but not all) of the aforementioned issues; there is no "second schema", for example, because the only schema used is that of the object definitions themselves. While many DBAs will faint dead away at the thought, in an increasingly service-oriented world, which eschews the idea of direct data access but instead requires all access go through the service gateway thus encapsulating the storage mechanism away from prying eyes, it becomes entirely feasible to imagine developers storing data in a form that's much easier for them to use, rather than DBAs.
3. Manual mapping. Developers simply accept that it's not such a hard problem to solve manually after all, and write straight relational-access code to return relations to the language, access the tuples, and populate objects as necessary. In many cases, this code might even be automatically generated by a tool examining database metadata, eliminating some of the principal criticism of this approach (that being, "It's too much code to write and maintain").
4. Acceptance of O/R-M limitations. Developers simply accept that there is no way to efficiently and easily close the loop on the O/R mismatch, and use an O/R-M to solve 80% (or 50% or 95%, or whatever percentage seems appropriate) of the problem and make use of SQL and relational-based access (such as "raw" JDBC or ADO.NET) to carry them past those areas where an O/R-M would create problems. Doing so carries its own fair share of risks, however, as developers using an O/R-M must be aware of any caching the O/R-M solution does within it, because the "raw" relational access will clearly not be able to take advantage of that caching layer.
5. Integration of relational concepts into the languages. Developers simply accept that this is a problem that should be solved by the language, not by a library or framework. For the last decade or more, the emphasis on solutions to the O/R problem have focused on trying to bring objects closer to the database, so that developers can focus exclusively on programming in a single paradigm (that paradigm being, of course, objects). Over the last several years, however, interest in "scripting" languages with far stronger set and list support, like Ruby, has sparked the idea that perhaps another solution is appropriate: bring relational concepts (which, at heart, are set-based) into mainstream programming languages, making it easier to bridge the gap between "sets" and "objects". Work in this space has thus far been limited, constrained mostly to research projects and/or "fringe" languages, but several interesting efforts are gaining visibility within the community, such as functional/object hybrid languages like Scala or F#, as well as direct integration into traditional O-O languages, such as the LINQ project from Microsoft for C# and Visual Basic. One such effort that failed, unfortunately, was the SQL/J strategy; even there, the approach was limited, not seeking to incorporate sets into Java, but simply allow for embedded SQL calls to be preprocessed and translated into JDBC code by a translator.
6. Integration of relational concepts into frameworks. Developers simply accept that this problem is solvable, but only with a change of perspective. Instead of relying on language or library designers to solve this problem, developers take a different view of "objects" that is more relational in nature, building domain frameworks that are more directly built around relational constructs. For example, instead of creating a Person class that holds its instance data directly in fields inside the object, developers create a Person class that holds its instance data in a RowSet (Java) or DataSet (C#) instance, which can be assembled with other RowSets/DataSets into an easy-to-ship block of data for update against the database, or unpacked from the database into the individual objects.
Linq To SQL using the dbml designer yes, otherwise Linq is just a set of extension methods for Enumerables.
We have a set of applications that work with multiple database engines including Sql Server and Access. The schemas for each are maintained separately and are not stored in text form making source control difficult. We are interested in moving to a system where the schema is stored in some text-based format (such as XML or YAML) with descriptions of field data types, foreign key relationhsips, etc.
When all is said and done, we want to have a single text file in source control that can be used to generate a clean database that works with both SQL Server, Access at least (and preferably is capable of working with Oracle, DB2 and other engines).
I'm certain that there are tools or libraries out there that can get us at least part of the way there. For one, I've found Altova MapForce that looks like it may do the trick but I'm interested in hearing about any alternative tools or libraries or even entirely different solutions for those in the same predicament.
Note: The applications are written in C++ and ORM solutions are both not readily available in C++ and would take far too long to integrate into our aging products.
If you don't use a object relational mapper that does this (and many other things for you) the easiest way might be to whip up a few structures to define your tables and attributes in some form of (static) code and write little generators to create actual databases from that description.
That makes it easy for source control, and if you're careful when designing those structures, you can easily re-use them for other DBs if need arises.
The consensus when I asked a similar (if rather more naive) question seem to be to use raw SQL, and to manage the RDMS dependencies with an additional layer. Good luck.
Tool you're looking for is liquibase. No support for Access though...