I'm trying to figure out how to make implementation of android database Cursor to wrap "ORMed" database layer.
To have ORM in MonoDroid we can use sqlite-net project (very lightweight ORM) or ServiceStack.OrmLite
My thoughts are to implement ICursor interface and "wrap" ORM
For now I just can't set it in my mind how it should work, and should it work ever or not.
Should it load "framed" set of results, or fetch it one by one?
Which is better for performance, how to get column values - reflection or..?
So, actually question is: is it possible ever?
Any thoughts will be appreciated.
Thanks.
I'm not sure what "problem" you're trying to solve with an ICursor implementation, perhaps you should be a little more specific as to what specific task you're trying to do. The entire point of an ORM (and you missed this one that also supports SQLite on Android) is to abstract away the whole RDBMS paradigm from the code and give you an object-oriented paradigm instead.
An ICursor gives you back an updatable resultset from a SQL query - which means you have to know about rows, resultsets, queries and all of that. An ORM gives back an object, or a collection of objects. If you want to update one, you update the object and send it back to the ORM.
Now I fully admit that there are times when an ORM's might not provide the cleanest mechanism to do something that a SQL query might do well. For example, if you logically wanted to "delete all parts built yesterday during second shift". A lightweight ORM might give you all parts and then you have to use LINQ or similar to filter that to the right day and shift and then iterate that resulting collection to delete each, whereas with a SQL query you just pass in a DELETE FROM Parts WHERE BornONDate BETWEEN #start AND #end, but that's one of the trade-offs you face.
In some cases the ORM might provide a facility to do what you want. For example in the OpenNETCF ORM linked above, you can cast your DataStore (if it isn't already) to a SQLDataStore and then you have access to the ExecuteNonQuery method, allowing you to pass in a direct SQL statement. If still doesn't have a means to pass you back a record set because, as I said, returning database rows is really the antithesis or an ORM.
There's also some inherent risk in using something like ExecuteNonQuery. If you want to change your backing store, from say a RDBMS like SQLite to something totally different like an object database, an XML file or whatever, then your code that builds and uses a SQL query breaks. Admittedly this might not be common, but if code portability and extensibility and on your radar, then it's at least something to keep in mind.
Related
I'm pretty new to database development in general and I've never used ORM before. I'm interested in the benefits of using one, specifically saving time writing boilerplate SQL queries. I'd like to use ORM for a project that I'm working on right now, but I'm not sure it's applicable.
This project is more akin to change tracking for very small (<= 500 characters) documents. I need to track edits and categorizations made by multiple users. Not really to see the specific changes they make, but more to see if the users agree with each other. I am using a SQL database for this (as opposed to actual document control) for a few reasons:
The documents are really small; and I'm only interested in the strings, not really in files.
I wanted the ability to perform ad-hoc queries against the data for development purposes, and didn't want an unpleasantly surprised halfway through that a particular document control package couldn't do what I wanted.
From most of what I've read it seems like you need a direct mapping from columns to data fields in an object to use ORM. What I have now does not even come close to this. To create objects representing documents in different stages of editing I have to cobble together data from columns in different tables, in different combinations.
So my question is: Does an ORM like Hibernate apply to this type of project? And if it does can one be added to an existing application/database?
If it makes a difference: I'm using Java, MySQL, and JDBC. The web app users have access to for edits is made with GWT and hosted via Tomcat6. If I need it, I have complete control of the webserver.
Thanks.
Does an ORM like Hibernate apply to
this type of project?
Yes
And if it does can one be added to an
existing application/database?
Yes
My opinion is that an ORM tool could be useful for you but you really need to delve into it to see for yourself. Remember when you use an ORM tool you are not forced to use only that to connect to your database. ORM tools in general make the most sense for applications that store data in a very object like structure. For instance your user code might be the place to start. Usually you only create 1 user at a time, you edit 1 user at a time, you check if 1 user is logged into. It also makes sense for things where you would return a list of results like Order Lines. Where I have run into issues with ORM tools is when you have complex data that requires multiple joins especially back to the table you started in. For those cases you might want to keep doing what you are doing. Overall, ORM tools are great but they are like a lot of other things in software development. Try them out on a small part of your code and use them where they work and don't where they don't. Ultimately, you are the one that will have to deal with and maintain whatever you make. Just educate yourself on Hibernate and I am sure you will know what to do!
I think that ORM (I would suggest using the JPA standard, probably with Hibernate as the provider) could suit your project.
It is fairly traditional, as you say, for database columns to map directly onto object fields. If you need to keep your existing database structure (which apparently doesn't map at all well to your objects), then you might find that its more trouble than it's worth to use ORM.
While it's certainly possible to use ORM to map to a specific database schema - perhaps because it's used by other systems - my view is that one of the biggest advantages of ORM is that you can almost ignore the schema. Once you design your objects, and tell hibernate about them, hibernate will create whatever tables it needs.
Currently I use a lot of the ado.net classes (SqlConnection, SqlCommand, SqlDataAdapter etc..) to make calls to our sql server. Is that a bad idea? I mean it works. However I see many articles that use an ORM, nHibernate, subsonic etc.. to connect to SQL. Why are those better? I am just trying to understand why I would need to change this at all?
Update:
I did check the following tutorial on using nHibernate with stored-procedures.
http://ayende.com/Blog/archive/2006/09/18/UsingNHibernateWithStoredProcedures.aspx
However it looks to me that this is way to overkill. Why would I have to create a mapping file? Even if I create a mapping file and lets say my table changes, then my code wont work anymore. However if I use ado.net to return a simple datatable then my code will still work. I am missing something here?
There's nothing wrong with using the basic ADO.NET classes.
You might just have to do a lot more manual work than necessary. If you e.g. select your top 10 customers from a table with SqlCommand and SqlDataReader, it's up to you go iterate over the results, pull out each and every single item of data (like customer number, customer name, and so forth), and you're dealing very closely with the database structures, e.g. rows and columns. That's fine for some scenarios, but too much work in others.
What an ORM gives you is a lot of this "grunt work" being handled for you. You just tell it to get a list of your top 10 customers - as "Customer" objects. The ORM will go off and grab the data (most likely using SqlCommand, SqlDataReader) and then pulling out the bits and pieces, and assemble nice, easy to use "Customer" objects for you, that are a lot easier to use, since they are what your code is dealing with - Customer objects.
So there's definitely nothing wrong with using ADO.NET and it's a good thing if you know how it works - but an ORM can save you a lot of tedious, repetitive and boring grunt work and let you focus on your real business problems on the object level.
Marc
First of all, the ORMs are likely to do a much better job at producing the SQL queries than your normal non-SQL specialized Joe :)
Secondly, ORMs are a great way to somewhat "standardize" your DALs, increasing flexibility over different projects.
And lastly, with a good ORM, you're likely to have an easier time substituting your underlaying data-source, as a good ORM will have many different dialects. Of course, this is just a side-bonus :)
ORM's are great to avoid code repetition. You can often find that your object model and database model are extremely close to each other and whenever you add a field you'll be adding it to the database, your objects, your sql statements as well as everywhere else. If you use an ORM then you change your code in one place and it builds the rest of it for you.
As for performance, this can go either way. You will probably find that a lot of the simple sql that is written for you is often extremely tailored with various shortcuts that you would have been too lazy to write, such as only returning the absolutely required data. On the other hand, if you have some extremely complex queries and joins that an automated system could not possibly build then you're better of keeping these written yourself.
In summary though, they're fantastic for fast builds!
You don't need to change. If SqlConnection, SqlCommand, etc. work for you then that's great.
They work just peachy fine for the DB app I'm developing, and I have dozens of concurrent users with no problems.
There's nothing wrong with using straight up ADO.Net, but using an ORM will save you time, both in development and maintenance. Thats the biggest benefit.
One thing to consider: will a future "new developer" be more inclined to know or learn a well documented and widely adopted OR/M or your custom data access layer?
The number one thing for me though is the time. Minutes with my favorite OR/M, nHibernate vs. hours/days writing a custom data access layer using ADO.NET.
I also favor OR/Ms because maintaining declarative XML mappings is way easier than maintaining potentially thousands of lines of imperative code... or worse thousands of lines of C# data access code on top of thousands of lines of stored procedure code. In my current project I have 58 objects mapped in 58 XML mapping files, each with less than 50 lines. I cringe when I think about writing/maintaining CRUD code for 58 entities in ADO.NET.
I must warn you to read the documentation. Many, dare I say most, folks with whom I've worked will jump on a tool like mice on cheese, but they'll never read the documentation and learn the technology. I recommend reading the docs BEFORE moving to a new technology like nHibernate. A good cup o' jo and an hour or two of hard reading before-hand will pay dividends.
I haven't find pointing to an general idea of ORM in any answer. The general idea of ORM is to perform an Object-Relational mapping and provide your business classes with persistence. It means that you will think only about you business logic and will let ORM tool to save its state for you. Sure there are a lot of different scenarios. As was already said, it is nothing bad in using pure ADO.NET and may be your application (that is already written in this stile won't get any benefit), but using ORM tool in new projects is a very good idea. As for other - I totally agree with other answers.
I am curious to know why should a Database person learn LINQ. How can it be useful.
LINQ has one big advantage over most other data access methods: it centers around the 'query expression' metaphor, meaning that queries can be passed around like objects and be altered and modified, all before they are executed (iterated). In practice what that means is that code can be modular and better isolated. The data access repository will return the 'orders' query, then an intermediate filter in the request processing pipeline will decorate this query with a filter, then it gets passed on to a display module that adds sorting and pagination etc. In the end, when its iterated, the expression has totally transformed into very specific SELECT ... WHERE ... ORDER BY ... LIMIT ... (or the other back-end specific pagination like ROW_NUMBER). For application developers, this is priceless and there simply isn't any viable alternative. This is why I believe LINQ is here to stay and won't vanish in 2 years. It is definitely more than just a fad. And I'm specifically referring to LINQ as database access method.
The advantage of manipulating query expression objects alone is enough of a factor to make LINQ a winning bid. Add to this the multiple container types it can manipulate (XML, arrays and collections, objects, SQL) and the uniform interface it exposes over all these disparate technologies, consider the parallel processing changes coming in .Net 4.0 that I'm sure will be integrated transparently into LINQ processing of arrays and collections, and is really no way LINQ will go away. Sure, today it sometimes produces unreadable, poorly performant and undebuggable SQL, and is every dedicated DBA nightmare. It will get better.
Linq provides "first class" access to your data. Meaning that your queries are now part of the programming language.
Linq provides a common way to access data objects of all kinds. For example, the same syntax that is used to access your database queries can also be used to access lists, arrays, and XML files.
Learning Linq will provide you with a deeper understanding of the programming language. Linq was the driver for all sorts of language innovations, such as extension methods, anonymous types, lambda expressions, expression trees, type inference, and object initializers.
I think a better question is why would one replace SQL with LINQ when querying a database?
"Know your enemy" ?
OTOH, out of interest. I'm learning more about XMl, XSLT, XSDs etc because I can see a use for them as a database developer.
Is this a real question? It can't be answered with a code snippet. Most of these discussion type questions get closed quickly.
I think one of the many reasons why one could consider linq it because linq is big productivity boost and huge time saver
This question also prompts me to ask, if LINQ becomes immensely popular, will server side developers know SQL as well as their predecessors did?
I come from a java background.
But I would like a cross-platform perspective on what is considered best practice for persisting objects.
The way I see it, there are 3 camps:
ORM camp
direct query camp e.g. JDBC/DAO, iBatis
LINQ camp
Do people still handcode queries (bypassing ORM) ? Why, considering the options available via JPA, Django, Rails.
There is no one best practice for persistence (although the number of people screaming that ORM is best practice might lead you to believe otherwise). The only best practice is to use the method that is most appropriate for your team and your project.
We use ADO.NET and stored procedures for data access (though we do have some helpers that make it very fast to write such as SP class wrapper generators, an IDataRecord to object translator, and some higher order procedures encapsulating common patterns and error handling).
There are a bunch of reasons for this which I won't go into here, but suffice to say that they are decisions that work for our team and that our team agrees with. Which, at the end of the day, is what matters.
I am currently reading up on persisting objects in .net. As such I cannot offer a best practice, but maybe my insights can bring you some benefit. Up until a few months ago I have always used handcoded queries, a bad habit from my ASP.classic days.
Linq2SQL - Very lightweight and easy to get up to speed. I love the strongly typed querying possibilities and the fact that the SQL is not executed at once. Instead it is executed when your query is ready (all the filters applied) thus you can split the data access from the filtering of the data. Also Linq2SQL lets me use domain objects that are separate from the data objects which are dynamically generated. I have not tried Linq2SQL on a larger project but so far it seems promising. Oh it only supports MS SQL which is a shame.
Entity Framework - I played around with it a little bit and did not like it. It seems to want to do everything for me and it does not work well with stored procedures. EF supports Linq2Entities which again allows strongly typed queries. I think it is limited to MS SQL but I could be wrong.
SubSonic 3.0 (Alpha) - This is a newer version of SubSonic which supports Linq. The cool thing about SubSonic is that it is based on template files (T4 templates, written in C#) which you can easily modify. Thus if you want the auto-generated code to look different you just change it :). I have only tried a preview so far but will look at the Alpha today. Take a look here SubSonic 3 Alpha. Supports MS SQL but will support Oracle, MySql etc. soon.
So far my conclusion is to use Linq2SQL until SubSonic is ready and then switch to that since SubSonics templates allows much more customization.
There is at least another one: System Prevalence.
As far as I can tell, what is optimal for you depends a lot on your circumstances. I could see how for very simple systems, using direct queries still could be a good idea. Also, I have seen Hibernate fail to work well with complex, legacy database schemata, so using an ORM might not always be a valid option. System Prevalence is supposed to unbeatingly fast, if you have enough memory to fit all your objects into RAM. Don't know about LINQ, but I suppose it has its uses, too.
So, as so often, the answer is: know a variety of tools for the job, so that you are able to use the one that's most appropriate for your specific situation.
The best practice depends on your situation.
If you need database objects in table structures with some sort of meaningful structure (so one column per field, one row per entity and so on) you need some sort of translation layer inbetween objects and the database. These fall into two camps:
If there's no logic in the database (just storage) and tables map to objects well, then an ORM solution can provide a quick and reliable persistence system. Java systems like Toplink and Hibernate are mature technologies for this.
If there is database logic involved in persistence, or your database schema has drifted from your object model significantly, stored procedures wrapped by Data Access Objects (with further patterns as you like) is a little more involved than ORM but more flexible.
If you don't need structured storage (and you need to be really sure that you don't, as introducing it to existing data is not fun), you can store serialized object graphs directly in the database, bypassing a lot of complexity.
I prefer to write my own SQL, but I apply all my refactoring techniques and other "good stuff" when I do so.
I have written data access layers, ORM code generators, persistence layers, UnitOfWork transaction management, and LOTS of SQL. I've done that in systems of all shapes and sizes, including extremely high-performance data feeds (forty thousand files totaling forty million transactions per day, each loaded within two minutes of real-time).
The most important criteria is destiny, as in control thereof. Don't ever let your ORM tool be an obstacle to getting your work done, or an excuse for not doing it right. Ultimately, all good SQL is hand-written and hand-tuned, but some decent tools can help you get a good first draft quickly.
I treat this issue the same way that I do my UI design. I write all my UIs directly in code, but I might use a visual designer to prototype some essential elements that I have in mind, then I tear apart the code it generates in order to kickstart my own.
So, use an ORM tool in any of its manifestations as a way to get a decent example--look at how it solves many of the issues that arise (key generation, associations, navigation, etc.). Tear apart its output, make it your own, then reuse the heck out of it.
I am working on a few PHP projects that use MVC frameworks, and while they all have different ways of retrieving objects from the database, it always seems that nothing beats writing your SQL queries by hand as far as speed and cutting down on the number of queries.
For example, one of my web projects (written by a junior developer) executes over 100 queries just to load the home page. The reason is that in one place, a method will load an object, but later on deeper in the code, it will load some other object(s) that are related to the first object.
This leads to the other part of the question which is what are people doing in situations where you have a table that in one part of the code only needs the values for a few columns, and another part needs something else? Right now (in the same project), there is one get() method for each object, and it does a "SELECT *" (or lists all the columns in the table explicitly) so that anytime you need the object for any reason, you get the whole thing.
So, in other words, you hear all the talk about how SELECT * is bad, but if you try to use a ORM class that comes with the framework, it wants to do just that usually. Are you stuck to choosing ORM with SELECT * vs writing the specific SQL queries by hand? It just seems to me that we're stuck between convenience and efficiency, and if I hand write the queries, if I add a column, I'm most likely going to have to add it to several places in the code.
Sorry for the long question, but I'm explaining the background to get some mindsets from other developers rather than maybe a specific solution. I know that we can always use something like Memcached, but I would rather optimize what we can before getting into that.
Thanks for any ideas.
First, assuming you are proficient at SQL and schema design, there are very few instances where any abstraction layer that removes you from the SQL statements will exceed the efficiency of writing the SQL by hand. More often than not, you will end up with suboptimal data access.
There's no excuse for 100 queries just to generate one web page.
Second, if you are using the Object Oriented features of PHP, you will have good abstractions for collections of objects, and the kinds of extended properties that map to SQL joins. But the important thing to keep in mind is to write the best abstracted objects you can, without regard to SQL strategies.
When I write PHP code this way, I always find that I'm able to map the data requirements for each web page to very few, very efficient SQL queries if my schema is proper and my classes are proper. And not only that, but my experience is that this is the simplest and fastest way to implement. Putting framework stuff in the middle between PHP classes and a good solid thin DAL (note: NOT embedded SQL or dbms calls) is the best example I can think of to illustrate the concept of "leaky abstractions".
I got a little lost with your question, but if you are looking for a way to do database access, you can do it couple of ways. Your MVC can use Zend framework that comes with database access abstractions, you can use that.
Also keep in mind that you should design your system well to ensure there is no contention in the database as your queries are all scattered across the php pages and may lock tables resulting in the overall web application deteriorating in performance and becoming slower over time.
That is why sometimes it is prefereable to use stored procedures as it is in one place and can be tuned when we need to, though other may argue that it is easier to debug if query statements are on the front-end.
No ORM framework will even get close to hand written SQL in terms of speed, although 100 queries seem unrealistic (and maybe you are exaggerating a bit) even if you have the creator of the ORM framework writing the code, it will always be far from the speed of good old SQL.
My advice is, look at the whole picture not only speed:
Does the framework improves code readability?
Is your team comfortable with writing SQL and mixing it with code?
Do you really understand how to optimize the framework queries? (I think a get() for each object is not the optimal way of retrieving them)
Do the queries (after optimization) of the framework present a bottleneck?
I've never developed anything with PHP, but I think that you could mix both approaches (ORM and plain SQL), maybe after a thorough profiling of the app you can determine the real bottlenecks and only then replace that ORM code for hand written SQL (Usually in ruby you use ActiveRecord, then you profile the application with something as new relic and finally if you have a complicated AR query you replace that for some SQL)
Regads
Trust your experience.
To not repeat yourself so much in the code you could write some simple model-functions with your own SQL. This is what I am doing all the time and I am happy with it.
Many of the "convenience" stuff was written for people who need magic because they cannot do it by hand or just don't have the experience.
And after all it's a question of style.
Don't hesitate to add your own layer or exchange or extend a given layer with your own stuff. Keep it clean and make a good design and some documentation so you feel home when you come back later.