I'm having some issues with dealing with indirectly associated models in cakephp. My current model setup is as follows:
Deliveries hasOne License
License belongsTo Delivery
License hasAndBelongsToMany Product (and vice-versa)
License hasAndBelongsToMany ProductOption (and vice-versa)
I'm trying to save information about ALL of these models inside ONE form. The shortcomings I'm running into are the following:
The form helper only seems able to see the field type one level deep.
saveAll() only seems able to save records one level deep (for multiple model forms).
I'm searching everywhere for the solutions to these issues, but since I'm new to CakePHP, I'm not sure what the newest methods or "correct" methods are for dealing with these issues.
Any advice is much appreciated. Thank you all!
EDIT: I've posted code to my failed attempt here: http://bin.cakephp.org/saved/58501
saveAll() only seems able to save records one level deep (for multiple model forms).
I have stumbled across this limitation in the past and chose, at the time, to work around it by breaking up my form into multiple smaller forms.
One thing to bear in mind when using saveAll and InnoDB tables is that you get atomic transactions, as Cake will perform a rollback if it is unable to commit changes to the database.
So, while you can obviously work around the issue with a few lines of your own code (since Cake's saveAll one-liner doesn't live up to expectations), you would have to spend more time if transactions were a requirement.
Hi I know this is an old post but thought I'd post this to help others, As of CakePHP 2.1 you can save multiple levels of model association using an option on the call the saveAll(), more details here: http://book.cakephp.org/2.0/en/appendices/new-features-in-cakephp-2-1.html#model-saveall-model-saveassociated-model-validateassociated
Example:
$this->MyModel->saveAll($this->request->data, array('deep'=>true));
As for the FormHelper limitation, I'm as lost as you are, I will probably resort to setting the input types manually as my fields don't need too much validation.
Related
I'm reaching out to gain perspective on possible solutions to this problem. I'll be using Angular and Rails, but really this problem is a bit more abstract and doesn't need to be answered in the context of these frameworks.
What are the best practices for managing complex nested SQL associations on the front-end?
Let's say you have posts and comments and comments are nested under posts. You send your posts to the front-end as JSON with comments nested under them. Now you can display them listed under each post, great. But then questions arise:
What if you want to display recent comments as well? Your comment service would need to have comments in a normalized collection or gain access to them in a fashion that allows them to be sorted by date.
Does this mean you make a separate API call for comments sorted by date? This would duplicate comments on the front-end and require you to update them in two places instead of one (one for the posts and one for the comment, assuming comments can be edited or updated).
Do you implement some kind of front-end data normalization? This meaning you have a caching layer that holds the nested data and then you distribute the individual resources to their corresponding service?
What if you have data that has varying levels of nesting? Continuing with the posts and comments example. What if your comments can be replied to up until a level of 10?
How does this effect your data model if you've made separate API calls for posts and comments?
How does this effect your caching layer if you choose that approach?
What if we're not just talking about posts? What if you can comment on photos and other resources?
How does this effect the two options for data-modeling patterns above?
Breaking from the example, what if we were talking about recursive relationships between friended users?
My initial thoughts and hypothetical solution
My initial thought and how I'd attack this is with a caching layer and normalize the data such that:
The caching layer handles any normalization necessary
The caching layer holds ONE canonical representation of each record
The services communicate with the caching layer to perform CRUD actions
The services generally don't care nor do they need to know how nested/complex the data model is, by the time the data reaches the services it is normalized
Recursive relationships would need to be finite at some point, you can't just continue nesting forever.
This all of course sounds great, but I see lots of potential pitfalls and wish to gain perspective. I'm finding it difficult to separate the abstract best practices from the concrete solutions to specific data models. I very interested to know how others have solved this problem and how they would go about solving it.
Thanks!
I assume you will use restful apis, attention I don't know rails but I will suggest you some general practices that you might consider
Let's say you have one page that shows 10 posts and their 10 comments sorted by date, make this response possible in one api call
There is one more page that shows only 5 posts and no comments use the same api end-point
Make this possible with some query parameters.
Try to optimize your response as much as you can.
You can have multiple response type in one end-point, in any programming languages, if we talking about APIs thats how I do the job.
If you query takes much time, and that query runs serveral times then of course you need to cache but talking about 10 posts in each api call doesn't need caching. It should not hard on database.
For nesting problem you can have a mechanism to make it possible i.e
I will fetch 10 posts and their all comments, I can send a query parameter that I want to include all comments of each post
like bar.com/api/v1/posts?include=comments
if I need only some customized data for their comments, I should be able to implement some custom include.
like bar.com/api/v1/posts?include=recent_comments
You API layer, should first match with your custom include if not found go on relation of the resources
for deeper references, like comments.publisher or recent_comments.publisher your API layer needs to know which resource are you currently working on it. You won't need this for normal include, but custom includes should describe that what model/resource they are point to that way it is possible to create endless chain
I don't know Rails, but you can make this pattern easily possible if you have a powerful ORM/ODM
Sometimes, you need to do some filtering same goes for this job too.
You can have filter query parameter and implement some custom filters
i.e
bar.com/api/v1/posts?include=recent_comments&filters=favorites
or forget about everything and make something below
bar.com/api/v1/posts?transformation=PageA
this will return 10 recent posts with their 10 recent comments
bar.com/api/v1/posts?transformation=PageB
this will return only 10 recent posts
bar.com/api/v1/posts?transformation=PageC
this will return 10 recent post and their all comments
I've a question for CakePHP about its architecture.
I've an application, which will have only two models(products and categories).
I've to make an online product browser. For me it makes no sens to have one controller "Products" and one controller "Categories", because I will have 20+ actions on the Products and none on the Categories.
I've several key functionnalities, can I have one controller for each of those functionnalities? e.g. One Controller "Search", which have action "SearchResult", "AdvancedSearch", "BasicSearch".
We will be several developers on this project and I don't wish we are always all blocked because one is doing some edit on the product controller(and I really don't like file with 1000+ lines of code).
Second question: What is the mecanism I should use to fragment view in several different sub-view?
E.g. For the view of the details of one product, I've a lot of things to display: Pictures section, description section, similar products, review, ... Is it possible to have sub-part?
Thank you very much
For "sub-views" look into Elements, which are just that. They are snippets of view code that can be included in other views. The proper structuring of elements can create some very elegant view/subview relationships.
http://book.cakephp.org/view/1081/Elements
As for your controller issue, it's one I've grappled with as well. From what I see, it sounds like you're misclassifying your actions. Think about alternative taxonomies for the actions you're doing (mixing all the search functions together into a controller is a great example).
Perhaps you should think about controllers as objects that classify various actions rather than aspects of your site on which various actions can be performed (as it seems you're doing). So rather than a CATEGORY controller on which you can search, list, buy, etc., you might have, the product controller (which handles viewing your products either in an aggregate - perhaps by category - or individually), and perhaps a user controller for your users accounts? It's hard to say without looking at your application specifics. Search would then be an action that would be performed on your product.
One thing that I have made the mistake of doing is picking my controllers based on pages i'll have rather than on models. Think about your data (models) and the actions that can be performed on them (actions in the controller), THEN think about your views.
I wish I could give you more specific help, but without knowing your application's specific needs, I can only give you guidelines on how to break down your controllers. Take a look at this question:
CakePHP - file structure confusion - different controllers, or all in same?
for more discussion on the topic.
Hope that helps!
You don't need a Categories controller if you have no methods/actions associated with it. I assume you may want an admin CRUD for administrating your categories?
Regarding searching; your core search functions should reside in the Products model with the various controllers calling the appropriate methods. Basic or Advanced searching on one model in most cases can be handled by one method, its only a matter of more search fields surely?
If you have several developers and are worried about conflicts on the same file you really need to be considering version control like GIT or SVN rather than approaches contrary to a MVC frameworks core methodology. It will cause you a lot less problems in the future.
I believe you question regarding 'sub views' is solved by using elements, which is in effect reusable sections of views.
good luck
I have been thinking and learning a lot about forms lately trying to add advanced extensions to the Spree ECommerce Platform: Subscriptions, Events, Donations, and all kinds of Surveys.
Every example I have ever encountered (in blogs, in the docs, in screencasts, in source code, etc.) make forms out of Models, but they never go to anything semi-structured or unstructured (or just really dynamic). So you have forms like:
Contact Form (User Model, maybe divided into an Address Model too)
Registration Form (User Model, Account Model, Address Model, etc.)
Blog Post Form (Post Model, Tag Model, etc.)
Checkout Form (Shipping Model, Order Model, LineItem Model, etc.)
All of those make perfect sense: They are the culmination of 10's of thousands, millions even, of man hours. Tons of people have slowly abstracted those things down into nearly universal "models" that could be saved into a database table. So now we all create models for them and make database tables for them.
But there are so many other things that can't be boiled down to those specific models. Things like a survey for a specific event, with form fields such as:
Are you Pregnant?
How many kids do you have?
Have you ever been sick?
What's your fastest mile?
If we started to save those things to the database in tables, we would have 100s and 1000s of database tables, one for each set of questions, or "survey".
So my thinking is, there's got to be some point at which you stop creating specific models like the "Post" and the "Order" and start just making a "Form" or "Survey" model (Form ~ Survey ~ Questionnaire to some degree).
Everything would be boiled down to these few models:
Survey
Question
Answer
ResponseSet (answers to questions in survey)
Response (specific response in response set)
And from those you could create any type of "Form" you wanted.
So my question is basically: When do you, in the most practical, day-to-day client projects, stop making forms with a bunch of models in them (a "Checkout" form is a form for the "Order" basically in Spree, but that easily requires 10 database models), and just start using Question/Answer or Field/Input or Key/Value? Practically?
I'm just looking for something like "when we built our online tutoring system, we didn't end up creating a bunch of SomeTutorialModel objects which extend TutorialModel, because that would've added too many tables to our database. Instead, we just used the Surveyor gem". Something along those lines :).
There's not much out there on this semi-structured type of data, but lots when you can boil it down to something super concrete.
It seems that if you used a Document Database, like CouchDB, you'd end up being able to create all kinds of Model objects in ruby for example, and could get them out with some clever view tricks. But with MySQL and the-like, it seems insane.
Your question is quite broad, so I will instead of giving direct answers mention these points:
1.) models often reflect the target (core) domain of the application, so the boundary between key/value and model is about the domain
2.) AFAIK e.g. Google uses relational databases even to store key/value data, so they can store everything as using document database
3.) all your questions are basically about modeling and abstraction, which is hard to explain shortly or in general
People seem to avoid building user interfaces that pull their information (names, field types, etc. as well as relationships) from a database; they instead hard-code forms (and tables, etc.) that have pretty much the same data names and types and things.
Am I making sense?
For instance, imagine an emumerated field in MySQL: why not just have the UI construct a drop-down list whenever it encounters an ENUM? Why put the same values in both the database and the code?
Perhaps I'm just missing something; perhaps there are projects out there that do this — sort of super-crud interfaces that can be pointed at any database and from it build a fully-functional relationally-aware user interface. Are there?
I'm possibly not quite conforming to the stackoverflow norms with this question; I shall summarise:
Can you please tell me of a project that constructs its user interface (solely) from analysis of the database schema?
Why is this not a common way to do it — surely it is good to only define data structure in one place (i.e. the database)?
Thank you, and may joyous code-love rain upon your IDE.
I'd like to point out that, last time I checked, .NET and Qt (and probably other environments) make it possible to use "database-aware widgets" (sometimes shortened to just data-aware widgets), which is probably the best pragmatic solution available. What I mean by data-aware widgets is that the widgets themselves know that they're linked directly to database fields, so you would have a combobox that knows that it's backed by an enum and fetches the possible values directly from the database at runtime, just like you suggested.
This is a really neat utility, and used well, it probably won't hurt anything. It still requires that you spend some time laying out widgets manually on a form, but then if you update the database to add a new value to that enum, you don't have to rebuild your app to see it show up in the UI.
But the reason most usability experts will cringe when they hear your question is because programmers tend to think that, well, why not just generate the entire UI, form layout and everything, from the database? And this is where it starts to get really nasty, really fast.
Let's say you have a simple Person table, with first_name, last_name, email_address, street_address, city, state, zip, and phone_number. You want to automatically generate a UI based on these fields. How do you sort the fields? I mean, ideally, first name and last name should be right next to one another. And it would look very silly if you had city and state before street address. So you have to add a new column to the table to specify sort order, if you go with the quickest method, or a new table to specify each field's order index to their field ID.
What if you want to group parts of the information separately? Then you have to add more UI-specific cruft into your database layout (to do this generically, you'll need a new table specifying which UI fields belong to which UI groupboxes). So you've only solved two problems and already your database layout has gotten twice as ugly, plus now instead of a simple O(1) layout operation when you load the UI, you've gotta do several database queries to find out what fields exist and dynamically lay them out while applying the correct widget order... and we haven't even dealt with sizing (should every field be the maximum size to fit its possible contents, or should all text fields be the same width? Wouldn't it be nice if you could say that some text fields should be one width and height, and some should be another combination? etc), or text justification, or formatting, or any other really common elementary usability requirements that will require further sacrifices from the clarity and simplicity of your database schema.
Most visual database editors. phpMyAdmin for instance.
Because the database structure isn't always a very good logical structure for a user to be using, especially in the case of databases that have been denormalised on purpose for efficiency reasons.
Yup, this route has already been traveled.
Simply pointing at a database will create an oversimplified UI, not giving much more than the CRUD of an Access UI. That's why Naked Objects (I'm one of its committers) builds its metamodel from a pojo domain model. This allows the UI to expose any public methods as menus ... we call this "behaviourally complete".
Per the comment about the UI not being suitable for end-users, I have two points:
distinguish between power users vs casual users. Most internal apps are for the former (we use Alan Coopers' term of a "sovereign application" for this), who understand the domain and don't want fancy UI stuff getting in the way. Most external apps, eg public web sites, are for the latter.
for the latter, there's nothing to prevent the autogenerated UI of a tool like Naked Objects being replaced with a custom or semi-customized viewer. One such viewer is Scimpi, I'm also working on an Eclipse RCP viewer that'll expose extension points. But even here, the auto gen UI is still very valuable for the development team and business analysts for exploration and prototyping.
Hope some of the above has piqued your interest. If you want more, google around, or you might want to check out my book on domain-driven design and NO, at pragprag.com.
HTH
Dan
List of projects that implement this idea.
.NET
dotObjects
Naked Objects
TrueView
Java
Domain Object Explorer
JMatter
Naked Objects
Sanssouci
Trails
Lablz
C++
Typical Objects
Specifically to your second question: Alot of it really depends on your data model. Some are very complicated and would lead to un-intuitive user interfaces. Perhaps for simply CRUD based systems, having your UI be a front end to the database would be preferable. In that case, I think that some of these tools would be great. However, for some more complicated systems where some db data needs to be hidden from the users, it would be better if you UI didn't mirror the db schema.
Microsoft Access has used this model for years - the database and UI development are very closely tied. You can auto-generate a form directly from a table definition with smart defaults and search built in. The model works well for developing applications with few concurrent users such as custom applications for small businesses where the amount of data stored is small.
If you are scaling to larger relational DBs with a number of concurrent users, or large databases then reliability and performance become more important, and separately constructed UI and databases make more sense. When more users are involved they often have different requirements so decoupling the UI from the DB schema makes it more efficient to develop.
Just a note on Java "projects that implement this idea" - tynamo is the new version of Trails framework
There are many systems that build an interface for you to edit stuff directly from table information. End-user interfaces, however, must be tweaked a little bit. You may not want to reveal to the user every field in your table.
Frameworks that make good use of the MVC design pattern can let you do all kinds of things with your models, which are the preferred way to build new systems (rather than creating database tables directly).
To answer your questions specifically:
django allows you to construct forms (and a complete admin CMS) out of models.
It is a common thing to do.
Naked Objects is about one step removed from this. They base the UI on an object model, and then persist the object model.
I think you are forgetting to consider the user in your design process if you are thinking like that. Bad mistake. Users don't like it when the interface changes, they would especially not like it if it changed frequently as they then wouldn't know what to do. Further, if you generate your UI on the fly based on the database structure, then what order would the objects be in? UIs need to have objects in an order that makes sense to the users not the database designers.
Further in a well-designed database there are fields that are not meant for the users to see. Things like numeric keys, insert date, last updated etc. You don't want to automatically expose these to the users and you certainly don't want them to have the ability to mess with the data in such fields.
Finally, if you don't think about the functionality of the page, then you aren't doing your job. A UI needs to be more than just a list of fileds that can be edited. You need to have constraints on who can see what, checks of the data before inserting to the database, business rules that need to be applied. You can't just autogenerate a lot of this (and you shouldn't even if you could!). Design needs thought and care.
Now as to drop down lists, of course you can generate them from the database and not the code, in fact it is the better choice. Just make the query the source for your particular object, not a list generated in code.
You can do it with the help of this cool tool from a developer in Philippines, it is called COBALT. You can download it here.
I'm currently looking at the Python framework Django for future db-based web apps as well as for a port of some apps currently written in PHP. One of the nastier issues during my last years was keeping track of database schema changes and deploying these changes to productive systems. I haven't dared asking for being able to undo them too, but of course for testing and debugging that would be a great feature. From other questions here (such as this one or this one), I can see that I'm not alone and that this is not a trivial problem. Also, I found many inspirations in the answers there.
Now, as Django seems to be very powerful, does it have any tools to help with the above? Maybe it's even in their docs and I missed it?
There are at least two third party utilities to handle DB schema migrations, South and Django Evolution. I haven't tried either one, but I have heard some good things about South, though Evolution has been around a little longer.
Also, look at SchemaEvolution on the Django wiki. It is just a wiki page about migrating the db.
Last time I checked (version 0.97), syncdb will be able to add tables to sync your DB schema with your models.py file, but it cannot:
Rename or add a column on a populated DB. You need to do that by hand.
Refactorize your model (like split a table into two) and repopulate your DB accordingly.
It might be possible though to write a Django script to make the migration by playing with the two different managers, but that might take ages if your DB is large.
There was a panel session on DB schema changes at the recent DjangoCon; there is a video of the session (thanks to Google), which should provide some useful information on a number of these utilities.
And now there's also dmigrations. From announcement:
django-evolution attempts to address this problem the clever way, by detecting changes to models that are not yet reflected in the database schema and figuring out what needs to be done to bring the two back in sync. In contrast, dmigrations takes the stupid approach: it requires you to explicitly state the changes in a sequence of migrations, which will be applied in turn to bring a database up to the most recent state that reflects the underlying models.
This means extra work for developers who create migrations, but it also makes the whole process completely transparent—for our projects, we decided to go with the simplest system that could possibly work.
(My bold)
I heard lot of good about Django Schema Evolution Branch and those were opions of actual users. It mostely works out of the box and do what it should do.
U should lookup Dmigrations, it functions a little bit diffrent from django-eveoltions.
It shows you everything it is doing and for compliccated things it asks you for your intervetnions. It should be great.