My app is working very slow. I am using Cakephp 2.3.1 version. Will it be beneficial to load Model, Components and Helpers in the functions where they are needed? Right now I am calling them in class. eg:
class PanelsController extends AppController {
public $name = 'Panel';
public $uses = array(list of models goes here);
public $components = array(list of components goes here);
.................
}
What other techniques do you suggest. Thanks
Here is something I would check if the site is performing slow
Speed Optimization
Enable caching
Compress JS and CSS. (A nice plugin that does this)
A good speed optimization checklist
Cake Practices
Cake conventions are your best guidelines, the framework has been designed to be scaled with its conventions.
Recursion and Containable, By default Cake fetches all related data when a query is fired. Both recursive level and containable behavior can limit the amount of data retrieved. If cake fetches all related data by default doesn't mean that you have to keep it that way.
Keep your DB normalized. This will allow you to defer many processes. For eg. when retrieving posts, cake automatically fetches all of its related data (Tags, comments). But when you have higher-order normalized DB, u can defer loading comments from an XHR/AJAX request. This will also allow you to serve comment related logic from comment's Model, Controller and View. Even if you bring related model data set limits for them.
You can also drop needs of counter query for related data by using Counter cache. More Details here
Cache your view
You can cache query results manually too,
Cache::write($this->Post->find("all"));
Try them out and you should be able experience amazing speed improvements.
Lastly, I do believe that Architecture of an application plays a big role in performance. Some times, we have to separate certain logic from a request Life cycle to boost the performance.
public $uses() does not matter. you can add as many as you want. Cake will only lazyload them if needed somewhere.
Just make sure you got recursive = -1 per default in your AppModel and only raise it or contain data you really need.
But your components will all be loaded and initialized right away.
You might want to reduce those.
Those two attributes cannot be your bottleneck, though. You must have some other serious issues.
Also don't make assumptions in debug mode. The real speed is measured/obseved with debug 0 where no additional debug info is gathered and the cache is not constantly replaced.
EDIT: Please note that my words above are only meant from a "speed point of view". It does not matter for speed. But it is not recommended to add models in $uses if you are able to reach those via relations and the relation chain.
So let's say you want to make a dashbard. In most cases you only need to add "User" model, since Profile, Images, and other models are usually directly accessable via $this->User->Profile->foo($bar) etc.
You can use caching technique in cakephp to reduce time , for documentation see here :http://book.cakephp.org/2.0/en/core-libraries/caching.html
Do not use load Model, with large data code it will create problem.
Here is an article for your reference:(Tips to speed up cakephp app)
http://www.dereuromark.de/2012/02/13/what-really-speeds-up-your-cakephp-app/
cakephp app slowness can be caused by a lot of reasons, have experienced so far:
mysql server that was trying to do a dns lookup
rendering a page with lot of links assembled via reverse routing
memory issues
best way to find out seems to install XDEBUG1
and examine the profiling information
Related
I'm reaching out to gain perspective on possible solutions to this problem. I'll be using Angular and Rails, but really this problem is a bit more abstract and doesn't need to be answered in the context of these frameworks.
What are the best practices for managing complex nested SQL associations on the front-end?
Let's say you have posts and comments and comments are nested under posts. You send your posts to the front-end as JSON with comments nested under them. Now you can display them listed under each post, great. But then questions arise:
What if you want to display recent comments as well? Your comment service would need to have comments in a normalized collection or gain access to them in a fashion that allows them to be sorted by date.
Does this mean you make a separate API call for comments sorted by date? This would duplicate comments on the front-end and require you to update them in two places instead of one (one for the posts and one for the comment, assuming comments can be edited or updated).
Do you implement some kind of front-end data normalization? This meaning you have a caching layer that holds the nested data and then you distribute the individual resources to their corresponding service?
What if you have data that has varying levels of nesting? Continuing with the posts and comments example. What if your comments can be replied to up until a level of 10?
How does this effect your data model if you've made separate API calls for posts and comments?
How does this effect your caching layer if you choose that approach?
What if we're not just talking about posts? What if you can comment on photos and other resources?
How does this effect the two options for data-modeling patterns above?
Breaking from the example, what if we were talking about recursive relationships between friended users?
My initial thoughts and hypothetical solution
My initial thought and how I'd attack this is with a caching layer and normalize the data such that:
The caching layer handles any normalization necessary
The caching layer holds ONE canonical representation of each record
The services communicate with the caching layer to perform CRUD actions
The services generally don't care nor do they need to know how nested/complex the data model is, by the time the data reaches the services it is normalized
Recursive relationships would need to be finite at some point, you can't just continue nesting forever.
This all of course sounds great, but I see lots of potential pitfalls and wish to gain perspective. I'm finding it difficult to separate the abstract best practices from the concrete solutions to specific data models. I very interested to know how others have solved this problem and how they would go about solving it.
Thanks!
I assume you will use restful apis, attention I don't know rails but I will suggest you some general practices that you might consider
Let's say you have one page that shows 10 posts and their 10 comments sorted by date, make this response possible in one api call
There is one more page that shows only 5 posts and no comments use the same api end-point
Make this possible with some query parameters.
Try to optimize your response as much as you can.
You can have multiple response type in one end-point, in any programming languages, if we talking about APIs thats how I do the job.
If you query takes much time, and that query runs serveral times then of course you need to cache but talking about 10 posts in each api call doesn't need caching. It should not hard on database.
For nesting problem you can have a mechanism to make it possible i.e
I will fetch 10 posts and their all comments, I can send a query parameter that I want to include all comments of each post
like bar.com/api/v1/posts?include=comments
if I need only some customized data for their comments, I should be able to implement some custom include.
like bar.com/api/v1/posts?include=recent_comments
You API layer, should first match with your custom include if not found go on relation of the resources
for deeper references, like comments.publisher or recent_comments.publisher your API layer needs to know which resource are you currently working on it. You won't need this for normal include, but custom includes should describe that what model/resource they are point to that way it is possible to create endless chain
I don't know Rails, but you can make this pattern easily possible if you have a powerful ORM/ODM
Sometimes, you need to do some filtering same goes for this job too.
You can have filter query parameter and implement some custom filters
i.e
bar.com/api/v1/posts?include=recent_comments&filters=favorites
or forget about everything and make something below
bar.com/api/v1/posts?transformation=PageA
this will return 10 recent posts with their 10 recent comments
bar.com/api/v1/posts?transformation=PageB
this will return only 10 recent posts
bar.com/api/v1/posts?transformation=PageC
this will return 10 recent post and their all comments
I've gone through the documentation on ATK4, trying to find a reference point how to handle caching - partial or full page.
Seems like there is no entry on the matter. Strange from a framework that is built for scalability. Is there a way to cache DB queries, pages, views, etc?
Thanks for your question. (I'm the author of ATK4).
In my opinion, scalability and caching are two different topics and can be addressed separately. Framework approaches scalability by optimising queries and minimising load for each request and also designing approach where multiple nodes can be used to seamlessly scale your application horizontally. There is also an option to add reverse proxy to cache pages before they even hit the web server.
Agile Toolkit has support for two types of caching:
View-level caching
As you read documentation on object render trees - framework initialise and render recursively, so if you add "caching" support to a Page level, you will be able to intercept and retrieve it's contents from cache. You can also cache views.
Here is a controller which can be used to implement caching for you:
https://github.com/agile55/viewcache
Model level caching
Sometimes you would want to cache your model data, so instead of retrieving data from the slow database, you can quickly fetch the data from a faster source. Agile Toolkit has support for multiple model data sources, where a cache would be queried first and refreshed if it didn't contain the data. Here you can find more information or ask further questions:
http://book.agiletoolkit.org/model/core.html#using-caching
http://forum.agiletoolkit.org/t/is-setcache-implemented/62
Other Ideas
Given the object-oriented nature of ATK4, you can probably come up with a new ways to cache data. If you have any interesting ideas, our c
95% of my app costs are related to write operations. In the last few days I've paid $150. And I wouldn't consider the amount of data stored to be that huge.
First I suspected that there may be a lot of write operations because of exploding indexes, but I read that this situation usually happens when you have two list properties in the same model. But during the model design phase those were limited to 1 per model max.
I also did try to go through my models and pass indexed=False to all properties that I will not need to order or filter by.
One other thing that I would need to disclose about my app is that I have bundled write operations in the sense that there are some entities that when they need to be stored, I usually call a proxy function that stores that entity and derivative entities along with it. Not in a transactional way since it's not a big deal if there's a write failure every now and then. But I don't really see a way around that given the logic of how the models are related.
So I'm eager to hear if someone else faced that problem and which approach / tools /etc they followed to solve it. Or if there are just some general things one can do..
You can try appstats tool. It can show you datastore calls stats, etc.
I need to implement master/slave/load balancing into an existing site.
Does anyone use these (or other) implementations for master/slave switching?
The resources I found on how to implement master/slave into Cake:.
(preferable) gamephase.net/posts/view/master-slave-datasource-behavior-cakephp
http://bakery.cakephp.org/articles/view/master-slave-support-also-with-multiple-slave-support
http://bakery.cakephp.org/articles/view/load-balancing-and-mysql-master-and-slaves-2
I'm getting number 1) to work most of the times but it has trouble with some of the joins.
I welcome new sources, hacks or mods for master/slave implementation as for now I can't get my head around it.
(Cake version I am using atm is 1.2)
(I'm cross posting this on CakePHP's google groups http://groups.google.co.uk/group/cake-php/browse_thread/thread/4b77af429759e08f)
Take a look at this tutorial in regards to Master/Slave over several nodes.
http://www.howtoforge.com/setting-up-master-master-replication-on-four-nodes-with-mysql-5-on-debian-etch
This may help you understand better.
As far as I can tell this happens if your model has relationships with models that do not use the same behaviour. Please correct me if this assumption is wrong.
All models have meta-data, which CakePHP accumulates using a DESCRIBE query on the database, if this data is not present your joins will be broken. This meta-data is database config specific.
CakePHP uses this meta-data to populate the $this->_schema property. SQL joins are built with data from the $this->_schema property and I guess this is where your issue lies, the database introduced by this MasterSlave switch behaviour do not have any model meta-data for tables associated with the model.
A solution would be to update your behaviour so that it only switches selectively on read and writes. Add this behaviour to all related models. i.e Any model that is related using hasOne, hasMany etc should also use the same behaviour.
In essence all models that are related should write to the same database and read from the same database.
The bonus of this solution is you will share the same database connections.
Your web app seems to be multi tier, you need to scale each tier individually:
The web layer, i.e. tha CakePHP app can be spread across multiple web servers. This is easy to do, as the code itself is idempotent. You should look into how to load balance apache servers, it is not a big deal. Webservers have quite high throughput though, so if you have a bottleneck here, you might improve your code/caching strategy instead. (Use memcache instead of file caches for example.) If you depend on the file system (uploads for example) this becomes a bit more complex, as it must become distributed or separated.
The data layer. There are various tutorials how to scale/load balance MySQL already linked by others.
Albeit first I would suggest to make benchmarks. (Premature optimization is the root of all evil.) You must know first where the bottlenecks are, where the throughput should scale. Often you can optimize queries, caching, or make thing cacheable in the first place. You must also be clear in your goals: scalability? fault tolerance?
I am currently working within a custom framework that uses the database to setup a Page Object which contains the information about Module, View, Controller, etc which is used by a Front Controller to handle routing and the like within an MVC (obviously) pattern.
The original reason for handling the pages within the database was because we needed to be able to create new landing pages on the fly from within a admin interface and because we also needed to create onLoad and onUnload events to which other dynamic objects could be attached.
However, after reading this post yesterday, it made me wonder if we shouldn't move this handling out of the database and make it all file structure and code driven like other frameworks so that pages can be tested without having the database be a component.
I am currently looking at whether to scrap the custom framework and go with one of the standard frameworks and extend it (which is what's most likely right now), but I'm wondering whether to extend the framework to handle page requests through database like we are now or if we should simply go with whatever routing / handling mechanism comes with the framework?
Usually I'm pretty lenient on what I will allow to go on in a "toy" application, but I think there are some bad habits that should be avoided no matter what. Databases are powerful tools, with reasonably powerful languages via stored procedures for doing whatever you need to have done... but they really should be used for storing and scaling access to data, and enforcing your low level data consistency rules.
Putting business logic in the data layer was common years ago, but separation of concerns really does help with the maintainability of an application over its lifespan.
Note that there is nothing wrong with using a database to store page templates instead of the file system. The line between the two will blur even further in the future and I have one system that all of the templates are in the database because of problems with the low budget hosting and how dynamically generated content need to be saved. As long as your framework can pull out a template as easily from a file or a field and process them, it isn't going to matter much either way.
On the other hand, the post from yesterday was about generating the UI layer elements directly from the database (at least, that's how I read it) and not an unusual storage location for templates. That is a deep concern for the reasons mentioned... the database becomes locked to web apps, and only web apps.
And on the third hand, never take other people's advice too much to heart if you have a system that works well and is easy to extend. Every use-case is slightly different. If your maintainability isn't suffering and it serves the business need, it is good enough.