I have built classified website using Yii php framework. Now it is getting a lot of traffic. So I want to using caching to optimize the performance of the website.
There are two controllers I want to optimize.
One is the thread list controller: (example) http://www.shichengbbs.com/category/view/id/15
The other one the the thread controller: (example) http://www.shichengbbs.com/info/view/id/67900
What I have done:
the thread list is cached for 3mins.(The other option is update the thread list only when new thread comes)
set the last-modified time HTTP header for the thread view. (expire time is not set, as some user complain that the page appears unchanged after editing)
Partial caching the categories navigation fragment.(It appears on the left side of every page)
Use htaccess to set expire header for img/html/css/js.
Considered database sql caching for the thread list, but not done. As I thought it is the same as 1.
What else can I do to improve the website performance?
I assume you have done the Performance Tuning guide point 1 and 3. It's really helpful.
For number 2 you can use the CHttpCacheFilter
class CategoryController extends Controller {
private $_categoryLastUpdate;
public function filters(){
return array(
array(
'CHttpCacheFilter + view',
'cacheControl' => " max-age=604800, must-revalidate",
'etagSeedExpression' => function() {
return $this->getCategoryLastUpdate();
}
'lastModifiedExpression' => function() {
return $this->getCategoryLastUpdate();
}
)
)
}
public function actionView($id){
$object = Category::model()->findByPk($_GET['id']);
$this->render('view', array('object' => $object));
}
public function getCategoryLastUpdate(){
if (!isset($this->_categoryLastUpdate)){
$obj = Category::model()->findByPk($_GET['id'], array('select' => 'lastUpdate'));
$this->_categoryLastUpdate
}
return $this->_categoryLastUpdate;
}
}
It basically will calculate the ETag and LastUpdate by the category. And to save the query, it will first only calculate the lastUpdate of the Category object.
And for number one, you can always use the CCacheDependency. Just make a field in the thread list object, e.g. lastUpdate. And when a new thread submitted, just update the field and use it for the CCacheDependency.
Since I see you are using a very large pagination, I think you want to read about Four Ways to Optimize Paginated Displays (if you use MySQL for your database and thread search/list).
Try using a Cache Manager with Memcache or APC. For example, http://code.google.com/p/memcache-flag/ . When you edit the list, then you can invalidate the cache item or tag. I suppose it could also just be done with regular APC / Memcache functions if you design is simple (set a key and delete it when it is no longer valid).
Use this to store serialized (or automatically serialized) data instead of retrieving it from mysql.
Related
I working on an application that has its own database and gets user information from another serivce (an LDAP is this case, through an API package).
Say I have a tables called Articles, with a column user_id. There is no Users table, instead a user or set of users is retrieved through the external API:
$user = LDAPConnector::getUser($user_id);
$users = LDAPConnector::getUsers([1, 2, 5, 6]);
Of course I want retrieving data from inside a controller to be as simple as possible, ideally still with something like:
$articles = $this->Articles->find()->contain('Users');
foreach ($articles as $article) {
echo $article->user->getFullname();
}
I'm not sure how to approach this.
Where should I place the code in the table object to allow integration with the external API?
And as a bonus question: How to minimise the number of LDAP queries when filling the Entities?
i.e. it seems to be a lot faster by first retrieving the relevant users with a single ->getUsers() and placing them later, even though iterating over the articles and using multiple ->getUser() might be simpler.
The most simple solution would be to use a result formatter to fetch and inject the external data.
The more sophisticated solution would a custom association, and a custom association loader, but given how database-centric associations are, you'd probably also have to come up with a table and possibly a query implementation that handles your LDAP datasource. While it would be rather simple to move this into a custom association, containing the association will look up a matching table, cause the schema to be inspected, etc.
So I'll stick with providing an example for the first option. A result formatter would be pretty simple, something like this:
$this->Articles
->find()
->formatResults(function (\Cake\Collection\CollectionInterface $results) {
$userIds = array_unique($results->extract('user_id')->toArray());
$users = LDAPConnector::getUsers($userIds);
$usersMap = collection($users)->indexBy('id')->toArray();
return $results
->map(function ($article) use ($usersMap) {
if (isset($usersMap[$article['user_id']])) {
$article['user'] = $usersMap[$article['user_id']];
}
return $article;
});
});
The example makes the assumption that the data returned from LDAPConnector::getUsers() is a collection of associative arrays, with an id key that matches the user id. You'd have to adapt this accordingly, depending on what exactly LDAPConnector::getUsers() returns.
That aside, the example should be rather self-explanatory, first obtain a unique list of users IDs found in the queried articles, obtain the LDAP users using those IDs, then inject the users into the articles.
If you wanted to have entities in your results, then create entities from the user data, for example like this:
$userData = $usersMap[$article['user_id']];
$article['user'] = new \App\Model\Entity\User($userData);
For better reusability, put the formatter in a custom finder. In your ArticlesTable class:
public function findWithUsers(\Cake\ORM\Query $query, array $options)
{
return $query->formatResults(/* ... */);
}
Then you can just do $this->Articles->find('withUsers'), just as simple as containing.
See also
Cookbook > Database Access & ORM > Query Builder > Adding Calculated Fields
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Custom Finder Methods
I want to use Yii2 and redis as database.
So far, I got Redis ActiveRecord Class for Yii2 from Here.
link1
link2
but, I got a problem. WHY THIS CLASS ADDS ANYTHING AS HASH IN REDIS????
Above that I cant Find in which pattern it Insert data. I add one user and it will add a user under user:xxx namespace and another record under s:user:xxx and so on but none of theme has any fields that i defined in attributes!! only contain IDs.
I know that a Key-value type database and RDBMS are different and also know how can implement relation like records in Redis, but I don't know why it will only save IDs.
I could not find any example of using redis ActiveRecords so far.
There is one in here and its not good enough.
So here is my main wuestion: how can add data to redis Using activeRecords and different data types In YII2?
And if its impossible with ActiveRecords what is the best solution? in this case
ANOTHER QUESTION: is it possible to use a Model instead and write my own model::save() method? and what is the best data validation solution at this rate?
Actually I want to make a telegram bot, so i should get messages and send them in RabitMQ and get data in a worker, do the process and save results to Redis, and finally send response to user through the RabitMQ.
So I need to do a lot of validations AND OF COURSE AUTHENTICATIONS and save and select and range and save to sets an lists and this and that ....
I want a good way to make Model or active record or the proper solution to validation, save and retrieve data to Redis and Yii2.
Redis DB can be declared as a cache component or as a database connection or both.
When it is declared as a cache component (using the yii/redis/cache) it is accessible within that component to store key/value pairs as shown here.
$cache = Yii::$app->cache;
// try retrieving $data from cache
$data = $cache->get($key);
// store $data in cache so that it can be retrieved next time
$cache->set($key, $data);
// one more example:
$access_token = Yii::$app->security->generateRandomString();
$cache->add(
// key
$access_token,
// data (can also be an array)
[
'id' => Yii::$app->user->identity->id
'name' => Yii::$app->user->identity->name
],
// expires
60*60*3
);
Also other components may start using it for caching proposes like session if configured to do so or like the yii\web\UrlManager which by default will try to cache the generated URL rules in whatever valid caching mechanism defined under the config file's cache component as explained here. So it is normal to find some stored data other than yours in that case.
When Redis is declared as a DB connection like in the links you provided which means using the yii\redis\Connection class you can make your model extending its \yii\redis\ActiveRecord class as any other ActiveRecord model in Yii. The only difference I know so far is that you need to define your attributes manually as there is no DB schema to parse for NoSQL databases. Then just define your rules, scenarios, relations, events, ... as any other ActiveRecord model:
class Customer extends \yii\redis\ActiveRecord
{
public function attributes()
{
return ['id', 'name', 'address', 'registration_date'];
}
public function rules()
{
return [
['name', 'required'],
['name', 'string', 'min' => 3, 'max' => 12, 'on' => 'register'],
...
];
}
public function attributeLabels() {...}
...
}
All available methods including save(), validate(), getErrors(), ... could be found here and should be used like any other ActiveRecord class as shown in the official guide.
I've two controllers one is "Upload" which deals with images uploads and other is "Page" whid deals with the creation of pages of CMS now if in my "Upload" controller I load both the models i.e 'image_m' which deals with image upload and "page_m" which deals with the pages creation I've highlighted the relevant code my problem is if I access the variables in the view
$this->data['images'] = $this->image_m->get(); sent by this I can access in foreach loop as "$images->image_title, $images->image_path" etc
But the variable sent by this line ***$this->data['get_with_images'] = $this->page_m->get_no_parents();*** as $get_with_images->page_name, $get_with_images->page_id etc produces given error
A PHP Error was encountered
Severity: Notice
Message: Trying to get property of non-object
Filename: upload/index.php
Line Number: 20
what is the difference between these two access levels one for $image & other for $get_with_images because I can only access its values as $get_with_images
class Upload extends Admin_Controller {
public function __construct() {
parent::__construct();
***$this->load->model('image_m');
$this->load->model('page_m');***
}
public function index($id = NULL) {
//var_dump($this->data['images'] = $this->image_m->get_with_images());
//$this->data['images'] = $this->image_m->get_with_images();
***$this->data['images'] = $this->image_m->get();***
$this->data['subview'] = 'admin/upload/index';
if ($id) {
$this->data['image'] = $this->image_m->get($id);
count($this->data['image']) || $this->data['errors'][] = 'Page Could not be found';
}
$id == NULL || $this->data['image'] = $this->image_m->get($id);
/*this calls the page_m model function to load all the pages from pages table*/
***$this->data['get_with_images'] = $this->page_m->get_no_parents();***
You are not posting all your code so its hard to tell but is it because you used $this-> in the controller, but you haven't done the same thing in the view?
In this case i would recommend not using $this-> because its not necessary. Also its much better to check for errors etc when you call the model so do something like
if ( ! $data['images'] = $this->image_m->get($id) ) {
// Failure -- show an appropriate view for not getting any images
// am showing $data in case you have other values that are getting passed
$this->load->view( 'sadview', $data ); }
else {
// Success -- show a view to display images
$this->load->view( 'awesomeview', $data ); }
so we are saying if nothing came back - the ! is a negative - then show the failure view. Else $data['images'] came back, and it will be passed to the view. note i have not had to use $this-> for anything and it won't be needed in the view.
Would also suggest using separate methods - have one method to show all images and a separate method like returnimage($id) to show an image based on a specific validated $id.
====== Edit
You can access as many models as you want and pass that data to the View. You have a different issue - the problem is that you are waiting until the View to find out - and then it makes it more difficult to figure out what is wrong.
Look at this page and make sure you understand the differences between query results
http://ellislab.com/codeigniter/user-guide/database/results.html
When you have problems like this the first thing to do is make a simple view, and echo out directly from the model method that is giving you problems. Its probably something very simple but you are having to look through so much code that its difficult to discover.
The next thing is that for every method you write, you need to ask yourself 'what if it doesn't return anything?' and then deal with those conditions as part of your code. Always validate any input coming in to your methods (even links) and always have fallbacks for any method connecting to a database.
On your view do a var_dump($get_with_images) The error being given is that you are trying to use/access $get_with_images as an object but it is not an object.
or better yet on your controller do a
echo '<pre>';
var_dump($this->page_m->get_no_parents());
exit();
maybe your model is not returning anything or is returning something but the data is not an object , maybe an array of object that you still need to loop through in some cases.
I'm building a dynamic view (Page) that consists of multiple elements (widgets) called via $this->element('messages_unread'). Some of these elements need data that is not related to the Page model.
In real life words: my users will be able to construct their own Page by choosing from a multitude of elements ("top 5 posts", "10 unread messages", etc...)
I get the data by calling $this->requestAction(array('controller'=>'events','action'=>'archive') from within the element, the url-variables differ per element .
I'm aware of the fact that requestAction() is expensive and I plan on limiting the costs by proper caching.
The actual question:
My problem is Pagination. When I'm in the Page view and call requestAction('/events/archive') the PaginatorHelper in the Page view will be unaware of the Event model and its paginator variables and $this->Paginator->next() etc... will not work.
How can I implement proper Pagination? I've tried to set the model by calling $this->Paginator->options(array('model'=>'Event')) but that doesn't work.
Do I maybe need to return custom defined Pagination variables in the requestAction and thus construct my own?
Or is there another approach that maybe even avoids requestAction()? And keep in mind here that the requested data is unrelated to the Page.
Kind regards,
Bart
[Edit] My temporary solution but still open for comments/solutions:
In the requestedAction Event/archive, return paginator variables along with the data like this:
return array('data'=>$this->paginate(), 'paging' => $this->params['paging']);
I've tinkered a bit more and the following works for me, and the PaginationHelper works:
In the element:
// requestAction returns an array('data'=>... , 'paging'=>...)
$data = $this->requestAction(array('controller'=>'events','action'=>'archive'));
// if the 'paging' variable is populated, merge it with the already present paging variable in $this->params. This will make sure the PaginatorHelper works
if(!isset($this->params['paging'])) $this->params['paging'] = array();
$this->params['paging'] = array_merge( $this->params['paging'] , $data['paging'] );
foreach($data['events'] as $event) {
// loop through data...
}
In the Controller:
public function archive() {
$this->paginate = array(
'limit' => 10
);
if ($this->params['requested'])
return array('events'=>$this->paginate('Event'), 'paging' => $this->params['paging']);
$this->set('events', $this->paginate('Event') );
}
I'm building a site with CodeIgniter, and my I have a model called Blog_model.
Within Blog_model, there are methods to pull a list of posts for a specific topic, for example, getPopularPosts().
getPopularPosts() queries the posts table for a list of posts with a topic_id matching the one specified and sorts them by popularity. So, that's one query against the entire table of posts (let's assume this will be very large eventually) to find all posts with topic_id x.
Then, foreach result as an individual post id, it creates a new Post object. The Post class constructs a post by setting the field id.
To return the contents of a Post, I assign $post->getPost();, which queries the posts table again to return the entire row for the given id.
This organization (AFAIK) follows a nice object oriented principle. But now, for every posts (again, let's assume thousands, millions, whatever...), I have to first query for a list of ids and then again to get each post's content. If I'm returning 30 posts, that means 31 separate queries.
Alternatively, I could break the object oriented pattern and pull * for each post in posts where topic_id = x. Then, I have one query that returns all 30 posts, but now I don't feel so object oriented.
What to do?
There is no reason to have that many queries. You're basically just looking for X number of posts that are from a particular topic ID... you should return this as one object and then iterate through the result in PHP because it is significantly faster to do it that way once you get to the point of having millions of rows
You should go about it more like this:
class blog_model extends CI_Model {
function __construct(){
parent::__construct();
}
function getPopularPosts($cat_id){
/* Using method chaining here since you sound like you
really want to utilize everything OO CI has to offer */
$posts = $this->db->select('id, title, post_info')
->where('topic_id', $topic_id)
->get('posts');
if($posts->num_rows() > 0){
return $posts;
}else{
return FALSE;
}
}
}
Then your controller would look like this:
class blog extends CI_Controller {
function __construct() {
parent::__construct();
}
function blog_posts($popular_post_id) {
$this->load->model('blog_model');
$posts = $this->blog_model->getPopularPosts($popular_post_id);
if(!empty($posts){
foreach($posts as $post){
echo $post->id;
echo $post->title;
echo $post->post_info;
}
}else{
echo 'There are no posts';
}
}
}
There is no benefit (and actually a big problem) with generating a ton of queries in the fashion that you currently have it set up, vs generating one object from the query and iterating through each of the rows in the controller and doing whatever you need with the data.