Here's my situation: we have master tables with relationships to attribute tables. Sometimes, we fetch a row all by itself:
my $row = $rs->search({ some_key => 'some_value' })->first;
and sometimes we join one or more tables:
my $row = $rs->search({ some_key => 'some_value' }, { join => 'attributes' });
We have "helper" methods that look up specific attributes:
sub get_x_attr {
my $obj = shift;
my $x_attr = $obj->attributes->search({ attribute_name => 'x' })->one_row;
return $x_attr ? $x_attr->attribute_value : 'default';
}
This seems to issue another query, and while it's a pretty low-impact query, these add up when you are doing that zillions of times a day.
Now, if the row was joined originally, I could write the helper as:
my #attrs = grep { $_->attribute_name eq 'x' } $obj->attributes->all;
my $x_attribute = $attrs[0] || return 'default';
# etc.
and there'd be no additional query.
Here's my question: is there a safe, reliable way to interrogate "$obj" to see if it's got attributes pre-fetched? And further, is there any way to tell after the fact if the join was conditional (e.g., 'WHERE attribute_name = 'some_other_value', which would make $obj->attributes rather useless here)?
(I did some digging, and found that $obj->{internals}{related_resultsets} has the answer to the first question, but since it's not part of the exposed API, I'm very much opposed to using it this way.)
Use the relationship methods, if the relationship is prefetched the cached related row is used, if not a query is automatically executed.
Note that joining won't populate the cache, only prefetch does.
Search will always issue a query and never restrict the resultset via Perl code like your grep example.
Related
I'm trying to sort multiple records for a model based on a field and store their ranks in DB. Like below:
$instances = Model::orderBy('field')->get();
$rank = 1;
foreach ($instances as $instance) {
$instance->update([
'rank' => $rank,
]);
$rank++;
}
I have two questions:
1- Is there any alternative ways to avoid using loop? for example I put the ranks in an array and update the whole records by only one magic method. For example:
$instances = Model::orderBy('field')->get();
$rank = 1;
$ranks_array = array();
foreach ($instances as $instance) {
array_push($ranks_array, $rank);
$rank++;
}
$instances->magicMethod($ranks_array);
2- Is it necessary at all to do so? are the loops have heavy effects on the server or not? need to say that the number of records I'm going to update may not exceed 50 at all.
For insert queries, inserting all records at once will go much faster than inserting them one by one. However for update queries, if you need to update specific rows with specific data, there is no other way than to update them one by one.
I recently came across a similar issue where I needed to update 90k+ row from my DB.
Because I needed to add specific values to each column I needed to individually update each column.
What I found was instead of doing
$DBModel = Model::get();
foreach ($DBModel as $row) {
$row->notify = $row->paid;
// the date is calculated from a specific carbon date from another column
// I did not include the logic here for the example
$row->due = "0000-00-00 00:00:00";
$row->save();
}
Running the previous query took 5m33s
But doing it like this
$DBModel = Model::get();
DB::beginTransaction();
foreach ($DBModel as $row) {
DB::update(update TABLE_NAME set due = ?, notify = ? where id = ?",
["0000-00-00 00:00:00", $row->paid, $row->id]
);
}
DB::commit();
The previous query took only 1m54s to execute.
I read the cookbook, but I can not figure out how to combine in a single query a matching() and a orWhere().
Example: I have Photo that belongs from Album. Both have the active field. So I'm trying to write a findInactive() method. A "inactive" photo has the active field as false or matching an album that has the active fields as false.
Something like this:
public function findInactive(Query $query, array $options)
{
$query->matching('Albums', function ($q) {
return $q->where(['Albums.active' => false]);
})
->orWhere(['Photos.active' => false])
->enableAutoFields(true);
return $query;
}
But that does not work:
'SELECT [...] FROM photos Photos INNER JOIN photos_albums Albums ON (Albums.active = :c0 AND Albums.id = (Photos.album_id)) WHERE Photos.active = :c1'
How to do? Thanks.
EDIT
Maybe a possible solution is usecontain():
$query->contain(['Albums => ['fields' => ['active']]])
->where(['Photos.active' => false])
->orWhere(['Albums.active' => false]);
But is it not possible to use matching() or innerJoinWith()?
Add the conditions to the main query
matching() or innerJoinWith() with conditions is the wrong way to go, as the conditions are being addded to the INNER joins ON clause, which will cause the Photo row to be excluded in case the Albums.active condition doesn't match.
If you want to only receive photos that belong to an album, then you want to use matching() or innerJoinWith(), but you'll have to add the conditions to the main query instead, ie:
$query
->innerJoinWith('Albums')
->where(['Albums.active' => false])
->orWhere(['Photos.active' => false])
// ...
In case a photo doesn't have to belong to an album, or it's not important whether it does, you could use either leftJoin(), leftJoinWith(), or even contain().
The latter however may use the INNER joinStrategy and/or the select strategy (which uses a separate query), so you'd need to take care of ensuring that it uses LEFT and join instead. Using containments however is usually only advised if you actually want to contain something, and given that your finder seems to be supposed to just filter things, I'd say go with leftJoinWith() instead:
$query
->leftJoinWith('Albums')
->where(['Albums.active' => false])
->orWhere(['Photos.active' => false])
// ...
See also
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Filtering by Associated Data Via Matching And Joins
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Retrieving Associated Data
Cookbook > Database Access & ORM > Associations - Linking Tables Together > BelongsTo Associations
Lets say I have a model, Article that has a large amount of columns and the database contains more than 100,000 rows. If I do something like var articles = db.Articles.ToList() it is retrieving the entire article model for each article in the database and holding it in memory right?
So if I am populating a table that only shows the date of the entry and it's title is there a way to only retrieve just these columns from the database using the entity framework, and would it be more efficient?
According to this,
There is a cost required to track returned objects in the object
context. Detecting changes to objects and ensuring that multiple
requests for the same logical entity return the same object instance
requires that objects be attached to an ObjectContext instance. If you
do not plan to make updates or deletes to objects and do not require
identity management , consider using the NoTracking merge options when
you execute queries.
it looks like I should use NoTracking since the data isn't being changed or deleted, only displayed. So my query now becomes var articles = db.Articles.AsNoTracking().ToList(). Are there other things I should do to make this more efficient?
Another question I have is that according to this answer, using .Contains(...) will cause a large performance drop when dealing with a large database. What is the recommended method to use to search through the entries in a large database?
It's called a projection and just translates into a SELECT column1, column2, ... in SQL:
var result = db.Articles
.Select(a => new
{
Date = a.Date,
Title = a.Title
})
.ToList();
Instead of a => new { ... } (creates a list of "anonymous" objects) you can also use a named helper class (or "view model"): a => new MyViewModel { ... } that contains only the selected properties (but you can't use a => new Article { ... } as an entity itself).
For such a projection you don't need AsNoTracking() because projected data are not tracked anyway, only full entity objects are tracked.
Instead of using Contains the more common way is to use Where like:
var date = DateTime.Now.AddYears(-1);
var result = db.Articles
.Where(a => date <= a.Date)
.Select(a => new
{
Date = a.Date,
Title = a.Title
})
.ToList();
This would select only the articles that are not older than a year. The Where is just translated into a SQL WHERE statement and the filter is performed in the database (which is as fast as the SQL query is, depending on table size and proper indexing, etc.). Only the result of this filter is loaded into memory.
Edit
Refering to your comment below:
Don't confuse IEnumerable<T>.Contains(T t) with string.Contains(string subString). The answer you have linked in your question talks about the first version of Contains. If you want to search for articles that have the string "keyword" in the text body you need the second Contains version:
string keyword = "Entity Framework";
var result = db.Articles
.Where(a => a.Body.Contains(keyword))
.Select(a => new
{
Date = a.Date,
Title = a.Title
})
.ToList();
This will translate into something like WHERE Body like N'%Entity Framework%' in SQL. The answer about the poor performance of Contains doesn't apply to this version of Contains at all.
I have been trying to figure out how to do this and it seems that its not something that many people are trying to do in cakephp or I am just completely misunderstanding the documentation.. I am using the following query below to get a field value so I get a value where the "findbycreated" is 0... this part works fine
$unregisteredemail = $this->Testmodel->findBycreated('0');
$emailaddress = $unregisteredemail['Testmodel']['emailaddress'] ;
$emailpassword = $unregisteredemail['Testmodel']['password'] ;
But now, after I do some things with this data that I retrieved, I want to mark a field, in the same row, in the same model / table as a value of '1' to indicate that an action has taken place (email address has been successfully created, for example)... I just can't figure out how to do this in cakephp despite my efforts of going through the documentation and searching, this should be rather simple, I am tempted, at this point, to just use a regular mysql query as its a simple query.. basically the query is (please excuse my syntax as I haven't used direct mysql queries in a while) "update (database / table) set 'created'='1' where 'emailaddress'=$emailaddress"
Or I could use the row ID, if needed, as cakephp seems to prefer this, but still can't get how to do this.. this is my attempt below that is not working:
// update database to show that email address has been created
$this->Testmodel->read('emailaddress', $this->Testmodel->findBycreated('0'))
$this->Testmodel->id = 1;
$this->Testmodel->set(array(
'created' => '1'
));
$this->Testmodel->save();
There are, as you can see from the previous answers, several ways to achieve the same end. I'd just like to explain a little about why your way didn't work.
In the model, CakePHP has abstracted the database row(s) into an array according its implementation of ORM . This provides us with a handy way of manipulating the data and chucking it around the MVC architecture.
When you say:
$this->Testmodel->set(array(
'created' => '1'
));
You are dealing directly with the model, but the data is actually stored, as an array, in a class variable called $data. To access and manipulate this data, you should instead say:
$this->data['Testmodel']['created'] => '1';
The reason for specifying the model name as the first index is that where associated tables have been retrieved, these can be accessed in the same way, so you might have , for instance:
Array([Testmodel] => Array ([id] => 1,
[created] => [1],
...
)
[Relatedmodel] => Array ([id] => 1,
[data] => asd,
...
)
)
...and so on. Very handy.
Now, when you use $this->MyModelName->save() with no parameters, it uses $this->data by default and uses the part of the array of data appropriate to the model you are calling the save method on. You can also pass an array of data, formatted in the same way if, for some reason, you don't (or can't) use $this->data.
Your use of the method read() is incorrect. The first parameter should be null, a string or an array of strings (representing fieldname(s)). The second parameter should be the id of the record you wish to read. Instead, for param 2, you are passing the result of a find, which will be an array. The result, which you are not capturing, will be empty.
I would write your code like:
$this->data = $this->Testmodel->read('emailaddress',1);
$this->data['Testmodel']['created'] = 1;
$this->Testmodel->save();
or more succinctly:
$this->Testmodel->id = 1;
$this->Testmodel->saveField('created', 1);
In this situation I would let Cake deal with the id's and just focus on changing the row data and resaving it to the database
$row = $this->Model->findBycreated(0);
$row['Model']['emailaddress'] = 1;
$this->Model->save($row);
This way, you don't have to worry about the id's, as the id will be in your dataset anyway, so just change what you want and then tell Cake to save it.
Ninja edit, Be sure that you are returning a full row with an id from your findBycreated() method.
There're many ways to do your work.I suggest you to read the cookbook about saving data in cakephp.And besides david's solution another simple way would be
$this->Testmodel->id = 1;
$this->Testmodel->saveField('created' =>'1');
Ok, I think I finally found the solution, I was able to get this to work:
$this->Test->updateAll(
array(
'Test.field' => 'Test.field+100'
),
array(
'Test.id' => 1
)
);
I think you have to use updateAll as anything else will just create a new row.. basically CakePHP, for whatever reason, neglected to include a function for updating just one field so you have to put it into an array with the updateAll to make it work...
the +100 is where the updated info goes, so in this case "100" would be what the field is updated to.
In cakephp 3.x the syntax seems to be different. This is what worked for me in 3.x:
$this->Tests->updateAll(
[
'Tests.field = Tests.field+100'
],
[
'Tests.id' => 1
]
];
The difference is that the entire expression needs to be in the value of the first array.
I have a n...n structure for two tables, makes and models. So far no problem.
In a third table (products) like:
id
make_id
model_id
...
My problem is creating a view for products of one specifi make inside my ProductsController containing just that's make models:
I thought this could work:
var $uses = array('Make', 'Model');
$this->Make->id = 5; // My Make
$this->Make->find(); // Returns only the make I want with it's Models (HABTM)
$this->Model->find('list'); // Returns ALL models
$this->Make->Model->find('list'); // Returns ALL models
So, If I want to use the list to pass to my view to create radio buttons I will have to do a foreach() in my make array to find all models titles and create a new array and send to the view via $this->set().
$makeArray = $this->Make->find();
foreach ($makeArray['Model'] as $model) {
$modelList[] = $model['title'];
}
$this->set('models', $models)
Is there any easier way to get that list without stressing the make Array. It will be a commom task to develops such scenarios in my application(s).
Thanks in advance for any hint!
Here's my hint: Try getting your query written in regular SQL before trying to reconstruct using the Cake library. In essence you're doing a lot of extra work that the DB can do for you.
Your approach (just for show - not good SQL):
SELECT * FROM makes, models, products WHERE make_id = 5
You're not taking into consideration the relationships (unless Cake auto-magically understands the relationships of the tables)
You're probably looking for something that joins these things together:
SELECT models.title FROM models
INNER JOIN products
ON products.model_id = models.model_id
AND products.make_id = 5
Hopefully this is a nudge in the right direction?
Judging from your comment, what you're asking for is how to get results from a certain model, where the condition is in a HABTM related model. I.e. something you'd usually do with a JOIN statement in raw SQL.
Currently that's one of the few weak points of Cake. There are different strategies to deal with that.
Have the related model B return all ids of possible candidates for Model A, then do a second query on Model A. I.e.:
$this->ModelB->find('first', array('conditions' => array('field' => $condition)));
array(
['ModelB'] => array( ... ),
['ModelA'] => array(
[0] => array(
'id' => 1
)
)
Now you have an array of all ids of ModelA that belong to ModelB that matches your conditions, which you can easily extract using Set::extract(). Basically the equivalent of SELECT model_a.id FROM model_b JOIN model_a WHERE model_b.field = xxx. Next you look for ModelA:
$this->ModelA->find('all', array('conditions' => array('id' => $model_a_ids)));
That will produce SELECT model_a.* FROM model_a WHERE id IN (1, 2, 3), which is a roundabout way of doing the JOIN statement. If you need conditions on more than one related model, repeat until you have all the ids for ModelA, SQL will use the intersection of all ids (WHERE id IN (1, 2, 3) AND id IN (3, 4, 5)).
If you only need one condition on ModelB but want to retrieve ModelA, just search for ModelB. Cake will automatically retrieve related ModelAs for you (see above). You might need to Set::extract() them again, but that might already be sufficient.
You can use the above method and combine it with the Containable behaviour to get more control over the results.
If all else fails or the above methods simply produce too much overhead, you can still write your own raw SQL with $this->Model->query(). If you stick to the Cake SQL standards (naming tables correctly with FROM model_as AS ModelA) Cake will still post-process your results correctly.
Hope this sends you in the right direction.
All your different Make->find() and Model->find() calls are completely independent of each other. Even Make->Model->find() is the same as Model->find(), Cake does not in any way remember or take into account what you have already found in other models. What you're looking for is something like:
$this->Product->find('all', array('conditions' => array('make_id' => 5)));
Check out the Set::extract() method for getting a list of model titles from the results of $this->Make->find()
The solution can be achieved with the use of the with operation in habtm array on the model.
Using with you can define the "middle" table like:
$habtm = " ...
'with' => 'MakeModel',
... ";
And internally, in the Model or Controller, you can issue conditions to the find method.
See: http://www.cricava.com/blogs/index.php?blog=6&title=modelizing_habtm_join_tables_in_cakephp_&more=1&c=1&tb=1&pb=1