I'm using Tree Behavior to store a ordered list of items.
When I add new elements to the tree I reorder it calling to the "reorder()" function. It is very slow. I've been trying a lot of things. My last test was a single table with 70 elements (1 parent and 69 childs). The time used for the reorder() function, was 1 minute & 20 seconds. I'm using MySQL and I have indexes in id, parent_id, rght and lft fields.
What could I be doing wrong?
Thanks
I've spent some time investigating this one as well.
It turns out that the reorder function is very inefficient. Basically it sets the left and right fields on the first item to high values, and then reorders the rest of the fields. Then it does the same for the second item, etc. You end up with tens of thousands of queries being run, for a tree with only a hundred items.
I sorted it out with the following code. What this does is only update each record once, and it preserves the gap between left and right values. It recursively goes through the items and reorders them properly.
Create a new file in app/Model/Behavior called "BRTreeBehavior.php", and then modify your model's $actsAs field to use BRTree instead of Tree.
<?php
/**
* Copyright 2015, Perthweb Pty Ltd
*
*/
App::uses('TreeBehavior', 'Model/Behavior');
/**
* BetterReorderTree Behavior.
*
* Improves reorder of tree behavior
*
*/
class BRTreeBehavior extends TreeBehavior {
/**
* Reorder method.
*
* Reorders the nodes (and child nodes) of the tree according to the field and direction specified in the parameters.
* This method does not change the parent of any node.
*
* Requires a valid tree, by default it verifies the tree before beginning.
*
* Options:
*
* - 'id' id of record to use as top node for reordering
* - 'field' Which field to use in reordering defaults to displayField
* - 'order' Direction to order either DESC or ASC (defaults to ASC)
* - 'verify' Whether or not to verify the tree before reorder. defaults to true.
*
* #param Model $Model Model instance
* #param array $options array of options to use in reordering.
* #return boolean true on success, false on failure
* #link http://book.cakephp.org/2.0/en/core-libraries/behaviors/tree.html#TreeBehavior::reorder
*/
public function reorder(Model $Model, $options = array()) {
$options = array_merge(array('id' => null, 'field' => $Model->displayField, 'order' => 'ASC', 'verify' => true, 'startIndex' => null), $options);
extract($options);
if ($verify && !$this->verify($Model)) {
return false;
}
$verify = false;
extract($this->settings[$Model->alias]);
$fields = array($Model->primaryKey, $field, $left, $right);
$sort = $field . ' ' . $order;
$nodes = $this->children($Model, $id, true, $fields, $sort, null, null, $recursive);
$cacheQueries = $Model->cacheQueries;
$Model->cacheQueries = false;
if ($nodes) {
if($id == null){
$index = 1;
}
else if($startIndex == null){
$index = $nodes[0][$Model->alias][$left];
}
else {
$index = $startIndex;
}
foreach ($nodes as $node) {
$id = $node[$Model->alias][$Model->primaryKey];
$difference = $node[$Model->alias][$right] - $node[$Model->alias][$left];
$nodeData = array(
$Model->alias => array(
$Model->primaryKey => $id,
$left => $index,
$right => $index + $difference
)
);
$Model->create();
$Model->save($nodeData, array('validate' => false, 'callbacks' => false, 'fieldList' => array($left, $right)));
$Model->clear();
$startIndex = $index + 1;
if ($difference != 1) {
$this->reorder($Model, compact('id', 'field', 'order', 'verify', 'startIndex'));
}
$index += $difference + 1;
}
}
$Model->cacheQueries = $cacheQueries;
return true;
}
}
Related
I recently installed CRM Core and all of its missing modules needed to run it. Sadly I need this module for the project that I am working on but the second I installed them I got this error.
Fatal error: Declaration of CRMCoreContactController::save($contact) must be compatible with EntityAPIController::save($entity, ?DatabaseTransaction $transaction = NULL) in /opt/lampp/htdocs/drupal/modules/crm_core/modules/crm_core_contact/includes/crm_core_contact.controller.inc on line 111
I went back in the code and I couldn't see what to change. Line 111 is the ver last line of the code. Ill paste the code as well maybe someone out there knows how to solve this, please.
<?php
/**
* CRM Contact Entity Class.
*/
class CRMCoreContactEntity extends Entity {
protected function defaultLabel() {
return crm_core_contact_label($this);
}
protected function defaultUri() {
return array(
'path' => 'crm-core/contact/' . $this->identifier(),
'options' => array(
'absolute' => TRUE,
),
);
}
/**
* Method for de-duplicating contacts.
*
* Allows various modules to identify duplicate contact records through
* hook_crm_core_contact_match. This function should implement it's
* own contact matching scheme.
*
* #return array
* Array of matched contact IDs.
*/
public function match() {
$checks = & drupal_static(__FUNCTION__);
$matches = array();
if (!isset($checks->processed)) {
$checks = new stdClass();
$checks->engines = module_implements('crm_core_contact_match');
$checks->processed = 1;
}
// Pass in the contact and the matches array as references.
// This will allow various matching tools to modify the contact
// and the list of matches.
$values = array(
'contact' => &$this,
'matches' => &$matches,
);
foreach ($checks->engines as $module) {
module_invoke($module, 'crm_core_contact_match', $values);
}
// It's up to implementing modules to handle the matching logic.
// Most often, the match to be used should be the one
// at the top of the stack.
return $matches;
}
}
/**
* #file
* Controller class for contacts.
*
* This extends the DrupalDefaultEntityController class, adding required
* special handling for contact objects.
*/
class CRMCoreContactController extends EntityAPIController {
public $revisionKey = 'vid';
public $revisionTable = 'crm_core_contact_revision';
/**
* Create a basic contact object.
*/
public function create(array $values = array()) {
global $user;
$values += array(
'contact_id' => '',
'vid' => '',
'uid' => $user->uid,
'created' => REQUEST_TIME,
'changed' => REQUEST_TIME,
);
return parent::create($values);
}
/**
* Update contact object before saving revision.
*/
protected function saveRevision($entity) {
if (!isset($entity->log)) {
$entity->log = '';
}
$entity->is_new_revision = TRUE;
$entity->uid = $GLOBALS['user']->uid;
return parent::saveRevision($entity);
}
/**
* Updates 'changed' property on save.
*/
public function save($contact) {
$contact->changed = REQUEST_TIME;
// Storing formatted contact label for autocomplete lookups.
$contact->name = crm_core_contact_label($contact);
return parent::save($contact);
}
}
Changing
public function save($contact)
to
public function save($contact, DatabaseTransaction $transaction = NULL)
should work.
You need to switch from PHP 7.x+ to PHP 5.6. This will resolve this error.
Can't give you more advice on how to downgrade without more details on what system you're running but there are many guides out there on this topic.
I was wondering if there is a way to search in entity without applying any filters. For Example I would like to build a textfiled in my template where a ajax post method is calling to a controller with purpose searching the whole entity.
My code:
$user = $this->getDoctrine()
->getRepository('AppBundle:QCE_SUBD')
->find('%'.$SearchParam.'%')
->getQuery();
$DSUB = $user->getArrayResult();
dump($DSUB);
I;m not sure how the function should be written, so if some one is willing to help it will be highly appreciate :)
You should just create a function that return a JsonResponse with an array of your result.
// In your controller
/**
* #Route("/ajax_action")
*/
public function ajaxAction(Request $request)
{
// Get the posted parameter from your ajax call
$searchParam = $request->get('searchParam');
// Request your entity
$user = $this->getDoctrine()
->getRepository('AppBundle:QCE_SUBD')
->createQueryBuilder('q')
->where('q.username LIKE :searchParam')
->orWhere('q.otherColumn LIKE :searchParam')
->setParameter('searchParam', '%'.$searchParam.'%')
->getQuery();
// Check if it's an ajax call
if ($request->isXMLHttpRequest()) {
return new JsonResponse($user->getArrayResult();
}
// Return an error
throw new \Exception('Wrong call!');
}
For the search part you need to implement a full text search, here is a tutorial on how to implement it :
http://ourcodeworld.com/articles/read/90/how-to-implement-fulltext-search-mysql-with-doctrine-and-symfony-3
P.S : You should be sure of what you need in your query. If you want it to be scalable, you should take a look at better search engine method as ElasticSearch or Solr.
You can inspire yourself from the following function. It iterates dynamically through all fields of the entity and depending on the type of the field a condition is applied to the query builder:
/**
* Creates the query builder used to get the results of the search query
* performed by the user in the "search" view with a given "keyword".
*
* #param array $entityConfig
* #param string $searchQuery
* #param string|null $sortField
* #param string|null $sortDirection
* #param string|null $dqlFilter
*
* #return DoctrineQueryBuilder
*/
public function createSearchQueryBuilder(array $entityConfig, $searchQuery, $sortField = null, $sortDirection = null, $dqlFilter = null)
{
/* #var EntityManager */
$em = $this->doctrine->getManagerForClass($entityConfig['class']);
/* #var DoctrineQueryBuilder */
$queryBuilder = $em->createQueryBuilder()
->select('entity')
->from($entityConfig['class'], 'entity')
;
$queryParameters = array();
foreach ($entityConfig['search']['fields'] as $name => $metadata) {
$isNumericField = in_array($metadata['dataType'], array('integer', 'number', 'smallint', 'bigint', 'decimal', 'float'));
$isTextField = in_array($metadata['dataType'], array('string', 'text', 'guid'));
if ($isNumericField && is_numeric($searchQuery)) {
$queryBuilder->orWhere(sprintf('entity.%s = :exact_query', $name));
// adding '0' turns the string into a numeric value
$queryParameters['exact_query'] = 0 + $searchQuery;
} elseif ($isTextField) {
$searchQuery = strtolower($searchQuery);
$queryBuilder->orWhere(sprintf('LOWER(entity.%s) LIKE :fuzzy_query', $name));
$queryParameters['fuzzy_query'] = '%'.$searchQuery.'%';
$queryBuilder->orWhere(sprintf('LOWER(entity.%s) IN (:words_query)', $name));
$queryParameters['words_query'] = explode(' ', $searchQuery);
}
}
if (0 !== count($queryParameters)) {
$queryBuilder->setParameters($queryParameters);
}
if (!empty($dqlFilter)) {
$queryBuilder->andWhere($dqlFilter);
}
if (null !== $sortField) {
$queryBuilder->orderBy('entity.'.$sortField, $sortDirection ?: 'DESC');
}
return $queryBuilder;
}
The source code comes from the EasyAdminBundle.
I'm beginner on Symfony2.
I have a Regions-Countries-States-Cities database with more of 2,000,000 results. I have 8 entities:
Region (recursive with itself) - RegionTranslation
Country - CountryTranslation
State (recursive with itself) - StateTranslation
City - CityTranslation
The thing is that when I want to load a countries list (only 250 registers in a pulldown, for example) Symfony+Doctrine load all entities structure (all states of all countries, and all cities of all states, with their respective translations).
I think that it spends a lot of memory.
What's the correct method to do it? Can I load only Country (and translations) with this structure? Any idea?
I have had the same problem for objected that are unassociated. Your best bet is to use select2's ajax loading (http://ivaynberg.github.com/select2/), which will give a limited number of items in the search box, and also narrow searches down by what is typed in the box.
A few things things need to be coded:
A javascript file:
$(document).ready(function(){
$('.select2thing').select2({
minimumInputLength:1
,width: "100%"
,ajax: {
url: <<path>> + "entity/json"
,dataType: 'jsonp'
,quitMillis: 100
,data: function (term, page) {
return {
q: term, // search term
limit: 20,
page: page
};
}
,results: function (data, page) {
var more = (page * 20) < data.total;
return { results: data.objects, more: more };
}
}
});
}
A jsonAction in the controller:
/**
* Lists all Thing entities return in json format
*
*/
public function jsonAction(Request $request)
{
$em = $this->getDoctrine()->getManager();
$rep = $em->getRepository('yourBundle:Thing');
$qb = $rep->createQueryBuilder('e');
$limit = $request->query->get('limit');
$current = $request->query->get('current');
$page=$request->query->get('page');
$queries=$request->query->get('q');
$qarray=explode(",", $queries);
$entities=$rep->getJSON($qarray, $page, $limit);
$total=$rep->getJSONCount($qarray);
$callback=$request->query->get('callback');
return $this->render('yourBundle:Thing:json.html.twig'
, array(
'entities' => $entities
,'callback' => $callback
,'total' => $total
)
);
}
A twig template (json.html.twig, possibly customized to display more)
{{callback}}(
{ "objects" :
[
{% for entity in entities %}
{ "id": "{{entity.id}}", "text": "{{entity}}""}
{% if not loop.last %},{% endif %}
{% endfor %}
],
"total": {{total}}
}
)
A transformer:
use Symfony\Component\Form\DataTransformerInterface;
use Symfony\Component\Form\Exception\TransformationFailedException;
use Doctrine\Common\Persistence\ObjectManager;
use yourBundle\Entity\Thing;
class ThingTransformer implements DataTransformerInterface
{
/**
* #var ObjectManager
*/
private $em;
/**
* #param ObjectManager $em
*/
public function __construct(ObjectManager $em)
{
$this->em = $em;
}
/**
* Transforms an object (thing) to a string (id).
*
* #param Issue|null $thing
* #return string
*/
public function transform($thing)
{
if (null === $thing) {return "";}
if (is_object($thing) && "Doctrine\ORM\PersistentCollection"==get_class($thing)){
$entity->map(function ($ob){return $ob->getId();});
return implode(",",$thing->toArray());
}
return $thing;
}
/**
* Transforms a string (id) to an object (thing).
*
* #param string $id
* #return Issue|null
* #throws TransformationFailedException if object (thing) is not found.
*/
public function reverseTransform($id)
{
if (!$id) {
return null;
}
//if (is_array($id)){
$qb=$this->em
->getRepository('yourBundle:Thing')
->createQueryBuilder('t');
$thing=$qb->andWhere($qb->expr()->in('t.id', $id))->getQuery()->getResult();
if (null === $entity) {
throw new TransformationFailedException(sprintf(
'A thing with id "%s" does not exist!',
$id
));
}
return $thing;
}
}
Your Controller using the select2 control will have to pass the 'em' to the form builder:
$editForm = $this->createForm(new ThingType()
,$entity
,array(
'attr' => array(
'securitycontext' => $sc
,'em' => $this->getDoctrine()
->getEntityManager()
)
)
);
And in your form type:
if (isset($options['attr']['em'])){ $em = $options['attr']['em'];} else {$em=null;}
$transformer = new ThingTransformer($em);
$builder->add(
$builder->create('thing'
,'hidden'
,array(
'by_reference' => false
,'required' => false
,'attr' => array(
'class' => 'select2thing'
)
)
)
->prependNormTransformer($transformer)
);
You can try to change the hydration mode, using array consumes less memory than creating objects.
Other way you can achieve this is using iterations to avoid memory problems:
Finally I think you can't load all without spending a lot of time and memory, so, why not make several queries to load the whole data?
I've been researching a bit and I found that CakePHP's form helper doesn't interpret ENUM fields correctly, so it simply outputs a text input. I found a post that suggested to use a helper for that specific purpose. Does anybody know a better way to achieve this? Or if CakePHP devs intend to correct this some day?
Thanks for reading!
Below is one of the helper extention.
App::uses('FormHelper', 'View/Helper');
/**
* APP/View/Helper/MySqlEnumFormHelper.php
* It extends FormHelper to implement ENUM datatype of MySQL.
*
* http://blog.xao.jp/blog/cakephp/implementation-of-mysql-enum-datatype-in-formhelper/
*
* created Oct. 15, 2012
* CakePHP 2.2.3
*/
class MySqlEnumFormHelper extends FormHelper
{
public function input($fieldName, $options = array())
{
if (!isset($options['type']) && !isset($options['options'])) {
$modelKey = $this->model();
if (preg_match(
'/^enum\((.+)\)$/ui',
$this->fieldset[$modelKey]['fields'][$fieldName]['type'],
$m
)) {
$match = trim($m[1]);
$qOpen = substr($match, 0, 1);
$qClose = substr($match, -1);
$delimiter = $qOpen . ',' . $qClose;
preg_match('/^'.$qOpen.'(.+)'.$qClose.'$/u', $match, $m);
$_options = explode($delimiter, $m[1]);
$options['type'] = 'select';
$options['options'] = array_combine($_options, $_options);
}
}
return parent::input($fieldName, $options);
}
}
Cake attempts to be database agnostic and therefore this issue won't be "corrected" since it's not a bug. For example, SQL server doesn't have an exact equivalent of MySQL's ENUM field type.
I would recommend getting your possible list of enum values like so:
YourController.php
// get column type
$type = $this->Model->getColumnType('field');
// extract values in single quotes separated by comma
preg_match_all("/'(.*?)'/", $type, $enums);
// enums
var_dump($enums[1]);
Then use a select field in your view and pass the enums as options. Your current value you'll already have. How does that sound?
I am new to cakephp I found some old code and pieced together an enum select box for you enjoy
/**
* Behavior with useful functionality around models containing an enum type field
*
* Copyright (c) Debuggable, http://debuggable.com
*
*
*
* #package default
* #access public
*
* reworked by Nathanael Mallow for cakephp 2.0
*
*/
/*
*Use case:Add this (EnumerableBehavior.php) to app/Model/Behavior/
* -->in the Model add public $actsAs = array('Enumerable');
* -->in the *_controller add $enumOptions = $this->Categorie->enumOptions('Section');
* -->in the view add print $this->Form->input('{db_field_name}', array('options' => $enumOptions, 'label' => 'here'));
*
*
*/
class EnumerableBehavior extends ModelBehavior {
/**
* Fetches the enum type options for a specific field
*
* #param string $field
* #return void
* #access public
*/
function enumOptions($model, $field) {
//Cache::clear();
$cacheKey = $model->alias . '_' . $field . '_enum_options';
$options = Cache::read($cacheKey);
if (!$options) {
$sql = "SHOW COLUMNS FROM `{$model->useTable}` LIKE '{$field}'";
$enumData = $model->query($sql);
$options = false;
if (!empty($enumData)) {
$enumData = preg_replace("/(enum|set)\('(.+?)'\)/", '\\2', $enumData[0]['COLUMNS']['Type']);
$options = explode("','", $enumData);
}
Cache::write($cacheKey, $options);
}
return $options;
}
}
?>
If you want to use MySqlEnumFormHelper instead of normal and call it by $this->Form-> instead by $this->MySqlEnumFormHelper . You should add this line in your controller to alias MySqlEnumFormHelper as Form.
public $helpers = array('Form' => array(
'className' => 'MySqlEnumForm'
));
/* comments about previus answers ***
Use case:Add this (EnumerableBehavior.php) to app/Model/Behavior/
-->in the Model add public $actsAs = array('Enumerable');
-->in the action of the *_controller add $enumOptions = $this->YourModelName->enumOptions('db_field_name'); $this->set('enumOptions',$enumOptions);
-->in the view add print $this->Form->input('{db_field_name}', array('options' => $enumOptions, 'label' => 'here'));
*
*/
i think the Behaviour way it's good...but the array keys are integer
so i have modified the function like this
function enumOptions($model, $field) {
//Cache::clear();
$cacheKey = $model->alias . '_' . $field . '_enum_options';
$options = Cache::read($cacheKey);
$enumOptions = array();
if (!$options) {
$sql = "SHOW COLUMNS FROM `{$model->useTable}` LIKE '{$field}'";
$enumData = $model->query($sql);
$options = false;
if (!empty($enumData)) {
$enumData = preg_replace("/(enum|set)\('(.+?)'\)/", '\\2', $enumData[0]['COLUMNS']['Type']);
$options = explode("','", $enumData);
foreach ($options as $option) {
$enumOptions["$option"] = $option;
}
}
Cache::write($cacheKey, $enumOptions);
}
return $enumOptions;
}
in order to be able to save the right value in the db field when the form is submitted
I created a function that goes into AppController to handle this. I combined some of the information provided above.
Usage:
$enumList = getEnumValues($ModelField) where ModelField is in this format: 'Model.Field'
Function that I put in AppController:
function getEnumValues($ModelField){
// split input into Model and Fieldname
$m = explode('.', $ModelField);
if ($m[0] == $ModelField) {
return false;
} else {
(! ClassRegistry::isKeySet($m[0])) ? $this->loadModel($m[0]): false;
$type = $this->$m[0]->getColumnType($m[1]);
preg_match_all("/'(.*?)'/", $type, $enums);
foreach ($enums[1] as $value){$enumList[$value] = $value;}
return $enumList;
}
}
Is there more optimal and shorter way to get nodes with certain conditions?
$query = new EntityFieldQuery;
$result = $query
->entityCondition('entity_type', 'node')
->propertyCondition('type', $node_type)
->propertyCondition('title', $title)
->fieldCondition('field_number', 'value', '1', '=');
->propertyCondition('status', 1, '=')
->execute();
// $result['node'] contains a list of nids where the title matches
if (!empty($result['node']) {
// You could use node_load_multiple() instead of entity_load() for nodes
$nodes = entity_load('node', array_keys($result['node']));
}
$query_two = new EntityFieldQuery;
$result_two = $query_two
->entityCondition('entity_type', 'node')
->propertyCondition('type', $node_type)
->propertyCondition('title', $title)
->fieldCondition('field_number', 'value', '2', '=');
->propertyCondition('status', 1, '=')
->execute();
// $result_two['node'] contains a list of nids where the title matches
if (!empty($result_two['node']) {
// You could use node_load_multiple() instead of entity_load() for nodes
$nodes_two = entity_load('node', array_keys($result_two['node']));
}
Well, you certainly could use ->fieldCondition('field_number', 'value', array(1, 2)), but other than that, not that I know of (and I wrote EntityFieldQuery). Even if you were to rewrite this to an SQL-storage only query, it would not be much simpler.
You do not need to specify = as the operator and you do not need to specify IN either, they are defaults for a scalar / array value.
I recently wrote a wrapper for EntityFieldQuery, since i just use it too much. Then the way I call it becomes to, the result can be an id or an array of ids. Hope this makes sense.
$ids = qplot_api_find_nodes2(
array(
'type' => 'content',
'field_content_project' => array('target_id', 10, '='),
),
array(
'created' => 'ASC'
),
TRUE
);
/**
* Returns node nid(s) with filters and sorts.
*
* #param array $conds
* Condition entries, there're three different type of conditions
* 1. prefixed with entity_, ex. 'entity_type' => 'node'
* 2. prefixed with field_, ex. 'field_project',
* two formats allowed, simple version 'field_tag' => 'abc',
* or long version, 'field_tag' => array('target_id', 11, '=')
* 3. no prefix or other prefix, 'title' => 'abc'
* Default $conds contains 'entity_type' => 'node' entry.
*
* #param array $sorts
* Sort entiries, there're two different type of sorts
* 1. prefixed with field_, ex. 'field_tag' => array('target_id', 'ASC')
* 2. no prefix or other prefix, 'title' => 'ASC'
* Default $sorts are empty
*
* #param bool $all
* If all matching nid are returned, or just the first one, default FALSE
*
* #return int
* The nid for the supplied id or 0 if not found.
* Or array of nids if $all = TRUE
*
* #author Fang Jin <windmaomao#gmail.com>
*/
function qplot_api_find_nodes2($conds, $sorts = NULL, $all = FALSE) {
$conds = array_merge(array('entity_type' => 'node'), $conds);
if (empty($sorts)) {
$sorts = array();
}
$query = new EntityFieldQuery();
// apply condition to query
foreach ($conds as $key => $value) {
$splits = explode('_', $key);
$type = $splits[0];
if (count($splits) == 1) {
$type = 'property';
}
switch ($type) {
case 'entity':
$query->entityCondition($key, $value);
break;
case 'field':
if (is_array($value)) {
$property = isset($value[1]) ? $value[0] : 'value';
$assigned = isset($value[1]) ? $value[1] : $value[0];
$operator = isset($value[2]) ? $value[2] : '=';
$query->fieldCondition($key, $property, $assigned, $operator);
} else {
$query->fieldCondition($key, 'value', $value);
}
break;
// rest of them are all property
default:
$query->propertyCondition($key, $value);
break;
}
}
// apply sort to query
foreach ($sorts as $key => $value) {
$splits = explode('_', $key);
$type = $splits[0];
if (count($splits) == 1) {
$type = 'property';
}
switch ($type) {
case 'field':
$query->fieldOrderBy($key, $value[0], $value[1]);
break;
default:
$query->propertyOrderBy($key, $value);
break;
}
}
$result = $query->execute();
$ctype = $conds['entity_type'];
if (!empty($result[$ctype])) {
$keys = array_keys($result[$ctype]);
if ($all) {
return $keys;
}
else {
return $keys[0];
}
} else {
if ($all) {
return array();
} else {
return 0;
}
}
}