iForels should support multiple fields for sorting. Hovewer, when I try to sort my iForels Database by 3 columns it always delete previouse features from sort config.
Here is my request body: {"config":{"filters":[],"sorting":{"63e162f4d9afbf6e733534e8":1}}}
I try to include multiple rules inside sorting config and expect it would not be removed automaticaly but return sorted datasets.
Yes, you can easily sort your data by multiple fields, and there are no limits. iForels Database is a kind of fork from Lucene technology. It saves docs in operational memory. That's why it works much faster than MongoDB or SQL databases, even with sorting by multiple fields.
Actually, your config for API works for me. Add as many fields as you'd like. Just make sure you add objects where the prop name is feature_id, and the value is 1 or -1.
If you have this issue on the front end, make sure you turn ON the multisport option for your view. Please, check the screenshots.
Open View Config:
Select Multisort:
I hope you’ll find it helpful!
Related
I'm currently trying to figure out if Solr is the right tool for me. I have the following setup:
There is the primary document type "blog". Then there are two additional document types "user" and "category". Both of these are parents of the "blog" document type.
Now when searching the "blog" documents, I not only want to search in those fields (e.g. title and content), but also in the parent fields (user>name and category>name.
Of course, I could just flatten that down to a single document for Solr, which would ease the search a lot. The downside to this is though, that when e.g. a user updates their name, I have to run through all blog posts of them and update the documents for that in Solr, instead of just updating a single document.
This becomes even worse when the user has another parent, on which I need to search as well.
Do you have any recommendations about how to handle this use case? Maybe my Google foo is just not good enough, but what I found (block joins, etc.) don't seem to do the trick.
The absolutely most performant and easiest solution would be to flatten everything to a single document. It turns out that these relations aren't updated as often as people think, and that searches are performed more often than the documents update. And even if one of the values that are identical across a large set of documents change, reindexing from the most recent documents (for a blog) and then going backwards will appear rather performant for most users. The assumes that you have to actually search the values and don't just need the values - which you could look up from secondary storage when displaying an item (and just store the never changing id in the document).
Another option is to divide this into a multi-search problem. One collection for blog posts, one collection for users and one collection for categories. You then search through each of the collections for the relevant data and merge it in your search model. You can also use [Streaming Expressions] to hand off most of this processing to a Solr cluster for you.
The reason why I always recommend flattening if possible is that most features in Solr (and Lucene) are written for a flat document structure, and allows you to fully leverage the features available. Since Lucene by design is a flat document store, most other features require special care to support blockjoins and parent/child relationships, and you end up experimenting a lot to get the correct queries and feature set you want (if possible). If the documents are flat, it just works.
I have a form that has an autocomplete field like this
$f = $this->add('Form');
$f->addField('autocomplete','item')->setValueList($this->api->db->dsql()->table('item')->field('nroitem,concat(nroitem,\'-\',detalle)')->do_getAssoc());
When I test the form, the response of the autocomplete field is very slow ( 3-4 seconds to bring the information of the table, and the screen gets freezed.), the same when I delete the first field to do a search.
The concat is to allow to search by the item number or the item name.
Is there any limit with this kind of fields in size of data that it takes from the table?
I have tried making some index but with no luck.
I have tried passing the array via session variable (setValueList($array) that I get with a previous query to the database).
Thanks.
Try to use latest ATK version (at least from 4.2.x branch).
Don't use setValueList for big arrays! Define Model and use $field->setModel($model) instead.
It is slow in PHP basically because you parse all records in array and then set this array as data source of autocomplete field. If there are many records, that will consume big amount of processing time and RAM.
It is slow in SQL because you search not in unmodified table fields, but in concatenation. That means, MySQL can't use any optimization and it should concatenate theses values for all records of your table and only then start to evaluate WHERE. That said WHERE nroitem LIKE '%abc%' OR detalle LIKE '%abc%' should work much faster than WHERE concat(nroitem,'-',detalle) LIKE '%abc%'
In theory there's no limit of records for autocomplete field. All depends on your database structure, server performance, indexing and mostly on your own code :) I would say 100`000 records should still work fine.
I have a fairly simple application (like CRM) which has a lot of contacts and associated tags.
A user can search giving lot of criteria (search-items) such as
updated_time in last 10 days
tags in xxx
tags not in xxx
first_name starts with xxx
first_name not in 'Smith'
I understand indexing and how filters (not in) cannot work on more than one property.
For me, since most of the times, reporting is done in a cron - I can iterate through all records and process them. However, I would like to know the best optimized route of doing it.
I am hoping that instead of querying 'ALL', I can get close to a query which can run with the appengine design limits and then manually match rest of the items in the query.
One way of doing it is to start with the first search-item and then get count, add another the next search-item, get count. The point it bails out, I then process those records with rest of the search-items manually.
The question is
Is there a way before hand to know if a query is valid programatically w/o doing a count
How do you determine the best of search-items in a set which do not collide (like not-in does not work on many filters) etc.
The only way I see it is to get all equal filters as one query, take the first in-equality filter or in, execute it and just iterate over the search entities.
Is there a library which can help me ;)
I understand indexing and how filters (not in) cannot work on more than one property.
This is not strictly true. You may create a "composite index" which allows you to perform filters on multiple fields. These consume additional data.
You may also generate your own equivalent of composite index by generating your own "composite field" that you can use to query against.
Is there a way before hand to know if a query is valid programatically w/o doing a count
I'm not sure I understand what kind of validity you're referring to.
How do you determine the best of search-items in a set which do not collide (like not-in does not work on many filters) etc.
A "not in" filter is not trivial. One way is to create two arrays (repeated fields). One with all the tagged entries and one with not all the tags. This would allow you to easily find all the entities with and without the tag. The only issue is that once you create a new tag, you have to sweep across the entities adding a "not in" entry for all the entities.
Description of Goal
Trying to make a CMS (for internal use, but on many sites) with CakePHP.
I'd like to be able to have a generic Content model, then have many different user-generated content types.
This would be simple to do, but my single contents table would eventually become massive if every piece of content on the site was in it. I'd like to split it into tables to help query times/site speed...etc.
My thought (not sure if possible) would be to somehow tell CakePHP that if the type field of the Content is "article", that it should use the content_articles table...etc These tables would be generated afterSave (I suppose) when creating a new content_type.
Would be nice to give them options of which fields the specific content-type would use - even manage this by adding/removing fields...etc, then only generate those fields in the table, and somehow do validation on them based on the content_fields table data.
//my thoughts on tables:
content_types //id, name, description, use_table
content_fields //id, name, content_type_id, required, field_type, max_chars
content_articles //generated by code
content_people //generated by code
Questions:
Is it even possible? Are there better ways to go about this?
Perhapse use a key value table for content rather than a standard table? The utils plugin from CakeDC can do just that with a supported RDBMS.
Or, you could set this model to use a key value data source like MongoDB, which is a great use case for using NoSQL. I'd probably take that approach if you are talking about massive key value stores and a changing schema. There's a plugin for MongoDb on github.
I have a location auto-complete field which has auto complete for all countries, cities, neighborhoods, villages, zip codes. This is part of a location tracking feature I am building for my website. So you can imagine this list will be in the multi-millions of rows. Expecting over 20 million atleast with all the villages and potal codes. To make the auto-complete work well I will use memcached so we dont hit the database always to get this list. It will be used a lot as this is the primary feature on the site. But the question is:
Is only 1 instace of the list stored in memcached irrespective of the users pulling the info or does it need to maintain a separate instance for each? So if say 20 million people are using it at the same time, will that differ from just 1 person using the location auto-complete? I am open to other ideas also on how to implement this location auto complete so it performs well.
Or can i do something like this: When a user logs in in the background I send them the list anyways, so by the time they reach the auto complete textfield their computer will have it ready to load instant?
Take a look at Solr (or Lucene itself), using NGram (or EdgeNGram) tokenizers you can get good autocomplete performance on massive datasets.