I'm having some issue with API Plateform performance, against a 35.7 GiB database i'm trying to expose.
I'm using the collection pagination feature, because it is a need. For most of my queries it is working ok because they contains filters limitating the set of results.
Only for the default query (without filter) the count query is taking a very long time (10k seconds). So i was hopping to find some way prevent the pagination to happen in this very case, as i'm not able to prevent this kind of API usages.
I looked into the README, the code and documentation about performance but didn't find a sufficent answer, so i tried overriding the PaginationExtension service:
I rewrote the isPartialPaginationEnabled() method just to replace the first line, but i'm not satisfied with the result.
So i'm asking if there is maybe a way i didn't see, or maybe an other approch to the problem. Any help would be welcome.
Thanks by advance.
I don't know if i get you right. But you can disable count query to improve pagination performance, check below my config for this situation.
#api_platform.yaml
collection:
pagination:
client_enabled: true
enabled_parameter_name: 'pagination' # optional
client_items_per_page: true # Disabled by default
items_per_page_parameter_name: itemsPerPage # Default value
partial: true
client_partial: true # Disabled by default
partial_parameter_name: 'partial' # Default value
Related
I am working on a project based on CKAN, and I am required to list in a page all the datasets that have the state "active" and "draft". When you go to the datasets page, you can only see the ones that have the state marked as "active", but not "draft".
If I use the API (call the package_list() method) or REST calls (http://localhost/api/3/action/package_list), CKAN only returns "active" datasets, but not "drafts". I double and triple checked the documentation, and apparently one cannot lists the datasets by their state.
Does anybody have a clue on how to do this? Has anybody done this already?
Thanks!
If nothing else, you could write an extension to do this. The database call itself will be pretty simple:
SELECT id,title,name FROM package WHERE state='active' OR state='draft';
I managed to modify CKAN core to list the datasets that do not have the state "draft" or "deleted", and it works, but I do no want to touch CKAN's core, I want to do this using a plugin, so the normal thing to do is to implement plugins.IActions and override the package_list method with a custom one. I have already written my own extension to try to modify CKAN behavior on method package_list(), but I can't seem to figure it out how to make it work.
Here is my code:
#side_effect_free
def package_list_custom(context, data_dict=None):
datasets = []
dataset_q = (model.Session.query(model.Package)
.join(model.PackageRole))
for dataset in dataset_q:
if dataset.state != 'draft' and dataset.state != 'deleted':
datasets.append(dataset)
return [dataset.id for dataset in datasets]
class Cnaf_WorkflowPlugin(plugins.SingletonPlugin):
plugins.implements(plugins.IActions)
def get_actions(self):
return {
'package_list' : package_list_custom
}
If I modify CKAN core it works very well, but the problem is that I am not to touch it, so I am obliged to do it via an extension.
EDIT: Ok, I managed to make it work, you need to decorate the method with #side_effect_free. I modified my code, and now it works.
The package_search API is capable of this, by searching for state:draft and setting the include_drafts=True flag. Something like this:
https://my-site.com/api/action/package_search?q=state:draft&include_drafts=True
You should be able to access this from a plugin with something like: ckan.plugins.toolkit.get_action('package_search')(context=context, data_dict={'q': 'state:draft', 'include_drafts': True}) (you'll need to assemble the context yourself, containing a 'user' key for the current username and a 'userobj' key for the current user object).
Then make a page from the results.
I would like to style a couple of elements using a filter to decipher if it should be yellow or red.
I understand filters should not carry logic operations in them as such so am guessing a service is the first port of call before i create any filter for it.
I am leveraging data from a backend (still a bit unsure of the Backend model here, but know I can leverage certain objects to obtain the data needed for working on) I mostly need to know if i`m on the right path by using a service to control the logical outcome and then a filter to provide 'filtration' of that outcome.
BTW: sorry, im waiting for my project to checkout from SVN at mo so cannot provide a skeleton attempt.
Will do in a bit though .....
Any advice before hand will be much appreciated
:) Gruffy - thanks for reading
You can directly set the class attribute if you want, so your filter can simply return the CSS class to apply:
<p class="{{'foo'|myFilter}}">Foo</p>
Here's a fiddle showing what I mean.
I am using the DC search Plugin in several Cake projects and generally it works very well. But for one of my pages I have the problem that the searches blows up the URL.
The starting URL is somethin like:
/lessons/abrechnung/10
When the search is used the resulting URL is something like:
/lessons/abrechnung/10/10/10/datumab:01.02.2014/datumbis:25.02.2014
The search itselfs works well - I get the results filtered by the search criteria.
But: As you can see the ID value is duplicated each time you search. This continues and after 3 or 4 searches the URL contains 50 or 100 times the ID.
How can I avoid this?
I guess this would happen on all actions where I have unnamed params in the URL - but I am not sure about this. BTW: The search params are NOT getting duplicated.
EDIT:
I use cakePHP 2.4.0 and Version 2.3 of teh Search Plugin.
Using 'paramType' => 'querystring' didn't help. But I see now that there is something wrong with my Session-Handling. I will check that and give further feedback.
My guess: Your form setup is incorrect.
Do not interfere with the URL of the posted form.
So use
echo $this->Form->create();
without modifying action/url keys.
This way the form will automatically post to itself, and the Search plugin auto-adds the search params in the PRG redirect.
Then there will be no duplication of passed params or alike.
Independent from this it is still better to use the query strings here (and for pagination then as well, of course).
I have a requirement to perform an indexed search across content which must include a couple of tags in the result. The tags must be a random selection. The platform is Drupal 7.12
I have created a view that manages the results of a SOLR search through the search_api. The view returns the required content and seems to work as intended. I have included a couple of Global: custom text fields as placeholders for the tag entries.
I am now looking for a solution to manage the requirement to randomise the tag values. The randomisation is not the issue, the issue is how to include the random values into the view result.
My current approach is to write a views_pre_render hook to intercept the placeholders which appear as fields ([nothing] and [nothing_1]). The test code looks like the following
function MODULE_views_pre_render( &$view )
{
$view_display = $view->display['default'];
$display_option = $view_display->display_options;
$fields = $display_option['fields'];
foreach( $view->result as $result )
{
$fields['nothing']['alter']['text'] = sprintf("test %d", rand(1,9));
}
}
I am currently not seeing any change in the placeholder when the view is rendered.
Any pointers to approach, alternate solutions etc would be gratefully received as this is consuming a lot of scarce time at the moment. Calling print_r( $view ) from within the hook dumps over 46M into a log file for a result set of 2 items.
There are two possible solutions for your task.
First approach is do everything on the template level. Define a template for the view field you want to randomize. In advanced settings of your display go to Theme: Information. Make sure that the proper theme is selected and find the template suggestions for your field. They are listed starting from most general to the most specific and you can choose whatever suits you better.
I guess the most specific template suggestion for your field would be something like this: views-view-field--[YOR VIEW NAME]--[YOUR DISPLAY NAME]--nothing.tpl.php. Create the file with that name in the theme templates directory and in this template you can render what ever you want.
By default this template has only one line:
print $output;
you can change this to:
print sprintf("test %d", rand(1,9));
or to anything else, whatsoever :)
Second approach is to go with Views PHP module. WIth this module you can add a custom PHP field in which you can do whatever you want. Even though the module hasn't been released it seems to work quite well for the most of the tasks and most certainly for such a simple task as randomizing numbers it will work out for sure.
I stumbled upon this while searching for another issue and thought I would contribute.
Instead of adding another module or modifying a template, just add a views "sort criteria" of "Global: Random".
I need to build a filter search that returns data using the criteria set.
My problem is building the search. It needs to be a OR query because anyone of them could be selected.
http://img200.imageshack.us/img200/2527/screenshot20100118at095.png
This is what will be searched.
Accommodation.space
Accommodation.room_type
Accommodation.facilities
Accommodation.speaks
The only way I can think of doing it is an if statement for each input, but that would be 15 if statements, is there a better way?
$Model->postConditions($this->data) [reference] can do this for you.
It will auto-generate Cake compatable conditions array, and you can even specify if you want it to be an "AND/OR" operation
This is a nice component which could likely be tweaked to work for you. It might even work right "out of the box"
http://www.jamesfairhurst.co.uk/posts/view/cakephp_filter_component/
Here is a new tutorial about this. Might help: http://www.youtube.com/watch?v=FAVuLXFVaCw