Mongoid supports soft deletion with
include Mongoid::Paranoia
Lets suppose i have soft deleted a document from one of the collection.
Now I need a query that includes a soft deleted document from that collection.
How can I do that?
Do I need to make a separate method for this to achieve?
Thanks
You can find all deleted documents by query
Model.deleted
and if you want to find deleted documents with specific condition then
Model.deleted.where(:field => value)
Related
We have around 100k products in our website and each product have around 30 attributes which are indexed. Most of the time we only update price of products but we still have to index the whole product. Is it possible in hybris to index only the price attribute(or description attribute) of all 100k products.
It is possible since Solr 4.0. This feature is called partial update, where you can update only the fields changed, in your case, price and description.
The official documentation is here.
Marco is right. You can do a Partial Update.
For Hybris, there is some documentation is in Creating and Configuring Indexed Types. SolrIndexerQuery.type attribute lets you choose partial_update.
You have the following values to choose from:
FULL: recreates the index
UPDATE: updates some documents in the index
PARTIAL_UPDATE: allows you to select the fields for the update
DELETE: deletes documents from the index
I have some documents, I want to update some fields by one field (indexed, but not id), it seems Solr could not support it, I know Solr can update by id. Anyone could give me answers?
There is no similar way to SQLs UPDATE collection SET field = 'foo' WHERE field = 'bar';, no. You'd have to implement this yourself by fetching the documents, changing the value and then reindexing the documents.
Unfortunately this is not supported. See Solr documentation about updating parts of documents.
I thought that this was not possible and that you had to reload all the data if you added new indexes.
Is this supposed to happen?
When Cloud Datastore builds a new index, it includes any existing entities that match the index, so there's no need to update your existing data.
If, however, you have inserted entities with unindexed properties and decide you want to define indexes on those properties, then you need to update each of those entities to mark the property as indexed.
I am trying to figure out whether I need to re-index a [very large] document base in Solr in the following scenarios:
I want to add a few new fields to the schema: none of the old Documents need to be updated to add values for these fields, only new documents that I will be adding after the schema update will have these fields. Do I still need to re-index Solr?
I want to remove couple of not-used fields from the schema (they were added prematurely ...): none of the existing documents has any of these fields. Do I still need to re-index the Solr after the schema update?
I saw many recommendations for updating existing documents when adding/modifying fields, but this is not the case for me - I only want to update the schema, not the existing documents.
Thanks!
Marina
Answer 1: You are correct, you can add new field, you do not need to reindex if you want only new documents going forward to have value for that new field.
Answer 2: Yes, you can remove field without rebuilding index if none of documents have value for that field. You can make sure by looking at that field under:
http://localhost:8080/admin/schema.jsp
If one of documents has value for field you want to remove, you have to rebuild index, else it will give error.
I was trying to figure out how to update the index for the deleted records. I'm indexing from the database. I search for documents in the database, put them in an array and index them by creating a SolrInputDocument.
So, I couldn't figure out how to update the index for the deleted records (because they don't exist in the database now).
I'm using the php-solr-pecl extension.
You need to handle the deletion of the documents separately from Solr.
Solr won't handle it for you.
In case of Incremental, You need to maintain the Documents deleted from the Database and then fire a delete query for the same to clean up the index.
For this you have to maintain a timestamp and delete flag to identify the documents.
In case of the Full, you can just clean up the index and reindex all.
However, in case of failures you may loose all the data.
Solr DIH provides a bit of handling for the same
create a delete trigger on the database table which will insert the deleted record id in another table.(or have boolean field "deleted" and mark the record instead of actually deleting it, considering the trade-offs I would choose the trigger)
Once in a while do a batch delete on index based on the "deleted" table, also removing them from the table itself.
We faced the same issue and came up with batch deletion approach.
We created a program that will delete the document from SOLR based on the uniqueid, if the unique id is present in SOLR but not in database you can delete that document from SOLR.
(Get the uniqueid list from SOLR) minus (uniqueid list from database)
You can just use SQL minus to get the list of uniqueid belonging to the documents that needs to be deleted.
Else you can do everything in JAVA side. Get the list from database, get the list from solr.. Do a comparison between the 2 list and delete based on that..This would be lost faster for huge number of documents. You can use binary search method to do the comparison..
Something like
Collections.binarySearch(DatabaseUniqueidArray, "SOLRuniqueid");