I have a Person collection that is made up of the following structure
{
"_id" : ObjectId("54ddd6795218e7964fa9086c"),
"_class" : "uk.gov.gsi.hmpo.belt.domain.person.Person",
"imagesMatch" : true,
"matchResult" : {
"_id" : null,
"score" : 1234,
"matchStatus" : "matched",
"confirmedMatchStatus" : "notChecked"
},
"earlierImage" : DBRef("image", ObjectId("54ddd6795218e7964fa9086b")),
"laterImage" : DBRef("image", ObjectId("54ddd67a5218e7964fa908a9")),
"tag" : DBRef("tag", ObjectId("54ddd6795218e7964fa90842"))
}
Notice that the "tag" is a DBRef.
I've got a Spring Data finder that looks like the following:
Page<Person> findByMatchResultNotNullAndTagId(#Param("tagId") String tagId, Pageable page);
When this code is executed the find query looks like the following:
{ matchResult: { $ne: null }, tag: { $ref: "tag", $id: ObjectId('54ddd6795218e7964fa90842') } } sort: {} projection: {} skip: 0 limit: 1
Which is fine, I get a collection of 1 person back (limit=1). However the page details are not correct. I have 31 persons in the collection so I should have 31 pages. What I get is the following:
"page" : {
"size" : 1,
"totalElements" : 0,
"totalPages" : 0,
"number" : 0
}
The count query looks like the following:
{ count: "person", query: { matchResult: { $ne: null }, tag.id: "54ddd6795218e7964fa90842" } }
That tag.id doesn't look correct to me compared with the equivalent find query above.
I've found that if I add a new method to org.springframework.data.mongodb.core.MongoOperations:
public interface MongoOperations {
public long count(Query query, Class<?> entityClass, String collectionName);
}
And then re-jig AbstractMongoQuery.execute(Query query) to use that method instead of the similar method without the entityClass parameter then I get the correct paging results.
Question: Am I doing something wrong or is this a bug in Spring Data Mongo?
Edit
Taking inspiration from Christoph I've added the following test code on Git https://github.com/tedp/Spring-Data-Test
The information contained in the Page returned depends on the query executed. Assuming a total number of 31 elements in you collection, only a few of them, or even just one might match the given criteria by referencing the tag with id: 54ddd6795218e7964fa90842. Therefore you only get the total elements that match the query, and not the total elements within your collection.
This bug was actually fixed DATAMONGO-1120 as pointed out by Christoph. I needed to override the spring data version to use 1.6.2.RELEASE until the next iteration of Spring Boot where presumably Spring Data will be up lifted to at least 1.6.2.RELEASE.
Related
I'm new to MongoDB. I've an object below
{
"_id" : "ABCDEFGH1234",
"level" : 0.6,
"pumps" : [
{
"pumpNo" : 1
},
{
"pumpNo" : 2
}
]
}
And I just want to move level field to pumps array's objects like this
{
"_id" : "ABCDEFGH1234",
"pumps" : [
{
"pumpNo" : 1,
"level" : 0.6
},
{
"pumpNo" : 2,
"level" : 0.6
}
]
}
I've check on MongoDB doc in Aggregation section but didn't found anything. In SQL by JOIN or SUB Query I'm able to do but here it's No-SQL
Could you please help me with this? Thankyou
Try this on for size:
db.foo.aggregate([
// Run the existing pumps array through $map and for each
// item (the "in" clause), create a doc with the existing
// pumpNo and bring in the level field from doc. All "peer"
// fields to 'pumps' are addressable as $field.
// By $projecting to a same-named field (pumps), we effectively
// overwrite the old pumps array with the new.
{$project: {pumps: {$map: {
input: "$pumps",
as: "z",
in: {pumpNo:"$$z.pumpNo", level:"$level"}
}}
}}
]);
Strongly recommend you explore the power of $map, $reduce, $concatArrays, $slice, and other array functions that make MongoDB query language different from the more scalar-based approach in SQL.
Currently we are performing full text search within MSSQL with query:
select * from contract where number like 'word%'
the problem is that contract number may be like
АА-1641471
TST-100069
П-5112-90-00230
001-1000017
1617/292/000001
and ES split all this into tokens.
How to configure ES not to split all this contract numbers into tokens and perform same search like SQL query above ?
the closest solution i've found is to perform query like this:
{
"size": 10,
"query": {
"regexp": {
"contractNumber": {
"value": ".*п-11.*"
}
}
}
}
this solution work same as MSSQL LIKE 'word%' with value like 1111,2568 etc, but fails with п-11
One option could be to use the wildcard query which can perform any type of wildcard combination i.e %val%, %val or val%
{
"query": {
"wildcard" : { "contractNumber" : "*11" }
}
}
NOTE: It's not recommended to start with a wildcard in the search. Could be extremely slow
To make this work with string values to prevent them from being tokenized, you need to update your index and tell the analyser to stay away. One way of doing that is to define the property as type keyword instead of text
PUT /_template/template_1
{
"index_patterns" : ["your_index*"],
"order" : 0,
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"your_document_type" : {
"properties" : {
"contractNumber" : {
"type" : "keyword"
}
}
}
}
NOTE: replace your_index with your index name and your_document_type with the document type.
When the mapping is added, delete the current index and recreate it, then it will use the template for properties and your contractNumber will be indexed as a keyword
I want to employ hashtag searching in combination with the standard text search.
Here is the kind of query I wish to be able to make:
"leather trousers #vintage #london"
So in effect I am wanting to strip off the #hashtaged elements and search for them by name, in a cumulative sense. Firstly I want it to prioritise on an exact match via the search string, then to ones with near match + hashtags, then lastly if no match with search string, via the hash tags.
So items with both Vintage and London would be placed higher than ones with either Vintage or London.
Here is my mapping
{
"title" : {
"type" : "string",
"analyzer" : "standard"
},
"hashtags" : {
"properties" : {
"id" : "integer",
"name" : "string"
}
}
}
So the query I want to make is
"exact or near match string" + "optional cumulative array match (preferably with fuzzyness)"
or in relation to my mapping
"near or exact match on 'title'" + "cumulative array match with fizzyness on hashtag.name"
I've tried a fuzzy match but get back too much results with not enough clarity. I've tried a simple simple_query_string but it returns weird results, and tried a bool match but get back nothing when I add the array.
Any help anyone can offer will be more than gratefully accepted. Let me know if you need more info or whatever? Many thanks in advance for your time to have even read this.
maybe a "dis_max" query can work for you. it enable to make multiples differents queries and concat the results. So her it make a first queries where "hashtags = 'vintage london'" then "hashtags = 'vintage'" then "hashtags = 'london'". you can also add wildcards (*) in the researched data like "hashtags = 'london*'"
{
"fields" : ["hashtags", "title"],
"query" : {
"dis_max" : {
"tie_breaker" : 0,
"queries" : [ {
"wildcard" : {
"hashtags" : "vintage london"
}
}, {
"wildcard" : {
"hashtags" : "vintage"
}
}, {
"wildcard" : {
"hashtags" : "london"
}
}
]
}
},
"sort" : {
"_score" : "desc"
} }
I have an array of objects, and I want to query in a MongoDB collection for documents that have elements that match any objects in my array of objects.
For example:
var objects = ["52d58496e0dca1c710d9bfdd", "52d58da5e0dca1c710d9bfde", "52d91cd69188818e3964917b"];
db.scook.recipes.find({products: { $in: objects }}
However, I want to know if I can sort the results by the number of matches in MongoDB.
For example, at the top will be the "recipe" that has exactly three elements matches: ["52d58496e0dca1c710d9bfdd", "52d58da5e0dca1c710d9bfde", "52d91cd69188818e3964917b"].
The second selected has two recipes: i.e. ["52d58496e0dca1c710d9bfdd", "52d58da5e0dca1c710d9bfde"], and the third one only one: i.e. ["52d58496e0dca1c710d9bfdd"]
It would be great if you could get the number of items it had.
By using the aggregation framework, I think that you should be able to get what you need by the following MongoDB query. However, if you're using Mongoose, you'll have to convert this to a Mongoose query. I'm not certain this will work exactly as is, so you may need to play with it a little to make it right. Also, this answer hinges on whether or not you can use the $or operator inside of the $project operator and that it will return true. If that doesn't work, I think you'll need to use map-reduce to get what you need or do it server side.
db.recipes.aggregate(
// look for matches
{ $match : { products : { $or : objects }}},
// break apart documents to by the products subdocuments
{ $unwind : "$products" },
// search for matches in the sub documents and add productMatch if a match is found
{ $project : {
desiredField1 : 1,
desiredField2 : 1,
products : 1,
// this may not be a valid comparison, but should hopefully
// be true or 1 if there is a match
productMatch : { "$products" : { $or : objects }}
}},
// group the unwound documents back together by _id
{ $group : {
_id : "$_id",
products : { $push : "$products" },
// count the matched objects
numMatches : { $sum : "$productMatch" },
// increment by 1 for each product
numProducts : { $sum : 1 }
}},
// sort by descending order by numMatches
{ $sort : { numMatches : -1 }}
)
This question already has answers here:
How to Update Multiple Array Elements in mongodb
(16 answers)
Closed 5 years ago.
I recently started using MongoDB and I have a question regarding updating arrays in a document.
I got structure like this:
{
"_id" : ObjectId(),
"post" : "",
"comments" : [
{
"user" : "test",
"avatar" : "/static/avatars/asd.jpg",
"text" : "....."
}
{
"user" : "test",
"avatar" : "/static/avatars/asd.jpg",
"text" : "....."
}
{
"user" : "test",
"avatar" : "/static/avatars/asd.jpg",
"text" : "....."
}
...
]
}
I'm trying to execute the following query:
update({"comments.user":"test"},{$set:{"comments.$.avatar": "new_avatar.jpg"}},false,true)
The problem is that it update all documents, but it update only the first array element in every document. Is there any way to update all array elements or I should try to do it manually?
Thanks.
You cannot modify multiple array elements in a single update operation. Thus, you'll have to repeat the update in order to migrate documents which need multiple array elements to be modified. You can do this by iterating through each document in the collection, repeatedly applying an update with $elemMatch until the document has all of its relevant comments replaced, e.g.:
db.collection.find().forEach( function(doc) {
do {
db.collection.update({_id: doc._id,
comments:{$elemMatch:{user:"test",
avatar:{$ne:"new_avatar.jpg"}}}},
{$set:{"comments.$.avatar":"new_avatar.jpg"}});
} while (db.getPrevError().n != 0);
})
Note that if efficiency of this operation is a requirement for your application, you should normalize your schema such that the location of the user's avatar is stored in a single document, rather than in every comment.
One solution could be creating a function to be used with a forEach and evaling it (so it runs quickly). Assuming your collection is "article", you could run the following:
var runUpdate = function(){
db.article.find({"comments.user":"test").forEach( function(article) {
for(var i in article.comments){
article.comments[i].avatar = 'new_avatar.jpg';
}
db.article.save(article);
});
};
db.eval(runUpdate);
If you know the indexes you want to update you can do this with no problems like this:
var update = { $set: {} };
for (var i = 0; i < indexesToUpdate.length; ++i) {
update.$set[`comments.${indexesToUpdate[i]}. avatar`] = "new_avatar.jpg";
}
Comments.update({ "comments.user":"test" }, update, function(error) {
// ...
});
be aware that must of the IDE's will not accept the syntax but you can ignore it.
It seems like you can do this:
db.yourCollection.update({"comments.user":"test"},{$set:{"comments.0.avatar": "new_avatar.jpg", "comments.1.avatar": "new_avatar.jpg", etc...})
So if you have a small known number of array elements, this might be a little easier to do. If you want something like "comments.*.avatar" - not sure how to do that. It is probably not that good that you have so much data duplication tho..