I'm using Spring 5 webflux with ReactiveMongoRepositories.
Suppose I have 2 collections "Race" and "RaceParticipation":
Race : {
"_id" : "ID",
"name" : "Name"
}
RaceParticipation : {
"_id" : "ID",
"userId" : "ID",
"raceId" : "ID"
}
I need to find a page of Races in which a given user has participated.
The solution I went with is call raceParticipationRepository.findByUserId("userId", PageRequest.of(0,10))
This returns a Flux<UserParticipation>.
The I should collect the ids from the response and call raceRepository.findAllById(ids).
Here is a snippet:
Flux<Race> findUserRaces(String userId, int page){
return raceParticipationRepository.findByUserId(userId, status,
PageRequest.of(page, 6))
.collectList()
.map(list -> list.stream()
.map(ParticipationRepository::getRaceId)
.collect(Collectors.toList()))
.flatMap(list -> raceRepository.findAllById(list));
}
This code doesn't compile. "Required Flux<Race> but flatMap was inferred to Mono..."
Any idea on what I'm missing or how should I do?
Your problem is the collectList() part, since it makes a Mono from your Flux, which you obviously don't want - you're making a stream from the list immediately after collection. This here should do the same, plus mapping the Mono to a Flux with flatMapMany:
raceParticipationRepository.findByUserId(userId, status,
PageRequest.of(page, 6))
.map(ParticipationRepository::getRaceId)
.collectList(Collectors.toList()) // returns Mono<List>
.flatMapMany(list -> raceRepository.findAllById(list)); // Mono -> Flux again
Related
I'm new to MongoDB. I've an object below
{
"_id" : "ABCDEFGH1234",
"level" : 0.6,
"pumps" : [
{
"pumpNo" : 1
},
{
"pumpNo" : 2
}
]
}
And I just want to move level field to pumps array's objects like this
{
"_id" : "ABCDEFGH1234",
"pumps" : [
{
"pumpNo" : 1,
"level" : 0.6
},
{
"pumpNo" : 2,
"level" : 0.6
}
]
}
I've check on MongoDB doc in Aggregation section but didn't found anything. In SQL by JOIN or SUB Query I'm able to do but here it's No-SQL
Could you please help me with this? Thankyou
Try this on for size:
db.foo.aggregate([
// Run the existing pumps array through $map and for each
// item (the "in" clause), create a doc with the existing
// pumpNo and bring in the level field from doc. All "peer"
// fields to 'pumps' are addressable as $field.
// By $projecting to a same-named field (pumps), we effectively
// overwrite the old pumps array with the new.
{$project: {pumps: {$map: {
input: "$pumps",
as: "z",
in: {pumpNo:"$$z.pumpNo", level:"$level"}
}}
}}
]);
Strongly recommend you explore the power of $map, $reduce, $concatArrays, $slice, and other array functions that make MongoDB query language different from the more scalar-based approach in SQL.
Why use of $[] with $ is causing write conflict ?
db.projectionTesting.updateOne({"metaData.title": "BCDe"} , {$set : {
"metaData.0.title" : "efde" ,
"metaData.$[].hasUpdateddd": 76
}} )
WriteError({
"index" : 0,
"code" : 40,
"errmsg" : "Updating the path 'metaData.$[].hasUpdateddd' would create a conflict at 'metaData'",
"op" : {
"q" : {
"metaData.title" : "BCDe"
},
"u" : {
"$set" : {
"metaData.0.title" : "efde",
"metaData.$[].hasUpdateddd" : 76
}
},
"multi" : false,
"upsert" : false
}
}) :
Why use of $[] with $ is causing write conflict ?
if these are working fine then why not the above one. I want to know what exactly the problem is with the use of $[] with $ in while performing updateOperation
db.projectionTesting.updateOne({"metaData.title": "BCDe"} ,
{$set : {"metaData.0.title" : "efde" ,
"metaData.$.hasUpdateddd": 76}} )
db.projectionTesting.updateOne({"metaData.title": "BCDe"} ,
{$set : {"metaData.$.title" : "efde" ,
"metaData.$.hasUpdateddd": 76}} )
In order to ensure consistency among replica set members, MongoDB replication requires that operations added to the oplog be idempotent. To achieve this update operations such as $inc, $dec, and most array operations are stored in the oplog as $set with the new value of the field.
When combining 2 updates using metaData.$, you are making updates to one specific element.
When combining metaData.0 and metadata.$[] you are making unrelated updates a single element and to the entire array.
The query executor has not been given logic to be able to ensure an idempotent operation in that situation, so it balks with a write conflict.
I have a Person collection that is made up of the following structure
{
"_id" : ObjectId("54ddd6795218e7964fa9086c"),
"_class" : "uk.gov.gsi.hmpo.belt.domain.person.Person",
"imagesMatch" : true,
"matchResult" : {
"_id" : null,
"score" : 1234,
"matchStatus" : "matched",
"confirmedMatchStatus" : "notChecked"
},
"earlierImage" : DBRef("image", ObjectId("54ddd6795218e7964fa9086b")),
"laterImage" : DBRef("image", ObjectId("54ddd67a5218e7964fa908a9")),
"tag" : DBRef("tag", ObjectId("54ddd6795218e7964fa90842"))
}
Notice that the "tag" is a DBRef.
I've got a Spring Data finder that looks like the following:
Page<Person> findByMatchResultNotNullAndTagId(#Param("tagId") String tagId, Pageable page);
When this code is executed the find query looks like the following:
{ matchResult: { $ne: null }, tag: { $ref: "tag", $id: ObjectId('54ddd6795218e7964fa90842') } } sort: {} projection: {} skip: 0 limit: 1
Which is fine, I get a collection of 1 person back (limit=1). However the page details are not correct. I have 31 persons in the collection so I should have 31 pages. What I get is the following:
"page" : {
"size" : 1,
"totalElements" : 0,
"totalPages" : 0,
"number" : 0
}
The count query looks like the following:
{ count: "person", query: { matchResult: { $ne: null }, tag.id: "54ddd6795218e7964fa90842" } }
That tag.id doesn't look correct to me compared with the equivalent find query above.
I've found that if I add a new method to org.springframework.data.mongodb.core.MongoOperations:
public interface MongoOperations {
public long count(Query query, Class<?> entityClass, String collectionName);
}
And then re-jig AbstractMongoQuery.execute(Query query) to use that method instead of the similar method without the entityClass parameter then I get the correct paging results.
Question: Am I doing something wrong or is this a bug in Spring Data Mongo?
Edit
Taking inspiration from Christoph I've added the following test code on Git https://github.com/tedp/Spring-Data-Test
The information contained in the Page returned depends on the query executed. Assuming a total number of 31 elements in you collection, only a few of them, or even just one might match the given criteria by referencing the tag with id: 54ddd6795218e7964fa90842. Therefore you only get the total elements that match the query, and not the total elements within your collection.
This bug was actually fixed DATAMONGO-1120 as pointed out by Christoph. I needed to override the spring data version to use 1.6.2.RELEASE until the next iteration of Spring Boot where presumably Spring Data will be up lifted to at least 1.6.2.RELEASE.
This question already has answers here:
How to Update Multiple Array Elements in mongodb
(16 answers)
Closed 5 years ago.
I recently started using MongoDB and I have a question regarding updating arrays in a document.
I got structure like this:
{
"_id" : ObjectId(),
"post" : "",
"comments" : [
{
"user" : "test",
"avatar" : "/static/avatars/asd.jpg",
"text" : "....."
}
{
"user" : "test",
"avatar" : "/static/avatars/asd.jpg",
"text" : "....."
}
{
"user" : "test",
"avatar" : "/static/avatars/asd.jpg",
"text" : "....."
}
...
]
}
I'm trying to execute the following query:
update({"comments.user":"test"},{$set:{"comments.$.avatar": "new_avatar.jpg"}},false,true)
The problem is that it update all documents, but it update only the first array element in every document. Is there any way to update all array elements or I should try to do it manually?
Thanks.
You cannot modify multiple array elements in a single update operation. Thus, you'll have to repeat the update in order to migrate documents which need multiple array elements to be modified. You can do this by iterating through each document in the collection, repeatedly applying an update with $elemMatch until the document has all of its relevant comments replaced, e.g.:
db.collection.find().forEach( function(doc) {
do {
db.collection.update({_id: doc._id,
comments:{$elemMatch:{user:"test",
avatar:{$ne:"new_avatar.jpg"}}}},
{$set:{"comments.$.avatar":"new_avatar.jpg"}});
} while (db.getPrevError().n != 0);
})
Note that if efficiency of this operation is a requirement for your application, you should normalize your schema such that the location of the user's avatar is stored in a single document, rather than in every comment.
One solution could be creating a function to be used with a forEach and evaling it (so it runs quickly). Assuming your collection is "article", you could run the following:
var runUpdate = function(){
db.article.find({"comments.user":"test").forEach( function(article) {
for(var i in article.comments){
article.comments[i].avatar = 'new_avatar.jpg';
}
db.article.save(article);
});
};
db.eval(runUpdate);
If you know the indexes you want to update you can do this with no problems like this:
var update = { $set: {} };
for (var i = 0; i < indexesToUpdate.length; ++i) {
update.$set[`comments.${indexesToUpdate[i]}. avatar`] = "new_avatar.jpg";
}
Comments.update({ "comments.user":"test" }, update, function(error) {
// ...
});
be aware that must of the IDE's will not accept the syntax but you can ignore it.
It seems like you can do this:
db.yourCollection.update({"comments.user":"test"},{$set:{"comments.0.avatar": "new_avatar.jpg", "comments.1.avatar": "new_avatar.jpg", etc...})
So if you have a small known number of array elements, this might be a little easier to do. If you want something like "comments.*.avatar" - not sure how to do that. It is probably not that good that you have so much data duplication tho..
Given the below competing schemas with up to 100,000 friends I’m interested in finding the most efficient for my needs.
Doc1 (Index on user_id)
{
"_id" : "…",
"user_id" : "1",
friends : {
"2" : {
"id" : "2",
"mutuals" : 3
}
"3" : {
"id" : "3",
"mutuals": "1"
}
"4" : {
"id" : "4",
"mutuals": "5"
}
}
}
Doc2 (Compound multi key index on user_id & friends.id)
{
"_id" : "…",
"user_id" : "1",
friends : [
{
"id" : "2",
"mutuals" : 3
},
{
"id" : "3",
"mutuals": "1"
},
{
"id" : "4",
"mutuals": "5"
}
]}
I can’t seem to find any information on the efficiency of the sub field retrieval. I know that mongo implements data internally as BSON, so I’m wondering whether that means a projection lookup is a binary O(log n)?
Specifically, given a user_id to find whether a friend with friend_id exists, how would the two different queries on each schema compare? (Assuming the above indexes) Note that it doesn’t really matter what’s returned, only that not null is returned if the friend exists.
Doc1col.find({user_id : "…"}, {"friends.friend_id"})
Doc2col.find({user_id : "…", "friends.id" : "friend_id"}, {"_id":1})
Also of interest is how the $set modifier works. For schema 1,given the query Doc1col.update({user_id : "…"}, {"$set" : {"friends.friend_id.mutuals" : 5}), how does the lookup on the friends.friend_id work? Is this a O(log n) operation (where n is the number of friends)?
For schema 2, how would the query Doc2col.update({user_id : "…", "friends.id" : "friend_id"}, {"$set": {"friends.$.mutuals" : 5}) compare to that of the above?
doc1 is preferable if one's primary requirements is to present data to the ui in a nice manageable package. its simple to filter only the desired data using a projection {}, {friends.2 : 1}
doc2 is your strongest match since your use case does not care about the result Note that it doesn’t really matter what’s returned and indexing will speed up the fetch.
on top of that doc2 permits the much cleaner syntax
db.doc2.findOne({user_id: 1, friends.id : 2} )
versus
db.doc1.findOne({ $and : [{ user_id: 1 }, { "friends.2" : {$exists: true} }] })
on a final note, however, one can create a sparse index on doc1 (and use $exists) but your possibility of 100,000 friends -- each friend needed a sparse index -- makes that absurd. opposed to a reasonable number of entries say demographics gender [male,female], agegroups [0-10,11-16,25-30,..] or more impt things [gin, whisky, vodka, ... ]