Bool query in array field - arrays

I have a very particular issue concerning querying over a boolean field and a string field which are nested to an array field. The index mapping is as follow:
indexes :string_field_1, type: 'string'
indexes :string_field_2, type: 'string'
indexes :boolean_field_1, type: 'boolean'
indexes :array_field_1 do
indexes :boolean_field_2, type: 'boolean'
indexes :string_field_3, type: 'string'
end
indexes :array_field_2 do
indexes :integer_field_1, type: 'integer'
end
indexes :array_field_3 do
indexes :integer_field_2, type: 'integer'
end
The document index also has many other fields which are not nested to the array field, but have to be included among the query fields.
I have tried an approach using filter and bool queries that is as follow:
"query":
{"bool":
{"must":
[
{"query_string":
{"query":"text which is being searched",
"fields":[
"string_field_1",
"string_field_2",
"array_field_1.string_field_3"
],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"}
}
],
"filter":[
{"bool":
{"must":
[
{"bool":
{"should":
[
{"term":{"boolean_field_1":false}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}]}},
{"bool":
{"should":
[
{"term":{"array_field_1.boolean_field_2":true}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}]}},
]
}
}
]
}
}
]
}
}
The problem with this query is that it is returning a document which, in my opinion, doesn't have to be returned.
The document, in this case, is the bellow:
_source": {
"string_field_1": "text 1",
"string_field_2": "text 2",
"boolean_field_1": false,
"array_field_1": [
{
"boolean_field_2": true,
"string_field_3": "some text which is not being searched"
},
{
"boolean_field_2": true,
"string_field_3": "some text which is not being searched"
},
{
"boolean_field_2": false,
"string_field_3": "text which is being searched"
},
{
"boolean_field_2": true,
"string_field_3": "some text which is not being searched"
}
],
"array_field_2": [
{
"integer_field_1": A
}
],
"array_field_3": [
{
"integer_field_2": B
}
]
}
As you can notice, the third item of array_field_1 contains boolean_field_2: false and also the text which is being searched. But, according to my filter: clause, only the documents which array_field_1.boolean_field_2 is true have to be retrieved, unless array_field_2.integer_field_1: or array_field_3.integer_field_1 occurs, which is not true, according to my query part.
It seems elastic is not considering that the array_field_1[2] is the one that the boolean_field_2 is false.
How can I make my query so that this document isn't retrieved?
Thanks is advance,
Guilherme

That was my solution:
"query":{
"bool":{
"should":
[
{
"query_string":
{
"query":"text which is being searched",
"fields":
[
"string_field_1",
"string_field_2"
],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"
}
},
{
bool: {
should:[
{
query:{
nested: {
path: 'array_field_1',
query: {
bool: {
must: [
{ match: { "array_field_1.string_field_3": "text which is being searched"} },
{term: {"array_field_1.boolean_field_2": true}}
]
}
}
}
}
},
{
bool:
{
must: [
{
query:{
nested: {
path: 'movimentos',
query: {
bool: {
must: [
{ match: { "array_field_1.string_field_3": "text which is being searched"} },
{term: {"array_field_1.boolean_field_2": false
]
}
}
}
}
},
{
query: {
bool: {
should: [
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}
]
}
}
}
]
}
}
]
}
}
]
}
}

Another approach consists of putting the array_field_1.string_field_3 query together with the bool query related to the boolean field:
"query":{
"bool":{
"should":
[
{
"query_string":
{
"query":"text which is being searched",
"fields":
[
"string_field_1",
"string_field_2"
],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"
}
},
{
"bool":{
"must":
[
{
"query_string":
{
"query":"text which is being searched",
"fields":["array_field_1.string_field_3"],
"fuzziness":"1","analyze_wildcard":true,"auto_generate_phrase_queries":false,"analyzer":"brazilian","default_operator":"AND"
}
},
{
"bool":{
"should":
[
{"term":{"array_field_1.boolean_field_2":true}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}
]
}
}
]
}
}
],
"filter":
[
{
"bool":{
"should":
[
{"term":{"boolean_field_1":false}},
{"terms":{"array_field_2.integer_field_1":[x,z]}},
{"term":{"array_field_3.integer_field_2":y}}
]
}
}
]
}
}
This query also retrieves the document, unfortunately. I really do not know how to build this query properly.
The query above is organized as:
(X) OR (A AND (B OR C OR D))

Related

Mongo Query to modify the existing field value with new value + array of objects

I want to update many documents based on the condition in MongoDB.
MODEL is the collection which has the document with below information.
"info": [
{
"field1": "String1",
"field2": "String2"
},
{
"field1": "String1",
"field2": "String_2"
}
],
"var": "x"
I need to update all the "String1" value of field1 with "STRING_NEW". I used the below query to update but not working as expected.
db.model.updateMany(
{ "info.field1": { $exists: true } },
[
{ "$set": {
"info": {
"$map": {
"input": "$info.field1",
"in": {
"$cond": [
{ "$eq": ["$$this.field1", "String1"] },
"STRING_NEW",
$$this.field1
]
}
}
}
} }
]
)
Please have a look and suggest if anything is to be modified in the above query.
Solution 1
With the update with aggregation pipeline, you should iterate the object in info array and update the iterated object by merging the current object with new field1 field via $mergeObjects.
db.model.updateMany({
"info.field1": "String1"
},
[
{
"$set": {
"info": {
"$map": {
"input": "$info",
"in": {
"$cond": [
{
"$eq": [
"$$this.field1",
"String1"
]
},
{
$mergeObjects: [
"$$this",
{
"field1": "STRING_NEW"
}
]
},
"$$this"
]
}
}
}
}
}
])
Demo Solution 1 # Mongo Playground
Solution 2
Can also work with $[<identifier>] positional filtered operator and arrayFilters.
db.model.updateMany({
"info.field1": "String1"
},
{
"$set": {
"info.$[info].field1": "STRING_NEW"
}
},
{
arrayFilters: [
{
"info.field1": "String1"
}
]
})
Demo Solution 2 # Mongo Playground

MongoDB - Iterate to Array of objects and update the field as array

I am new to MongoDB. I have a collection that has multiple documents in which a particular document has a nested array structure. I need to iterate through this nested array and change the data type of the value of the iterated field.
Nested Array structure:
[
{
"identifier":{
"type":"xxxx",
"value":"1111"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":"yyyyy",
"value":"222"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":"zzzzz",
"value":"3333"
},
"origin":"https://olkdghf.com",
"score":8.0
}
]
The problem is I need to change the datatype without replacing the existing field value. But I am getting a new empty value instead of the original value.
My query:
db.SourceEntityv8test.find({"hasIdentifier.identifier.type": {$exists:true}}).sort({_id:1}).skip(0).limit(100).forEach(function(x)
{
db.SourceEntityv8test.update({_id: x._id}, {$set:{"hasIdentifier.$[].identifier.type":[]}} );
});
Expected output:
[
{
"identifier":{
"type":[xxxx],
"value":"1111"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[yyyyy],
"value":"222"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[zzzzz],
"value":"3333"
},
"origin":"example.com",
"score":8.0
}
]
Achieved output:
[
{
"identifier":{
"type":[],
"value":"1111"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[],
"value":"222"
},
"origin":"example.com",
"score":8.0
},
{
"identifier":{
"type":[],
"value":"3333"
},
"origin":"example.com",
"score":8.0
}
]
A bit complex. Expect that you need to achieve it by update with aggregation pipeline.
Concept:
Update whole hasIdentifier array by 1.1.
1.1. Merge each document in hasIdentifier array with 1.1.1.
1.1.1. Merge identifier object with the document with type array.
db.SourceEntityv8test.update({_id: x._id},
[
{
$set: {
hasIdentifier: {
$map: {
input: "$hasIdentifier",
in: {
$mergeObjects: [
"$$this",
{
"identifier": {
$mergeObjects: [
"$$this.identifier",
{
type: [
"$$this.identifier.type"
]
}
]
}
}
]
}
}
}
}
}
])
Sample Mongo Playground

Concat a string prefix to the field of an object in an array in MongoDb

I have many mongoDb documents like so
{
store:"Jacks Pizza",
storeNumbers:[
{
"chef":"Giovanni",
"number":"7203305544"
}
]
},
store:"Felicias Kitchen",
storeNumbers:[
{
"chef":"Gina",
"number":"+19161214594"
}
]
I would like to append a "+1" prefix to all such numbers that don't have a +1 country code attached to them.
Here's what I have tried-
db.users.updateMany({
"storeNumbers.number" : {
$exists: true,
$ne:"",
$regex: /^(?!\+)^(?![a-zA-Z])/
}
},
[ {
$set : {
"storeNumbers.$.number" : {
"$concat": [ "+1" , "$storeNumbers.$.number"]
}
}
}
]
);
This gives me an error saying that I cannot perform concat on elements in an array.
How would you do this?
There is no straight way to do this, you can use update with aggregation pipeline starting from MongoDB 4.2,
match query if regex does not match "+1" string at the start of the number
$map to iterate loop of storeNumbers
$cond check condition if number does not match "+1" string at the start of the number and number field is not string and string type then concat "+1" before number using $concat otherwise do nothing
$mergeObjects to merge current object with new update number field
db.users.updateMany({
"storeNumbers.number": {
$exists: true,
$ne: "",
$not: {
$regex: "^\\+1"
}
}
},
[
{
$set: {
storeNumbers: {
$map: {
input: "$storeNumbers",
in: {
$mergeObjects: [
"$$this",
{
$cond: [
{
$and: [
{
$not: {
$regexMatch: {
input: "$$this.number",
regex: "^\\+1"
}
}
},
{
$ne: ["$$this.number", ""]
},
{
$eq: [{ $type: "$$this.number" }, "string"]
}
]
},
{
number: {
$concat: ["+1", "$$this.number"]
}
},
{}
]
}
]
}
}
}
}
}
])
Playground

How to perform Greater than Operator in MongoDb Compass Querying inside inner object collection

Here is my Json in Mongo DB Compass. I am just querying greater than rating products from each collection.
Note: if I am doing with pageCount it is working fine because that is not inside a collection.
{PageCount:{gte:2}} -- works.
Problem with inner arrays collection of collection if anyone matches it displays all.
When we are doing the below query if anyone of the index have greater than 99 it shows all the values.
{"ProductField.ProductDetailFields.ProductDetailInfo.ProductScore.Rating": {$exists:true, $ne: null , $gte: 99}}
----- if I perform above query, I am getting this output.
How to iterate like foreach kind of things and check the condition in MongoDB querying
{
"_id":{
"$oid":"5fc73a7b3fb52d00166554b9"
},
"ProductField":{
"PageCount":2,
"ProductDetailFields":[
{
"PageNumber":1,
"ProductDetailInfo":[
{
"RowIndex":0,
"ProductScore":{
"Name":"Samsung",
"Rating":99
},
},
{
"RowIndex":1,
"ProductScore":{
"Name":"Nokia",
"Rating":96
},
},
{
"RowIndex":2,
"ProductScore":{
"Name":"Apple",
"Rating":80
},
}
]
}
]
}
},
{
"_id":{
"$oid":"5fc73a7b3fb52d0016655450"
},
"ProductField":{
"PageCount":2,
"ProductDetailFields":[
{
"PageNumber":1,
"ProductDetailInfo":[
{
"RowIndex":0,
"ProductScore":{
"Name":"Sony",
"Rating":93
}
},
{
"RowIndex":1,
"ProductScore":{
"Name":"OnePlus",
"Rating":93
}
},
{
"RowIndex":2,
"ProductScore":{
"Name":"BlackBerry",
"Rating":20
}
}
]
}
]
}
}
#Misky How to run this query execute:
While run this query in Mongo Shell - no sql client throws below error. we are using 3.4.9 https://www.nosqlclient.com/demo/
Is this somewhat close to your idea
db.collection.aggregate({
$addFields: {
"ProductField.ProductDetailFields": {
$map: {
"input": "$ProductField.ProductDetailFields",
as: "pdf",
in: {
$filter: {
input: {
$map: {
"input": "$$pdf.ProductDetailInfo",
as: "e",
in: {
$cond: [
{
$gte: [
"$$e.ProductScore.Rating",
99
]
},
{
$mergeObjects: [
"$$e",
{
PageNumber: "$$pdf.PageNumber"
}
]
},
null
]
}
}
},
as: "i",
cond: {
$ne: [
"$$i",
null
]
}
}
}
}
}
}
},
{
$addFields: {
"ProductField.ProductDetailFields": {
"$arrayElemAt": [
"$ProductField.ProductDetailFields",
0
]
}
}
})
LIVE VERSION

Updating data type to an Object in mongoDB

I have changed one of the fields of my collection in mongoDB from an array of strings to an array of object containing 2 strings. New documents get inserted without any problem, but when a get method is called to get , querying all the documents I get this error:
Failed to decode 'Students'. Decoding 'photoAddresses' errored
with: readStartDocument can only be called when CurrentBSONType is
DOCUMENT, not when CurrentBSONType is STRING.
photoAddresses is the field that was changed in Students.
I was wondering is there any way to update all the records so they all have the same data type, without losing any data.
The old version of photoAdresses:
"photoAddresses" : ["something","something else"]
This should be updated to the new version like this:
"photoAddresses" : [{photoAddresses:"something"},{photoAddresses:"something else"}]
The following aggregation queries update the string array to object array, only if the array has string elements. The aggregation operator $map is used to map the string array elements to objects. You can use any of the two queries.
db.test.aggregate( [
{
$match: {
$expr: { $and: [ { $isArray: "$photo" },
{ $gt: [ { $size: "$photo" }, 0 ] }
]
},
"photo.0": { $type: "string" }
}
},
{
$project: {
photo: {
$map: {
input: "$photo",
as: "ph",
in: { addr: "$$ph" }
}
}
}
},
] ).forEach( doc => db.test.updateOne( { _id: doc._id }, { $set: { photo: doc.photo } } ) )
The following query works with MongoDB version 4.2+ only. Note the update operation is an aggregation instead of an update. See updateMany.
db.test.updateMany(
{
$expr: { $and: [ { $isArray: "$photo" },
{ $gt: [ { $size: "$photo" }, 0 ] }
]
},
"photo.0": { $type: "string" }
},
[
{
$set: {
photo: {
$map: {
input: "$photo",
as: "ph",
in: { addr: "$$ph" }
}
}
}
}
]
)
[EDIT ADD]: The following query works with version MongoDB 3.4:
db.test.aggregate( [
{
$addFields: {
matches: {
$cond: {
if: { $and: [
{ $isArray: "$photoAddresses" },
{ $gt: [ { $size: "$photoAddresses" }, 0 ] },
{ $eq: [ { $type: { $arrayElemAt: [ "$photoAddresses", 0 ] } }, "string" ] }
] },
then: true,
else: false
}
}
}
},
{
$match: { matches: true }
},
{
$project: {
photoAddresses: {
$map: {
input: "$photoAddresses",
as: "ph",
in: { photoAddresses: "$$ph" }
}
}
}
},
] ).forEach( doc => db.test.updateOne( { _id: doc._id }, { $set: { photoAddresses: doc.photoAddresses } } ) )

Resources