I have a collection structured like this (with sample data):
{
name: "Billie Jean",
active: false,
ranges: [
{ sfeMin: -6.75 , sfeMax: -7 , cilMin: 0.75 , cilMax: 0.75 },
{ sfeMin: -7.25 , sfeMax: -7.5 , cilMin: 1.25 , cilMax: 1.25 },
{ sfeMin: -7.5 , sfeMax: -7.75 , cilMin: 1.5 , cilMax: 1.5 },
{ sfeMin: -7.75 , sfeMax: -8 , cilMin: 1.75 , cilMax: 1.75 },
{ sfeMin: -6.5 , sfeMax: -6.75 , cilMin: 0.5 , cilMax: 0.5 },
{ sfeMin: -7 , sfeMax: -7.25 , cilMin: 0 , cilMax: 0 }
]
},
{
name: "Louis Armstrong",
active: true,
ranges: [
{ sfeMin: -8 ,sfeMax: 8 , cilMin: 0 , cilMax: 6 },
{ sfeMin: -8 ,sfeMax: 8 , cilMin: 0 , cilMax: 6 },
{ sfeMin: -8 ,sfeMax: 8 , cilMin: 0 , cilMax: 6 },
{ sfeMin: -8 ,sfeMax: 8 , cilMin: 0 , cilMax: 6 }
]
}
(this is some examples of data inserted into the collection).
What I need is to search if an item inside this collection is active and have a value, let's name it x, that falls between sfeMin and sfeMax and another value, let's name it y, that falls between cilMin and cilMax.
Having x=-8 and y=0, using this filter:
{ $and: [{ active: true }, { 'ranges.sfeMin': { $gte: -8 } }, { 'ranges.sfeMax': { $lte: -8 } }, { 'ranges.cilMin': { $gte: 0 } }, { 'ranges.cilMax': { $lte: 0 } } ]}
should not find any value.
But removing {active: false} or switching the value of all records to true for active, the query returns the collection's item with name "Billie Jean", which should not (at least in my desired behavior).
Let's work out the query step by step for the record in question:
all sfeMin are >= -8
the only sfeMax <= -8 is the fourth entry of the ranges array (-8)
for this row (with sfeMin = -7.75 and sfeMax= -8) cilMin is >=0 (in fact is 1.75) but cilMax is NOT <= 0 (again, 1.75).
Seems like MongoDB treat the ranges array as a single, flattened record instead of running the query in array's row by row. Considering the array as a single, big picture is OK to consider the query valid for Billie, because all values are present and correct but spanned in different array's rows.
This is not what I'm looking for.
The problem the way you've done it is the query will NOT know that ranges filters have to be part of the same object, it will look over all object of the array regardless there are in the same object.
Your query queries for documents where the ranges array has at least one embedded document that contains the field sfeMin gte to -8 and at least one embedded document (but not necessarily the same embedded document) that contains the field sfeMax equal to -8 and so on
I haven't tested it but I think this is the way it should be done:
{
active: true,
ranges: { $elemMatch: { sfeMin: { $gte: -8 }, sfeMax: { $lte: -8 }, cilMin: { $gte: 0 }, cilMax: { $lte: 0 } }}
}
Related
I need to accumulate previous values from an array of objects:
"registries": [
{
"period_range": "06:00 - 07:00",
"timestamp": "2021-11-09T09:45:00.000Z",
"production": {
"D": 4,
"P": 3
},
"accumulated": {
"D": 0,
"P": 0
},
},
]
Inside the "accumulated" object, I need to get all the previous values of "production", and sum with the current production
Example:
If production D is 5 at 5 a.m, and 3 at 6 a.m, the accumulated at 7 a.m need to be 5 + 3 and + the production of the current period.
Object.values() to get the values, Array.prototype.reduce() to add all the values.
let registries = [
{
"period_range": "06:00 - 07:00",
"timestamp": "2021-11-09T09:45:00.000Z",
"production": {
"D": 4,
"P": 3
},
"accumulated": {
"D": 0,
"P": 0
},
},
];
const reducer = (previousValue, currentValue) => previousValue + currentValue;
const addAllValues = (where) => Object.values(where).reduce(reducer);
let reg = registries[0];
let result = addAllValues(reg.production) + addAllValues(reg.accumulated);
console.log(result);
I have 2 variables called "score" and "scoreRange" and assuming "score" value is 0.185 and I want to compare it with 0.3 and 0.9. my condition is as below.
if score is less than 0.3 (score <0.3) -> mark scoreRange as LOW,
0.3 to 0.9 (0.3 <= score < 0.9) --> mark scoreRange as MEDIUM
and larger than 0.9 (score >= 0.9) --> mark scoreRaneg as HIGH
How to compare 0.185 with 0.3 or 0.9 in jolt.
I tried creating an array with values [0.3, 0.9] and add the "score" to that array "array" : [ 0.3, 0.9, 0.185 ]. Then took out the min and max of that array. If Score is in min position or equal to min then scoreRange is low. If Score is in max position or equal to max then scoreRange is high. If both fails then medium. But currently i'm not getting how to compare with normal or array values.
My Input ( Irrelevant )
{
"sampleArray": [
4,
2,
8
]
}
Tried out spec
[
{
"operation": "default",
"spec": {
"array": [0.3, 0.9]
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"score": 0.185,
"array": {
"[2]": "#(2,sumIntData)"
},
"minAB": "=min(#(1,array))",
"maxAB": "=max(#(1,array))"
}
}
]
My current output.
{
"sampleArray" : [ 4, 2, 8 ],
"array" : [ 0.3, 0.9, 0.185 ],
"score" : 0.185,
"minAB" : 0.185,
"maxAB" : 0.8
}
I'm new to jolt. Any guidance in comparing values will be helpful to me.
And how to get a specific value from an array in jolt. elementAt option from jolt was not helpful
I suppose that you can have such an input
{
"score": 0.185,
"sampleArray": [
0.3,
0.9
]
}
and through use of shift transformation having conditionals for comparisons of values generated by using doubleSubtract and abs functions within the modify-overwrite-beta transformation such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"minAB": "=min(#(1,sampleArray))",
"maxAB": "=max(#(1,sampleArray))",
"diff1": "=doubleSubtract(#(1,score),#(1,minAB))",
"diff2": "=doubleSubtract(#(1,score),#(1,maxAB))",
"adiff1": "=abs(#(1,diff1))",
"adiff2": "=abs(#(1,diff2))",
"absdiff1": "=doubleSubtract(#(1,diff1),#(1,adiff1))",
"absdiff2": "=doubleSubtract(#(1,diff2),#(1,adiff2))"
}
},
{
"operation": "shift",
"spec": {
"absdiff1": {
"0.0": {
"#(2,absdiff2)": {
"0.0": {
"#HIGH": "scoreRange"
},
"*": {
"#MEDIUM": "scoreRange"
}
}
},
"*": {
"#LOW": "scoreRange"
}
}
}
}
]
I have a collection or orders with the following format:
"createTime" : ISODate("2021-04-16T08:01:39.000Z"),
"statusDetails" : [
{
"createTime" : ISODate("2021-04-16T08:01:39.000Z"),
"createUser" : "FOP-SYSTEM",
"stateOccurTimeStr" : "2021-04-16 15:01:39",
"status" : 27,
"statusDesc" : "Shipped"
}
]
where createTime is showing that when the order has been created
statusDetails.status = 27 showing that order has been shipped
statusDetails.createTime showing that when the order has been shipped
The result which I need is something like this:
Order Date
0-4 Hours
4-8 Hours
8-12 Hours
12-24 Hours
> 24 Hours
Total Orders
01-Jan-21
15
10
4
1
1
31
This shows that on "1-Jan-2021" after order creation,
15 orders shipped between 0-4 hours,
10 orders shipped between 4-8 hours,
4 orders shipped between 8-12 hours
and so on.
What I have done so far is:
db.orders.aggregate([
{ $unwind : "$statusDetails"},
{$match: {"statusDetails.status": { "$exists": true, "$eq": 24 }}},
{$project : { createTime: 1,statusDetails:1,
dateDiff: {"$divide" : [{ $subtract: ["$statusDetails.createTime","$createTime" ] },3600000]}}},
{$sort:{"createTime":-1}}
])
But this is showing time difference of each individual record, but I need group by
Edit
I have updated my query and now it is showing records using $group but still I need to add an another pipeline to group the current data.
db.orders.aggregate([
{ $unwind : "$statusDetails"},
{$match: {"statusDetails.status": { "$exists": true, "$eq": 27 }}},
{$project : { createTime: 1,statusDetails:1, dateDiff:{"$floor": {"$divide" : [{ $subtract: ["$statusDetails.createTime","$createTime" ] },3600000]}}}},
{
$group:
{ _id: { year : { $year : "$createTime" }, month : { $month : "$createTime" }, day : { $dayOfMonth : "$createTime" }},
shippedTime: { $push: "$dateDiff" },
count: { $sum: 1 }
}
},
{$sort:{"createTime":-1}}
])
$unwind to deconstruct statusDetails array
$match your conditions
$addFields to add slot field on the base of your date calculation hour slot from equation and get total
$group by date as per your format using $dateToString and slot
$sort by slot in ascending order
$group by only date and construct array of slots and get total orders
db.collection.aggregate([
{ $unwind: "$statusDetails" },
{ $match: { "statusDetails.status": 27 } },
{
$addFields: {
slot: {
$multiply: [
{
$floor: {
$divide: [
{
$abs: {
"$divide": [
{ $subtract: ["$statusDetails.createTime", "$createTime"] },
3600000
]
}
},
4
]
}
},
4
]
}
}
},
{
$group: {
_id: {
date: {
$dateToString: {
date: "$createTime",
format: "%d-%m-%Y"
}
},
slot: "$slot"
},
total: {
$sum: 1
}
}
},
{ $sort: { "_id.slot": 1 } },
{
$group: {
_id: "$_id.date",
hours: {
$push: {
slot: "$_id.slot",
total: "$total"
}
},
totalOrders: { $sum: "$total" }
}
}
])
Playground
Result would be:
[
{
"_id": "16-04-2021",
"hours": [
{ "slot": 0, "total": 1 }, // from 0 to 4
{ "slot": 4, "total": 1 }, // from 4 to 8
{ "slot": 8, "total": 2 } // from 8 to 12
],
"totalOrders": 4
}
]
I have a rather unusual structure, that looks something like this (it is actually more nested in reality):
{
"2019-12-05": {
"10": {
"us": {
"6631_10_s902381": {
revenue: 30,
approved: 14,
clicks: 20,
hosts: 10
}
},
"fr": {
"6631_10_s902381": {
revenue: 60,
approved: 4,
clicks: 2,
hosts: 1
},
"2631_11_s902381": {
revenue: 20,
approved: 7,
clicks: 3,
hosts: 0
}
}
},
"14": {
"us": {
"5630_12_9502345": {
revenue: 20,
approved: 4,
clicks: 0,
hosts: 0
}
}
}
},
"2019-12-06": ...
}
Initially I thought about using mysql for this structure, but there would be a lot of duplicates as the deepest object contains the data that is specific to all the parameters above it (there is no overlapping in data).
If I did this on Mysql, it would be like this:
date hour country sub revenue approved clicks hosts
2019-12-05 10 us "6631_10_s902381" 30 14 20 10
2019-12-05 10 fr "6631_10_s902381" 60 4 2 1
2019-12-05 10 fr "2631_11_s902381" 20 7 3 0
2019-12-05 14 us "5630_12_9502345" 20 4 0 0
2019-12-06 ...
I would have to create a unique/primary index on the combination of all fields until "revenue". As you can see there would be a lot of overlapping.
However, I will need to query the database things like, "get me all data of date == 2019-12-05 and country == us (so ignore the hour and the sub parameters)", and I don't know how I can achieve that with mongodb.
I tried:
db.reports.find({"2019-12-05.$.us": { $exists: true}})
But it didn't work.
My questions are:
Do you have a suggestion for a better database or schema for my needs?
Is there a way to get my query to work in mongodb? Can you match every key in an object as I tried to apply on the 2-level-nested key?
You can't use identifiers $ or $[identifier] in non-arrays for mongodb. I would suggest a schema restructure such as this:
{
{
"date": ISODate(date)
"us": [
"10": {...},
"14": {...}
],
},
{
"date": ISODate(date)
"fr": [
"10": {...},
"14": {...}
],
}
}
Note that I changed the date to an ISODate so that Mongo can more easily query for a date or date range when you would like to. Please also note that I don't know the significance of the number 10 and 14 in your data set, so I don't know if this structure is exactly right for you.
However, with this new data structure, you could now run a query like this:
db.collection.find({"us": {$exists: true}, date: {$gte: new Date(date at midnight)}, {$lte: new Date(date at 11:59:59pm) })
I have collection path_test with 2 documents in it
Document 1
{
"_id" : 1,
"tpc" : 5,
"path" : [
{
"nids" : [ 0, 10, 11 ],
"ctc" : 2
},
{
"nids" : [ 0, 10 ],
"ctc" : 2
},
{
"nids" : [ 0, 10, 21 ],
"ctc" : 1
}
]
}
Document 2
{
"_id" : 2,
"tpc" : 5,
"path" : [
{
"nids" : [ 0, 10, 110 ],
"ctc" : 1
},
{
"nids" : [ 0, 10, 11 ],
"ctc" : 2
},
{
"nids" : [ 0, 5 ],
"ctc" : 2
}
]
}
What I'm trying to get as a result are documents with path array in which all elements have nids like [0, 10, *]. Order is important, so [10, 0, *] will be wrong.
It should find Document 1, but not Document 2. Was hoping I can resolve this with a query, before I start using map-reduce or aggregation.
This is what I've tried so far
Query1
db.getCollection('path_test').find( {
"path": { $not: { $elemMatch: { "nids.0": { $nin: [0] }, "nids.1": { $nin: [10] } } } }
});
Query 2
db.getCollection('path_test').find( {
"path.nids": { $not: { $elemMatch: { $nin: [0, 10] } } }
});
but both queries give me results where only 0 is in or where only 10 is in, but I need both and in that exact order.
Is that possible?
not at least one means noone
Query 1
For simplification, lets assign
A = "nids.0": { $ne: 0 }
B = "nids.1": { $ne: 10 }
C = { A, B }
then
{ "path" : { $elemMatch: C } }
will find documents where at least one element in path array satisfies condition C, while
{ "path" : { $not: { $elemMatch: C } } }
will find documents where there are no element in path array that satisfies condition C.
Document 1 and Document 2 don't have elements in their path arrays that satisfy condition C, thus the Query1 output contains both of them. If, f.e, you add to the path array of the Document 1
{ "nids": [ 1, 11, 110], "ctc" : 1 }
then Document 1 will not be in the output of Query 1 becase this added element satisfies C.
Query 2
For simplification, lets assign
C = { $nin: [0, 10] }
then
{ "path.nids" : { $not: { $elemMatch: C } } }
will find documents where there are no element in path.nids array that satisfies condition C.
Document 1 and Document 2 in their path.nids arrays have elements that satisfy condition C, thus the Query 2 output contains neither of them. If, f.e, you add to you collection document
{ "_id" : 6, "tpc" : 5, "path" : [ { "nids" : [ 0, 10 ], "ctc" : 1 } ] }
then it will be in the output of Query 2 because in path.nids array there are no elements that satisfy C.
Solution
In Query 1 replace
{ $elemMatch: { "nids.0": { $nin: [0] }, "nids.1": { $nin: [10] } } }
with
{ $elemMatch: { $or: [ { "nids.0": { $ne: 0 } }, { "nids.1": { $ne: 10 } } ] } }
This new Query will find documents where there are no element in path array that satisfies at least one of conditions A and B. So, it will find Document 1, but not Document 2 (where "nids" : [ 0, 5 ] does not satisfy condition B.
Note that { $ne: 10 } is equivalent to { $nin: [10] }.