MongoDB: how to group by date with mongo date aggregration using java driver - spring-data-mongodb

I have been struggling writing to convert a mongo shell command to java query on top of the java mongo driver in order to call it in my webapp project: the command is as follow:
db.post.aggregate(
{
$match: { dateCreated:
{
"$gt": new ISODate("2013-08-09T05:51:15.000Z"),
"$lt": new ISODate("2013-08-09T05:51:20.000Z")
}
}
},
{
$group: {
_id: {
hour: {$hour: "$dateCreated"},
minute: {$minute: "$dateCreated"},
second: {$second: "$dateCreated"}
},
cnt: {$sum : 1}
}
}
)
The query above outputs result in the format below in the mongo shell:
{
"result" : [
{
"_id" : {
"hour" : 5,
"minute" : 51,
"second" : 19
},
"cnt" : 26
},
{
"_id" : {
"hour" : 5,
"minute" : 51,
"second" : 18
},
"cnt" : 29
},
{
"_id" : {
"hour" : 5,
"minute" : 51,
"second" : 17
},
"cnt" : 27
},
{
"_id" : {
"hour" : 5,
"minute" : 51,
"second" : 16
},
"cnt" : 25
},
{
"_id" : {
"hour" : 5,
"minute" : 51,
"second" : 15
},
"cnt" : 16
}
],
"ok" : 1
}
I failed in writing the same query in java using java mongo driver . below is my query:
DBObject matchStart = new BasicDBObject("$match",new BasicDBObject("dateCreated",new BasicDBObject("$gt",startTime).append("$lt",endTime)));
DBObject field = new BasicDBObject("dateCreated",1);
field.put("h", new BasicDBObject("$hour","$dateCreated"));
field.put("m", new BasicDBObject("$minute","$dateCreated"));
field.put("s", new BasicDBObject("$second","$dateCreated"));
DBObject project = new BasicDBObject("$project",field);
DBObject groupField = new BasicDBObject("_id","$s");
groupField.put("count", new BasicDBObject("$sum",1));
DBObject group = new BasicDBObject("$group",groupField);
AggregationOutput output = mongoOperations.getCollection("post").aggregate(matchStart,project,group);
return output;
it returns a resultset below :
{"result" : [
{ "_id" : 19 , "count" : 26} ,
{ "_id" : 18 , "count" : 29} ,
{ "_id" : 17 , "count" : 27} ,
{ "_id" : 16 , "count" : 25} ,
{ "_id" : 15 , "count" : 16}
] ,
"ok" : 1.0}
I am having challenges making the query include the minute part and the hour part. How can tweak my query to output the same resultset as in the mongo shell one.
Thanks for looking at that

Java code for given query is as follows :
DBCollection coll = ...
Date startDate = ...
Date endDate = ...
DBObject dateQuery = new BasicDBObject();
dateQuery.put("$gt", startDate);
dateQuery.put("$lt", endDate);
DBObject match = new BasicDBObject();
match.put("dateCreated", dateQuery);
DBObject id = new BasicDBObject();
id.put("hour", new BasicDBObject("$hour", "$dateCreated"));
id.put("minute", new BasicDBObject("$minute", "$dateCreated"));
id.put("second", new BasicDBObject("$second", "$dateCreated"));
DBObject group = new BasicDBObject();
group.put("_id", id);
group.put("cnt", new BasicDBObject("$sum", 1));
AggregationOutput output = coll.aggregate(new BasicDBObject("$match", match), new BasicDBObject("$group", group));
if (output != null) {
for (DBObject result : output.results()) {
Integer count = (Integer) result.get("cnt");
DBObject idObj = (DBObject) result.get("_id");
Integer hour = (Integer) idObj.get("hour");
Integer minute = (Integer) idObj.get("minute");
Integer second = (Integer) idObj.get("second");
}
}

Related

How to get documents whose array does not include "apple" element but if it includes other elements except apple then it should return the doc

let me give an example.
There is a collection vehicle_list
`[
{
"driver" : 1,
"driver_vehicles" : [
{
"name" : "Car1",
"state" : 1
"vehicledates" : []
},
{
"name" : "Car2",
"vehicledates" : [
{
"trip" : ObjectId("6374c5521e0056579430e15e"),
"trip_date" : "Nov 28, 2022"
},
{
"trip" : ObjectId("6374c3c65c106e84520e2bda"),
"trip_date" : "Nov 30, 2022"
},
{
"trip" : ObjectId("63789e73aab4b1b08dac4d23"),
"trip_date" : "Dec 1, 2022"
}
],
"state" : 1
},
{
"name" : "Car3"
"vehicledates" : [
{
"trip_id" : ObjectId("63776d5b9061c736099501ac"),
"trip_date" : "Dec 2, 2022"
}
],
"state" : 1
}
],
}
]
`
What i want is to have the Driver who does not have a vehicle that does not have a trip on Dec 1, 2022 and also state:1
So i made a query
db.vehicle_list.find({"driver_vehicles.vehicledates.trip_date": {$nin: ["Dec 1, 2022"]},"state":1})
This doesn't return document _id 1
Now i understand that car2 has Dec 1, 2022 so it should not return.
But Car1 and Car3 doesn't have Dec 1, 2022 and it also matches the condition state:1 so it should return the document.
Hope you undestand what i mean. Let me know if you have any doubts.
I tried this query
db.vehicle_list.find({"driver_vehicles.vehicledates.trip_date": {$nin: ["Dec 1, 2022"]},"state":1})
This doesn't return document _id 1
But it doesn't return Driver 1
Instead, I think the $elemMatch operator will work as what you need.
db.collection.find({
"driver_vehicles": {
$elemMatch: {
"vehicledates.trip_date": {
$nin: [
"Dec 1, 2022"
]
},
"state": 1
}
}
})
Demo # Mongo Playground

Need help in querying mongodb

I have a a few documents that have the following structure. See attached image.
document structure
Each document includes an array of 'FileMeta' objects and each FileMeta object includes an array of 'StatusHistory' objects. I'm trying to get only the FileMetas that contain StatusCode equal to 4 and that the TimeStamp is greater than a certain datetime.
Tried the following query but it only returns the first FileMeta element of each document.
db.getCollection('Collection').find({'ExternalParams.RequestingApplication':'aaa.bbb'},
{ "FileMeta": { $elemMatch: { "StatusHistory":{ $elemMatch:{ "StatusCode": 4, "TimeStamp": { $gt: ISODate("2020-06-28T11:02:26.542Z")} } } } }} )
What am I doing wrong?
here is the document structure:
{
"_id" : ObjectId("5ef84e2ec08abf38b0043ab4"),
"FileMeta" : [
{
"StatusHistory" : [
{
"StatusCode" : 0,
"StatusDesc" : "New File",
"TimeStamp" : ISODate("2020-06-28T11:00:46.286Z")
},
{
"StatusCode" : 2,
"StatusDesc" : "stby",
"TimeStamp" : ISODate("2020-06-28T11:02:20.400Z")
},
{
"StatusCode" : 4,
"StatusDesc" : "success",
"TimeStamp" : ISODate("2020-06-28T11:02:26.937Z")
}
]
},
{
"StatusHistory" : [
{
"StatusCode" : 0,
"StatusDesc" : "New File",
"TimeStamp" : ISODate("2020-06-28T11:00:46.286Z")
},
{
"StatusCode" : 2,
"StatusDesc" : "stby",
"TimeStamp" : ISODate("2020-06-28T11:02:20.617Z")
},
{
"StatusCode" : 4,
"StatusDesc" : "success",
"TimeStamp" : ISODate("2020-06-28T11:02:26.542Z")
}
]
}
],
}
I want to return only the FileMeta objects that include a StatusHistory that match the following conditions: StatusCode = 4 and TimeStamp > SomeDateTime
Sorry for the delay, mate, I've been quite busy lately. Hope you already solved your problem. Anyway, I think that I found the solution.
As you can see on this link, the example shows that by default the $elemMatch operator returns the whole array in case of match on any element.
For instance, consider the following collection:
{ _id: 1, results: [ { product: "abc", score: 10 }, { product: "xyz", score: 5 } ] }
{ _id: 2, results: [ { product: "abc", score: 8 }, { product: "xyz", score: 7 } ] }
{ _id: 3, results: [ { product: "abc", score: 7 }, { product: "xyz", score: 8 } ] }
If you do the following query, for example:
db.survey.find(
{ results: { $elemMatch: { product: "xyz", score: { $gte: 8 } } } }
)
The output will be:
{ "_id" : 3, "results" : [ { "product" : "abc", "score" : 7 }, { "product" : "xyz", "score" : 8 } ] }
Not:
{ "_id" : 3, "results" : [{ "product" : "xyz", "score" : 8 }]}
That said, if you want to return only the document in the array that matches the specified query, you must use the db.collection.aggregate() function with the $unwind and $match operator.
The query below shall give you what you want.
Query:
db.collection.aggregate([
{"$unwind" : "$FileMeta"},
{"$unwind" : "$FileMeta.StatusHistory"},
{
"$match" : {
"FileMeta.StatusHistory.StatusCode" : 4,
"FileMeta.StatusHistory.TimeStamp" : {"$gte" : ISODate("2020-06-28T11:02:26.937Z")}
}
}
]).pretty()
Result:
{
"_id" : ObjectId("5ef84e2ec08abf38b0043ab4"),
"FileMeta" : {
"StatusHistory" : {
"StatusCode" : 4,
"StatusDesc" : "success",
"TimeStamp" : ISODate("2020-06-28T11:02:26.937Z")
}
}
}
One last tip. Consider changing your modeling to something that looks like the unwinded document, and remember that one document should be equivalent to one row in a normal relational database. So avoid storing information that should be on "several rows" on a single document.
Useful links:
The $elemMatch operator.
The $unwind operator.

Remove duplicates from MongoDB 4.2 data base

I am trying to remove duplicates from MongoDB but all solutions find fail.
My JSON structure:
{
"_id" : ObjectId("5d94ad15667591cf569e6aa4"),
"a" : "aaa",
"b" : "bbb",
"c" : "ccc",
"d" : "ddd",
"key" : "057cea2fc37aabd4a59462d3fd28c93b"
}
Key value is md5(a+b+c+d).
I already have a database with over 1 billion records and I want to remove all the duplicates according to key and after use unique index so if the key is already in data base the record wont insert again.
I already tried
db.data.ensureIndex( { key:1 }, { unique:true, dropDups:true } )
But for what I understand dropDups were removed in MongoDB > 3.0.
I tried also several of java script codes like:
var duplicates = [];
db.data.aggregate([
{ $match: {
key: { "$ne": '' } // discard selection criteria
}},
{ $group: {
_id: { key: "$key"}, // can be grouped on multiple properties
dups: { "$addToSet": "$_id" },
count: { "$sum": 1 }
}},
{ $match: {
count: { "$gt": 1 } // Duplicates considered as count greater than one
}}
],
{allowDiskUse: true} // For faster processing if set is larger
).forEach(function(doc) {
doc.dups.shift(); // First element skipped for deleting
doc.dups.forEach( function(dupId){
duplicates.push(dupId); // Getting all duplicate ids
}
)
})
and it fails with:
QUERY [Js] uncaught exception: Error: command failed: {
“ok“: 0,
“errmsg“ : “assertion src/mongo/db/pipeline/value.cpp:1365“.
“code“ : 8,
“codeName" : “UnknownError“
} : aggregate failed
I haven't change MongoDB settings, working with the default settings.
This is my input collection dups, with some duplicate data (k with values 11 and 22):
{ "_id" : 1, "k" : 11 }
{ "_id" : 2, "k" : 22 }
{ "_id" : 3, "k" : 11 }
{ "_id" : 4, "k" : 44 }
{ "_id" : 5, "k" : 55 }
{ "_id" : 6, "k" : 66 }
{ "_id" : 7, "k" : 22 }
{ "_id" : 8, "k" : 88 }
{ "_id" : 9, "k" : 11 }
The query removes the duplicates:
db.dups.aggregate([
{ $group: {
_id: "$k",
dups: { "$addToSet": "$_id" },
count: { "$sum": 1 }
}},
{ $project: { k: "$_id", _id: { $arrayElemAt: [ "$dups", 0 ] } } }
] )
=>
{ "k" : 88, "_id" : 8 }
{ "k" : 22, "_id" : 7 }
{ "k" : 44, "_id" : 4 }
{ "k" : 55, "_id" : 5 }
{ "k" : 66, "_id" : 6 }
{ "k" : 11, "_id" : 9 }
As you see the following duplicate data is removed:
{ "_id" : 1, "k" : 11 }
{ "_id" : 2, "k" : 22 }
{ "_id" : 3, "k" : 11 }
Get the results in an array:
var arr = db.dups.aggregate([ ...] ).toArray()
The arr has the array of the documents:
[
{
"k" : 88,
"_id" : 8
},
{
"k" : 22,
"_id" : 7
},
{
"k" : 44,
"_id" : 4
},
{
"k" : 55,
"_id" : 5
},
{
"k" : 66,
"_id" : 6
},
{
"k" : 11,
"_id" : 9
}
]

Swift 4 json array - how to get value

I have this code in Swift 4 and Alamofire:
Alamofire.request("http://xxxx.pl/id=1", method: .get, parameters: nil)
.responseJSON { response in
let jsonResponse = JSON(response.result.value!)
let resData = jsonResponse["ranking"].array
print("XXXX: \(jsonResponse)")
}
.responseString { response in
if let error = response.result.error {
print(error)
}
if let value = response.result.value {
print(value)
}
}
After running this code, I get a json with the following parameters:
XXXX: {
"ranking" : {
"dataWidoczneOd" : {
"second" : 0,
"year" : 2018,
"month" : 2,
"hourOfDay" : 0,
"dayOfMonth" : 1,
"minute" : 0
}
"opis" : "cx",
"id" : 50971,
"dataWidoczneDo" : {
"second" : 0,
"year" : 2018,
"month" : 2,
"hourOfDay" : 0,
"dayOfMonth" : 31,
"minute" : 0
},
"grupy" : [
{
"nazwa" : "yyy",
"kod" : "yyy",
"id" : 51032,
"idkiPlikowGrafiki" : [
"51034"
],
"gracze" : [
{
"zakonczonaGra" : false,
"imieINazwisko" : "zzzz yyyy",
"email" : "tertretera#cccc.com",
"liczbaZdobytychPunktow" : "0.0",
"czasGry" : "0 min"
}
]
},
{
"nazwa" : "ttt",
"kod" : "ttt",
"id" : 50981,
"idkiPlikowGrafiki" : [
"50983",
"50986"
],
"gracze" : [
]
}
],
"nazwa" : "grupowy",
"idkiPlikowGrafiki" : [
"50976"
],
"typ" : "GRUPA",
"dataOd" : {
"second" : 0,
"year" : 2018,
"month" : 2,
"hourOfDay" : 0,
"dayOfMonth" : 1,
"minute" : 0
}
}
}
How can I get the values from this json array and save in variables:
- gracze (all values)
- idkiPlikowGrafiki (all values)
- typ (all values)
- kod (all values)
- dataWidoczneOd (all values)
?
Please help :)
You should create a model class for this response and parse it with, for example, Codable. Here is a good example.

Find documents matching multiple fields in an object array in MongoDB

Let's say I have an array of objects (let us call that array A) and I need a query to find a collection in MongoDB for all documents matching one of it's fields to one of the properties of object 1 in array A and another field to some other property in the same object in array A.
The documents do not have all the properties that the objects in array A have.
To make things clear...
Array A would look something like this...
[{
id_bus:1,
id_bus_variation:13,
....
},{
id_bus:2,
id_bus_variation:184,
....
},{
id_bus:3,
id_bus_variation:13,
....
}]
The documents in my database include those two properties and I need to match those two at the same time. For example, I need to find in my database the docs that have id_bus == 1 and id_bus_variation == 13, and also the ones that have id_bus == 2 and id_bus_variation == 184 but not the ones that id_bus == 4 and id_bus_variation == 13.
I really don't have any idea of how to do this using a single query, the only way around it I found is to go through array A and execute a query for each of it's elements, matching all the fields I need, but that doesn't seem efficient.
It sounds like you want to match the structure of a subdocument in an array to one of many possible structures specified by an array. I'll give an example of how to do this in the mongo shell:
> db.test.insert({
"_id" : 0,
bus : [
{ "id_bus" : 1, "id_bus_variation" : 1 },
{ "id_bus" : 2, "id_bus_variation" : 2 },
{ "id_bus" : 3, "id_bus_variation" : 3 }
]
})
> db.test.insert({
"_id" : 1,
bus : [
{ "id_bus" : 1, "id_bus_variation" : 3 },
{ "id_bus" : 2, "id_bus_variation" : 2 },
{ "id_bus" : 3, "id_bus_variation" : 1 }
]
})
> db.test.insert({
"_id" : 2,
bus : [
{ "id_bus" : 1, "id_bus_variation" : 1 },
{ "id_bus" : 2, "id_bus_variation" : 3 },
{ "id_bus" : 3, "id_bus_variation" : 2 }
]
})
If we want to return all documents where (id_bus = 2 and id_bus_variation = 3) or (id_bus = 3 and id_bus_variation = 3), as specified in an array
> var match = [{ "id_bus" : 2, "id_bus_variation" : 3 }, { "id_bus" : 3, "id_bus_variation" : 3 }];
We can construct the query programmatically:
> var query = { "$or" : [] }
> for (var i = 0; i < match.length; i++) {
query["$or"].push({ "bus" : { "$elemMatch" : match[i] } });
}
> db.test.find(query, { "_id" : 1 }) // just get _id's for easy reading
{ "_id" : 0 }
{ "_id" : 2 }
We get the expected results.
I don't know if I understand your Question.
Your collection could be like
{
"_id" : ObjectId("53de54c1560b7815e123792f"),
"bus" : [
{
"id_bus" : 1,
"id_bus_variation" : 13
},
{
"id_bus" : 2,
"id_bus_variation" : 184
},
{
"id_bus" : 3,
"id_bus_variation" : 13
}
]
}
And you want retrieve the document only if id_bus and id_bus_variation are "true"
You can try it
db.stack.find({$and:[{ "bus.id_bus" : 1,"bus.id_bus_variation" : 13},{"bus.id_bus" : 2,"bus.id_bus_variation" : 184}]})
and retrieve the Document only if bus.id_bus and bus.id_bus_variation are in Document.
For Example
db.stack.find({$and:[{ "bus.id_bus" : 1,"bus.id_bus_variation" : 13},{"bus.id_bus" : 2,"bus.id_bus_variation" : 184},{"bus.id_bus":4}]})
you haven't any result.
If you want exactly the element inside Object
db.stack.find ( { bus: { "$elemMatch" : { id_bus:1, id_bus_variation : 13} } } )
The document return only if both value are "true"

Resources