I have a collection with data like below:
{
"_id": ObjectId("95159a08a27971c35a2683f"),
"Date": ISODate("2018-04-03T07:00:00Z"),
"Employee": "Bill",
"Hours": 7.5
}
{
"_id": ObjectId("372c6be4912fdd32398382f"),
"Date": ISODate("2018-04-05T07:00:00Z"),
"Employee": "Bill",
"Hours": 2
}
And I would like to get the total hours per week, but the week needs to start on Saturday and end on Friday. If I were working with a standard week I would just do:
db.myCollection.aggregate(
{$match: {
Employee: "Bill",
Date: {
$gte: ISODate("2018-03-15T07:00:00Z"),
$lte: ISODate("2018-04-06T07:00:00Z")
}
}},
{$group: {
_id: {$week: "$Date"},
hours: {$sum: "$Hours"}
}
}
)
which works fine for standard weeks that start on Sunday and end Saturday.
How would I modify this to work with the non-standard week that I described? Would I be better off querying the data and aggregating it manually at the code level? Could I aggregate by week for Date - 1 day or would that be too weird?
UPDATE:
If anyone needs to make this work across years they can use the updated version of Neil's answer below:
{ "$group": {
"_id": {
"week":{
"$let": {
"vars": {
"satWeek": {
"$cond": {
"if": { "$eq": [ { "$dayOfWeek": "$Date" }, 7 ] },
"then": { "$add": [ { "$week": "$Date" }, 1 ] },
"else": { "$week": "$Date" }
}
}
},
"in": {
"$cond": {
"if": { "$gt": ["$$satWeek", 52] },
"then": 0,
"else": "$$satWeek"
}
}
}
},
"year": { "$year": "$Date" }
},
"hours": { "$sum": "$hours" }
}}
Well $week returns based on the "input" date, so Date - 1 day would still be in the "previous week" based on what it would return from that input even if you adjusted it by one day.
It's certainly always best to make the "server" do this type of thing, otherwise you're just pulling a lot of data "over the wire" which kind of obviates the point of using a database in the first place.
In brief, "starting on a Saturday" simply means that what was Week 1 becomes Week 2, etc when the day is Saturday. So week + 1. And the only real caveat is that anything above Week 52 becomes Week 0.
Therefore:
{ "$group": {
"_id": {
"$let": {
"vars": {
"satWeek": {
"$cond": {
"if": { "$eq": [ { "$dayOfWeek": "$Date" }, 7 ] },
"then": { "$add": [ { "$week": "$Date" }, 1 ] },
"else": { "$week": "$Date" }
}
}
},
"in": {
"$cond": {
"if": { "$gt": ["$$satWeek", 52] },
"then": 0,
"else": "$$satWeek"
}
}
}
},
"hours": { "$sum": "$hours" }
}}
The main adjusting point there being based around the test for $dayOfWeek, which returns 7 for Saturday.
Of course whether it's 52 or 53 as the boundary depends on whether it's a leap year or not, but since you want to aggregate per "week" then I presume you only want a "year at most", and then can adjust that as an input parameter based on whether your date selection is within a leap year or not.
Or of course, adjust the coding to be even more considerate of that. But the basic principle is adjust the "output week" instead of the "input Date"
As an alternate case, then I guess Date plus 1 day would actually get you the same result be skewing all dates forward:
{ "$group": {
"_id": { "$week": { "$add": [ "$Date", 1000 * 60 * 60 * 24 ] } },
"hours": { "$sum": : "$hours" }
}}
And if you needed local timezone adjustment, then from MongoDB 3.6 you can simply include the timezone information to adjust by:
{ "$group": {
"_id": {
"$week": {
"date": { "$add": [ "$Date", 1000 * 60 * 60 * 24 ] },
"timezone": "America/Chicago"
}
},
"hours": { "$sum": : "$hours" }
}}
Also note as of MongoDB 3.4 there is $isoWeek and like functions, where you get some different handling:
Returns the week number in ISO 8601 format, ranging from 1 to 53. Week numbers start at 1 with the week (Monday through Sunday) that contains the year’s first Thursday.
So all math would be based from Monday instead of Sunday and in consideration of which day of the year is counted as the "first week", as well as starting from 1 instead of 0.
Related
Here is my data. I wanted to change year but It should effective to only the first item of the document array
{
"_id": {
"$oid": "62053aa8aa1cfbe8c4e72662"
},
"school": "Test",
"reports": [
{
"year": "2020", // This has to be changed to 2019
"createdAt": {
"$date": "2022-02-10T17:05:25.682Z"
},
"pid": {
"$oid": "620545d5097761628f32365a"
},
"details": {
"end_date": {
"$date": "2020-03-31T00:00:00.000Z" // when end date is prior to July 01 of the $year mentioned.
}
}
}, {
"year": "2020",
"createdAt": {
"$date": "2022-03-14T19:08:38.125Z"
},
"pid": {
"$oid": "622f92b68a408531d4b784de"
},
"details": {
"end_date": {
"$date": "2021-03-31T00:00:00.000Z"
}
}
}
]
}
In the above data, I want to reduce the year to the previous year, if details.end_date is prior to July 01 of the current mentioned year. But It should change only the first item of the embedded array.
For example,
If Year is 2020 and details.end_date is prior to 01-July-2020, then change the year to 2019
If Year is 2020 and details.end_date is after 01-July-2020, then do not change the year
If Year is 2021 and details.end_date is prior to 01-July-2021, then change the year to 2020
If Year is 2021 and details.end_date is after 01-July-2021, then do not change the year
You can do the followings in an aggregation pipeline:
isolate the first elem of the reports array for easier processing using $arrayElemAt
use $cond to derive the year value with $month. Use $toInt and $toString for type conversion
use $concatArrays to append back the processed first Elem back to the reports array. Keep only the "tail" (i.e. without the first elem) using $slice
$merge to update the result back to the collection
db.collection.aggregate([
{
"$addFields": {
"firstElem": {
"$arrayElemAt": [
"$reports",
0
]
}
}
},
{
"$addFields": {
"firstElem.year": {
"$cond": {
"if": {
$lt: [
{
"$month": "$firstElem.end_date"
},
7
]
},
"then": {
"$toString": {
"$subtract": [
{
"$toInt": "$firstElem.year"
},
1
]
}
},
"else": "$firstElem.year"
}
}
}
},
{
"$addFields": {
"reports": {
"$concatArrays": [
[
"$firstElem"
],
{
"$slice": [
"$reports",
1,
{
"$subtract": [
{
"$size": "$reports"
},
1
]
}
]
}
]
}
}
},
{
"$project": {
firstElem: false
}
},
{
"$merge": {
"into": "collection",
"on": "_id",
"whenMatched": "replace"
}
}
])
Here is the Mongo playground for your reference.
I have a collection or orders with the following format:
"createTime" : ISODate("2021-04-16T08:01:39.000Z"),
"statusDetails" : [
{
"createTime" : ISODate("2021-04-16T08:01:39.000Z"),
"createUser" : "FOP-SYSTEM",
"stateOccurTimeStr" : "2021-04-16 15:01:39",
"status" : 27,
"statusDesc" : "Shipped"
}
]
where createTime is showing that when the order has been created
statusDetails.status = 27 showing that order has been shipped
statusDetails.createTime showing that when the order has been shipped
The result which I need is something like this:
Order Date
0-4 Hours
4-8 Hours
8-12 Hours
12-24 Hours
> 24 Hours
Total Orders
01-Jan-21
15
10
4
1
1
31
This shows that on "1-Jan-2021" after order creation,
15 orders shipped between 0-4 hours,
10 orders shipped between 4-8 hours,
4 orders shipped between 8-12 hours
and so on.
What I have done so far is:
db.orders.aggregate([
{ $unwind : "$statusDetails"},
{$match: {"statusDetails.status": { "$exists": true, "$eq": 24 }}},
{$project : { createTime: 1,statusDetails:1,
dateDiff: {"$divide" : [{ $subtract: ["$statusDetails.createTime","$createTime" ] },3600000]}}},
{$sort:{"createTime":-1}}
])
But this is showing time difference of each individual record, but I need group by
Edit
I have updated my query and now it is showing records using $group but still I need to add an another pipeline to group the current data.
db.orders.aggregate([
{ $unwind : "$statusDetails"},
{$match: {"statusDetails.status": { "$exists": true, "$eq": 27 }}},
{$project : { createTime: 1,statusDetails:1, dateDiff:{"$floor": {"$divide" : [{ $subtract: ["$statusDetails.createTime","$createTime" ] },3600000]}}}},
{
$group:
{ _id: { year : { $year : "$createTime" }, month : { $month : "$createTime" }, day : { $dayOfMonth : "$createTime" }},
shippedTime: { $push: "$dateDiff" },
count: { $sum: 1 }
}
},
{$sort:{"createTime":-1}}
])
$unwind to deconstruct statusDetails array
$match your conditions
$addFields to add slot field on the base of your date calculation hour slot from equation and get total
$group by date as per your format using $dateToString and slot
$sort by slot in ascending order
$group by only date and construct array of slots and get total orders
db.collection.aggregate([
{ $unwind: "$statusDetails" },
{ $match: { "statusDetails.status": 27 } },
{
$addFields: {
slot: {
$multiply: [
{
$floor: {
$divide: [
{
$abs: {
"$divide": [
{ $subtract: ["$statusDetails.createTime", "$createTime"] },
3600000
]
}
},
4
]
}
},
4
]
}
}
},
{
$group: {
_id: {
date: {
$dateToString: {
date: "$createTime",
format: "%d-%m-%Y"
}
},
slot: "$slot"
},
total: {
$sum: 1
}
}
},
{ $sort: { "_id.slot": 1 } },
{
$group: {
_id: "$_id.date",
hours: {
$push: {
slot: "$_id.slot",
total: "$total"
}
},
totalOrders: { $sum: "$total" }
}
}
])
Playground
Result would be:
[
{
"_id": "16-04-2021",
"hours": [
{ "slot": 0, "total": 1 }, // from 0 to 4
{ "slot": 4, "total": 1 }, // from 4 to 8
{ "slot": 8, "total": 2 } // from 8 to 12
],
"totalOrders": 4
}
]
I have a field called delDate and I want to use project to get the end of the week which is next Saturday and I want to use the end of week to group the revenue/ How do I do this in MongoDB?
weekEnd: {
$subtract: [{
$dateFromParts: {
"year": { $year: "$$NOW" },
"month": { $month: "$$NOW" },
"week": { $add: [{ $week: "$$NOW" }, 1] },
}
}, 86400000]
}
Demo - https://mongoplayground.net/p/ID0P83Au0ul
This Will get you next Saturday
Today + days for next Saturday
7 is Saturday, so 7 - today's day number * 1 day time
db.collection.aggregate([
{
"$addFields": {
weekEnd: {
$add: [ "$$NOW", { "$multiply": [ 86400000, { "$subtract": [ 7, { $dayOfWeek: "$$NOW" } ] } ] } ]
}
}
}
])
Update
Demo - https://mongoplayground.net/p/eGpsaGNTvJ_
Next Saturday without time
db.collection.aggregate([
{"$addFields":{
weekEnd:{
$dateFromString:{
dateString:{
$dateToString:{
format:"%Y-%m-%d",
date:{$add:["$$NOW",{"$multiply":[86400000,{"$subtract":[7,{$dayOfWeek:"$$NOW"}]}]}]}}
}
}
}
}
}
])
If you want to use node.js
You can use moment().startOf('isoWeek').day("saturday");
Here, date is register date and with simple group by I got result like this
[
{ date: '2019-09-01', count: 1 },
{ date: '2019-09-02', count: 3 },
{ date: '2019-09-04', count: 2 },
{ date: '2019-09-05', count: 5 },
// ...
]
But I want each and every date if on that date user is not register that display count as 0
[
{ date: '2019-09-01', count: 1 },
{ date: '2019-09-02', count: 3 },
{ date: '2019-09-03', count: 0 },
{ date: '2019-09-04', count: 0 },
// ...
]
If the user is not registered on 3 and 4 dates then displays 0 counts.
monthalldate = [ '2019-09-1', '2019-09-2', '2019-09-3', '2019-09-4', '2019-09-5', '2019-09-6', '2019-09-7', '2019-09-8', '2019-09-9',
'2019-09-10', '2019-09-11', '2019-09-12', '2019-09-13',.......,
'2019-09-30' ]
User.aggregate([
{ "$group": {
"_id": { "$substr": ["$createdOn", 0, 10] },
"count": { "$sum": 1 },
"time": { "$avg": "$createdOn" },
}},
{ "$sort": { "_id": 1 } },
{ "$project": { "date": "$_id", "createdOn": "$count" }},
{ "$group": { "_id": null, "data": { "$push": "$$ROOT" }}},
{ "$project": {
"data": {
"$map": {
"input": monthalldate,
"in": {
"k": "$$this",
"v": { "$cond": [{ "$in": ["$$this", "$data.date" ] }, 1, 0 ] }
}
}
}
}},
{ "$unwind": "$data" },
{ "$group": { "_id": "$data.k", "count": { "$sum": "$data.v" }}}
]).exec(function (err, montlysub) {
// console.log(montlysub);
});
But I got the wrong result
My user collection
{ "_id" : ObjectId("5a0d3123f954955f15fe88e5"), "createdOn" : ISODate("2019-11-16T06:33:07.838Z"), "name":"test" },
{ "_id" : ObjectId("5a0d3123f954955f15fe88e6"), "createdOn" : ISODate("2019-11-17T06:33:07.838Z"), "name":"test2" }
$project transforms input documents. If there is no user record for a particular month, there are no input documents to transform and you won't have any output for that month.
Ways around this:
Create a collection containing the months (only), then start by retrieving the desired date range from this collection and joining user documents to each month.
Add the missing zero data points in the application, either as the results are iterated or as a post-processing step prior to result rendering.
This keeps a record of each time the worker has worked on that project, and there may be several records in a single day because he works on several things.
I have this collection "WORKSDAYS":
{
"_id" : "00007cASDASDASDAS32423423",
"workDate" : ISODate("2017-01-20T00:00:00.000+00:00"),
"hours" : 6.0,
"worker" : {
"$ref" : "T_WORKERS",
"$id" : "f3f849d1-8777-4066-bdc6-64625475bbff"
},
"project" : {
"$ref" : "T_PROJECTS",
"$id" : "03b80b87-dc3b-4dc3-88c4-2b3334758459"
}
}
I need something similar to this for each project:
{
_id: { $ref: T_WORKERS, $id: id},
january: 168,
febrary: 120,
...
}
Or instead of January put 0 or 1, february = 02, etc.
I tried this that the workers gives me and the total hours of the project, but I would like the hours for each month:
aggregate([
{
$match: {
"project.$id": "03b80b87asdasdasdasd9"
}
},{
$group: {
"_id": "$worker",
"hours":{"$sum":"$hours"}
}
}
])
I had also thought, in passing him the start and end date (example: January and December and to join them)
.aggregate([
{$match: {
"project.$id": "03b80b8asdasdasdasdasd7"
}
}, {
$group: {
"_id": "$worker",
"month": {
"$sum": {
"$cond": [
{ "$gt": [
{ "$subtract": [ISODate("2018-01-01T00:00:00.000Z"), ISODate("2017-01-01T00:00:00.000Z") ] },
new Date().valueOf() - ( 1000 * 60 * 60 * 24 )
]},
"$hours",
0
]
}
}
}
}])