I have following dynamoDB Schema:
{
"Id": {"N": "789"},
"ProductCategory": {"S": "Home Improvement"},
"productReviews": {
"M": {
"FiveStar": {
"L": [
{ "S": "Best product ever!" }
]
},
"FourStar": {
"L": [
{ "S": "Good product" },
{ "S": "Another Review" }
]
}
}
}
}
So basically I have a map productReviews which has key as - "FourStar", "FiveStar", "TwoStar" etc. and value as List of reviews.
I want add new reviews to this table i.e. if a fiveStar reviews comes I will add/append it in the list of 'FiveStar' of productReviews. If a key does not exist in the map, I would like to just append the key value.
Is this possible in DynamoDB or I have to merge the list on my own and then update at each write.
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.SET
If you want this model, you'll need to merge and update on your own.
It might be simpler to make each star value its own item (give them all the same PK with an SK indicative of the star value) so you can update them directly as a single item with a string list to append to.
Depending on your access patterns, you might even want each review as its own item (still under the same PK). For example, if the item gets a thousand reviews coming in over time, you don't want to repeatedly update an ever growing item if you can just blindly add a small item each time.
On retrieval it'll still be efficient with a Query for the PK.
Related
I am practicing arangodb in company.
When I want to express the following relation between the user and the user, I want to delete the data of the corresponding following relation when one user is deleted.
user collection
{
"_key": "test4",
"_id": "users/test4",
"_rev": "_V8yGRra---"
},
{
"_key": "test2",
"_id": "users/test2",
"_rev": "_V8whISG---"
},
{
"_key": "test1",
"_id": "users/test1",
"_rev": "_V8whFQa---"
},
{
"_key": "test3",
"_id": "users/test3",
"_rev": "_V8yoFWO---",
"userKey": "test3"
}
follow collection[edge]
{
"_key": "48754",
"_id": "follow/48754",
"_from": "users/test1",
"_to": "users/test2",
"_rev": "_V8wh4Xe---"
}
{
"_key": "57447",
"_id": "follow/57447",
"_from": "users/test2",
"_to": "users/test3",
"_rev": "_V8yHGQq---"
}
If used properly, the ArangoDB system ensures the integrity of named graphs (GRAPHs).
To delete a specific user (say "users/test4") and the corresponding edges in follow manually, an AQL query along the following lines should suffice to delete the edges:
for v,e IN 1..1 ANY "users/test4" follow
REMOVE e IN follow
COLLECT WITH COUNT INTO counter
RETURN counter
Assuming "users/test4" is not referenced elsewhere, the node can then safely be deleted, e.g. by
REMOVE "test4" in users
The important point here is that when manually deleting nodes, all the relevant edge collections must be identified and managed explicitly.
First you should create a graph with your vertex and edge collection. Working with graphs you can use the REST API to remove a vertex. This way all edges pointing to this vertex and the vertex itself get removed.
DELETE /_api/gharial/{graph-name}/vertex/{collection-name}/{vertex-key}
You can find the documentation including an example under https://docs.arangodb.com/3.2/HTTP/Gharial/Vertices.html#remove-a-vertex.
It is also possible to achieve this with an AQL query, for example deleting test1 from the users collection:
LET keys = (
FOR v, e IN 1..1 ANY 'users/test1' GRAPH 'your-graph-name' RETURN e._key)
LET r = (FOR key IN keys REMOVE key IN follow) REMOVE 'test1' IN users
A graph traversal is used to get _key attributes of all edges pointing to test1, then these edges are removed from the follow collection and test1 is removed from the users collection.
My goal is to find songs that match user's choices. User has the opportunity to exclude genres he doesn't like.
This is basically a duplicate of this question, except that I'd like to be able to do this in an Aggregate operation instead of a Find, as I need to add other stages.
Right now, I am trying in an aggregation to exclude songs that belong to genres user doesn't like.
My song json is as such (keep in mind this is dummy data - but this song is really good though):
{
"_id": {
"$oid": "5890aa3b0a9f110011698fac"
},
"artist": "Beach House",
"songName": "Master Of None",
"genres": [
{
"$oid": "58624b4298fba881a46663a01"
},
{
"$oid": "58624b9d98fba772a46663a05"
}
]
}
A song can have multiple genres, stored in an array of objects as references to Genre documents in a different collection.
Basically, user's disliked genres are in an array of genre $oid.
Say
dislikedGenres = [ "786761gg1G176ga1", "78676187g1G176hsj3", "78676187g1G1761sj4" ]
What I'm trying to do is say "if you find a song with any of these genres, exclude it".
Any idea how to achieve this? I feel like I'm missing something dramatically obvious here...
Thanks in advance for the help! Much appreciated.
Please leave a comment if you need extra info.
I'm just starting to experiment with Firebase. It's a real head bender when you're used to relational databases!
I'm trying to design an app that will allow users to search for meals by barcode or name and retrieve the number of calories. Additionally, I need to be able to store the meals eaten by a user, and finally retrieve the food eaten by a user each day, week or month.
I was thinking each meal would have a unique ID (e.g. M1234 for Pizza), then I'd have 2 lookup sections - one by barcode and one by name, so that should hopefully cover the search functionality.
Each user would have the meals eaten stored in the eaten 'table' (what is the correct term for 'table' in a Firebase database?) by date, just referencing the meal by ID.
This is how I've designed the database.
{
// Here are the users.
"users": {
"mchen": {
"name": "Mary Chen",
"email": "mary#chen.com",
}
},
...
},
// Here are the meals eaten by date.
"eaten": {
"mchen": {
// index Mary's meals in her profile /eaten/mchen/meals/20161217 should return 'M1234' (pizza) and 'M8765' (chips)
"meals": {
"20161217": {
"M1234": true,
"M8765": true
},
"20161218": {
"M2222": true,
"M8765": true
}
},
...
},
// Here are the meals with calorie information.
"meals": {
"M1234": {
"name": "Pizza"
"calories": 400
},
"M2222": {
"name": "Curry"
"calories": 250
},
"M8765": {
"name": "Chips"
"calories": 100
},
},
// Here is the barcode lookup
"barcode-lookup": {
"12345678": {
"id": "M1234"
},
"87654321": {
"id": "M2222"
},
"11223344": {
"id": "M8765"
}
},
// Here is the name lookup
"name-lookup": {
"Chips": {
"id": "M8765"
},
"Pizza": {
"id": "M1234"
},
"Curry": {
"id": "M2222"
}
}
}
Does it seem reasonable or are there any obvious flaws?
You will want to leverage .childByAutoId() and let Firebase create the parent key names. It's best practice to disassociate your child data from the parent node and allowing Firebase to create 'random' key's for the parents will make that work.
Along with that, it's customary to create a /users node and the parent nodes for each user would be the uid which was created by Firebase when the user was first created.
In your original structure, there's a barcode and name lookup which I have integrated into the following structure to reduce complexity.
users
uid_0
name: "Mary Chen",
email: "mary#chen.com"
uid_1
name: "Larry David"
email: "ldavid#david.com"
and then the dining
dining
-Yuiia09skjspo
dining_timestamp: "20161207113010"
Y79joa90ksss: true
Yjs9990kokod: true
user: uid_0
uid_timestamp: "uid_0_ 20161207113010"
-Yi9sjmsospkos
dining_timestamp: "20161207173000"
Y79joa90ksss: true
Yjs9990kokod: true
user: uid_1
uid_timestamp: "uid_1_ 20161207173000"
and the meals the user can choose from
meal
-Y79joa90ksss
name: "Pizza"
calories: "400"
barcode: "008481816164"
-Yjs9990kokod
name: "Burger"
calories: "520"
barcode: "991994411815"
As you can see, the dining node contains a dining event for each user (so all of the dining events are in one node)
This enables you to query for all kinds of things:
All dining for all users by date or range of dates.
All dining that contain a certain meal
All meals by a user
->The cool one<- all dining for a specific user within a date range.
The one omission is a search for dining that contains two meals, however, the solution to that is also in this answer.
All in all, your structure is sound - just needs a little tweaking.
The structure looks fine (though I would let firebase generate the ids). The only thing that won't work like what you're expecting is searching. Based on your data if I searched for pizza you couldn't write a query that would return the Pizza entry. My suggestion would be to either use Algolia (or something similar) for searching or to roll another key with your name lowerCased to make it possible for a query to work. The only issue with running your own is you won't be able to search for things like izz and have Pizza turn up. See my answer Firebase - How can I filter similarly to equalTo() but instead check if it contains the value? for how to do a search.
I have a reviews collection and a users collection. In the users collection I have docs that have an array called reviews that contains the _ids of the review from the reviews collection. Sometimes I go to the mongo terminal and delete reviews from the reviews collection that I don't want from the reviews collection. for example I would delete all that don't have a specific field. the problem is I would also like to delete the review id in the reviews array. I do a simple db.collection.remove on the reviews collection so that is easy.(that would delete like 50 reviews). I was wondering how to remove ids in the reviews array easily. Any ideas? I use mongoose but I was wondering if there was an easy way to do in the terminal. I think the only association is the ids.
I no how to use $pull but I want to make it pull the ids every time a specific review is removed.
You certainly can't do this with a single atomic update as you have two collections to update. What you can do is carry out two operations in series i.e. the removal of the specific reviews from the reviews collection and then use the $pullAll operator to delete the values from reviews array in the users collection.
The following mongo shell example shows this:
var reviewIds = [
ObjectId("582ebf010936ac3ba5cd00e2"),
ObjectId("582ebf010936ac3ba5cd00e3"),
...
];
db.reviews.remove({ "_id": { "$in": reviewIds } });
db.users.updateMany(
{ "reviews": { "$in": reviewIds } },
{ "$pullAll": { "reviews": reviewIds } }
);
For a single review id, you can just use the $pull operator as
db.reviews.remove({ "_id": reviewId });
db.users.updateMany(
{ "reviews": reviewIds },
{ "$pull": { "reviews": reviewId } }
);
what seemed a simple task, came to be a challenge for me.
I have the following mongodb structure:
{
(...)
"services": {
"TCP80": {
"data": [{
"status": 1,
"delay": 3.87,
"ts": 1308056460
},{
"status": 1,
"delay": 2.83,
"ts": 1308058080
},{
"status": 1,
"delay": 5.77,
"ts": 1308060720
}]
}
}}
Now, the following query returns whole document:
{ 'services.TCP80.data.ts':{$gt:1308067020} }
I wonder - is it possible for me to receive only those "data" array entries matching $gt criteria (kind of shrinked doc)?
I was considering MapReduce, but could not locate even a single example on how to pass external arguments (timestamp) to Map() function. (This feature was added in 1.1.4 https://jira.mongodb.org/browse/SERVER-401)
Also, there's always an alternative to write storedJs function, but since we speak of large quantities of data, db-locks can't be tolerated here.
Most likely I'll have to redesign the structure to something 1-level deep, like:
{
status:1,delay:3.87,ts:138056460,service:TCP80
},{
status:1,delay:2.83,ts:1308058080,service:TCP80
},{
status:1,delay:5.77,ts:1308060720,service:TCP80
}
but DB will grow dramatically, since "service" is only one of many options which will append each document.
please advice!
thanks in advance
In version 2.1 with the aggregation framework you are now able to do this:
1: db.test.aggregate(
2: {$match : {}},
3: {$unwind: "$services.TCP80.data"},
4: {$match: {"services.TCP80.data.ts": {$gte: 1308060720}}}
5: );
You can use a custom criteria in line 2 to filter the parent documents. If you don't want to filter them, just leave line 2 out.
This is not currently supported. By default you will always receive the whole document/array unless you use field restrictions or the $slice operator. Currently these tools do not allow filtering the array elements based on the search criteria.
You should watch this request for a way to do this: https://jira.mongodb.org/browse/SERVER-828
I'm attempting to do something similar. I tried your suggestion of using the GROUP function, but I couldn't keep the embedded documents separate or was doing something incorrectly.
I needed to pull/get a subset of embedded documents by ID. Here's how I did it using Map/Reduce:
db.parent.mapReduce(
function(parent_id, child_ids){
if(this._id == parent_id)
emit(this._id, {children: this.children, ids: child_ids})
},
function(key, values){
var toReturn = [];
values[0].children.forEach(function(child){
if(values[0].ids.indexOf(product._id.toString()) != -1)
toReturn.push(child);
});
return {children: toReturn};
},
{
mapparams: [
"4d93b112c68c993eae000001", //example parent id
["4d97963ec68c99528d000007", "4debbfd5c68c991bba000014"] //example embedded children ids
]
}
).find()
I've abstracted my collection name to 'parent' and it's embedded documents to 'children'. I pass in two parameters: The parent document ID and an array of the embedded document IDs that I want to retrieve from the parent. Those parameters are passed in as the third parameter to the mapReduce function.
In the map function I find the parent document in the collection (which I'm pretty sure uses the _id index) and emit its id and children to the reduce function.
In the reduce function, I take the passed in document and loop through each of the children, collecting the ones with the desired ID. Looping through all the children is not ideal, but I don't know of another way to find by ID on an embedded document.
I also assume in the reduce function that there is only one document emitted since I'm searching by ID. If you expect more than one parent_id to match, than you will have to loop through the values array in the reduce function.
I hope this helps someone out there, as I googled everywhere with no results. Hopefully we'll see a built in feature soon from MongoDB, but until then I have to use this.
Fadi, as for "keeping embedded documents separate" - group should handle this with no issues
function getServiceData(collection, criteria) {
var res=db[collection].group({
cond: criteria,
initial: {vals:[],globalVar:0},
reduce: function(doc, out) {
if (out.globalVar%2==0)
out.vals.push({doc.whatever.kind.and.depth);
out.globalVar++;
},
finalize: function(out) {
if (vals.length==0)
out.vals='sorry, no data';
return out.vals;
}
});
return res[0];
};