Say person can have multiple cars, and car can have multiple accidents. So we could have:
# Person with no cars
person:
name: "Misha"
cars: []
# Person with free-accident car
person:
name "Arlen"
cars:
0:
name: "Toyota"
accidents: []
Firebase stores these people as:
person:
name: "Misha"
and
person:
name "Arlen"
cars:
0:
name: "Toyota"
So in JavaScript I have to do following to restore the empty arrays: (CoffeeScript)
if person.cars?
for car in person.cars
car.accidents = [] unless car.accidents?
else
person.cars = []
Is there a better way to handle empty arrays in Firebase without writing this needless JavaScript code?
I think that, if I understand the core question, the short answer is that there is no way to force an empty array into Firebase. However, there are some paradigms that might work better than what you have above.
Keep in mind that Firebase is a real-time environment. The number of cars and accidents can (and will) change at any time. It's best to treat everything as new data arriving in real time and avoid even thinking about exists or doesn't exist.
// fetch all the people in real-time
rootRef.child('people').on('child_added', function(personSnapshot) {
// monitor their cars
personSnapshot.ref().child('cars', 'child_added', function(carSnapshot) {
// monitor accidents
carSnapshot.ref().child('accidents', 'child_added', function(accidentSnapshot) {
// here is where you invoke your code related to accidents
});
});
});
Note how there is no need for if exists/unless type logic. Note that you would probably also want to monitor child_removed on cars and people and call ref.off() to stop listening to specific children.
If for some reason you want to stick with the static model, then forEach will become your friend:
// fetch all the people as one object, asynchronously
// this won't work well with many thousands of records
rootRef.child('people').once('value', function(everyoneSnap) {
// get each user (this is synchronous!)
everyoneSnap.forEach(function(personSnap) {
// get all cars (this is asynchronous)
personSnap.ref().child('cars').once('value', function(allCars) {
// iterate cars (this is synchronous)
allCars.forEach(function(carSnap) { /* and so on */ });
});
});
});
Note how, even with forEach, there is no need for "exists or unless" sort of logic.
I usually use the DataSnapshot function numChildren() to see if it's empty of not, like this
var fire = new Firebase("https://example.firebaseio.com/");
fire.once('value', function(data){if (data.numChildren() > 0){ /*Do something*/ });
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 10 months ago.
Improve this question
Im going to create a achievement system in Mongodb. But im not sure how i would format/store it in the database.
As of the users should have a progress (on each achievement they would have some progress value stored), im really confused what would be the best way to perform this, and without having an performence issue.
what should i do?, cause i dont know, what i had in mind, was maybe something like:
Should i store each achievement in an unique row in a Achievement collection, and an user array within that row, containing object with userid and achievement progress?
Would i then get an performance issue when its 1000+ achievements, that is beeing checked fairy often?
or should i do something else?
example schema for the option above:
{
name:{
type:String,
default:'Achievement name'
},
users:[
{
userid:{
type:String,
default:' users id here'
},
progress:{
type:Number,
default:0
}
}
]
}
Even though the question is specifically about the database design, I will give a solution for the tracking/awarding logic as well to establish more accurate context for the db design.
I would store the achievements progress separately from the already awarded achievements for cleaner tracking and discovery.
The whole logic is event based and has multiple layers of event handling. This gives you TONS of flexibility on how you track your data and also gives you a pretty good mechanism to track history. Basically, you can look at it as a form of logging.
Of course, your system design and contracts are highly dependent on the information you're gonna be tracking and its complexity. A simple progress field may not suffice for each case(you might want to track something more complex, not a simple number between X and Y). There is also the case of tracking data which updates quite frequently(as distance travelled in games, for example). You didn't give any context on the topic of your achievement system so we're gonna stick with a generic solution. It's just a couple of things that you should take a note about as it will affect the design.
Okay, so, let's start from the top and track the entire flow for a tracked piece of data and its eventual achievement progress. Let's say we're tracking consecutive days of user login and we're gonna award him with an achievement when he reaches [10].
Note that everything below is just a pseudo-code.
So, let's say today is [8th of July, 2017]. For now, our User entity looks like this:
User: {
id: 7;
trackingData: {
lastLogin: 7 of July, 2017 (should be full DateTime object, but using this for brevity),
consecutiveDays: 9
},
achievementProgress: [
{
achievementID: 10,
progress: 9
}
],
achievements: []
}
And our achievements collection contains the following entity:
Achievement: {
id: 10,
name: '10 Consecutive Days',
rewardValue: 10
}
The user tries to login(or visit the site). The application handler takes note of that and after handling the login logic fires an event of type ACTION:
ACTION_EVENT = {
type: ACTION,
name: USER_LOGIN,
payload: {
userID: 7,
date: 8 of July, 2017 (should be full DateTime object, but using this for brevity)
}
}
We have an ActionHandler which listens for events of type ACTION:
ActionHandler.handleEvent(actionEvent) {
subscribersMap = Map<eventName, handlers>;
subscribersMap[actionEvent.name].forEach(subscriber => subscriber.execute(actionEvent.payload));
}
subscribersMap gives us a collection of handlers that should respond to each specific action(this should resolve to USER_LOGIN for us). In our case we can have 1 or 2 that concern themselves with updating the user tracking information of lastLogin and consecutiveDays tracking properties in the user entity. The handlers in our case will update the tracking information and fire new events further down the line.
Once again, for brevity, we're gonna incorporate both into one:
updateLoginHandler: function(payload) {
user = db.getUser(payload.userID);
let eventType;
let eventValue;
if (date - user.trackingData.lastLogin > 1 day) {
user.trackingData = 1;
eventType = 'PROGRESS_RESET';
eventValue = 1;
}
else {
const newValue = user.trackingData.consecutiveDays + 1;
user.trackingData.consecutiveDays = newValue;
eventType = 'PROGRESS_INCREASE';
eventValue = newValue;
}
user.trackingData.lastLogin = payload.date;
/* DISPATCH NEW EVENT OF TYPE ACHIEVEMENT_PROGRESS */
AchievementProgressHandler.dispatch({
type: ACHIEVEMENT_PROGRESS
name: eventType,
payload: {
userID: payload.userID,
achievmentID: 10,
value: eventValue
}
});
}
Here, PROGRESS_RESET have the same contract as the PROGRESS_INCREASE but have a different semantic meaning and I would keep them separate for history/tracking purposes. If you wish, you can combine them into a single PROGRESS_UPDATE event.
Basically, we update the tracked fields that are dependent on the lastLogin date and fire a new ACHIEVEMENT_PROGRESS event which should be handled by a separate handler with the same pattern(AchievementProgressHandler). In our case:
ACHIEVEMENT_PROGRESS_EVENT = {
type: ACHIEVEMENT_PROGRESS,
name: PROGRESS_INCREASE
payload: {
userID: 7,
achievementID: 10,
value: 10
}
}
Then, in AchievementProgressHandler we follow the same pattern:
AchievementProgressHandler: function(event) {
achievementCheckers = Map<achievementID, achievementChecker>;
/* update user.achievementProgress code */
switch(event.name): {
case 'PROGRESS_INCREASE':
achievementCheckers[event.payload.achievementID].execute(event.payload);
break;
case 'PROGRESS_RESET':
...
}
}
achievementCheckers contains a checker function for each specific achievement that decides if the achievement has reached its desired value(a progress of 100%) and should be awarded. This enables us to handle all kinds of complex cases. If you only track a single X out of Y scenario, you can share the function between all achievements.
The handler basically does this:
achievementChecker: function(payload) {
achievementAwardHandler;
achievement = db.getAchievement(payload.achievementID);
if (payload.value >= achievement.rewardValue) {
achievementAwardHandler.dispatch({
type: ACHIEVEMENT_AWARD,
name: ACHIEVEMENT_AWARD,
payload: {
userID: payload.userID,
achievementID: achievementID,
awardedAt: [current date]
}
});
/* Here you can clear the entry from user.achievementProgress as you no longer need it. You can also move this inside the achievementAwardHandler. */
}
}
We once again dispatch an event and use an event handler - achievementAwardHandler. You can skip the event creation step and award the achievement to the user directly but we keep it consistent with the whole history logging flow.
An added benefit here is that you can use the handler to defer the achievement awarding to a specific later time thus effectively batching awarding for multiple users, which serve a couple of purposes including performance enhancement.
Basically, this pseudo code handles the flow from [a user action] to [achievement rewarding] with all intermediate steps included. It's not set in stone, you can modify it as you like but all in all, it gives you a clean separation of concerns, cleaner entities, it's performant, let's you add complex checks and handlers which are easy to reason about while in the same time provide a great history log of the user overall progress.
Regarding the DB schema entities, I would suggest the following:
User: {
id: any;
trackingData: {},
achievementProgress: {} || [],
achievements: []
}
Where:
trackingData is an object that contains everything you're willing
to track about the user. The beauty is that properties here are
independent from achievement data. You can track whatever and eventually use it for achievement purposes.
achievementProgress: a map of <key: achievementID, value: data> or
an array containing the current progress for each achievement.
achievements: an array of awarded achievements.
and Achievement:
Achievement: {
id: any,
name: any,
rewardValue: any (or any other field/fields. You have complete freedom to introduce any kind of tracking with the approach above),
users?: [
{
userID: any,
awardedAt: date
}
]
}
users is a collection of users who have been rewarded the given achievement. This is optional and is here only if you have the use for it and query for this data frequently.
What you might be looking for is a Badge style implementation. Just like Stack Overflow rewards it's users with badges for specific achievements.
Method 1: You can have flags in the user profile for each badge. Since you're doing it in NoSQL database, you just have to set a flag for each badge.
const badgeSchema = new mongoose.Schema({
badgeName: {
type: String,
required: true,
},
badgeDescription: {
type: String,
required: true,
}
});
const userSchema = new mongoose.Schema({
userName: {
type: String,
required: true,
},
badges: {
type: [Object],
required: true,
}
});
If your application architecture is event based, you can trigger awarding badges to users. And that operation is just inserting Badge object with progress in User badges array.
{
badgeId: ObjectId("602797c8242d59d42715ba2c"),
progress: 10
}
Update operation will be to find and update the badges array with progress percentage number
And while displaying user achievements on user interface, you can just loop over badges array to show the badges this user has achieved and their progress with it.
Method 2: Have a separate mongo collection for Badge and User Mapping. Whenever a user achieves a badge you insert a record in that collection. It will be one to one mapping of user _id and badge _id and progress value. But as the table will grow huge you will need to do indexing to efficiently query user and badge mapping.
You will have to do analysis on best approach according to your specific use case.
MongoDB is flexible enough to allow teams develop applications quickly, and involve their model with litter friction as the application needs it. In cases where you need a robust model from day one, theirs is a flexible methodology that can guide you through the process of modeling your data.
The methodology is composed of:
Workload: This stage is about gathering as much information as possible to understand your data. This will allow you formulate assumptions about, you data size the operations that will be performance against it (reads and writes), quantify operations and qualify operations.
You can get this by:
Scenarios
Prototype
Production Logs & Stats (if you are migrating).
Relationships: Identify the relationship between the different entities in your data, quantify those relationships and apply embedding or linking. In general you should prefer embedding by default, but remember that arrays should not grow without bound (6 Rules of Thumb for MongoDB Schema Design: Part 3).
Patterns: Apply schema design patterns. Take a look at Building with Patterns: A Summary, it presents a matrix that highlights the pattern that could be useful for a given use case.
Finally, the goal of this methodology is help you create a model, that can scale and perform well under stress.
If you design the achievement schema like this:
{
name: {
type: String,
default: "Achievement name",
},
userid: {
type: String,
default: " users id here",
},
progress: {
type: Number,
default: 0,
},
}
}
When an achievement is gained you just add another entry
for getting achievements Map-Reduce is a good candidate for running map reduce on the database. you can run them on a less regular basis, using them for offline computation of the data that you want.
based on documentation you can do like the following photo
i'm building an Gmail-like email browser client app prototype and i need a little help/advice structuring my React/Flux app. I decided to use pure Flux to get a better idea of how it works.
It's a simple email client with a list of letters, grouped by folders and tags and an ability to add letters to favorites.
So, i have a LettersStore containing an array of letters. The single letter data object looks something like this
{
id: 0,
new: true, //unread
checked: false,
starred: false,
folder: "inbox", //could be 'sent', 'spam', 'drafts'
sender: "Sender Name",
subject: "Re:",
snippet: "Hello there, how are you...",
body: "Hello there, how are you doing, Mike?",
date: "02.19.2016 16:30",
tags:["personal", "urgent"]
}
So what i'm trying to achieve is to let users navigate through folders (inbox, sent, drafts, spam) and filters (starred, tag, etc.)
In both folders and filters there has to be a way to select (check) some/all letters. The view state depends on how many letters are selected (the Select-all checkbox update, just like on Gmail). When the user selects a letter, the Flux action is being triggered and the state of the app updates.
The controller-view on top of the app does all the calls to the LettersStore public methods and passes the data down as props, but i'm not sure, what public methods the LettersStore should have. Currently it has:
emitChange()
addChangeListener()
removeChangeListener()
getAll() //returns array
areSomeLettersInFolderChecked(folderName) //returns bool
areAllLettersInFolderChecked(folderName) //returns bool
countNewLettersInAllFolders() //returns object
This works ok with folders, but when it comes to filters, it doesn't make sense anymore, since a starred letter is in some folder, and i feel like it's not the right thing to add specific methods like areSomeLettersInFilterChecked(filterType) etc.
Also, just like in Gmail, there has to be a way to select letter in the "Starred" filter, which belongs to the "Inbox" folder, then navigate to "Inbox" folder and keep that letter selected.
Maybe i should move the areSomeLettersInFolderChecked-like stuff to the component level?
I'm sure here has to be a proper way of doing it. Thanks in advance!
Rather than trying to encapsulate all the possible states and filters into your letter objects, keep it dumb. Normalize it and use supporting data structures to represent the other characteristics.
I'd strip it down to just the following properties:
{
id: 0,
sender: "Sender Name",
subject: "Re:",
snippet: "Hello there, how are you...",
body: "Hello there, how are you doing, Mike?",
date: "02.19.2016 16:30",
tags:["personal", "urgent"]
}
Your LetterStore can stay the same, or alternatively you could use an object or map to store letters against their id's for quick lookups later.
Now we need to represent the properties we removed from the message.
We can use individual sets to determine whether a message belongs to the new, checked and starred categories.
For instance, to star a message, just add it's id to the starred set.
var starred = new Set();
starred.add(message.id);
You can easily check whether a message is starred later on.
function isStarred(message) {
return starred.has(message.id);
}
The pattern would be the same for checked and unread.
To represent folders you probably want to use a combination of objects and sets.
var folders = {
inbox: new Set(),
sent: new Set(),
spam: new Set(),
drafts: new Set()
}
Simplifying your structures into these sets makes designing queries quite easy. Here are some examples of the methods you talked about implemented with sets.
function checkAll() {
messages.forEach(function(message) {
checked.add(message.id);
});
return checked;
}
function isChecked(message) {
return checked.has(message.id);
}
function inFolder(name, message) {
return folders[name].has(message.id);
}
// is message checked and in inbox
if(isChecked(message) && inFolder('inbox', message)) {
// do something
}
It becomes easy to construct complex queries, simply by checking whether messages belong to multiple sets.
I am trying to make a Meteor app to let users push a value to the database. It works ok, but there a small issue. As soon a certain user has pushed his information, i don't want to let the same user create another entry. Or this must be blocked, or the value the user is pushing must be overwritten for the value he is posting the second time. Now I get multiple entry's of the same user.
Here is my code. Hope you can help me here. Thanks in advance.
Estimations.update(userstory._id, {
$addToSet: {
estimations: [
{name: Meteor.user().username, estimation: this.value}
]
}
});
From the mongo docs
The $addToSet operator adds a value to an array unless the value is
already present, in which case $addToSet does nothing to that array.
Since your array elements are objects the value is the entire object, not just the username key. This means a single user can create multiple name, estimation pairs as long as the estimation value is different.
What you can do is remove any value for the user first, then reinsert:
var username = Meteor.user().username;
Estimations.update({ userstory._id },
{ $pull: { estimations: { name: username }}}); // if it doesn't exist this will no-op
Estimations.update({userstory._id },
{ $push: { estimations: { name: username, estimation: this.value }}});
By way of commentary, you've got a collection called Estimations that contains an array called estimations that contains objects with keys estimation. This might confuse future developers on the project ;) Also if your Estimations collection is 1:1 with UserStorys then perhaps the array could just be a key inside the UserStory document?
I've been working with react/flux for a few weeks now and while I feel like I've got a pretty good handle on everything from async loading to updating props/states/etc, one thing that is still bothering me is how to handle save states.
For example, when loading data, I just have an isLoading boolean parameter in my store that gets passed to my components. But when I try and post an updated object to the server, it's trivial to:
fire the update action
display a "save in progress" state
but figuring out the result of the update action seems to be way more difficult.
Probably the most applicable post I've seen on this is in Fluxxor's async data guide, but their solution (adding/modifying a status property on the object) feels error-prone to me.
onAddBuzz: function(payload) {
var word = {id: payload.id, word: payload.word, status: "ADDING"};
this.words[payload.id] = word;
this.emit("change");
},
onAddBuzzSuccess: function(payload) {
this.words[payload.id].status = "OK";
this.emit("change");
},
onAddBuzzFail: function(payload) {
this.words[payload.id].status = "ERROR";
this.words[payload.id].error = payload.error;
this.emit("change");
}
Is there a better way to manage save states or is adding a status property to the object the best way?
I recommend keeping your "model stores" and "ui stores" separate, or at least accessed via different cursor positions in the same store. So, in your case you'd have one store or branch for your "word model" and then another store or branch for "word status."
While this adds some complexity in the form of breaking up logic across stores and reacting twice to the AddBuzz action, it ends up reducing (more) complexity by confining model store updates to true changes in model data and managing ui states separately.
EDIT
This is what Relay will more-or-less be doing, keeping persisted data in a separate self-managed store, leaving custom stores for nothing but ui state. Some other libraries like https://github.com/Yomguithereal/baobab also recommend this approach. The idea is that these are fundamentally different kinds of state. One is domain-specific persisted data and the other is ui-specific ephemeral application state.
It might look something like this:
model_store.js:
onAddBuzz: function(payload) {
var word = {id: payload.id, word: payload.word};
this.words[payload.id] = word;
this.emit("change");
}
ui_store.js:
onAddBuzz: function(payload) {
this.wordStates[payload.id] = 'ADDING';
this.emit("change");
}
my_view_controller.js:
// listen to both stores and pass down both words and wordStates to your views
my_word_view.js:
...
render: function() {
var word = this.props.word,
wordState = this.props.wordState;
...
}
If you don't want to emit two change events, have one waitFor the other and emit the change only from the second one.
Ok Im starting out fresh with Firebase. I've read this: https://www.firebase.com/docs/data-structure.html and I've read this: https://www.firebase.com/blog/2013-04-12-denormalizing-is-normal.html
So I'm suitably confused as one seems to contradict the other. You can structure your data hierarchically, but if you want it to be scalable then don't. However that's not the actual problem.
I have the following structure (please correct me if this is wrong) for a blog engine:
"authors" : {
"-JHvwkE8jHuhevZYrj3O" : {
"userUid" : "simplelogin:7",
"email" : "myemail#domain.com"
}
},
"posts" : {
"-JHvwkJ3ZOZAnTenIQFy" : {
"state" : "draft",
"body" : "This is my first post",
"title" : "My first blog",
"authorId" : "-JHvwkE8jHuhevZYrj3O"
}
}
A list of authors and a list of posts. First of all I want to get the Author where the userUid equals my current user's uid. Then I want to get the posts where the authorId is the one provided to the query.
But I have no idea how to do this. Any help would be appreciated! I'm using AngularFire if that makes a difference.
Firebase is a NoSQL data store. It's a JSON hierarchy and does not have SQL queries in the traditional sense (these aren't really compatible with lightning-fast real-time ops; they tend to be slow and expensive). There are plans for some map reduce style functionality (merged views and tools to assist with this) but your primary weapon at present is proper data structure.
First of all, let's tackle the tree hierarchy vs denormalized data. Here's a few things you should denormalize:
lists you want to be able to iterate quickly (a list of user names without having to download every message that user ever wrote or all the other meta info about a user)
large data sets that you view portions of, such as a list of rooms/groups a user belongs to (you should be able to fetch the list of rooms for a given user without downloading all groups/rooms in the system, so put the index one place, the master room data somewhere else)
anything with more than 1,000 records (keep it lean for speed)
children under a path that contain 1..n (i.e. possibly infinite) records (example chat messages from the chat room meta data, that way you can fetch info about the chat room without grabbing all messages)
Here's a few things it may not make sense to denormalize:
data you always fetch en toto and never iterate (if you always use .child(...).on('value', ...) to fetch some record and you display everything in that record, never referring to the parent list, there's no reason to optimize for iterability)
lists shorter than a hundred or so records that you always as a whole (e.g. the list of groups a user belongs to might always be fetched with that user and would average 5-10 items; probably no reason to keep it split apart)
Fetching the author is as simple as just adding the id to the URL:
var userId = 123;
new Firebase('https://INSTANCE.firebaseio.com/users/'+userId);
To fetch a list of posts belonging to a certain user, either maintain an index of that users' posts:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
fb.child('/my_posts/'+userId).on('child_added', function(indexSnap) {
fb.child('posts/'+indexSnap.name()).once('value', function(dataSnap) {
console.log('fetched post', indexSnap.name(), dataSnap.val());
});
});
A tool like Firebase.util can assist with normalizing data that has been split for storage until Firebase's views and advanced querying utils are released:
/posts/$post_id/...
/my_posts/$user_id/$post_id/true
var fb = new Firebase('https://INSTANCE.firebaseio.com');
var ref = Firebase.util.intersection( fb.child('my_posts/'+userId), fb.child('posts') );
ref.on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val();
});
Or simply store the posts by user id (depending on your use case for how that data is fetched later):
/posts/$user_id/$post_id/...
new Firebase('https://INSTANCE.firebaseio.com/posts/'+userId).on('child_added', function(snap) {
console.log('fetched post', snap.name(), snap.val());
});