React + Redux recommend saving data normalized and using selectors to get derived data. So in our store we save Users and Tags which have a many to many relationship.
type Store = {
entities: {
users: User[];
tags: Tag[];
userTagMapping: { userId: string, tagId: string }[]
}
}
And in our view we want to show some derived data for this many to many relation-ship. For example we want to calculate the total users with some tag or the online-users with some tag. Right now we solved this using rselect. The only problem is that calculating these things becomes quite tedious. For example we have a selector which returns a map from a tag-id to a list of users to show which users belong to this tag (and vice versa a selector from user-id to tag list).
const selectUsersPerTag = createSelector(
selectUsers, selectTags, selectUserTagMapping,
(users, tags, mapping) => {
let result = {};
for (const tag on tags) {
const isUserMappedToTag = user => ({userId, tagId}) => userId == user.id && tagId === tag.id
result[tag.id] = users.filter(user => mapping.some(isUserMappedToTag(user)))
}
return result
}
)
and in my opinion this looks quite ugly and is a bit too hard to read for my taste.
My questions are:
Are we understanding the recommendations correctly (to use normalization and selectors)?
Is using a map the correct way to process our data and show it in the view or is there a better way? I am asking because this basically copies our data (slightly modified) many times into the props of our React components
Is there a nicer way to do this mapping (which is basically a SQL like join)? Because I really don't like this imperative approach and would find a functional one much nicer.
Related
I am working through a simple logical problem, but I cannot seem to have things work smoothly. Let me share my most convincing code experiment and then I'll share some thoughts.
useEffect(() => {
firebase
.firestore()
.collection("someCollection")
.orderBy("date", "desc")
.onSnapshot(docs => {
let documents = []
if (canGetUpdatesFromFirestore.current) {
docs.forEach((doc) => {
documents.push(doc.data())
})
if(documents.length > 3) {
documents.splice(4, 0, {questionPostId: 0})
documents.splice(5, 0, {questionPostId: 1})
}
setAllQuestions(documents)
setUsers(documents)
}
})
if (searchValue.length > 2) {
canGetUpdatesFromFirestore.current = false;
functions.searchForSearchVal(searchValue, "Sexuality")
.then((result) => {
setAllQuestions(result);
})
} else {
canGetUpdatesFromFirestore.current = true
}
}, [searchValue])
function setUsers(docs){
let arrFinal = []
let copyOfAllQuestions = ""
for(let i = 0; i< docs.length; i++) {
console.log("HERE")
if (docs[i].postedBy) {
docs[i].ref.get().then(userFire => {
copyOfAllQuestions = {
...allQuestions,
...{hasPremium: userFire.data().hasPremium}
}
})
arrFinal.push(copyOfAllQuestions)
}
}
setAllQuestions(arrFinal)
}
Let me share some of my current state and what I am trying to accomplish.
I have a that display allQuestions. Each question data has a ref to its user document in firestore. For each question I need to check if that user hasPremium. How should I go about doing that the correct way?
The problem currently is that I can get the data from my Users collection through the ref, but I have to refresh my state in order for it all to show.
Could someone help me get on the right path / think correctly on this one please?
One approach that I put forward is to embrace data denormalization. That is, rather than putting references to other documents (Users) inside of the Questions document, put all the relevant user information directly into the Questions document.
This is antithetical to SQL database approaches, but that's okay because Firestore is "NoSQL". Embrace the anti-SQL-idity!!
Essentially, in your Question document you want to copy in whatever information is required in your app when working with a Question, and avoid doing "joins" by fetching other documents. You don't need to copy in all of the User document into a Question document - just the elements needed when your app is working with a Question.
For example, maybe in the question all you need is:
question: {
name: ...,
type: ...,
lastUpdated: ...,
postedBy: {
email: ...,
displayName: ...,
avatarUrl: ...,
hasPremium: true,
}
}
With data duplicated, you often need a mechanism to keep duplicate data up-to-date from its "source". So you might consider a Cloud Function trigger for onUpdate() of User documents, and when a relevant value is modified (email, displayName, avatarUrl, and/or hasPremium) then you would loop through all questions that are postedBy that user and update accordingly.
The rules-of-thumb here are:
all data needed for one screen/function in your app goes into a SINGLE document
NoSQL document stores are used where reads are frequent and writes are infrequent
NoSQL data stores (typically) do not have "joins" - so don't design your app to require them (which is what your code above is doing: joining Question and Users)
often you don't care about updating ALL instances of duplicated data (e.g. if a user updates their displayName today, should you update a Question they posted 3 years ago? -- different apps/business needs will give different answers)
I am currently working on a blogging app, in which users can create their own blogs and each blog has blogposts within that. I'm ideating about architecting a database that is scalable when each blog has a lot of blogposts.
So is it better to structure my database as this:
blog1 : {
blogname : 'blog1',
blogposts: [array of blogposts]
},
blog2 : {
blogname : 'blog2',
blogposts: [array of blogposts]
}
Or should I create a separate collection with all the blogposts, something like this:
blogpost1: {
id: 'blogpost1',
content: {blogpost content in json format}
},
blogpost2: {
id: 'blogpost2',
content: {blogpost content in json format}
}
and reference them in the blog collection.
I want to know which choice would be superior when there are a lot of blogposts. Because I remember reading somewhere in MongoDB docs that it's not recommended to have arrays within document that can grow beyond bounds, so approach #1 is not ideal, right?
When creating databases, I find it useful to think about the requests I would be making.
A blogging app user would want to search all blogs or find a blogger by some criteria.
In this case separate collections for bloggers and blogs would work best. Then structure your documents so that the bloggers link to their blogs and vice versa.
This can be done with Mongoose Schemas (https://mongoosejs.com/docs/index.html).
// models/blogger.js
const mongoose = require('mongoose')
const bloggerSchema = mongoose.Schema({
blogs: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'Blog'
}
],
name: String
})
bloggerSchema.set('toJSON', {
transform: (document, returnedObject) => {
const blogger = returnedObject
blogger.id = blogger._id.toString()
delete blogger._id
delete blogger.__v
}
})
module.exports = mongoose.model('Blogger', bloggerSchema)
Then use populate with your request:
// controllers/bloggers.js
const bloggersRouter = require('express').Router()
const Blogger = require('../models/blogger')
bloggersRouter.get('/', async (request, response) => {
const bloggers = await Blogger.find({}).populate(
'blogs', {
title: 1
}
)
response.json(bloggers.map(blogger => blogger.toJSON()))
})
module.exports = bloggersRouter
This way you don't have to add the blogs in their entirety to the blogger document, you can just include the title or anything else that you need on the bloggers initial view.
You could also think about limiting the length of a blog, so you can have more control over the data and then think about the options Joe suggested.
Why does it have to be one or the other?
Storing the blog posts in the same document as the blog is great as long as the individual posts are not very large, and there aren't very many of them.
Storing the posts in a separate collection is good for bigger posts and busy blogs but adds an additional query or lookup to retrieve.
I would think it is expected that your users' output will run the gamut from sparse to prolific, and individual posts will range from a few dozen bytes to many megabytes.
For small posts on not very active blogs, store the posts in the blog document for efficient retrieval.
For busy blogs, store them in an archive collection. Perhaps store the most recent couple of posts, or the most popular posts, in the blog document so you don't have to refer to the other collection every time.
You will also need to figure out how to split a post between documents. MongoDB has a 16MB limit on a single document, so if any of your users make huge posts, you'll need to be able to store them somewhere.
Your question as written seems to be asking whether it is better to follow a relation model or a strict document model. I think in reality neither is a perfect fit for this and a hybridized and flexible approach would work out better.
I've got an app using mobx-state-tree that currently has a few simple stores:
Article represents an article, either sourced through a 3rd party API or written in-house
ArticleStore holds references to articles: { articles: {}, isLoading: bool }
Simple scenario
This setup works well for simple use-cases, such as fetching articles based on ID. E.g.
User navigates to /article/{articleUri}
articleStoreInstance.fetch([articleUri]) returns the article in question
The ID is picked up in render function, and is rendered using articleStoreInstance.articles.get(articleUri)
Complex scenario
For a more complex scenario, if I wanted to fetch a set of articles based on a complex query, e.g. { offset: 100, limit: 100, freeTextQuery: 'Trump' }, should I then:
Have a global SearchResult store that simply links to the articles that the user has searched for
Instantiate a one-time SearchResult store that I pass around for as long as I need it?
Keep queries and general UI state out of stores altogether?
I should add that I'd like to keep articles in the stores between page-loads to avoid re-fetching the same content over and over.
Is there a somewhat standardized way of addressing this problem? Any examples to look at?
What you need might be a Search store which keeps track of following information:
Query params (offset, limit, etc.)
Query results (results of the last search)
(Optional) Query state (isLoading)
Then to avoid storing articles in 2 places, the query results should not use Article model, but reference to Article model. Anytime you query, the actual result will be saved in existing store ArticleStore, and Search only holds references:
import { types, getParent, flow } from 'mobx-state-tree'
const Search = types.model({
params: // your own params info
results: types.array(types.reference(Article))
}).views(self => ({
get parent() {
return getParent(self) // get root node to visit ArticleStore
}
})).actions(self => ({
search: flow(function*(params) {
this.params = params // save query params
const result = yield searchByQuery(query) // your query here
this.parent.articleStore.saveArticles(result) // save result to ArticleStore
this.results = getArticleIds(result) // extract ids here for references
})
}))
Hope it's what you are looking for.
In an architecture where objects have many complex relationships, what are some maintainable approaches to dealing with
Resolving Dependencies
Optimistic Updates
in react applications?
For example, given this type of schema:
```
type Foo {
...
otherFooID: String,
bars: List<Bar>
}
type Bar {
...
bizID: String,
}
type Biz {
...
}
```
A user might want to save the following ->
firstBiz = Biz();
secondBiz = Biz();
firstFoo = Foo({bars: [Bar({biz: firstBiz})]
secondFoo = Foo({bars: [Bar({biz: secondBiz})] otherFooId: firstFooId.id})
First Problem: Choosing real ids
The first problem with above is having the correct id. i.e in order for secondFoo to save, it needs to know the actual id of firstFoo.
To solve this, we could make the tradeoff, of letting the client choose the id, using something like a uuid. I don't see anything terribly wrong this this, so we can say this can work
Second Problem: Saving in order
Even if we determine id's from the frontend, the server still needs to receive these save requests in order.
```
- save firstFoo
// okay. now firstFoo.id is valid
- save secondFoo
// okay, it was able to resolve otherFooID to firstFoo
```
The reasoning here is that the backend must guarantee that any id that is being referenced is valid.
```
- save secondFoo
// backend throws an error otherFooId is invalid
- save firstfoo
// okay
```
I am unsure what the best way to attack this problem is
The current approaches that come to mind
Have custom actions, that do the coordination via promises
save(biz).then(_ => save(Bar).then(_ => save(firstFoo)).then(_ => save(second)
The downside here is that it is quite complex, and the number of these kinds of combinations will continue to grow
Create a pending / resolve helper
const pending = {}
const resolve = (obj, refFn) => {
return Promise.all(obj, refFn(obj));
}
const fooRefs = (foo) => {
return foo.bars.map(bar => bar.id).concat(foo.otherFooId);
}
pending[firstFoo].id = resolve(firstFoo, fooRefs).then(_ => save(firstFoo))
```
The problem with 2. is that it can cause a bunch of errors easily, if we forget to resolve or to add to pending.
Potential Solutions
It seems like Relay or Om next can solve these issues, but i would like something less high power. Perhaps something that can work in with redux, or maybe it's some concept I am missing.
Thoughts much appreciated
I have a JS/PHP implementation of such a system
My approach is to serialize records both on the client and server using a reference system
For example unsaved Foo1 has GUID eeffa3, and a second Foo references its id key as {otherFooId: '#Foo#eeffa3[id]' }
Similarily you can reference a whole object like this
Foo#eefa3:{bars['#Baz#ffg4', '#Baz#ffg5']}
Now the client-side serializer would build a tree of relations and model attributes like this
{
modelsToSave:[
'Foo#effe3':{
attribs:{name:'John', title:'Mr.'},
relations:{bars:['#Bar#ffg4']}
},
'Bar#ffg4':{
attribs:{id:5}
relations:{parentFoo:'#Foo#effe3'}
},
]
}
As you can see in this example I have described circular relations between unsaved objects in pure JSON.
The key here is to hold these "record" objects in client-side memory and never mutate their GUID
The server can figure out the order of saving by saving first records without "parent" dependencies, then records which depend on those parents
After saving, the server wil return the same reference map, but now the attribs will also include primary keys and foreign keys
JS walks the received map twice (first pass just update server-received attributes, second pass substitute record references and attribute references to real records and attributes).
So there are 2 mechanisms for referencing a record, a client-side GUID and a server-side PK
When receiving a server JSON, you match your GUID with the server primary key
I've been working with react/flux for a few weeks now and while I feel like I've got a pretty good handle on everything from async loading to updating props/states/etc, one thing that is still bothering me is how to handle save states.
For example, when loading data, I just have an isLoading boolean parameter in my store that gets passed to my components. But when I try and post an updated object to the server, it's trivial to:
fire the update action
display a "save in progress" state
but figuring out the result of the update action seems to be way more difficult.
Probably the most applicable post I've seen on this is in Fluxxor's async data guide, but their solution (adding/modifying a status property on the object) feels error-prone to me.
onAddBuzz: function(payload) {
var word = {id: payload.id, word: payload.word, status: "ADDING"};
this.words[payload.id] = word;
this.emit("change");
},
onAddBuzzSuccess: function(payload) {
this.words[payload.id].status = "OK";
this.emit("change");
},
onAddBuzzFail: function(payload) {
this.words[payload.id].status = "ERROR";
this.words[payload.id].error = payload.error;
this.emit("change");
}
Is there a better way to manage save states or is adding a status property to the object the best way?
I recommend keeping your "model stores" and "ui stores" separate, or at least accessed via different cursor positions in the same store. So, in your case you'd have one store or branch for your "word model" and then another store or branch for "word status."
While this adds some complexity in the form of breaking up logic across stores and reacting twice to the AddBuzz action, it ends up reducing (more) complexity by confining model store updates to true changes in model data and managing ui states separately.
EDIT
This is what Relay will more-or-less be doing, keeping persisted data in a separate self-managed store, leaving custom stores for nothing but ui state. Some other libraries like https://github.com/Yomguithereal/baobab also recommend this approach. The idea is that these are fundamentally different kinds of state. One is domain-specific persisted data and the other is ui-specific ephemeral application state.
It might look something like this:
model_store.js:
onAddBuzz: function(payload) {
var word = {id: payload.id, word: payload.word};
this.words[payload.id] = word;
this.emit("change");
}
ui_store.js:
onAddBuzz: function(payload) {
this.wordStates[payload.id] = 'ADDING';
this.emit("change");
}
my_view_controller.js:
// listen to both stores and pass down both words and wordStates to your views
my_word_view.js:
...
render: function() {
var word = this.props.word,
wordState = this.props.wordState;
...
}
If you don't want to emit two change events, have one waitFor the other and emit the change only from the second one.