Say I am building an API with a call that returns a collection of cities, each of which has a relationship to a state. A state has many cities, but a city has only one state.
I can imagine flattening the relationship and obscuring the underlying structure of the data like this,
{ cities : [
{ id: 1,
name: "Los Angeles",
state: "CA" }
]}
Or I can imagine structuring the JSON such that the relationship between cities and states is apparent,
{ cities : [
{ id: 1,
name: "Los Angeles",
state: { id: 1,
name: "CA" }
]}
The consumer of the API, as of now, only ever needs to know the name of the state. He does not need to know its ID or a way to get more information about the state. What are the pros and cons in structuring the JSON in either way?
In my opinion you should not add useless information to you api, but as #kgb said if your api is prone to be expanded you should design it that way. You asked about the relationship between the cities and states, and in my opinion this relationship is already defined in both of them.
So if you are 100% sure your api will not expand the state functionality, you should go with option 1. If there is only a slight chance, i propose you do this:
{ cities : [{
id: 1,
name: "Los Angeles",
state: { name: "CA" }
}]
}
That depends on the other consumers. Do you have any? Are you planning to?
An API is a machine interface, it is equally easy for the consumer's developer to use both structures. If the "state" entity is not a compound entity(no usable properties except name), it might me a good idea to show just the name, not a structure with the id.
If there is a possibility that state id might be useful in future, or a possibility that a new property is going to be added to state entity, then you should use the second approach from the start. Changing API in any way breaks the software already written, so changing it will make you support two different versions. Change between approaches 1 and 2 is not backward-compatible.
I would go with approach 2. It is not much more complex than 1 and leaves a possibility to extend state entity.
There is also a third approach. But it is noticeably more complex(and more extendable). Return state id only and create a method for state entity retrieval.
Related
I am new to Redux and RTKQ and believe that I have missed something. The use case I am going to describe just seems so standard that I cannot believe there isn't an inbuilt solution.
The High Level View:
Fetch data from the backend.
The user then updates parts of that data.
At some point the user clicks a button to persist the changes.
A bit more details
The fetched data is a list of profiles. These profiles have a name and a list of contacts.
Here is an example:
[
{
uuid: '<some id>',
name: '<SomeName>',
contacts: [
{
uuid: '<some id>',
label: 'Email',
type: 'email',
value: '<some email>',
},
],
},
{
uuid: '<some other id>',
name: '<SomeName>',
contacts: [
{
uuid: '<some other id>',
label: 'Email',
type: 'email',
value: '<some email>',
},
],
},
];
The user can add new profiles, alter existing profiles or contacts and can add new contacts to profiles.
As is typical in react the data structure is displayed with the help of various components.
Here is a tree of my current setup:
EditProfiles (handles saving/deletion of profiles)
-> ProfileList
-> ProfileListItem (transiently stores changes to the name)
-> EditContacts (transiently stores updates to channels or the creation of new channels)
-> EditContactList
-> EditContactListItem
The EditProfiles component fetches the data from the backend and sends any updates to the server (when the save or delete button is clicked).
But how do you correctly persist the updates before sending them?
After fetching the data RTKQ stores it. So far so good.
Now the user wants to change part of the fetched data. And here the trouble starts. Where do I store the changes before sending them?
I cannot store them as react state bc it's a complex object and there are all kinds of pitfalls if done incorrectly.
I could update them in RTKQ but there any changes would be overriden if the query is sent again for any reason.
Or, and this is what I ultimately went with: I can create a new slice and persist any changed data there. Because it creates a parallel structure the user's changes are save even if the query is sent again. Any changes to this state also cause a rerender (if listened to) no matter the complexity of the object.
But this has drawbacks of it's own:
I have to manually handle this slice, meaning I must make sure to add changes and also to delete them, after the user persisted them. I lose a lot of the neat automatisation of RTKQ.
I have to store duplicated data. Most of the time users only want to change part of a object, I still need to store all of it.
Because of 1. there is a lot of error potential that I do not have when just using RTKQ.
So the question basically is: Is there a more RTKQ way of solving this I am just unaware of?
You didn't miss anything - the job of RTKQ is to mirror the data you have on the server, and to keep that as sync as possible. It does only that one thing.
I would recommend using some kind of form library (formik, react hook form, react final form, ...) to handle that state while the user is editing it - or, if local state is really not an option, to go for a separate slice.
I'm writing an sort of project management app that'll have a large number of "tasks" with a lot of properties within them that'll be rendered throughout the app. I am looking at using React Query to prefetch, cache locally, and update this data from the server.
One key architectural thing I want to get right is that when I query or mutate Tasks[123] that I affect a single underlying object in the state and not get stuck with duplicate data everywhere. On first glance React Query seems to be perfect for this job if the Query Keys are setup right. However in their examples they don't seem to do this (or I'm failing to understand).
In their Basic Example they fetch some Posts on start and query using queryClient.getQueryData(["post", post.id]). As far as I can tell this is causing the data to be duplicated if I look at the provided ReactQueryDevtools window in the example.
Am I correct in thinking the example should be rewritten to use something like queryClient.getQueryData(["posts", {id: post.id} ])?
That is indeed the way I am setting up my query keys, so that I can do: queryClient.invalidateQueries(['posts']) and it invalidates all of them. But sometimes, you need more fine granular control. If that's the case, I'd do:
["posts", "list", { filter: "all" }]
["posts", "list", { filter: "published" }]
["posts", "detail", 1]
["posts", "detail", 2]
that way, I can still tackle everything with ["posts"], all lists with ["posts", "list"], all details with ["posts", "detail"] and a specific detail with ["posts", "detail", id] etc.
It is also good practice to have a queryKeyFactory to create those keys, something like:
const postKeys = {
prefix: "posts",
lists: [postKeys.prefix, "list"],
list: (filter) => [...postKeys.lists, { filter }],
details: [postKeys.prefix, "detail"],
detail: (id) => [...postKeys.details, id]
}
Of course, I'm talking about "at scale" here. None of this is really needed for a todo app :)
Intro
I have a FireStore database similar to a social media db, with 3 collections Users, Events, and EventUpdates. My goal is to create a feed with eventUpdates created by me and my friends. So i have to expand my database with friendship connections. But i struggle with 3 problems, and hopefully somebody here can push me in the right direction to solve these.
Problem/Question 1:
I added username and user image to the EventUpdate model so it's easier to query. I've heard denormalise is the way to go in a NoSQL database. But if a user updates his user image, i've to update all eventUpdates created by that user. Sounds like something you don't wanne do. But is there a better way to do this?
Problem/Question 2:
How can i create a data structure that is optimised for performing the following query: get eventUpdates from me and my friends ordered by date.
Problem/Question 3:
How to store likes? I can keep a counter in a eventUpdate. But this becomes a problem when i denormalise eventUpdates (see current solution underneath EDIT)..
Data structure example .
{
"users": {
"1": { "name": "Jack", "imageUrl": "http://lorempixel.nl" }
},
"events": {
"A": {
"name": "BeerFestival",
"date": "2018/09/05",
"creatorId": "1"
}
},
"eventUpdates": {
"1": {
"timestamp": "13243543",
"creatorId: "1",
"creatorName": "Jack",
"creatorImageUrl": "http://lorempixel.nl",
"eventId": "A",
"message": "Lorem ipsum"
}
}
}
EDIT
OK, after some trial and error i ended up with the following structure. This structure seems work, but my problem with this solution is that i need to make a lot of write calls to update a single eventUpdate because of all the copies in each feed (1000 followers means 1000 copies). And it looks like i need to do that a lot.
I would like for example to add a like button to each event update. This trigger an update on all EventUpdate copies. For me it looks like firebase is not suited for my project and i'm thinking of replacing it with a SQL DB, or can anyone here change my mind with a better solution?
{
"users": {
"user1": { "name": "Jack",
"imageUrl": "http://lorempixel.nl",
"followers": ["user1"]
}
},
"feeds": {
"user1": {
"eventUpdates": {
"1": {
"timestamp": "13243543",
"creatorId: "1",
"eventId": "A",
"message": "Lorem ipsum"
}
},
"following": {
"user1": {
"name": "Jack",
"imageUrl": "http://lorempixel.nl",
"followers": ["user1"]
}
}
},
"events": {
"A": {
"name": "BeerFestival",
"date": "2018/09/05",
"creatorId": "1"
}
}
}
I added username and user image to the EventUpdate model so it's easier to query. I've heard denormalise is the way to go in a NoSQL database.
That's right, denormalization and is a common practice when it comes to Firebase. If you are new to NoQSL databases, I recommend you see this video, Denormalization is normal with the Firebase Database for a better understanding. It is for Firebase realtime database but same rules apply to Cloud Firestore.
But if a user updates his user image, i've to update all eventUpdates created by that user. Sounds like something you don't wanne do. But is there a better way to do this?
Yes, that's also correct. You need to update all the places where that image exists. Because you have chosen google-cloud-firestore as a tag, I recommend you see my answer from this post because in case of many write operations, Firestore might be a little costly. Please also see Firestore pricing plans.
Regarding Firestore, instead of holding an entire object you can only hold a reference to a picture. In this case, there is nothing that you need to update. It's always a trade between these two techniques and unfortunately there is no way between. You either hold objects or only references to objects. For that, please see my answer from this post.
How can i create a data structure that is optimised for performing the following query: get eventUpdates from me and my friends ordered by date.
As I see, your schema is more a Firebase realtime database schema more than a Cloud Firestore. And to answer to your question, yes you can create. So talking about Firestore, you can create a collection named eventUpdates that can hold eventUpdate objects and to query it according to a timestamp, a query like this is needed:
FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference eventUpdatesRef = rootRef.collection("eventUpdates");
Query query = eventUpdatesRef.orderBy("timestamp", Query.Direction.ASCENDING);
But please note that the timestamp field should be of type Date and not long. Please also take a look at my answer from this post, to see how you can add a date property in a Cloud Firestore database.
How to store likes? I can keep a counter in a eventUpdate. But this becomes a problem when i denormalise eventUpdates (see current solution underneath EDIT)
You can simply add likes but I recommend you see the last part of my answer from this post. So you might consider adding that count in a Firebase realtime database rather than in Cloud Firestore. Both databases work very well together.
This structure seems work, but my problem with this solution is that i need to make a lot of write calls to update a single eventUpdate because of all the copies in each feed (1000 followers means 1000 copies). And it looks like i need to do that a lot.
You might also take a look at my answer from this post.
For me it looks like firebase is not suited for my project and i'm thinking of replacing it with a SQL DB, or can anyone here change my mind with a better solution?
I don't think this way. There are many apps out there that have the exact mechanism as yours and are working very well.
If you want your feed items to be in sync with the real users data (new profile image when the user changes it for example) you can simply store the user ID in the eventUpdate document. This way you don't have to keep them in sync manually, and every time you have to display the item in the feed you could simply fetch user data, and easily query many eventUpdates on userId and created_at fields ( assuming you have them ).
To implement likes in your feed the solution depends on a bunch of things like traffic volume.
The simplest way is to update a likes field with a transaction, but Firestore has a maximum updates frequency on a single document of 1 second. Plus, a transaction can easily fail if more than 5 transactions are trying to update the same document.
To implement a more solid likes system take a look at this page from the official Firebase docs.
Firestore has a different approach to the NoSQL world. Once you know the data you will use (as You already do) there are some very important points about what architecture the data will have. And It depends a lot about how the data grows, what kind of queries you will need and how often you will use them. Some cases You can create a root collection that aggregates data and queries might be easier.
There is a great video from Firebase Channel that might help. Check it out!
How to Structure Your Data | Get to Know Cloud Firestore #5
https://www.youtube.com/watch?v=haMOUb3KVSo
[UPDATED] December 26th
Others videos that might help to model and query your data is these videos:
How to Connect Firebase Users to their Data - 3 Methods
https://www.youtube.com/watch?v=jm66TSlVtcc
How to NOT get a 30K Firebase Bill
https://www.youtube.com/watch?v=Lb-Pnytoi-8
Model Relational Data in Firestore NoSQL
https://www.youtube.com/watch?v=jm66TSlVtcc
I am trying to normalize my state and can't quite figure out how to handle partial & complete resources, namely whether should they share the same slice of state or be relegated to their own slices.
A little more info:
When I make a request for a list of things, the server responds with a list of partials.
[ { id: 1, name: "thing1"},
{ id: 2, name: "thing2"},
...
{ id: N, name: "thingN"} ]
When I make a request for a single thing, the server responds with a more complete object.
{
id: 1,
name: "thing1",
img_url: "url1",
description: "description1",
... and more fields
}
Both the partial list and detailed information need to be present at the same time.
So should these partial and complete things all be in the same slice of state?
If so what is a best practice for managing whether a given thing has been partially or completely fetched?
Thanks for the help!
What's the purpose of fetching a single item? Probably showing it? Why not set up your state like:
this.state = {
allThings: [],
singleThing: {}
};
And fill it up accordingly.
You can go about this a few ways:
Have an incomplete object like so:
{
id: 1,
name: 'thing1',
detail: null //this would be a JS object when populated
}
How you would use this: in a component that uses the detail view, you check for null on the state. This means that you can lazy load details if you will need multiple in the future, as well as not have to retrieve them again if needed. However, this isn't great if information needs to be updated (you would have to resolve both null and the last retrieve date, can be risky for creating stale data).
Manage the two as completely different nodes on your state, where the detail view is either an array or just a singular object. This approach would work better if you want to be always updating the object on a lifecycle event (e.g. on a component mount you always want to refresh the data).
My personal preference is #1 - I use Firebase a lot and this works really well as the real time DB handles when the data should be updated, and then having them in the state means I can display data super fast to the user if it's pre-loaded.
If you are using an SQL database, it's very straightforward to develop a user interface for CRUD operations. Since the schema is defined, it's obvious how many inputs you need in a form, etc.
But when using a schema-less NoSQL approach for storage, how do you build interfaces since you don't know exactly what to expect from the types of data being stored? For example if you had a database of cars:
var cars = [
{ model: "BMW", color: "Red", manufactured: 2016 },
{ model: "Mercedes", type: "Coupe", color: "Black", manufactured: “1-1-2017” }
];
If you needed to create a user interface so you could access and edit this data, you have no clue how many inputs you need since there is no schema. How do UI developers solve this problem?
Do you have a bunch of if statements to test if every possible attribute exists in the record before showing the proper inputs?
// psuedo code
if ($car.hasKey("model") ) {
// Show the "Model" input form element
}
if ($car.hasKey("type") ) {
// Show the "Type" input form element
}
if ($car.hasKey("color") ) {
// Show the "Color" input form element
}
if ($car.hasKey("manufactured") ) {
// Show the "Manufactured" input form element
}
If you needed to create a user interface so you could access and edit this data, you have no clue how many inputs you need since there is no schema. How do UI developers solve this problem?
You solve this by reasoning from feature requirements. Emphatically, you do not try to generate forms (automatically or otherwise) from schemas: that is a recipe for poor UX even if you do have a comprehensive, canonical and unequivocal schema to hand.
Instead: you know what your 'use cases' are (you ask users) and then you build exactly those.
So the question becomes:
What do you do when your data item/instance does not contain a particular object/field/key which you did expect?
What do you do when your instance contains fields which you do not know what to do with?
The answer for #1 is pretty straightforward, and basically just the same as dealing with schema changes: assume/present sane defaults or handle null values gracefully. That is: permit your users to add such fields later where they make sense and do not choke on objects that lack them.
The answer for #2 is more complicated and it is going to depend heavily on the application and its environment (for example: is it the sole consumer of the data, or are there other consumers to consider as well). One option could be normalisation: you prune such extraneous fields on saving, so objects become normalised over time as they are updated by the users. An alternative could be preservation: you keep any fields you do not know as-is, and you take pains to preserve them through every layer of your application.