My partner and I are working on some app, and recently we've come out with a problem merging frontend user data to the backend database data. That also raises the issue of how to keep this data consistent.
He's developing a frontend app using React and TypeScript, and I'm developing a backend API using Flask.
Before I found out that it would be pretty compute-intensive to merge this data, he wanted to send me the whole data he has every 5 seconds.
The user data looks like this. A user has one root folder, and there can be many folders nested in a folder. Also, there can be collections of texts nested in a folder.
interface Article {
id: string;
text: string;
}
interface Collection {
id: string;
name: string;
articles: Array<Article>;
}
interface Folder {
id: string;
name: string;
children: Array<Folder | Collection>;
}
Example:
{
"id": 1,
"name": "name1",
"children": [{
"id": 2,
"name": "name2",
"children": [{
"id": 3,
"name": "name3",
"articles": [{
"id": 4,
"text": "text"
}]
}]
}]
}
My MySQL db model looks like this:
DB Model
My partner thinks I should decline using SQL DB. Instead, we need to use MongoDB, so he will send to me a user data and I just put it in root field of every user in DB.
He insists that Backend should know nothing about user data model. The main argument is that if he would want to add some field there won't be any need to change Backend code, because he would write DB migrations himself and I just put it in the DB.
So, my main questions are:
How to keep data consistent between an SPA and DB, so there won't be any loading screens for user?
Should we really switch to MongoDB?
We have pretty nested data. Is it better to just put this Frontend user data model into DB, or should we use some kind of patches to DB model instead?
My partner says that I violate the single-responsibility principle according to SOLID, because I parse his user data model to store it in the DB or get it from there. Is it really so, or it's OK to do so?
Related
I created a Json Server Database like this:
"Time":
[
{
"id":1,
"name":
[
{
"id":1,
"checkin":
[
{
"id":1,
"date":"123",
"time":"123"
},
{
"id":2,
"date":"123",
"time":"123"
}
]
},
{
"id":2,
"checkout":
[
{
"id":1,
"date":"123",
"time":"123"
}
]
}
]
}
]
I don't want to get the entire Database and go through it. I just want to tell the Database where exactly my Object is and have it returned.
How would I call the call for example the first Check-in Object?
I use the Angular HttpClient like this:
this.http.get(endpoint, JSON.stringify(time), this.httpOptions))
So I need the Exact Endpoint in a format like: endpoint/id/id or similar
I imagined it like this: endpoint/time/1/1
With output:
[
{
"id":1,
"date":"123",
"time":"123"
}
]
If this is not possible please tell me anyways.
PS: The question from this thread is essentially the same as mine. Also the JSON documentation doesn't real help either, it just says you need custom routes for multilayer JSON strings but not how to implement these routes.
I'm not sure if I understand correctly where you are returning the data from. If you meant json-server, just look at the documentation (here) and then you could use an endpoint like "/posts?Id=2"
However, if you mean your own API, which does not have an endpoint that returns one record, e.g. by its ID, the only convenient solution is to create a service that will map the result from the server and return the desired value.
You can do all this in one place, but for clearer code, I recommend dividing it into:
service that will download data from the server
service that will map the data on the basis of a given parameter
component that will display the data
Below is a working example on Stackblitz.
example
Note that in the app-component I pass the ID 32 to the method from the mapping service as the parameter. The mapping service then calls a method that sends the request for all the data.
The important thing is that all data is returned to the application, not just one record. If the API you are using does not provide such an endpoint, it is not possible to return only one record.
Apparently a request like I wanted to call is still not possible. The only way to come close is to fake it with custom Routes and flattening the JSON structure like in this old thread.
I am thinking to move to graphQL from a standard REST API architecture. Currently I have a data wrapper which preloads a dataset after initialization with user info, and other meta data.
Is there a way for apollo client to build a single query requested by multiple components and send it to graphQL server after the components are loaded instead of sending separate queries for each component and form a data layer that is necessary for application to operate ?
What are the possibilities eventually other options to do this and make it better with GraphQL then the current implementation.
Queries in Apollo client have an option fetchPolicy with default value cache-first which means Apollo will try to read from your local cache first before attempting a network fetch. If you have multiple components requesting data that has been fetched already then no network calls will happen. Just ensure to return the id field of the requested data in your query even if your component doesn't use it to ensure proper caching.
Let's assume you have the following graphql schema:
type Query {
question(questionId: Int!): Question
tag(tagId: Int!): Tag
}
type Question {
questionId: Int!
content: String!
}
type Tag {
tagId: Int!
tagName: String!
}
If you want to retrieve 2 objects in one request you can write the following qraphql request:
{
question(questionId: 1) {
questionId
content
}
tag(tagId:1){
tagId
tagName
}
}
As output, you will receive the following JSON:
{
"data": {
"question": {
"questionId": 1,
"content": "how to code in js?"
},
"tag": {
"tagId": 1,
"tagName": "js"
}
}
}
I will implement my fan out model in Google's Firebase, but my question is only theoretical so the answer doesn't need to be in Firebase terms.
I am creating an app that I think should have a data structure similar to Tinder. The idea is only one post shows in your feed at a time; you then accept it or reject it and another one pops up and so on. My question is how exactly to structure the data so that it remains fast when the app scales up.
What I have right now is one node called "Posts" that contains every post that has ever been made. The app then queries for a post which is checked against the user Node of "viewedPosts" so that if the queried post has already been accepted/rejected by the user another one is queried until an unseen one is found. This obviously isn't a great solution because if there are a lot of posts, a query through them will be slow (especially if a lot of them have already been seen and the query has to be repeated multiple times).
I came across this article: The Firebase Blog: Client-side fan-out for data consistency which gave me the idea of creating a node inside each user which is "unseen posts" and every time a new post is uploaded by someone, to put it in the unseen folder for every user. This solves the problem on the side of the viewer, but to upload, one would have to download the list of all users in the app and then write to every single one of them.
So the question is, is there a middle ground between these which I can use to accurately do this?
Thank you.
EDIT:
Someone asked for my data structure:
{
"posts": {
"jkldsahjfkds": {
"title": "Simple Post",
"description": "This is my first post",
"numberOfImages": "2",
"price": "14.99",
"timestamp": "51782345",
"postedBy": "-hjd673bbewi7n",
"name": "Ryan Jacobs"
}
"-nisd7enskwes" : {...}
"-asdjfhk7385i" : {...}
"-sdfh49506ndk" : {...}
}
"users": {
"user1": {
"postsViewed": {
"-nisd7enskwes": 51784645,
"-sdfh49506ndk": 51782329
}
"postsLiked": {
"-sdfh49506ndk": 51782329
}
"userData": {
"name": "Albert Jones",
"bio": "Hi! Jow is everyone doing!?",
"location": "London"
}
}
}
}
[I am copying the exact question from last year which still has not been answered, and extend it a bit]
I'm developing a REST api using node.js + mongoose (MEAN.io). I have a model which has few nested arrays within it. I have a put endpoint to update the document. However to add an object into the sub-array, do I need a separate endpoint or is there a way to use the same put endpoint?
companyName: String,
city: String,
pincode: Number,
managers: [{
name: String,
dob: Date,
gender: String,
highestEducation: String,
email: String,
phoneNumbers: [{phoneNumber: Number}],
}],
I have an endpoint ../api/customer for updating the document. It replaces the existing document with the json that I'm supplying. So, if I want to add a manager (not replace the existing manager), do I need a separate endpoint just for this? What is the optimized solution?
At the moment, I have created another put api like this:
../api/customer/:id/managers
and at the server side of api, I am using customer.managers.push(updates); instead of _.merge(customer, updates).
Your API design is ultimately your choice. Personally I prefer to keep things quite shallow, maybe one relationship deep if that. Generally I will have an endpoint per resource.
In terms of your example about the new phone number for existing manager for an existing customer. A general flow using a somewhat shallow api could be:
Frontend:
Goto URL site/users/1 (this loads up the basic view for the user).
API:
Action: GET
Url: site/api/v1/users/1
Desc: SHOW user 1
Response: { id: 1, name: "bob", age: 22 }
API:
Action: GET
Url: site/api/v1/managers?query=user_id=1
Desc: INDEX all manager for user 1
Response: { [ { id: 55, name: "mana1", phone: 4444 }, { id: 66, name: "mana2", phone: 333 }, .. ] }
Frontend: List managers as a bunch of forms with some hidden id's
Frontend: Enter phone number hit save
API:
Action: PATCH
Url: site/api/v1/managers/55
BODY: { phone: 9898 }
Desc: PATCH manager 55 with new phone data
Response: { ..updated data.. }
I personally really like this for API's:
https://github.com/WhiteHouse/api-standards
It's not rest, but its simple, repeatable and usable.
Say I have an API endpoint at /users and another one at /cars. Naturally, a GET request to either of them will get all users or cars available. Now, a GET /users/74/cars should return all cars belonging to user 74.
But my app has many models related to cars, not just users, so more endpoints exist like /shops/34/cars and /mechanics/12/cars. For simplicity, I want all PUT/PATCH requests to be made to the main /cars endpoint.
At the moment of performing the save, Restangular will by default do a PUT request to the endpoint through which the item was loaded. But that endpoint do not exist.
It also provides a nice Restangular.setParentless(['cars']) method that will discard the first part of the url. However, I don't want to do this globally, but specifically for a particular element.
The neatest would actually do it globally, but restrict it for a specific method, like: Restangular.setParentless(['cars'], ['PUT']).
Anything like that around? Or am I overcomplicating it?
So far I tried stuff I don't like:
delete car.parentResource;
I would recommend using self reference links. A self reference link is a link which stores the route which should be used for GET/PUT/DELETE etc. on the item, rather than the URL from which it was pulled.
Example, update the mileage on one of user id 74's cars:
First, configure Restangular to look for a self link property called 'self' on each object.
RestangularProvider.setRestangularFields({
selfLink: 'self'
});
Next, make your call to get the cars. I'll assume that you have already modified your API to return a property called 'self' on each object that has the URL to it's proper API endpoint.
GET /users/74/cars
[
{
"id": 12,
"model": "Camaro",
"make": "Chevrolet",
"year": 1969,
"color": "red",
"odometer": 67294,
"license": "ABC12345",
"self": "/cars/12"
},
{
"id": 14,
"model": "Gallardo",
"make": "Lamborghini",
"year": 2015,
"color": "black",
"odometer": 521,
"license": "XYZ34567",
"self": "/cars/14"
}
]
We want to add some miles to one of them, and then save it. The entire Restangular code would look like:
Restangular.one('users', 74).all('cars').getList().then(function(cars){
cars[1].odometer = 613;
cars[1].put();
});
The PUT will go to /cars/14 instead of /users/74/cars/14. This is very useful for applications like yours that relate models as a graph rather than a strict hierarchical tree.