I will implement my fan out model in Google's Firebase, but my question is only theoretical so the answer doesn't need to be in Firebase terms.
I am creating an app that I think should have a data structure similar to Tinder. The idea is only one post shows in your feed at a time; you then accept it or reject it and another one pops up and so on. My question is how exactly to structure the data so that it remains fast when the app scales up.
What I have right now is one node called "Posts" that contains every post that has ever been made. The app then queries for a post which is checked against the user Node of "viewedPosts" so that if the queried post has already been accepted/rejected by the user another one is queried until an unseen one is found. This obviously isn't a great solution because if there are a lot of posts, a query through them will be slow (especially if a lot of them have already been seen and the query has to be repeated multiple times).
I came across this article: The Firebase Blog: Client-side fan-out for data consistency which gave me the idea of creating a node inside each user which is "unseen posts" and every time a new post is uploaded by someone, to put it in the unseen folder for every user. This solves the problem on the side of the viewer, but to upload, one would have to download the list of all users in the app and then write to every single one of them.
So the question is, is there a middle ground between these which I can use to accurately do this?
Thank you.
EDIT:
Someone asked for my data structure:
{
"posts": {
"jkldsahjfkds": {
"title": "Simple Post",
"description": "This is my first post",
"numberOfImages": "2",
"price": "14.99",
"timestamp": "51782345",
"postedBy": "-hjd673bbewi7n",
"name": "Ryan Jacobs"
}
"-nisd7enskwes" : {...}
"-asdjfhk7385i" : {...}
"-sdfh49506ndk" : {...}
}
"users": {
"user1": {
"postsViewed": {
"-nisd7enskwes": 51784645,
"-sdfh49506ndk": 51782329
}
"postsLiked": {
"-sdfh49506ndk": 51782329
}
"userData": {
"name": "Albert Jones",
"bio": "Hi! Jow is everyone doing!?",
"location": "London"
}
}
}
}
Related
I created a Json Server Database like this:
"Time":
[
{
"id":1,
"name":
[
{
"id":1,
"checkin":
[
{
"id":1,
"date":"123",
"time":"123"
},
{
"id":2,
"date":"123",
"time":"123"
}
]
},
{
"id":2,
"checkout":
[
{
"id":1,
"date":"123",
"time":"123"
}
]
}
]
}
]
I don't want to get the entire Database and go through it. I just want to tell the Database where exactly my Object is and have it returned.
How would I call the call for example the first Check-in Object?
I use the Angular HttpClient like this:
this.http.get(endpoint, JSON.stringify(time), this.httpOptions))
So I need the Exact Endpoint in a format like: endpoint/id/id or similar
I imagined it like this: endpoint/time/1/1
With output:
[
{
"id":1,
"date":"123",
"time":"123"
}
]
If this is not possible please tell me anyways.
PS: The question from this thread is essentially the same as mine. Also the JSON documentation doesn't real help either, it just says you need custom routes for multilayer JSON strings but not how to implement these routes.
I'm not sure if I understand correctly where you are returning the data from. If you meant json-server, just look at the documentation (here) and then you could use an endpoint like "/posts?Id=2"
However, if you mean your own API, which does not have an endpoint that returns one record, e.g. by its ID, the only convenient solution is to create a service that will map the result from the server and return the desired value.
You can do all this in one place, but for clearer code, I recommend dividing it into:
service that will download data from the server
service that will map the data on the basis of a given parameter
component that will display the data
Below is a working example on Stackblitz.
example
Note that in the app-component I pass the ID 32 to the method from the mapping service as the parameter. The mapping service then calls a method that sends the request for all the data.
The important thing is that all data is returned to the application, not just one record. If the API you are using does not provide such an endpoint, it is not possible to return only one record.
Apparently a request like I wanted to call is still not possible. The only way to come close is to fake it with custom Routes and flattening the JSON structure like in this old thread.
My partner and I are working on some app, and recently we've come out with a problem merging frontend user data to the backend database data. That also raises the issue of how to keep this data consistent.
He's developing a frontend app using React and TypeScript, and I'm developing a backend API using Flask.
Before I found out that it would be pretty compute-intensive to merge this data, he wanted to send me the whole data he has every 5 seconds.
The user data looks like this. A user has one root folder, and there can be many folders nested in a folder. Also, there can be collections of texts nested in a folder.
interface Article {
id: string;
text: string;
}
interface Collection {
id: string;
name: string;
articles: Array<Article>;
}
interface Folder {
id: string;
name: string;
children: Array<Folder | Collection>;
}
Example:
{
"id": 1,
"name": "name1",
"children": [{
"id": 2,
"name": "name2",
"children": [{
"id": 3,
"name": "name3",
"articles": [{
"id": 4,
"text": "text"
}]
}]
}]
}
My MySQL db model looks like this:
DB Model
My partner thinks I should decline using SQL DB. Instead, we need to use MongoDB, so he will send to me a user data and I just put it in root field of every user in DB.
He insists that Backend should know nothing about user data model. The main argument is that if he would want to add some field there won't be any need to change Backend code, because he would write DB migrations himself and I just put it in the DB.
So, my main questions are:
How to keep data consistent between an SPA and DB, so there won't be any loading screens for user?
Should we really switch to MongoDB?
We have pretty nested data. Is it better to just put this Frontend user data model into DB, or should we use some kind of patches to DB model instead?
My partner says that I violate the single-responsibility principle according to SOLID, because I parse his user data model to store it in the DB or get it from there. Is it really so, or it's OK to do so?
I have a GQL cursor based pagination and have implemented the frontend in react using the apollo client library. This is my current server response, and things work well when requesting the next or the previous page.
{
"data": {
"users": {
"pageInfo": {
"hasNextPage": true,
"hasPreviousPage": false,
"startCursor": "1",
"endCursor": "6"
},
"edges": [
{
"node": {
"id": "1"
}
},
{
"node": {
"id": "6"
}
}
]
}
}
}
The issue which I have is;
I have a page count dropdown, 20, 50 and 100, which determines the number of record to be show on a page. So, if i start with 50 records per page, and let's say record number 74 is on page 2, when i switch to 20 records per page, the apollo client does not request the server, instead loads the data from the cache. This is good as caching helps avoid additional request, but now I do not know which page record 74 belongs to.
Solution I am after;
I am after a solution/library that can handle pagination in react which also takes into account the fact that the server wont be contacted always and therefore should keep track of the page.
So, after some deep dive into apollo and graphql cursors, there isn't a way for react to "guess" the start cursor for a particular page, and therefore is not practically possible for pagination to place nice when the cache is in play.
For this reason, I had to disable cache for this specific query using the following
const { loading, error, data } = useQuery(GET_GREETING, {
variables: { ... },
fetchPolicy: "network-only"
});
This will make sure the query is always fetched from the network and is not cached.
I'm experimenting with a Conversation where I would like to modify the output in a couple of different ways:
different output for speech or text
different output depending on the tone of the conversation
It looks like I can add extra output details which make it through to the client ok. For example, adding speech alongside text...
{
"output": {
"speech": {
"Hi. Please see my website for details."
},
"link": "http://www.example.com",
"text": {
"Hi. Please see http://www.example.com for details."
}
}
}
For the tone, I wondered about making up a custom selection policy, unfortunately it seems to treat it the same as a random selection policy. For example...
{
"output": {
"text": {
"values": [
"Hello. Please see http://www.example.com for more details.",
"Hi. Please see http://www.example.com for details."
]
},
"append": false,
"selection_policy": "tone"
}
}
I could just add a separate tone-sensitive object to output though so that's not a big problem.
Would there be any issues adding things to output in this way?
You can definitely use the output field to specify custom variables you want your client app to see with the benefit that these variables will not persist across multiple dialog rounds (which they would if you would add them to the context field).
Now currently there is no "easy" way how to define your custom selection policy (apart from the random and sequential supported by the runtime right now) - but you could still return an array of possible answers to the client app with some attribute telling the client app which selection policy to use and you would implement this policy in the client app.
Say I have an API endpoint at /users and another one at /cars. Naturally, a GET request to either of them will get all users or cars available. Now, a GET /users/74/cars should return all cars belonging to user 74.
But my app has many models related to cars, not just users, so more endpoints exist like /shops/34/cars and /mechanics/12/cars. For simplicity, I want all PUT/PATCH requests to be made to the main /cars endpoint.
At the moment of performing the save, Restangular will by default do a PUT request to the endpoint through which the item was loaded. But that endpoint do not exist.
It also provides a nice Restangular.setParentless(['cars']) method that will discard the first part of the url. However, I don't want to do this globally, but specifically for a particular element.
The neatest would actually do it globally, but restrict it for a specific method, like: Restangular.setParentless(['cars'], ['PUT']).
Anything like that around? Or am I overcomplicating it?
So far I tried stuff I don't like:
delete car.parentResource;
I would recommend using self reference links. A self reference link is a link which stores the route which should be used for GET/PUT/DELETE etc. on the item, rather than the URL from which it was pulled.
Example, update the mileage on one of user id 74's cars:
First, configure Restangular to look for a self link property called 'self' on each object.
RestangularProvider.setRestangularFields({
selfLink: 'self'
});
Next, make your call to get the cars. I'll assume that you have already modified your API to return a property called 'self' on each object that has the URL to it's proper API endpoint.
GET /users/74/cars
[
{
"id": 12,
"model": "Camaro",
"make": "Chevrolet",
"year": 1969,
"color": "red",
"odometer": 67294,
"license": "ABC12345",
"self": "/cars/12"
},
{
"id": 14,
"model": "Gallardo",
"make": "Lamborghini",
"year": 2015,
"color": "black",
"odometer": 521,
"license": "XYZ34567",
"self": "/cars/14"
}
]
We want to add some miles to one of them, and then save it. The entire Restangular code would look like:
Restangular.one('users', 74).all('cars').getList().then(function(cars){
cars[1].odometer = 613;
cars[1].put();
});
The PUT will go to /cars/14 instead of /users/74/cars/14. This is very useful for applications like yours that relate models as a graph rather than a strict hierarchical tree.