I have a get route on my server-side that responds with two random records from MongoDB. I currently have a couple records hard-wired as excluded records that will never be returned to the client.
app.get("/api/matchups/:excludedrecords", (req, res) => {
const ObjectId = mongoose.Types.ObjectId;
Restaurant.aggregate([
{
$match: {
_id: { $nin: [ObjectId("5b6b5188ed2749054c277f95"), ObjectId("50mb5fie7v2749054c277f36")] }
}
},
{ $sample: { size: 2 } }
]).
This works, but I don't want to hard-wire the excluded records, I want to dynamically pass the ObjectIds from the client side. I want the user to be able to exclude multiple records from the random query. I have an action creator that pushes the ObjectId the user wishes to exclude through a reducer so that it becomes part of the store, and the store is an array that includes all the ObjectIds of the records the user wishes to exclude. Here's my action that fetches the random records, taking the excluded records from the store as an argument:
export function fetchRecords(excludedrecords) {
const excludedarray = JSON.stringify(excludedrecords); // Don't currently have this, but feel like I need to.
const request =
axios.get(`http://localhost:3000/api/matchups/${excludedarray}`);
return {
type: "FETCH_MATCHUP_DATA",
payload: request
};
}
I have the sense I need to stringify the array on the client side and parse it on the server side, but I'm not sure how. I've started something like:
app.get("/api/matchups/:excludedrecords", (req, res) => {
const excludedRecords = JSON.parse(req.params.excludedrecords);
const ObjectId = mongoose.Types.ObjectId;
Restaurant.aggregate([
{
$match: {
_id: { $nin: [excludedRecords] } //
}
},
But how do I get ObjectId() to wrap around each record number that is passed in params? I've tried inserting the number on the client side into a template string, like ObjectId('${excludedrecord}'), which results in me passing an array that looks like what I want, but when it gets stringified and parsed it doesn't quite work out.
Sorry if this question is a bit messy.
First of all you should pass the array of string as a body of the http request and not as a part of the url. You do not pass an array parameter as part of the url or in the query string.
Second, you must transcode the to $nin: [ObjectId(excludedRecord1), ObjectId(excludedRecord2)]
at the server side.
Hope it helps!
Related
I am not too confident working with Firestore and have trouble with more complex API calls to get data. Usually I use SQL backends in my apps.
For the section that I am working on, I would like to combine three collections to get an array of ToDos with the involved users and the category the current user labelled this ToDo with. Every involved person can label the ToDo like they prefer, which makes things a little more complicated. Broken down the collections are structured as follows.
todo: Firestore Database Document
{
title: string,
involved: string[], //user ids
involvedCategory: string[] //category ids mapped by index to involved
}
(I tried to have an array of objects here instead of the two arrays, but it seems I would not be able to query the array for the current user´s ID, like mentioned here, so this is a workaround)
category: Firestore Database Document
{
title: string,
color: string
}
user: Firebase Authentication User
{
uid: string,
displayName: string,
photoURL: string,
...
}
THE GOAL
An array of ToDo items like this:
{
id: string,
title: string,
involved: User[],
category?: {
title: string,
color: string
}
}
As I am working with TypeScript, I created an interface to use a converter with. My code looks like this so far:
import {
DocumentData,
FirestoreDataConverter,
WithFieldValue,
QueryDocumentSnapshot,
SnapshotOptions,
query,
collection,
where,
} from 'firebase/firestore'
import { store } from '../firebase'
import { useCollectionData } from 'react-firebase-hooks/firestore'
import { User } from 'firebase/auth'
import { useCategories } from './categories'
import { useAuth } from '../contexts/AuthContext'
interface ToDo {
id: string
title: string
involved: User[]
category?: {
title: string
color: string
}
}
const converter: FirestoreDataConverter<ToDo> = {
toFirestore(todo: WithFieldValue<ToDo>): DocumentData {
return {} //not implemented yet
},
fromFirestore(
snapshot: QueryDocumentSnapshot,
options: SnapshotOptions
): ToDo {
const data = snapshot.data(options)
return {
id: snapshot.id,
title: data.title,
category: undefined, //?
involved: [], //?
}
},
}
export function useToDos() {
const { currentUser } = useAuth()
const { categories } = useCategories() //needed in converter
const ref = query(
collection(store, 'habits'),
where('involved', 'array-contains', currentUser.uid)
).withConverter(converter)
const [data] = useCollectionData(ref)
return {
todos: data,
}
}
Is there any way I can do this? I have a Hook that returns all of the user´s categories, but I obviously can´t call that outside the
useToDos-Hook. And creating the const in the hook does not help, either, as it results in an infinite re-render.
I know this is a long one, but does anyone have tips how I could approach this? Thanks in advance ^^
UPDATE:
I had to make two small adjustments to #ErnestoC ´s solution in case anyone is doing something similar:
First, I changed the calls for currentUser.id to currentUser.uid.
Afterwards I got the very missleading Firestore Error: PERMISSION_DENIED: Missing or insufficient permissions, which made me experiment a lot with my security rules. But that is not where the error originated. Debugging the code line by line, I noticed the category objects resolved by the promise where not correct and had a weird path with multiple spaces at the beginning and the end of their ids. When I removed them before saving them in the promises array, it worked. Although I do not see where the spaces came from in the first place.
promises.push(
getDoc(
doc(
store,
'categories',
docSnap.data().involvedCategory[userCatIndex].replaceAll(' ', '')
)
)
)
The general approach, given that Firestore is a NoSQL database that does not support server-side JOINS, is to perform all the data combinations on the client side or in the backend with a Cloud Function.
For your scenario, one approach is to first query the ToDo documents by the array membership of the current user's ID in the involved array.
Afterwards, you fetch the corresponding category document the current user assigned to that ToDo (going by index mapping between the two arrays). Finally, you should be able to construct your ToDo objects with the data.
const toDoArray = [];
const promises = [];
//Querying the ToDo collection
const q = query(collection(firestoreDB, 'habits'), where('involved', 'array-contains', currentUser.id));
const querySnap = await getDocs(q);
querySnap.forEach((docSnap) => {
//Uses index mapping
const userCatIndex = docSnap.data().involved.indexOf(currentUser.id);
//For each matching ToDo, get the corresponding category from the categories collection
promises.push(getDoc(doc(firestoreDB, 'categories', docSnap.data().involvedCategory[userCatIndex])));
//Pushes object to ToDo class/interface
toDoArray.push(new ToDo(docSnap.id, docSnap.data().title, docSnap.data().involved))
});
//Resolves all promises of category documents, then adds the data to the existing ToDo objects.
await Promise.all(promises).then(categoryDocs => {
categoryDocs.forEach((userCategory, i) => {
toDoArray[i].category = userCategory.data();
});
});
console.log(toDoArray);
Using the FirestoreDataConverter interface would not be that different, as you would need to still perform an additional query for the category data, and then add the data to your custom objects. Let me know if this was helpful.
I need to make a series of api calls, defined by 2 parameters. Let's call them users and foods. Each is an array of strings. For each user, and for each food, I need to construct a unique api call, which calls a route. I have a utility saga to do this:
function* getUserFoodDetails(requestParams) {
const { user, food } = requestParams
const response = yield call(
axios.get,
`api/foodstats/${user}/${food}`,
{
params: {
startDate: 'some datestring',
endDate: 'some datestring'
}
}
);
return response;
}
(Yes, I know this API design is not great, but its what I have to work with.) This saga can call one route at a time. I read redux-saga: How to create multiple calls/side-effects programmatically for yield?, and I had once asked my own question Getting results from redux-saga all, even if there are failures for how to implement error handling in this type of scenario. The general consensus was to use the utility saga with yield all to call many routes at once. If there was only one parameter, we could do this
const responses = yield users.map(user => call(getUserDetails, { user }));
And we would end up with an array of user data mapped from the original users array.
Now I am in a situation where I have a 2-dimensional dataset, and I need to make a call for every combination of user and food. For example,
const users = ['me', 'you', 'someone'];
const foods = ['bananas', 'oranges', 'apples'];
I am working within an existing app and I need to conform to the end-result data structure. The structure the rest of the UI expects is an array of arrays of result data. The first level array corresponds to each user, and the next level corresponds to each user's food data. (While I may want to change this, there's only so much legacy code refactoring I want to deal with at once). The structure should end up like this:
[
[ // 'me' data:
{ noEaten: 44, enjoyment: 5 }, // 'me' 'bananas' data
{ noEaten: 14, enjoyment: 2 }, // 'me' 'oranges' data
{ noEaten: 22, enjoyment: 4 }, // 'me' 'apples' data
],
[ // 'you' data:
{ noEaten: 12, enjoyment: 2 }, // 'you' 'bananas' data
{ noEaten: 334, enjoyment: 12 }, // 'you' 'apples' data
],
[ // 'someone' data
{ noEaten: 14, enjoyment: 2 }, // 'someone' 'oranges' data
{ noEaten: 22, enjoyment: 4 }, // 'someone' 'apples' data
]
]
Its not a very semantic structure, but its what I'm trying to get to.
The previous code achieves this with a nested map call within promises
const allUsers = user.map(user =>
const all = foods.map(food =>
fetch(`api/foodstats/${user}/${food}`)
.then(res => res.json())
)
return Promise.allSettled(all)
)
Promise.all(allUsers).then(res => doSomethingWithData(res));
This nesting of Promise.all and allSettled is very strange to me, but it does result in the above data structure.
I need to recreate this with sagas. Within my saga, I try to do a nested map as well:
const data = yield all(
users.map(user =>
foods.map(food =>
call(getUserFoodDetails, {
user,
food
})
)
)
);
However this does't work. I have 2 layers deep of .map, but only 1 layer deep of all and call. what I end up with an array of redux saga objects:
While I understand why this is happenining, I'm not sure how to fix it.
How can I nest .map statements with redux sagas, such that the api calls are made, and the data is returned in the same nested structure with which I made the calls? Is this possible? Is it worth bothering? Or is it better to come up with some intermediate data structures to obtain a single-layer array, and then restructure back to the 2-layer array that's needed in the components?
The solution was right in front of me - I'll leave it here in case anyone ever needs something similar. If a single call to yield all(users.map(() => {)) gives a 1-layer array of data, then I needed a 2-layer call to yield all to get a 2 layer array of data. Its a little convoluted, but 2 utiliy sagas, each which takes a single parameter:
// Utility saga to call route once, for 1 user's single food
function* getFoodDetails(requestParams) {
const { user, food, params } = requestParams;
const response = yield call(
axios.get,
`api/foodstats/${user}/${food}`,
{ ...params }
);
return response;
}
// Utility saga to call route for every food for a single user
function* getUserDetails(requestParams) {
const { user, foods, params } = requestParams;
yield all(foods.map(food =>
call(getFoodDetails, { user, food, params })
))
return response;
}
// Saga to call route for every user
function* getAllUserFoodData(requestParams) {
const { users, foods, params } = requestParams;
const response = yield call(
yield all(users.map(user =>
call(getUserDetails, { user, foods, params })
))
);
return response;
}
So getAllUserFoodData maps over all users, creating the first level of the array, which is one item per user. In each of those map calls, getUserDetails maps over each food for a given user, creating the second level of the array, which is one item per food. Finally getFoodDetails is called, which calls the route for a single user's single food.
I have created several user accounts on mongodb and i want to sort them out by user name. I compare the user names in the database against a string provided through aaxios request with a body value that is taken from an input value, like this:
frontend
const findUsers = async () => {
try {
const response = await axios.post(`http://localhost:8080/search-users/${_id}`, { searchValue });
setReturnedUser(response.data.matchedUser);
} catch (error) {
console.log(error);
}
}
findUsers();
backend
exports.sort = (req, res) => {
let result;
User.find({ name: req.body.searchValue }).exec((error, users) => {
if (error) {
return res.status(400).json({
message: error,
});
}
result = users;
res.status(200).json({
message: 'Description added successfully',
matchedUser: result,
});
});
};
The problem with this approach is that the users are returned only after I type in the entire name.
What I want is that the users to get returned as I type in the name, so several matching users will het returned when I start typing and as I continue the list will narrow down, until only the matching user remains.
I have successfully achieved this on the react side, but that was possible only by fetching all the users from the database, which would be a very bad idea with a lot of users. The obvious solution is to do the sorting on the server.
Filtering on the client-side is possible but with some tweaks to your architecture:
Create an end-point in node that returns all the users as JSON. Add caching onto this end-point. A call to origin would only occur very infrequently. YOu can then filter the list easily.
Use something like GraphQL and Appollo within node. This will help performance
To do the filtering in node you can use a normal array.filter()
I woul do the filter in mongo as the quick approach and then change it if you notice performance issues. It is better no to do pre-optimisation. As Mongo is NoSQL it wshould be quick
I am writing an angular service that calls an API endpoint to get some basic metadata for an object. My struggle is around having to complete 2 API calls to get all of the data that I need. The first API call returns a list of object ID's, and the second API call uses those ID's to get the data.
I need to map over the entries array returned by getFiles and call another endpoint to get the data. I don't want the subscribers to get a bunch of different responses and have to build an object on their end, I want a single response.
//getFiles response
{
total_count: 2
offset: 0,
limit: 30,
entries: [{ id: '1' }, { id: '2' }]
}
getFiles() {
return this.http('call my endpoint')
}
getFileDetails(fileId) {
return this.http('call my endpoint using fileId')
}
getFilesWithDetails() {
this.getFiles()
.pipe(
// ????
);
}
I want my consumers of the service to call getFilesWithDetails and have it return hydrated files. Is this possible?
You can use forkJoin to obtain an array of file details from an array of file ids and you can then use mergeMap to emit the array of details into the observable stream:
import { forkJoin } from 'rxjs/observable/forkJoin';
getFilesWithDetails() {
return this.getFiles().pipe(
mergeMap(result => forkJoin(
result.entries.map(e => this.getFileDetails(e.id))
))
);
}
forkJoin accepts an array of observables and emits an array that contains the last emitted value for each of the passed observables.
In my react App I have a input element. The search query should be memoized, which means that if the user has previously searched for 'John' and the API has provided me valid results for that query, then next time when the user types 'Joh', there should be suggestion for the user with the previously memoized values(in this case 'John' would be suggested).
I am new to react and am trying caching for the first time.I read a few articles but couldn't implement the desired functionality.
You don't clarify which API you're using nor which stack; the solution would vary somewhat depending on if you are using XHR requests or something over GraphQL.
For an asynchronous XHR request to some backend API, I would do something like the example below.
Query the API for the search term
_queryUserXHR = (searchTxt) => {
jQuery.ajax({
type: "GET",
url: url,
data: searchTxt,
success: (data) => {
this.setState({previousQueries: this.state.previousQueries.concat([searchTxt])
}
});
}
You would run this function whenever you want to do the check against your API. If the API can find the search string you query, then insert that data into a local state array variable (previousQueries in my example).
You can either return the data to be inserted from the database if there are unknowns to your view (e.g database id). Above I just insert the searchTxt which is what we send in to the function based on what the user typed in the input-field. The choice is yours here.
Get suggestions for previously searched terms
I would start by adding an input field that runs a function on the onKeyPress event:
<input type="text" onKeyPress={this._getSuggestions} />
then the function would be something like:
_getSuggestions = (e) => {
let inputValue = e.target.value;
let {previousQueries} = this.state;
let results = [];
previousQueries.forEach((q) => {
if (q.toString().indexOf(inputValue)>-1) {
result.push(a);
}
}
this.setState({suggestions: results});
}
Then you can output this.state.suggestions somewhere and add behavior there. Perhaps some keyboard navigation or something. There are many different ways to implement how the results are displayed and how you would select one.
Note: I haven't tested the code above
I guess you have somewhere a function that queries the server, such as
const queryServer = function(queryString) {
/* access the server */
}
The trick would be to memorize this core function only, so that your UI thinks its actually accessing the server.
In javascript it is very easy to implement your own memorization decorator, but you could use existing ones. For example, lru-memoize looks popular on npm. You use it this way:
const memoize = require('lru-memoize')
const queryServer_memoized = memoize(100)(queryServer)
This code keeps in memory the last 100 request results. Next, in your code, you call queryServer_memoized instead of queryServer.
You can create a memoization function:
const memo = (callback) => {
// We will save the key-value pairs in the following variable. It will be our cache storage
const cache = new Map();
return (...args) => {
// The key will be used to identify the different arguments combination. Same arguments means same key
const key = JSON.stringify(args);
// If the cache storage has the key we are looking for, return the previously stored value
if (cache.has(key)) return cache.get(key);
// If the key is new, call the function (in this case fetch)
const value = callback(...args);
// And save the new key-value pair to the cache
cache.set(key, value);
return value;
};
};
const memoizedFetch = memo(fetch);
This memo function will act like a key-value cache. If the params (in our case the URL) of the function (fetch) are the same, the function will not be executed. Instead, the previous result will be returned.
So you can just use this memoized version memoizedFetch in your useEffect to make sure network request are not repeated for that particular petition.
For example you can do:
// Place this outside your react element
const memoizedFetchJson = memo((...args) => fetch(...args).then(res => res.json()));
useEffect(() => {
memoizedFetchJson(`https://pokeapi.co/api/v2/pokemon/${pokemon}/`)
.then(response => {
setPokemonData(response);
})
.catch(error => {
console.error(error);
});
}, [pokemon]);
Demo integrated in React