we have a project, with a nextjs, redux saga, typescript setup.
SSR is very important for our web app, but we also have a lot of relevant widgets on every page.
This is why our page is structured in a modular way, where every widget (1-2 per page) loads the data it needs.
These widgets are relevant for SEO reasons now.
My problem is, that the API requests are not made on the serverside though. Right now it only returns the defaultState of every widget on the server and only loads them on the client.
I have searched and found a lot of instructions on how to do it, but they all rely on the fact that nextjs waits for the "getInitialProps" method until it returns the result from the server.
Since that lifecycle method is only available in the "pages" folder, that doesn't work for me.
Also if I block the "getInitialProps" component, the component is never really rendered.
Our pages are structured like below:
- pages/Home
- <HomeContainer ...> (fetches data for the main page)
- <HomeComponent >
- <Widget1Container ...> (fetches data)
- <Widget2Container ...> (fetches data)
What I want is for the serverside to wait for all the requests provided, before it returns the page to the user.
Because of the complex nature of different widgets on a page, it is not possible to create "combined" endpoint where we get the data in the "pages/Home" folder.
I know it's not ideal, but how could we make sure that the server actually makes all 3 requests (homecontainer, widget1container, widget2container) and awaits there responses, before returning?
I would like to have it like angular-universal does it. Just wait until there are not open requests or promises and then just render.
Any ideas?
Any help or ideas are deeply appreciated.
thanks
since you are redux sage, in getServerSideProps, send the start signal to saga
export const getServerSideProps = wrapper.getServerSideProps(
async (context) => {
store.dispatch(
fetchWidgetsStart("add payload")
);
store.dispatch(END);
await (store as SagaStore).sagaTask.toPromise();
const state = store.getState();
// I am making up the reducer name
const widgetListState = state.widgetsList ? state.widgetList : null;
return { props: { productListState } };
}
);
inside saga function:
export function* fetchWidgetsStart() {
yield takeLatest(
WidgetListActionTypes.WIDGET_LIST_START,
fetchWidgetssAsync
);
}
function* fetchWidgetsAsync(action: IFetchWidgetssStart) {
try {
const res: AxiosResponse<IWidget[] | []> = Promise.all([
yield axios.get(fetchWidget1),
yield axios.get(fetchWidget2),
yield axios.get(fetchWidget3)
])
yield put(fetchWidgetSuccess(res));
} catch (e: any) {
yield put(
fetchWidgetFailure(
e.response && e.response.data.detail
? e.response.data.detail
: e.message
)
);
}
}
The Promise.all() method takes an iterable of promises as an input, and returns a single Promise that resolves to an array of the results of the input promises. This returned promise will resolve when all of the input's promises have resolved, or if the input iterable contains no promises. It rejects immediately upon any of the input promises rejecting or non-promises throwing an error, and will reject with this first rejection message / error.
Related
I was facing a problem for sometime, that was I'm unable to clear cache using RTK query.
I tried in various ways but cache data is not clear.
I used invalidatesTag in my mutation query and it called the api instantly. But in this case I want to refetch multiple api again, but not from any rtk query or mutation. I want to make the api call after some user activity like click.
How can I solve this problem?
I made a separate function where I return api.util.invalidateTags(tag) or api.util.resetApiState().
this is my code-snipet:-
` const api = createApi({.....})
export const resetRtkCache = (tag?: String[]) => {
const api =
if (tag) {
return api.util.invalidateTags(tag)
} else {
return api.util.resetApiState()
}
}`
& I called it using dispatch method from other files
`const reloadData = () => {
dispatch(resetRtkCache())
}`
but here cache data is not removed.I think dispatch funtion is not working. I don't see the api call is being sent to server in the browser network.
But in this case I want to refetch multiple api again, but not from
any rtk query or mutation. I want to make the api call after some user
activity like click. How can I solve this problem?
So if I understood correctly what you want to achieve is to fetch some api that you have in RTK only after some kind of user interaction?
Can't you just define something like this?
const { data } = useGetYourQuery({ skip: skipUntilUserInteraction })
Where skipUntilUserInteraction is a component state variable that you will set to true and update to false based on the user interaction you need? (e.g. a click of a button).
So essentially on component render that specific endpoint will be skipped but will be fetched after the interaction that you want will happen?
wow, you actually asking so many questions at once. but I think you should definitely read the documentation because it covers all the questions you have.
so trying to answer your questions one by one.
I used invalidatesTag in my mutation query and it called the api instantly.
invalidating with Tags is one of the ways to clear the cache.
you should first set the tagTypes for your API then use those tags in mutation queries and tell the RTK query which part of entities you want to clear.
I want to refetch multiple APIs again
you can customize the query inside of a mutation or query like this example and by calling one function query you can send multiple requests at once and if you want to fetch the API again after the cache removed you do not need to do anything because RTK query will do it for you.
I want to make the API call after some user activity like click
every mutation gives u a function that you can pass to onClick like below:
import { use[Mymutation]Mutation } from 'features/api';
const MyComponenet() {
const [myMutationFunc, { isLoading, ...}] = use[Mymutation]Mutation();
return <button type='button' onClick={myMutationFunc}>Click for call mutaion</button>
}
and remember if you set providesTags for your endpoint which you were defined in tagTypes by clicking on the button and firing up the myMutationFunc you will be clearing the cache with those tags.
and if you looking for an optimistic update for the cache you can find your answer in here.
async onQueryStarted({ id, ...patch }, { dispatch, queryFulfilled }) {
const patchResult = dispatch(
api.util.updateQueryData('getPost', id, (draft) => {
Object.assign(draft, patch)
})
)
try {
await queryFulfilled
} catch {
patchResult.undo()
}
}
I have a chrome extension that stores data in Firestore and populates that data to the frontend. I always have to refresh the page to see newly added data, which isn’t a user friendly experience. How can I update the UI to show the newly updated data without having to refresh the page?
So far, I've tried using useEffect to get the data. Inside of it, I'm using a function that gets data from Firestore cached inside of chrome local storage.
Here is my code
const getFolderData = () => {
getDataFromChrome("docId").then((res: any) => {
setDocId(res.docId);
});
getDataFromChrome("content").then((res: any) => {
//console.log("getting in mainfolder",res);
// for (const item of res.content) {
// if (item.type.toLowerCase() === "subfolder") {
// // console.log(item)
// getSubFolder(item.id);
// }
// }
for (const item of res.content) {
setTiersContent((pre: any) => [...pre, item]);
}
});
};
useEffect(() => {
getFolderData();
}, []);
I also get this error. I'm also using the chrome extension API to communicate with a background script. It could be related to the problem
Uncaught (in promise) Error: A listener indicated an asynchronous response by returning true, but the message channel closed before a response was received
I've never used firebase so I'm not sure what your functions do, I can only guess. A few things wrong from what I can see:
Your useEffect is set to only run on page load since the dep array is empty, I assume you want to refetch on some condition.
If any of the 2 functions is supposed to be a subscription, your useEffect needs to return a cancel function.
Refetch data when needed is not a new problem, packages like React Query has tools that optimize your requests and refetch when needed. I suggest you give it a shot if your app has more than 2-3 fetch requests.
I have to pretty weird case to handle.
We have to few boxes, We can call some action on every box. When We click the button inside the box, we call some endpoint on the server (using axios). Response from the server return new updated information (about all boxes, not the only one on which we call the action).
Issue:
If user click submit button on many boxes really fast, the request call the endpoints one by one. It's sometimes causes errors, because it's calculated on the server in the wrong order (status of group of boxes depends of single box status). I know it's maybe more backend issue, but I have to try fix this on frontend.
Proposal fix:
In my opinion in this case the easiest fix is disable every submit button if any request in progress. This solution unfortunately is very slow, head of the project rejected this proposition.
What we want to goal:
In some way We want to queue the requests without disable every button. Perfect solution for me at this moment:
click first button - call endpoint, request pending on the server.
click second button - button show spinner/loading information without calling endpoint.
server get us response for the first click, only then we really call the second request.
I think something like this is huge antipattern, but I don't set the rules. ;)
I was reading about e.g. redux-observable, but if I don't have to I don't want to use other middleware for redux (now We use redux-thunk). Redux-saga it will be ok, but unfortunately I don't know this tool. I prepare simple codesandbox example (I added timeouts in redux actions for easier testing).
I have only one stupid proposal solution. Creating a array of data needs to send correct request, and inside useEffect checking if the array length is equal to 1. Something like this:
const App = ({ boxActions, inProgress, ended }) => {
const [queue, setQueue] = useState([]);
const handleSubmit = async () => { // this code do not work correctly, only show my what I was thinking about
if (queue.length === 1) {
const [data] = queue;
await boxActions.submit(data.id, data.timeout);
setQueue(queue.filter((item) => item.id !== data.id));
};
useEffect(() => {
handleSubmit();
}, [queue])
return (
<>
<div>
{config.map((item) => (
<Box
key={item.id}
id={item.id}
timeout={item.timeout}
handleSubmit={(id, timeout) => setQueue([...queue, {id, timeout}])}
inProgress={inProgress.includes(item.id)}
ended={ended.includes(item.id)}
/>
))}
</div>
</>
);
};
Any ideas?
I agree with your assessment that we ultimately need to make changes on the backend. Any user can mess with the frontend and submit requests in any order they want regardless how you organize it.
I get it though, you're looking to design the happy path on the frontend such that it works with the backend as it is currently.
It's hard to tell without knowing the use-case exactly, but there may generally be some improvements we can make from a UX perspective that will apply whether we make fixes on the backend or not.
Is there an endpoint to send multiple updates to? If so, we could debounce our network call to submit only when there is a delay in user activity.
Does the user need to be aware of order of selection and the impacts thereof? If so, it sounds like we'll need to update frontend to convey this information, which may then expose a natural solution to the situation.
It's fairly simple to create a request queue and execute them serially, but it seems potentially fraught with new challenges.
E.g. If a user clicks 5 checkboxes, and order matters, a failed execution of the second update would mean we would need to stop any further execution of boxes 3 through 5 until update 2 could be completed. We'll also need to figure out how we'll handle timeouts, retries, and backoff. There is some complexity as to how we want to convey all this to the end user.
Let's say we're completely set on going that route, however. In that case, your use of Redux for state management isn't terribly important, nor is the library you use for sending your requests.
As you suggested, we'll just create an in-memory queue of updates to be made and dequeue serially. Each time a user makes an update to a box, we'll push to that queue and attempt to send updates. Our processEvents function will retain state as to whether a request is in motion or not, which it will use to decide whether to take action or not.
Each time a user clicks a box, the event is added to the queue, and we attempt processing. If processing is already ongoing or we have no events to process, we don't take any action. Each time a processing round finishes, we check for further events to process. You'll likely want to hook into this cycle with Redux and fire new actions to indicate event success and update the state and UI for each event processed and so on. It's possible one of the libraries you use offer some feature like this as well.
// Get a better Queue implementation if queue size may get high.
class Queue {
_store = [];
enqueue = (task) => this._store.push(task);
dequeue = () => this._store.shift();
length = () => this._store.length;
}
export const createSerialProcessor = (asyncProcessingCallback) => {
const updateQueue = new Queue();
const addEvent = (params, callback) => {
updateQueue.enqueue([params, callback]);
};
const processEvents = (() => {
let isReady = true;
return async () => {
if (isReady && updateQueue.length() > 0) {
const [params, callback] = updateQueue.dequeue();
isReady = false;
await asyncProcessingCallback(params, callback); // retries and all that include
isReady = true;
processEvents();
}
};
})();
return {
process: (params, callback) => {
addEvent(params, callback);
processEvents();
}
};
};
Hope this helps.
Edit: I just noticed you included a codesandbox, which is very helpful. I've created a copy of your sandbox with updates made to achieve your end and integrate it with your Redux setup. There are some obvious shortcuts still being taken, like the Queue class, but it should be about what you're looking for: https://codesandbox.io/s/dank-feather-hqtf7?file=/src/lib/createSerialProcessor.js
In case you would like to use redux-saga, you can use the actionChannel effect in combination with the blocking call effect to achieve your goal:
Working fork:
https://codesandbox.io/s/hoh8n
Here is the code for boxSagas.js:
import {actionChannel, call, delay, put, take} from 'redux-saga/effects';
// import axios from 'axios';
import {submitSuccess, submitFailure} from '../actions/boxActions';
import {SUBMIT_REQUEST} from '../types/boxTypes';
function* requestSaga(action) {
try {
// const result = yield axios.get(`https://jsonplaceholder.typicode.com/todos`);
yield delay(action.payload.timeout);
yield put(submitSuccess(action.payload.id));
} catch (error) {
yield put(submitFailure());
}
}
export default function* boxSaga() {
const requestChannel = yield actionChannel(SUBMIT_REQUEST); // buffers incoming requests
while (true) {
const action = yield take(requestChannel); // takes a request from queue or waits for one to be added
yield call(requestSaga, action); // starts request saga and _waits_ until it is done
}
}
I am using the fact that the box reducer handles the SUBMIT_REQUEST actions immediately (and sets given id as pending), while the actionChannel+call handle them sequentially and so the actions trigger only one http request at a time.
More on action channels here:
https://redux-saga.js.org/docs/advanced/Channels/#using-the-actionchannel-effect
Just store the promise from a previous request and wait for it to resolve before initiating the next request. The example below uses a global variable for simplicity - but you can use smth else to preserve state across requests (e.g. extraArgument from thunk middleware).
// boxActions.ts
let submitCall = Promise.resolve();
export const submit = (id, timeout) => async (dispatch) => {
dispatch(submitRequest(id));
submitCall = submitCall.then(() => axios.get(`https://jsonplaceholder.typicode.com/todos`))
try {
await submitCall;
setTimeout(() => {
return dispatch(submitSuccess(id));
}, timeout);
} catch (error) {
return dispatch(submitFailure());
}
};
I'm building an app with Next.js... we have 100k+ pages and content changes daily, so using SSR and getServerSideProps.
Some of our data is coming from a headless CMS provider that charges by the request. I'd like to cache the API responses from this server for 24hrs.
What is the best way of going about this?
Is there a common library most folks use to do this?
Just looking for suggestions of approaches I should investigate (or great examples of how to do this).
I used this npm package:
https://www.npmjs.com/package/memory-cache
And then something like this:
import cacheData from "memory-cache";
async function fetchWithCache(url, options) {
const value = cacheData.get(url);
if (value) {
return value;
} else {
const hours = 24;
const res = await fetch(url, options);
const data = await res.json();
cacheData.put(url, data, hours * 1000 * 60 * 60);
return data;
}
}
Then if you want to fetch something with using the cache just call this function. Or it can be used as a midware in the requests. It checks if the data is already in the cache and returns it, or if not - it puts the data into the cache under the key. The key can be anything, I am using the url for instance.
In addition to Tobias Lins' answer:
At least if deploying on Vercel, you can use set Cache-Control headers in getStaticProps, getServerSideProps, API routes, etc to cache responses on Vercel's edge network. This solution does not require any additional dependencies and very minimal code.
api route example - source Vercel
// pages/api/user.js
export default function handler(req, res) {
res.setHeader('Cache-Control', 's-maxage=86400');
res.status(200).json({ name: 'John Doe' });
}
Example in getServerSideProps - Source NextJS
// This value is considered fresh for ten seconds (s-maxage=10).
// If a request is repeated within the next 10 seconds, the previously
// cached value will still be fresh. If the request is repeated before 59 seconds,
// the cached value will be stale but still render (stale-while-revalidate=59).
//
// In the background, a revalidation request will be made to populate the cache
// with a fresh value. If you refresh the page, you will see the new value.
export async function getServerSideProps({ req, res }) {
res.setHeader(
'Cache-Control',
'public, s-maxage=10, stale-while-revalidate=59'
)
return {
props: {},
}
}
I believe you'd want to use:
res.setHeader('Cache-Control', 's-maxage=1440000')
Here are some other useful links for caching on Vercel:
https://vercel.com/docs/concepts/functions/edge-caching
https://vercel.com/docs/concepts/edge-network/overview
https://vercel.com/docs/concepts/edge-network/caching
https://vercel.com/docs/concepts/edge-network/headers
For your specific case, you also may want to look into using getStaticPaths with getStaticProps. You can use fallback: true on getStaticPaths to only build pages when they're visited (you can still build your post popular pages at initial build time).
https://nextjs.org/docs/basic-features/data-fetching#the-fallback-key-required
I know this is an old post, but for others googling (at least those deploying on Vercel), these solutions should help where revalidate in getStaticProps does not.
You could use getStaticProps from Next.js for SSG
They currently have a revalidate property that you can return, that defines how often the content should be re-fetched.
Take a look here:
https://nextjs.org/blog/next-9-5#stable-incremental-static-regeneration
This is how we did it without any 3rd party libraries, as in our use-case we only had to cache a relatively smaller amount of global data(header/footer menus) which was shared across the site.
The data was coming from a CMS via GraphQL.
We ran an async method getGlobalData on each page from on getStaticProps method and then returned the cached data to the page component via props.
import fs from 'fs';
import path from 'path';
// Cache files are stored inside ./next folder
const CACHE_PATH = path.join(__dirname, 'globalData.json');
export default async function getGlobalData() {
let cachedData;
// #1 - Look for cached data first
try {
cachedData = JSON.parse(fs.readFileSync(CACHE_PATH, 'utf8'));
} catch (error) {
console.log('❌ CACHE NOT INITIALIZED');
}
// #2 - Create Cache file if it doesn't exist
if (!cachedData) {
// Call your APIs to-be-cached here
const data = await fetchGlobalData();
// Store data in cache files
// this always rewrites/overwrites the previous file
try {
await fs.writeFileSync(
CACHE_PATH,
JSON.stringify(data),
err =>throw err
);
console.log('💾 CACHE FILE WRITTEN SUCCESSFULLY');
} catch (error) {
console.log('❌ ERROR WRITING MEMBERS CACHE TO FILE\n', error);
}
cachedData = data;
}
return cachedData;
}
Call getGlobalData method from getStaticProps.
export async function getStaticProps({ preview = false }) {
const globalData = await getGlobalData();
// call other page-specific/non-shared APIs here
// ...
return { props: { globalData } };
}
References
https://flaviocopes.com/nextjs-cache-data-globally/
Note if you get an error saying fs or path is unknown or invalid, then please understand that, the above code is supposed to be running or referenced "serverside" i.e only inside getStaticProps or getServerSideProps. If you import and reference it "browser-side", say somewhere inside your components or on the page (other than methods mentioned above), then you will get an error, as there is no filesystem(fs) or path modules on browser. They are only available on node.
I'm learning Redux-Saga and having a bit of trouble wrapping my head round the correct flow for connectng people to a chat service (Chatkit by Pusher) when they log in and disconnecting them on logout.
So far I have an "auth" saga which waits for a LOGIN_REQUEST action, logs in to a REST api using axios then stores a username and token in the store by calling a USER_SET action.
My question is, when the login happens and the credentials are stored, should I PUT a new action called something like CHAT_CONNECT which would kick off another saga to connect to Chatkit, or should I get the chat saga to listen to the LOGIN_SUCCESS being fired and act on that? Is there even any practical difference in these two approaches.
As a bonus question, what's the best way to receive and act on new websocket messages from Chatkit using Redux Sagas? Here's the boilerplate code for connecting and receiving events from chatkit.
chatManager
.connect()
.then(currentUser => {
currentUser.subscribeToRoom({
roomId: currentUser.rooms[0].id,
hooks: {
onNewMessage: message => {
console.log(`Received new message: ${message.text}`)
}
}
});
})
.catch(error => {
console.error("error:", error);
})
Regarding your first question:
My question is, when the login happens and the credentials are stored, should I PUT a new action called something like CHAT_CONNECT which would kick off another saga to connect to Chatkit, or should I get the chat saga to listen to the LOGIN_SUCCESS being fired and act on that?
With the information provided its difficult to decide which approach is ideal because either will accomplish the same functionality. The biggest difference I see between the two proposed approaches is the direction of dependency. You have two different "modules" (features, packages, ...whatever you call your chunks of code that handle a single responsiblity), lets call them log-in and connect-chat.
If you dispatch an action CHAT_CONNECT from within the log-in saga, your log-in module will be dependent to the connect-chat module. Presumably, the connect-chat action will live in the connect-chat module.
Alternatively, if your connect-chat saga waits for LOGIN_SUCCESS, then your connect-chat module will be dependent on your log-in module. Presumably, the LOGIN_SUCCESS will live in the log-in module.
There's nothing wrong with either approach. Which is best depends on your applications needs and functionality.
If you might want to connect to chat any other time then after successfully logging in, then it might make sense to dispatch CHAT_CONNECT from within your log-in saga. Because chat is no longer dependent on log in. There are several scenarios where either approach will work better than the other, but it really depends on how the rest of your application is set up.
Regarding your bonus questions:
One approach to hooking external events in redux-saga is accomplished via eventChannels. Docs: https://redux-saga.js.org/docs/api/#eventchannelsubscribe-buffer-matcher
There's a bit of boiler plate, but I found this approach makes testing easier and truly encapsulates external functionality. Here's a quick example of how I might hook up an event channel to the code snippet you provided:
export const createOnMessageChannel = () =>
eventChannel((emit) => {
chatManager
.connect()
.then(currentUser => {
currentUser.subscribeToRoom({
roomId: currentUser.rooms[0].id,
hooks: {
onNewMessage: message => emit({ message }),
}
});
})
.catch(error => emit({ error }));
return () => {
// Code to unsubscribe, e.g. chatManager.disconnet() ?
};
});
export function* onMessage({ message, error }) {
if (error) {
yield put(handleError(error));
return;
}
yield put(handleMessage(message));
}
// this is what you pass to your root saga
export function* createOnMessageSaga() {
// using call because this makes it easier to test
const channel = yield call(createOnMessageChannel);
if (!channel) return;
yield takeEvery(channel, onMessage);
}