Next.js: Reduce data fetching and share data between pages - reactjs

I'm looking for solutions for better data fetching in a Next.js app. In this question I'm not just looking for a solution, I'm looking for multiple options so we can look at the pros and cons.
The problem I have
Right now I have a few pages that all include a component that displays som static content and a that have some dynamic content that is fetched from an API. Each page do a fetch() in their getInitialProps() to get their own page data, but also the footer data, which is the same for all pages.
This of course works, but there is a lot of duplicated data fetching. The footer data will always be displayed for all pages and always be the same. It will also rarely be changed in the API, so no need for revalidate the data.
The answers I'm looking for
I'm not just looking to solve this one problem, I'm looking for an overview to learn some new practice for future projects as well. I like writing "obvious" code, so not looking for too hacky solutions, like writing to the window object etc. Simple solutions with less dependancies are preferred. The goal is a fast site. It's not that important to reduce network usage/API calls.
What I have thought so far
This is the possible solutions I've come up with, somewhat sorted from simple/obvious to more complex.
Do a fetch inside the Footer component (client side)
Do a fetch in getInitialProps (server side & client side) on all /pages
Do a fetch in _app.js with a HOC and hooking into it's getInitialProps() and add it to props, so data is available for all pages
Use zeit/swr and data prefetching to cache data
Use redux to store a global state
All of these "work", but most of them will refetch the data unnecessarily, and/or adds a bit more complexity. Here are the pros/cons as I see it (numbers are the same as above):
πŸ‘ Simple! Fetch code is only in one place, it's located where it's used. πŸ‘Ž Data is fetched after page is loaded, so the content "jumps" in to view. Data is refetched all the time.
πŸ‘ Simple! Data is fetched on the server, so content is available before the page is rendered. πŸ‘Ž Data is refetched for each page. We have to remember to fetch the same footer data for each page in their getInitialProps().
πŸ‘ We can do the fetch in one place and add it to all the pages props, so footer data is automatically available for all pages' props. πŸ‘Ž Might be a bit more complex for some to easily understand what's going on, as it requires a bit more understanding of how Next.js/React works. Still refetches the data for all pages. We now do two fetch() calls after each other (first in _app.js to load footer content, then in each page to get custom content), so it's even slower.
πŸ‘ Somewhat simple. We can use the prefetching to load data to cache even before the JS is loaded. After first page load, we will have fast data fetching. Can have fetch code directly in footer component. πŸ‘Ž The rel="preload" prefetching technique won't work with all types of fetching (for instance Sanity's client using groq). To not have "jumpy" content where the data is loaded after initial page load, we should provide useSWR() with initialData which still will require us to fetch data in getInitialProps(), but it would be enough to just do this on the server side. Could use the new getServerSideProps().
πŸ‘ We can load data once(?) and have it available throughout the application. Fast and less/no refetching. πŸ‘Ž Adds external dependency. More complex as you'll have to learn redux, even to just load one shared data object.
Current solution, using the solution described in bullet point number 2.
const HomePage = (props) => {
return (
<Layout data={props.footer}>
<Home data={props.page} />
</Layout>
)
}
// Not actual query, just sample
const query = `{
"page": *[_type == "page"][0],
"footer": *[_type == "footer"][0]
}`
HomePage.getInitialProps = async () => {
const data = await client.fetch(query)
return {
page: data.page
footer: data.footer
}
}
export default HomePage
Would love some more insight into this. I'm a missing something obvious?

O'right! I found this thread while I was looking for something else. But since I had to work on similar issues, I can give you some directions, and I will do my best to make it clear for you.
So there are some data which you want to have it share, across your app (pages/components).
Next.js uses the App component to initialize pages. You can override it and control the page initialization. to achieve that simply create _app.js file in root of pages directory. For more information follow this link: https://nextjs.org/docs/advanced-features/custom-app
Just like the way you can use getInitialProps in your pages to fetch data from your API, you can also use the same method in _app.js. So, I would fetch those data which I need to share them across my app and eliminate my API calls.
Well, Now I can think of two ways to share the data across my app
Using of createContext hooks.
1.1. Create a DataContext using createContext hooks. and wrap <Component {...pageProps} /> with your <DataContext.Provider>.
Here is a code snippet to give you a better clue:
<DataContext.Provider value={{ userData, footerData, etc... }}>
<Component {...pageProps} />
</DataContext.Provider>
1.2. Now in other pages/components you can access to your DataContext like following:
const { footerData } = useContext(DataContext);
And then you are able to do the manipulation in your front-end
populates props using getInitialProps
2.1. getInitialProps is used to asynchronously fetch some data, which then populates props. that would be the same case in _app.js.
The code in your _app.js would be something like this:
function MyApp({ Component, pageProps, footerData }) {
//do other stuffs
return (
<Component {...pageProps} footerData={footerData} />
;
}
MyApp.getInitialProps = async ({ Component, ctx }) => {
const footerRes = await fetch('http://API_URL');
const footerData = await footerRes.json();
let pageProps = {};
if (Component.getInitialProps) {
pageProps = await Component.getInitialProps(ctx);
}
return { pageProps, footerData };
};
2.2. Now in your pages (not in your components) you can access to props including those you have shared from _app.js
and you can start to do you manipulation.
Hope I could give you a clue and direction. Have fun exploring.

Related

What is proper way to detect device in Next.js SSR?

I have <MobileLayout />, <DesktopLayout />. I'm using Next.js for Server Side Rendering.
And I noticed there are many famous ui library has mobile detection components like <Respnosive /> component in Semantic-UI-React. But all of this is client side method, not working properly on SSR
I read some documents the conclusion is I should check user-agent of server side req.headers. In Next.js, What is proper way to detect device and conditonally render one of MobileLayout / DesktopLayout?
What I tried
in _app.js
import isMobile from 'ismobilejs'
...
function Homepage({ Component, pageProps, mobile }){
return (
mobile ?
<MobileLayout><Component {...pageProps} /></MobileLayout> :
<DesktopLayout><Component {...pageProps} /></DesktopLayout>
)
}
HomePage.getInitialProps = async (appContext) => {
const userAgent = appContext.ctx.req.headers['user-agent']
const mobile = isMobile(userAgent).any
const appProps = await App.getInitialProps(appContext)
return { ...appProps, mobile }
}
But the problem is getIntialProps on _app.js executed every page load. with moving page with client, the appContext.ctx is undefined so it will omit error. and I think this method might block some nextjs builtin optimizations.
Error in error page getInitialProps: TypeError: Cannot read
property 'headers' of undefined
So what is propery way to check device in Next.js?
If you want to detect the user's device using userAgent, your best bet is this answer:
IndexPage.getInitialProps = ({ req }) => {
let userAgent;
if (req) { // if you are on the server and you get a 'req' property from your context
userAgent = req.headers['user-agent'] // get the user-agent from the headers
} else {
userAgent = navigator.userAgent // if you are on the client you can access the navigator from the window object
}
}
(Note you should actually be using getServerSideProps or getStaticProps when possible, if you have Next 9.3 or newer, but sometimes there is no replacement for the getInitialProps functionality.)
However, the folks at Mozilla advise:
It's worth re-iterating: it's very rarely a good idea to use user
agent sniffing. You can almost always find a better, more broadly
compatible way to solve your problem!
The maker of the isMobile package you're importing even warns:
You might not need this library. In most cases, responsive design
solves the problem of controlling how to render things across
different screen sizes.
So, see if you can use CSS3 media queries to conditionally render certain elements or change their size, etc., rather than having completely separate mobile and desktop layout components. But it's possible you have an edge case where you can't make any alternative option work.
If you are going to keep your current setup and use your two layouts on other pages, you might consider combining them into a parent <Layout> component that conditionally renders one or the other so you don't have to copy that logic into every page:
export const Layout = (props) => {
return (
props.mobile ?
<MobileLayout>{props.children}</MobileLayout> :
<DesktopLayout>{props.children}</DesktopLayout>
)
}
you can use "next-useragent"
Give access to user-agent details anywhere using withUserAgent method.
next-useragent npm package

Best method of persisting API data when using react-router

I have a problem with latency with my current React application. As it currently is written, I have a top level react-router which routes between several protected routes. Like so:
export default function IndexRouter(): JSX.Element {
return (
<Switch>
<Route path="/routeA" component={ComponentA} />
<Route path="/routeB" component={ComponentB} />
<Route path="/routeC" component={ComponentC} />
...
</Switch>
);
}
Component A loads a lot of data from several API calls, including multiple data arrays and images. This is done with a custom Axios hook. The approximate structure of component A is as follows:
export default function ComponentA(): JSX.Element {
const [data, loading, error, fetch] = useAxios(...);
return (
<Sub1 />
<Sub2 />
);
}
export default function Sub1(): JSX.Element {
const [data, loading, error, fetch] = useAxios(...);
const [data1, loading1, error1, fetch1] = useAxios(...);
return (
... more sub-components
);
}
And it is nested like so.
So basically a lot of data is obtained when the user navigates to componentA with react-router. My problem is that the user frequently navigates away from componentA (which is the main page), only to come back to componentA each time. Each time, componentA loads all of the data that was loaded previously, with only some minor changes made from users. This repeated loading is causing a ton of latency during navigation.
I don't feel I can take advantage of useMemo or useCallback since componentA is being unmounted during navigation by react-router. I don't feel persisting data in localStorage would be a good long term fix. I also feel that abstracting all the deep API calls within componentA to a higher order component would not be a great fix either and would cause a mess of the code with prop drilling.
So my question is how does one fix this issue?
save fetched data into redux, or create a CacheContext in the top, and create interceptor for axios to save fetched data on response, try use cache in request
I don't feel persisting data in localStorage would be a good long term fix
I don't think there's something wrong with using localStorage for this use-case.
not be a great fix either and would cause a mess of the code with prop drilling
React offers a painless way to handle prop drilling, it's native and built-in to the language and it's called Context API. There are some caveats to this as this requires optimizations and if not used properly, could lead to issues such as (but not limited to) re-rendering unnecessarily. State management libraries like MobX gives you a way to handle this in a less painful way by simply connecting your component to the state tree which could be persisted in your local storage.
Another way to handle this is to simply cache your API's response, there are libraries out there like SWR (stale-while-revalidate) built to simplify this for you.
A snippet from SWR's documentation:
SWR first returns the data from cache (stale), then sends the fetch request (revalidate), and finally comes with the up-to-date data again.
There are also parameters in the library that allows you to dictate when to re-validate.

Global variables in React

I know Redux solves this but I came up with an idea.
Imagine I have an app that gets some JSON on start. Based on this JSON I'm setting up the environment, so let's assume the app starts and it downloads an array of list items.
Of course as I'm not using Redux (the app itself is quite simple and Redux feels like a huge overkill here) if I want to use these list items outside of my component I have to pass them down as props and then pass them as props again as deep as I want to use them.
Why can't I do something like this:
fetch(listItems)
.then(response => response.json())
.then(json => {
window.consts = json.list;
This way I can access my list anywhere in my app and even outside of React. Is it considered an anti-pattern? Of course the list items WON'T be changed EVER, so there is no interaction or change of state.
What I usually do when I have some static (but requested via API) data is a little service that acts kind like a global but is under a regular import:
// get-timezones.js
import { get } from '../services/request'
let fetching = false
let timez = null
export default () => {
// if we already got timezones, return it
if (timez) {
return new Promise((resolve) => resolve(timez))
}
// if we already fired a request, return its promise
if (fetching) {
return fetching
}
// first run, return request promise
// and populate timezones for caching
fetching = get('timezones').then((data) => {
timez = data
return timez
})
return fetching
}
And then in the view react component:
// some-view.js
getTimezones().then((timezones) => {
this.setState({ timezones })
})
This works in a way it will always return a promise but the first time it is called it will do the request to the API and get the data. Subsequent requests will use a cached variable (kinda like a global).
Your approach may have a few issues:
If react renders before this window.consts is populated you won't
be able to access it, react won't know it should re-render.
You seem to be doing this request even when the data won't be used.
The only downside of my approach is setting state asynchronously, it may lead to errors if the component is not mounted anymore.
From the React point of view:
You can pass the list from top level via Context and you can see docs here.
Sample of using it is simple and exists in many libraries, such as Material UI components using it to inject theme across all components.
From engineering concept of everything is a trade of:
If you feel that it's gonna take so much time, and you are not going to change it ever, so keep it simple, set it to window and document it. (For your self to not forget it and letting other people know why you did this.)
If you're absolutely certain they won't ever change, I think it's quite ok to store them in a global, especially if you need to access the data outside of React. You may want to use a different name, maybe something like "appNameConfig"..
Otherwise, React has a feature called Context, which can also be used for "deep provision" - Reference

React Redux: More containers v.s. less containers

As I get further into implementing redux + react into a fairly complex app which depends on many API requests to load a single page, I'm having trouble deciding whether it's better to have a single container component at the root of the page which handles all async stuff and passes props down to dumb components, v.s. having multiple container components which concern themselves only with the data they need, as well as fetching the data they need. I've gone back and forth between these two patterns and found that they each have pros/cons:
If I put a single container component at the top:
pro: All isFetching props and fetchSomeDependency() actions can be handled in one place.
con: the downside which is really annoying is that I find myself having to forward props and callbacks through multiple components, and certain components in the middle of the tree end up being tied up to this.
Here's a visual example of the issue that shows the relationships required props-wise:
<MyContainer
data={this.props.data}
isFetchingData={this.props.isFetchingData}
fetchData={this.props.fetchData}
>
{!isFetchingData &&
<MyModal
someData={props.data}
fetchData={props.fetchData}
>
<MyModalContent
someData={props.data}
fetchData={props.fetchData}
>
<SomethingThatDependsOnData someData={props.someData} />
<SomeButtonThatFetchesData onClick={props.fetchData} />
</MyModalContent>
</MyModal>
}
</MyContainer>
As you can see, <MyModal /> and <MyModalContent /> now need to be concerned with props that have nothing to do with it, seeing as a modal should be able to be re-used and only be concerned with stylistic qualities of a modal.
At first the above seemed great but once I got to 100+ components it all felt very tangled, and I found the complexity of these top-level container components to be too high for my liking, seeing as most of them (in the app I'm working on) depend on responses from 3+ API requests.
Then I decided to try multiple containers:
pro: Completely removes the need to forward props. It still makes sense to do it in some cases, but it's a lot more flexible.
pro: Way easier to refactor. I'm surprised at how I can significantly move around and refactor components without anything breaking, whereas in the other pattern things broke a lot.
pro: The complexity of each container component is much less. My mapStateToProps and mapDispatchToProps is more specific to the purpose of the component it's in.
con: Any component that depends on async stuff will always need to handle isFetching state in itself. This adds complexity that is not necessary in the pattern where its handled in a single container component.
So the main dilemma is that if I use one container, I get this un-necessary complexity in components between the root container and the leaf components. If I use multiple containers, I get more complexity in the leaf components, and end up with buttons that need to worry about isFetching even though a button should not be concerned about that.
I'd like to know if anyone has found a way to avoid both cons, and if so, what is the "rule of thumb" you follow to avoid this?
Thanks!
The way I have always seen it is to have your containers at the top most component of a logical components group other than your root/app component.
So if we have a simple search app that display results and lets assume the component heiarchy is such
<Root> <- setup the app
<App>
<Search/> <- search input
<Results/> <- results table
</App>
</Root>
I would make Search and Results redux aware containers. Because react component are suppose to be composable. You might have other components or pages that need display Results or Search. If you delegate the data fetch and store awareness to the root or app component, it make the components become dependent on each other/app. This make it harder down the line when you have to implement changes, now you have to change all the places that use them.
The exception to this is probably if you do have really tightly coupled logic between components. Even then, I would say then you should create a container that wraps your tightly coupled components since they won't be abled to be used realistically without each other.
Redux author Dan Abramov suggests that you use container components when you need them. That is, once you get to have too many props wiring up and down between components it's time to use containers.
He calls it an "ongoing process of refactoring".
See this article: https://medium.com/#dan_abramov/smart-and-dumb-components-7ca2f9a7c7d0
I wouldn't even consider using a single container approach. It pretty much entirely negates all advantages of redux. There is no need whatsoever to have a state management system if all your state is in one place and all your callbacks are in one place (root component).
I think there's a thin line to walk, though. I'm making an app where I've been at it for about 5 weeks (part time) and it's up to 3000 lines right now. It has 3 levels of view nesting, a routing mechanism i implemented myself, and components that are 10+ levels of nesting deep that need to modify state. I basically have one redux container for each big screen and it works marvelously.
However, if I click on my "clients" view, I get a clients listing which is fine, since my clients view is inside a redux container and gets the list of clients passed as props. However, when I click on one client, I'm really hesitant to do another redux container for the individual client's profile since it's only one additional level of passing props. It seems that depending on the scope of the app, you might want to pass props up to 1-2 levels past the redux container and if it's any more than that, then just create another redux container. Then again, if it's an even more complex app, then the mixing of sometimes using redux containers and some other times not using them could be even worse for maintainability. In short, my opinion is trying to minimize redux containers wherever possible but definitely not at the expense of complex prop chains, since that's the main point of using redux to begin with.
So it's been over 2 years since I've posted this question, and this whole time
I have been consistently working with React/Redux. My general rule of thumb now
is the following: Use more containers, but try to write components in such a way where they don't need to know about isFetching.
For example, here is a typical example of how I would have built a to-do list before:
function Todos({ isFetching, items }) {
if (isFetching) return <div>Loading...</div>
return (
<ul>
{items.map(item =>
<li key={item.id}>...</li>
)}
</ul>
)
}
Now I would do something more like:
function Todos({ items }) {
if (!items.length) return <div>No items!</div>
return (
<ul>
{items.map(item =>
<li key={item.id}>...</li>
)}
</ul>
)
}
This way, you only have to connect the data, and the component has no concerns about states of asynchronous API calls.
Most things can be written this way. I rarely need isFetching, but when I do it is typically because:
I need to prevent, for example, a submit button from being clicked a second time, which makes an API call, in which case the prop should probably be called disableSubmit rather than isFetching, or
I want to explicitly show a loader when something is waiting for an asynchronous response.
Now, you might think, "wouldn't you want to show a loader when items are being fetched in the above todos example?" but in practice, actually I wouldn't.
The reason for this is that in the above example, let's say you were polling for new todos, or when you add a todo, you "refetch" the todos. What would happen in the first example is that every time this happened, the todos would disappear and get replaced with "Loading..." frequently.
However, in the second example that is not concerned with isFetching, the new items are simply appended/removed. This is much better UX in my opinion.
In fact, before posting this, I went through all the UI code for an exchange interface I wrote which is quite complex and did not find a single instance of having to connect isFetching to a container component that I wrote.
You don't have to dispatch AND load your state in the same place.
In other words, your button can dispatch the async request, while another component can check if you're loading.
So for example:
// < SomeButtonThatFetchesData.js>
const mapDispatchToProps = (dispatch) => ({
onLoad: (payload) =>
dispatch({ type: DATA_LOADED, payload })
});
You'll need to have some middleware to handle a loading state. It needs to update isFetching when you're passing an async payload.
For example:
const promiseMiddleware = store => next => action => {
if (isPromise(action.payload)) {
store.dispatch({ type: ASYNC_START, subtype: action.type });
Then you can use it wherever you want:
// <MyContainer.js>
const mapStateToProps = (state) => ({
isFetching: state.isFetching
});
And load the data in your inner nested component:
// <SomethingThatDependsOnData.js>
const mapStateToProps = (state) => ({
someData: state.someData
});

How to logically combine react-router and redux for client- and server-side rendering

I'd like my React based SPA to render on server side (who's not these days). Therefore I want to combine React with react-router, redux and some build layer like isomorphic starterkit.
There is hapi universal redux which joins all together, but I am struggling with how to organize my flow. My data is coming from multiple endpoints of a REST API. Different components have different data needs and should load data just in time on the client. On the server instead, all data for a specific route (set of components) has to be fetched, and the necessary components rendered to strings.
In my first approach I used redux's middleware to create async actions, which load the data, return a promise, and trigger a SOME_DATA_ARRIVED action when the promise resolves. Reducers then update my store, components re-render, all good. In principle, this works. But then I realized, that the flow becomes awkward, in the moment routing comes into play.
Some component that lists a number of data records has multiple links to filter the records. Every filtered data set should be available via it's own URL like /filter-by/:filter. So I use different <Link to={...}> components to change the URL on click and trigger the router. The router should update the store then according to the state represented by the current URL, which in turn causes a re-render of the relevant component.
That is not easy to achive. I tried componentWillUpdate first to trigger an action, which asynchronously loaded my data, populated the store and caused another re-render cycle for my component. But this does not work on the server, since only 3 lifecycle methods are supported.
So I am looking for the right way to organize this. User interactions with the app that change the apps state from the users perspective should update the URL. IMO this should make the router somehow load the necessary data, update the store, and start the reconciliation process.
So interaction -> URL change -> data fetching -> store update -> re-render.
This approach should work on the server also, since from the requested URL one should be able to determine the data to be loaded, generate initial state and pass that state into the store generation of redux. But I do not find a way to properly do that. So for me the following questions arise:
Is my approach wrong because there is something I do not understand / know yet?
Is it right to keep data loaded from REST API's in redux's store?
Is'nt it a bit awkward to have components which keep state in the redux store and others managing their state by themselfs?
Is the idea to have interaction -> URL change -> data fetching -> store update -> re-render simply wrong?
I am open for every kind of suggestion.
I did set up exactly the same thing today. What we already had, was a react-router and redux. We modularized some modules to inject things into them – and viola – it works. I used https://github.com/erikras/react-redux-universal-hot-example as a reference.
The parts:
1. router.js
We return a function (location, history, store) to set up the router using promises. routes is the route definition for the react-router containing all your components.
module.exports = function (location, history, store) {
return new Bluebird((resolve, reject) => {
Router.run(routes, location, (Handler, state) => {
const HandlerConnected = connect(_.identity)(Handler);
const component = (
<Provider store={store}>
{() => <HandlerConnected />}
</Provider>
);
resolve(component);
}).catch(console.error.bind(console));
});
};
2. store.js
You just pass the initial state to createStore(reducer, initialState). You just do this on the server and on the client. For the client you should make the state available via a script tag (ie. window.__initialstate__).
See http://rackt.github.io/redux/docs/recipes/ServerRendering.html for more information.
3. rendering on the server
Get your data, set up the initial state with that data (...data). createRouter = router.js from above. res.render is express rendering a jade template with the following
script.
window.csvistate.__initialstate__=!{initialState ? JSON.stringify(initialState) : 'null'};
...
#react-start
!= html
var initialState = { ...data };
var store = createStore(reducer, initialState);
createRouter(req.url, null, store).then(function (component) {
var html = React.renderToString(component);
res.render('community/neighbourhood', { html: html, initialState: initialState });
});
4. adapting the client
Your client can then do basically the same thing. location could be HistoryLocation from React-Router
const initialState = window.csvistate.__initialstate__;
const store = require('./store')(initialState);
router(location, null, store).then(component => {
React.render(component, document.getElementsByClassName('jsx-community-bulletinboard')[0]);
});
To answer your questions:
Your approach seems right. We do the same. One could even include the url as part of the state.
All state inside of the redux store is a good thing. This way you have one single source of truth.
We are still working out what should go where right now. Currently we request the data on componentDidMount on the server it should already be there.

Resources