Next.js: How to cache initial data on the server (same for all users) - reactjs

I'm learning Next.js and started building one of my first applications - https://www.codart.io/
If you visit the site, you'll notice a delay until the collections become available: The application makes an initial graphql request to retrieve a json file with a number of NFT collections, which takes a few seconds. This data needs to be available across different components, so I built a global context to store the data (I'm not sure this is the best approach):
https://github.com/bartomolina/codart/blob/main/app/components/collections-context.tsx
const ArtBlocksContext = createContext({
aBCollections: [] as IABCollection[],
cACollections: [] as ICACollection[],
fetchCACollections: () => {},
});
export const useArtBlocks = () => useContext(ArtBlocksContext);
export const ArtblocksProvider = ({ children }: React.PropsWithChildren) => {
...
useEffect(() => {
fetchCACollections();
execute(ArtblocksCollectionsDocument, {}).then((result) => {
I wonder what would be the best way to cache this data (as the data doesn't change often, it could be cached at build time, or ideally, indicating an expiration date i.e. 24 hours). Note that the cache will be shared between all users.
I've been looking at some similar posts, and I got some ideas, although it seems there isn't a clear and simple way to do this:
Using Redux - Seems an overkill for a small project like this.
Use some custom caching libraries - I'd rather not use any external libraries.
Use getStaticProps along with the Context API - It seems you can't getStaticProps within the _app.ts page, so you would need to call it in every page where the context is used?
Use SWR and an API call - SWR will cache the data, but on a per-user basis? i.e. the first time a user visits the site, it will still take a few seconds to load.

Related

Storing E-Commerce Product Data On LocalStorage

I am busy working on a "Headless" E-Commmerce Application in ReactJS and I Have stumbled upon an issue regarding performance.
My application uses a serverless approach with commercejs meaning I fetch my products and every other data via API calls instead of a traditional approach that involves me setting up a database and having other backend tools.
I already have this:
const App = () => {
const [products, setProducts ] = useState([]);
const getProducts = async () => {
const { data } = await commerce.products.list();
setProducts(data)
}
useEffect(() => {
getProducts()
},[])
}
Which is all used to get products and assign them to the products variable which I map through to display them inside divs and this works perfectly.
Here's what I need help with:
Is it be a good idea for me to use localStorage to store products instead of making a new commerce.products.list() every time the same user visits a page that need to display products?
Also, if so, how would one go about creating a function that knows when to update the products localStorage if there has been any changes (say a new product has been added or there's been price change for a certain product) if the products are now being fetched from localStorage?
Surely the answer to number 2, if is a yes, will be something like: make api requests that will be called on intervals but my main question/concern is how to know exactly when localStorage should be updated in an event like this without calling the API every now and then?
Also the use of sessionStorage did cross my mind, and it seemed like a better idea as the products data will be updated every time a user visits the application but I have considered the possibility of a user resuming a current window which was left open in the background for weeks and not see anything new.
I know this isn't a typical code/error/solution question but still any form of guidance/assistance will be highly appreciated.

Algolia and Next.js - Index Updates and getStaticProps Side Effects

I've got a fairly complex Next.js site that is mostly statically rendered and I'm adding Algolia search into the site. The mechanics of Algolia require that you use their API and notify them of additional content every time it gets published.
My question is where in the Next app you notify Algolia. Given that my site is using getStaticProps to fetch data, I figured this is the logical place to notify and update Algolia. It works but wondering what others have done, best practice, tradeoffs, etc.
The lifecycle looks like this:
Get data from database via GraphQL (using headless CMS Prismic)
Normalize data inside Next before sending to Algolia
Send data to Algolia (their system reconciles old/new records be referencing uuid)
Code that makes this happen:
// Use nextjs getStaticProps, destructure default params
export async function getStaticProps({ params, preview = false, previewData }) {
// Data fetch (assume getAllNewsForLandingPage gets an array of obj)
pageData.data = await getAllNewsForLandingPage(params.uid, previewData)
// Format and normalize results
const algoliaFormattedData = dataFormatter(pageData.data)
// Send data back to Algolia, who will reconcile old and new data automatically
await AlgoliaIndex.saveObjects(algoliaFormattedData)
}

Best way to speed up a big React project by using "Snapshots" of critical routes (static renders)

We have a large and complex traditional React app that we've been building for the last couple of years. It loads an index.html injects javascript and gets data from an API as is usual. Unfortunately, cold load times are pretty bad (5 - 7 seconds on average). Once everything loads, it's snappy as usual but the cold load times are killing us in specific "critical" pages. These are our public user pages, in the format of: https://mywebsite/userId
We're looking for a way to dramatically speed up loading times for these routes, with methods that go beyond code-splitting or resource optimization. We already do those, and are serving our app off a CDN.
We've looked at creating static "snapshots" of these user pages, that we need to load very fast using something like react-static, and serving them as static versions and hydrating them later. Rewriting our project using something like next.js or gatsby is not an option as it would entail too much work. SSR is also not an option as our entire backend is coded in Django rather than Node.js
Are we on the right track? Is it possible / worth it to use react-static (or a similar tool) to do this? There is a LOT of documentation on how to create react-static projects from scratch but nothing on how to convert an existing project over, even if it's just a small subset of routes like we need.
Also, once the data changes on our user pages, how do we trigger a "rebuild" of the appropriate snapshot? Users don't update their data that often, about 3 of 4 times per month, but we have 3K users, so maybe 15 updates per hour would be the average. Can we trigger only a rebuild of the routes that actually changed?
Like you said, you could use react-static.
They have a feature which fills exactly with your need ( user's specific pages ).
In their example they use an array of posts to generate a specific static file for each of them.
This have a huge lesser amount of time taken to load, as it's only html static files.
Imagine having this scenario:
[
{
id: 'foo',
...
},
{
id: 'bar',
...
},
...
]
Following the example below this would generate something like this ( at runtime ):
- src
- pages
- blog
- posts
- foo // Specific post page
- bar // Specific post page
Look at into the example:
//static.config.js
export default {
// resolves an array of route objects
getRoutes: async () => {
// this is where you can make requests for data that will be needed for all
// routes or multiple routes - values returned can then be reused in route objects below
// ATTENTION: In here, instead of posts you'd fetch your users json data
const { data: posts } = await axios.get(
"https://jsonplaceholder.typicode.com/posts"
);
return [
// route object
{
// React Static looks for files in src/pages (see plugins below) and matches them to path
path: "/blog",
// function that returns data for this specific route
getData: () => ({
posts
}),
// an array of children routes
// in this case we are mapping through the blog posts from the post variable above
// and setting a custom route for each one based off their post id
children: posts.map(post => ({
path: `/post/${post.id}`,
// location of template for child route
template: "src/containers/Post",
// passing the individual post data needed
getData: () => ({
post
})
}))
},
];
},
// basic template default plugins
plugins: [
[
require.resolve("react-static-plugin-source-filesystem"),
{
location: path.resolve("./src/pages")
}
],
require.resolve("react-static-plugin-reach-router"),
require.resolve("react-static-plugin-sitemap")
]
};
You can use a Service Worker.
Load the important fast pages as static, then in the background, using the Service worker load the longer resources.
You can also use a Service Worker for smart caching.
For example, the server can set a cookie with the current resource version (comes with that first page), and the Service worker can compare this to it’s resource version , and decide whether to loat it from cache or go to the server.

Should I build a local data layer/app state to maintain state in a React Native/Firestore App?

A main selling point of Firestore is the ability to use it as a online/offline source of truth. I'm using it in this way right now: updating the Firestore document directly on an action, then listening to Firestore DB changes and mapping this back to local state. However, relying on this latency compensation and mapping back to local state is not sufficient for quick updates (taps, toggles even with a small document size). For example, toggles will "jitter" as the RN toggle presumptively shifts on tap, and the local state hasn't been updated until it already returns see video example. It appears worse on Android and the problem isn't strictly limited to basic toggles.
Does document size or query result size have a bigger impact on latency compensation? Our document size is very small right now with a worst case ~1000 query result set. We could make the documents 1000x bigger (100kb) and have a query result set of size 1. Update: Testing appears inconsistent here, latency compensation is not ideal in either case
Which of the following other things may impact latency compensation?
Using queries with custom indexes. Note: we're not currently reading from cache, we're using the JS SDK
Multiple writes. Would multiple writes to the same document make it worse (4 quick writes vs. 2 quick writes). Update: not clear this makes a big difference.
Using the native vs. JS module. We're currently using the Firestore Web SDK with an Expo app. Update: switching to native module via React-Native Firestore has no apparent performance improvement.
Is it common for people to build a local data shim layer / local app state with React Native / Firestore apps to help improve local performance speed? Are there any suggested libraries for this?
On app load, mount the listener, and export the result to context to be used through the app
const [user, setUser] = useState();
firebase.firestore().collection(`users/${user.uid}`).onSnapshot(qs => setUser(oldState => {
const newState = {};
qs.docChanges().forEach(change => {
if (change.type === "added" || change.type === "modified") {
newState[change.doc.id] = {
docid: change.doc.id,
...change.doc.data(),
};
} else if (change.type === "removed") {
delete oldState[change.doc.id];
}
});
return {
...oldState,
...newState,
};
}))
Sample component and function to toggle notifications: (switch is jittery)
const toggleNotifications = (user, value) => {
firebase.firestore().doc(`users/${user.uid}`).update({
wantNotifications: value,
});
};
const TestComponent = () => {
//gets from context, set in listener mounted on app load
const { user } = useUserContext();
return (
<Switch
value={user.wantNotifications}
onValueChange={value => toggleNotifications(user, value)}
/>
);
};
This not an answer, just a long comment :)
#learningAngular For example, in toggleNotifications just need to call an async action creator and don't worry about putting any logic inside react component.
Instead Redux pattern gives space to do some logics, in this case because user's last moment decision is source of truth so dispatch function would always set a local tempState and updatingStatus before start updating firestore, then after firestore promise either resolved or rejected dispatches an action to reducer to reset updatingStatus. Then a selector would check if updatingStatus is true to just rely on local tempState otherwise rely on listened firestore state. Finally, react component use selector result as currently valid state.
I am not answering this question because I don't have that much experience. I am also curious if a good solution is out there, but this is what I think is the solution at this moment.
I updated the answer with specific learnings, but after a lot of testing, my biggest general learnings so far are
Latency compensation can be very inconsistent even with the same data and environment. Listeners can take time to "warm up", as is mentioned in other questions. It is hard to have a standard metric here.
Document size DOES impact latency compensation. Everything else so far is inconclusive.

Next.js: Reduce data fetching and share data between pages

I'm looking for solutions for better data fetching in a Next.js app. In this question I'm not just looking for a solution, I'm looking for multiple options so we can look at the pros and cons.
The problem I have
Right now I have a few pages that all include a component that displays som static content and a that have some dynamic content that is fetched from an API. Each page do a fetch() in their getInitialProps() to get their own page data, but also the footer data, which is the same for all pages.
This of course works, but there is a lot of duplicated data fetching. The footer data will always be displayed for all pages and always be the same. It will also rarely be changed in the API, so no need for revalidate the data.
The answers I'm looking for
I'm not just looking to solve this one problem, I'm looking for an overview to learn some new practice for future projects as well. I like writing "obvious" code, so not looking for too hacky solutions, like writing to the window object etc. Simple solutions with less dependancies are preferred. The goal is a fast site. It's not that important to reduce network usage/API calls.
What I have thought so far
This is the possible solutions I've come up with, somewhat sorted from simple/obvious to more complex.
Do a fetch inside the Footer component (client side)
Do a fetch in getInitialProps (server side & client side) on all /pages
Do a fetch in _app.js with a HOC and hooking into it's getInitialProps() and add it to props, so data is available for all pages
Use zeit/swr and data prefetching to cache data
Use redux to store a global state
All of these "work", but most of them will refetch the data unnecessarily, and/or adds a bit more complexity. Here are the pros/cons as I see it (numbers are the same as above):
πŸ‘ Simple! Fetch code is only in one place, it's located where it's used. πŸ‘Ž Data is fetched after page is loaded, so the content "jumps" in to view. Data is refetched all the time.
πŸ‘ Simple! Data is fetched on the server, so content is available before the page is rendered. πŸ‘Ž Data is refetched for each page. We have to remember to fetch the same footer data for each page in their getInitialProps().
πŸ‘ We can do the fetch in one place and add it to all the pages props, so footer data is automatically available for all pages' props. πŸ‘Ž Might be a bit more complex for some to easily understand what's going on, as it requires a bit more understanding of how Next.js/React works. Still refetches the data for all pages. We now do two fetch() calls after each other (first in _app.js to load footer content, then in each page to get custom content), so it's even slower.
πŸ‘ Somewhat simple. We can use the prefetching to load data to cache even before the JS is loaded. After first page load, we will have fast data fetching. Can have fetch code directly in footer component. πŸ‘Ž The rel="preload" prefetching technique won't work with all types of fetching (for instance Sanity's client using groq). To not have "jumpy" content where the data is loaded after initial page load, we should provide useSWR() with initialData which still will require us to fetch data in getInitialProps(), but it would be enough to just do this on the server side. Could use the new getServerSideProps().
πŸ‘ We can load data once(?) and have it available throughout the application. Fast and less/no refetching. πŸ‘Ž Adds external dependency. More complex as you'll have to learn redux, even to just load one shared data object.
Current solution, using the solution described in bullet point number 2.
const HomePage = (props) => {
return (
<Layout data={props.footer}>
<Home data={props.page} />
</Layout>
)
}
// Not actual query, just sample
const query = `{
"page": *[_type == "page"][0],
"footer": *[_type == "footer"][0]
}`
HomePage.getInitialProps = async () => {
const data = await client.fetch(query)
return {
page: data.page
footer: data.footer
}
}
export default HomePage
Would love some more insight into this. I'm a missing something obvious?
O'right! I found this thread while I was looking for something else. But since I had to work on similar issues, I can give you some directions, and I will do my best to make it clear for you.
So there are some data which you want to have it share, across your app (pages/components).
Next.js uses the App component to initialize pages. You can override it and control the page initialization. to achieve that simply create _app.js file in root of pages directory. For more information follow this link: https://nextjs.org/docs/advanced-features/custom-app
Just like the way you can use getInitialProps in your pages to fetch data from your API, you can also use the same method in _app.js. So, I would fetch those data which I need to share them across my app and eliminate my API calls.
Well, Now I can think of two ways to share the data across my app
Using of createContext hooks.
1.1. Create a DataContext using createContext hooks. and wrap <Component {...pageProps} /> with your <DataContext.Provider>.
Here is a code snippet to give you a better clue:
<DataContext.Provider value={{ userData, footerData, etc... }}>
<Component {...pageProps} />
</DataContext.Provider>
1.2. Now in other pages/components you can access to your DataContext like following:
const { footerData } = useContext(DataContext);
And then you are able to do the manipulation in your front-end
populates props using getInitialProps
2.1. getInitialProps is used to asynchronously fetch some data, which then populates props. that would be the same case in _app.js.
The code in your _app.js would be something like this:
function MyApp({ Component, pageProps, footerData }) {
//do other stuffs
return (
<Component {...pageProps} footerData={footerData} />
;
}
MyApp.getInitialProps = async ({ Component, ctx }) => {
const footerRes = await fetch('http://API_URL');
const footerData = await footerRes.json();
let pageProps = {};
if (Component.getInitialProps) {
pageProps = await Component.getInitialProps(ctx);
}
return { pageProps, footerData };
};
2.2. Now in your pages (not in your components) you can access to props including those you have shared from _app.js
and you can start to do you manipulation.
Hope I could give you a clue and direction. Have fun exploring.

Resources