Best way to speed up a big React project by using "Snapshots" of critical routes (static renders) - reactjs

We have a large and complex traditional React app that we've been building for the last couple of years. It loads an index.html injects javascript and gets data from an API as is usual. Unfortunately, cold load times are pretty bad (5 - 7 seconds on average). Once everything loads, it's snappy as usual but the cold load times are killing us in specific "critical" pages. These are our public user pages, in the format of: https://mywebsite/userId
We're looking for a way to dramatically speed up loading times for these routes, with methods that go beyond code-splitting or resource optimization. We already do those, and are serving our app off a CDN.
We've looked at creating static "snapshots" of these user pages, that we need to load very fast using something like react-static, and serving them as static versions and hydrating them later. Rewriting our project using something like next.js or gatsby is not an option as it would entail too much work. SSR is also not an option as our entire backend is coded in Django rather than Node.js
Are we on the right track? Is it possible / worth it to use react-static (or a similar tool) to do this? There is a LOT of documentation on how to create react-static projects from scratch but nothing on how to convert an existing project over, even if it's just a small subset of routes like we need.
Also, once the data changes on our user pages, how do we trigger a "rebuild" of the appropriate snapshot? Users don't update their data that often, about 3 of 4 times per month, but we have 3K users, so maybe 15 updates per hour would be the average. Can we trigger only a rebuild of the routes that actually changed?

Like you said, you could use react-static.
They have a feature which fills exactly with your need ( user's specific pages ).
In their example they use an array of posts to generate a specific static file for each of them.
This have a huge lesser amount of time taken to load, as it's only html static files.
Imagine having this scenario:
[
{
id: 'foo',
...
},
{
id: 'bar',
...
},
...
]
Following the example below this would generate something like this ( at runtime ):
- src
- pages
- blog
- posts
- foo // Specific post page
- bar // Specific post page
Look at into the example:
//static.config.js
export default {
// resolves an array of route objects
getRoutes: async () => {
// this is where you can make requests for data that will be needed for all
// routes or multiple routes - values returned can then be reused in route objects below
// ATTENTION: In here, instead of posts you'd fetch your users json data
const { data: posts } = await axios.get(
"https://jsonplaceholder.typicode.com/posts"
);
return [
// route object
{
// React Static looks for files in src/pages (see plugins below) and matches them to path
path: "/blog",
// function that returns data for this specific route
getData: () => ({
posts
}),
// an array of children routes
// in this case we are mapping through the blog posts from the post variable above
// and setting a custom route for each one based off their post id
children: posts.map(post => ({
path: `/post/${post.id}`,
// location of template for child route
template: "src/containers/Post",
// passing the individual post data needed
getData: () => ({
post
})
}))
},
];
},
// basic template default plugins
plugins: [
[
require.resolve("react-static-plugin-source-filesystem"),
{
location: path.resolve("./src/pages")
}
],
require.resolve("react-static-plugin-reach-router"),
require.resolve("react-static-plugin-sitemap")
]
};

You can use a Service Worker.
Load the important fast pages as static, then in the background, using the Service worker load the longer resources.
You can also use a Service Worker for smart caching.
For example, the server can set a cookie with the current resource version (comes with that first page), and the Service worker can compare this to it’s resource version , and decide whether to loat it from cache or go to the server.

Related

Next.js: How to cache initial data on the server (same for all users)

I'm learning Next.js and started building one of my first applications - https://www.codart.io/
If you visit the site, you'll notice a delay until the collections become available: The application makes an initial graphql request to retrieve a json file with a number of NFT collections, which takes a few seconds. This data needs to be available across different components, so I built a global context to store the data (I'm not sure this is the best approach):
https://github.com/bartomolina/codart/blob/main/app/components/collections-context.tsx
const ArtBlocksContext = createContext({
aBCollections: [] as IABCollection[],
cACollections: [] as ICACollection[],
fetchCACollections: () => {},
});
export const useArtBlocks = () => useContext(ArtBlocksContext);
export const ArtblocksProvider = ({ children }: React.PropsWithChildren) => {
...
useEffect(() => {
fetchCACollections();
execute(ArtblocksCollectionsDocument, {}).then((result) => {
I wonder what would be the best way to cache this data (as the data doesn't change often, it could be cached at build time, or ideally, indicating an expiration date i.e. 24 hours). Note that the cache will be shared between all users.
I've been looking at some similar posts, and I got some ideas, although it seems there isn't a clear and simple way to do this:
Using Redux - Seems an overkill for a small project like this.
Use some custom caching libraries - I'd rather not use any external libraries.
Use getStaticProps along with the Context API - It seems you can't getStaticProps within the _app.ts page, so you would need to call it in every page where the context is used?
Use SWR and an API call - SWR will cache the data, but on a per-user basis? i.e. the first time a user visits the site, it will still take a few seconds to load.

In Next.js 13 app directory, how do I incrementally generate new pages?

I have multiple items from a CMS. /items/1 all the way to /items/9999. The content is immutable, so I don't have to worry about revalidateing them.
However, items do get added to the CMS frequently, maybe multiple times in a day. I want to make a static website. How can I add new static pages incrementally?
The CMS isn't handled by me, so there's no way I can add a hook.
As per the docs, by default, route segment parameters that were not statically generated at build-time by generateStaticParams function will be generated on demand. These non-generated segments will use Streaming Server Rendering. This is basically the equivalent to fallback: true on getStaticPaths function on pages folder page components.
Just make sure to perform the appropriate checks on your page component in case the requested data doesn't exist in the CMS. That way you can throw a Not Found error and render a 404 UI making use of the not-found.js file. Example from the docs:
import { notFound } from 'next/navigation';
export default async function Profile({ params }) {
const user = await fetchUser(params.id);
if (!user) {
notFound();
}
// ...
}

Algolia and Next.js - Index Updates and getStaticProps Side Effects

I've got a fairly complex Next.js site that is mostly statically rendered and I'm adding Algolia search into the site. The mechanics of Algolia require that you use their API and notify them of additional content every time it gets published.
My question is where in the Next app you notify Algolia. Given that my site is using getStaticProps to fetch data, I figured this is the logical place to notify and update Algolia. It works but wondering what others have done, best practice, tradeoffs, etc.
The lifecycle looks like this:
Get data from database via GraphQL (using headless CMS Prismic)
Normalize data inside Next before sending to Algolia
Send data to Algolia (their system reconciles old/new records be referencing uuid)
Code that makes this happen:
// Use nextjs getStaticProps, destructure default params
export async function getStaticProps({ params, preview = false, previewData }) {
// Data fetch (assume getAllNewsForLandingPage gets an array of obj)
pageData.data = await getAllNewsForLandingPage(params.uid, previewData)
// Format and normalize results
const algoliaFormattedData = dataFormatter(pageData.data)
// Send data back to Algolia, who will reconcile old and new data automatically
await AlgoliaIndex.saveObjects(algoliaFormattedData)
}

Routing localization with NextJS

I'm migrating a website developed with Gatsby to NextJS, but something I could achieve using Gatsby's createPage API is localizing the app's routes, but until now I couldn't achieve this with NextJS APIs.
I'm using Next v10.0.1 for this.
As I see in other threads regarding this type of resource, this is actually kinda confusing of what it actually means, so here goes an example of what is the desired result:
User access route /my-data/1234 (where the NextJS equivalent routing would be: /my-data/[dataId].js)
User must be able to access the same page but translated in the URL /pt/meus-dados/1234 (using, for example, portuguese translation).
Some guesses on how to achieve that keeping Next's static optimizations (Static rendering and Incrementing Static Rendering)?
I actually found an answer which is pretty useful for my use case, I'm using NextJS rewrites for that, maybe not the best solution, but fits my needs well.
I use a single file for each route, so the directory structure should be something like this:
pages
-- my-data
-- [id].js
then I'll have some kind of internationalization, in my case I'm using react-i18next, won't think about the aspects of implementations for the library here, it could also be achieved with any other.
Next step is to set a translation somewhere for the pages routes, for example, add an entry for the i18next messages named routes containing a key-value pair for the routes translations. e.g (intl/pt.json):
{
...
"routes": {
"/my-data/:id": "/meus-dados/:id"
}
...
}
and then use NextJS rewrites, so you will have to import the intl messages (e.g: from intl/pt.json) and map them as rewrites in next.config.js:
# next.config.js
...
async rewrites() {
// languages here is a key-value pair object containing the structure { [language]: { routes: {...} } }
// in my case, imported from `intl/pt.json`
const intlRewrites = Object.entries(languages).reduce((rewrites, [ language, { routes } ]) => {
return [
...rewrites,
...Object.entries(pages).map(([ path, translatedPath ]) => {
const source = translatedPath
const destination = path // here you can write a logic to add a language prefix if needed
return { source, destination }
})
]
}, [])
return intlRewrites
}
From my experience, optimizations like ISG work fine. Should work with SSG too, but I haven't tested it.
I've been tackling this exact problem, and whilst I don't yet have a solution that integrates directly into NextJS, this can be achieved fairly simply before your project is compiled.
If you were to organise your pages directory as follows, it should work as you expect:
// Before
pages
-- my-data
-- [id].js
// After
pages
-- pt
-- meus-dados
-- [id].js
-- en
-- my-data
-- [id].js
However the developer experience here isn't nice. So what I have done to solve this currently is written a simple build step that runs before next build. It takes a regular pages directory and converts it to the above format, allowing next build to run against a version that works as intended for the translated paths. This allows SSG and ISG to work as expected.
Ideally I'd like to hook into the Next ecosystem so this works seamlessly for dev and build, but I haven't yet gotten that far

Next.js: Reduce data fetching and share data between pages

I'm looking for solutions for better data fetching in a Next.js app. In this question I'm not just looking for a solution, I'm looking for multiple options so we can look at the pros and cons.
The problem I have
Right now I have a few pages that all include a component that displays som static content and a that have some dynamic content that is fetched from an API. Each page do a fetch() in their getInitialProps() to get their own page data, but also the footer data, which is the same for all pages.
This of course works, but there is a lot of duplicated data fetching. The footer data will always be displayed for all pages and always be the same. It will also rarely be changed in the API, so no need for revalidate the data.
The answers I'm looking for
I'm not just looking to solve this one problem, I'm looking for an overview to learn some new practice for future projects as well. I like writing "obvious" code, so not looking for too hacky solutions, like writing to the window object etc. Simple solutions with less dependancies are preferred. The goal is a fast site. It's not that important to reduce network usage/API calls.
What I have thought so far
This is the possible solutions I've come up with, somewhat sorted from simple/obvious to more complex.
Do a fetch inside the Footer component (client side)
Do a fetch in getInitialProps (server side & client side) on all /pages
Do a fetch in _app.js with a HOC and hooking into it's getInitialProps() and add it to props, so data is available for all pages
Use zeit/swr and data prefetching to cache data
Use redux to store a global state
All of these "work", but most of them will refetch the data unnecessarily, and/or adds a bit more complexity. Here are the pros/cons as I see it (numbers are the same as above):
πŸ‘ Simple! Fetch code is only in one place, it's located where it's used. πŸ‘Ž Data is fetched after page is loaded, so the content "jumps" in to view. Data is refetched all the time.
πŸ‘ Simple! Data is fetched on the server, so content is available before the page is rendered. πŸ‘Ž Data is refetched for each page. We have to remember to fetch the same footer data for each page in their getInitialProps().
πŸ‘ We can do the fetch in one place and add it to all the pages props, so footer data is automatically available for all pages' props. πŸ‘Ž Might be a bit more complex for some to easily understand what's going on, as it requires a bit more understanding of how Next.js/React works. Still refetches the data for all pages. We now do two fetch() calls after each other (first in _app.js to load footer content, then in each page to get custom content), so it's even slower.
πŸ‘ Somewhat simple. We can use the prefetching to load data to cache even before the JS is loaded. After first page load, we will have fast data fetching. Can have fetch code directly in footer component. πŸ‘Ž The rel="preload" prefetching technique won't work with all types of fetching (for instance Sanity's client using groq). To not have "jumpy" content where the data is loaded after initial page load, we should provide useSWR() with initialData which still will require us to fetch data in getInitialProps(), but it would be enough to just do this on the server side. Could use the new getServerSideProps().
πŸ‘ We can load data once(?) and have it available throughout the application. Fast and less/no refetching. πŸ‘Ž Adds external dependency. More complex as you'll have to learn redux, even to just load one shared data object.
Current solution, using the solution described in bullet point number 2.
const HomePage = (props) => {
return (
<Layout data={props.footer}>
<Home data={props.page} />
</Layout>
)
}
// Not actual query, just sample
const query = `{
"page": *[_type == "page"][0],
"footer": *[_type == "footer"][0]
}`
HomePage.getInitialProps = async () => {
const data = await client.fetch(query)
return {
page: data.page
footer: data.footer
}
}
export default HomePage
Would love some more insight into this. I'm a missing something obvious?
O'right! I found this thread while I was looking for something else. But since I had to work on similar issues, I can give you some directions, and I will do my best to make it clear for you.
So there are some data which you want to have it share, across your app (pages/components).
Next.js uses the App component to initialize pages. You can override it and control the page initialization. to achieve that simply create _app.js file in root of pages directory. For more information follow this link: https://nextjs.org/docs/advanced-features/custom-app
Just like the way you can use getInitialProps in your pages to fetch data from your API, you can also use the same method in _app.js. So, I would fetch those data which I need to share them across my app and eliminate my API calls.
Well, Now I can think of two ways to share the data across my app
Using of createContext hooks.
1.1. Create a DataContext using createContext hooks. and wrap <Component {...pageProps} /> with your <DataContext.Provider>.
Here is a code snippet to give you a better clue:
<DataContext.Provider value={{ userData, footerData, etc... }}>
<Component {...pageProps} />
</DataContext.Provider>
1.2. Now in other pages/components you can access to your DataContext like following:
const { footerData } = useContext(DataContext);
And then you are able to do the manipulation in your front-end
populates props using getInitialProps
2.1. getInitialProps is used to asynchronously fetch some data, which then populates props. that would be the same case in _app.js.
The code in your _app.js would be something like this:
function MyApp({ Component, pageProps, footerData }) {
//do other stuffs
return (
<Component {...pageProps} footerData={footerData} />
;
}
MyApp.getInitialProps = async ({ Component, ctx }) => {
const footerRes = await fetch('http://API_URL');
const footerData = await footerRes.json();
let pageProps = {};
if (Component.getInitialProps) {
pageProps = await Component.getInitialProps(ctx);
}
return { pageProps, footerData };
};
2.2. Now in your pages (not in your components) you can access to props including those you have shared from _app.js
and you can start to do you manipulation.
Hope I could give you a clue and direction. Have fun exploring.

Resources