API caching for next JS - reactjs

Question

I'm building an app with Next.js... we have 100k+ pages and content changes daily, so using SSR and getServerSideProps.
Some of our data is coming from a headless CMS provider that charges by the request. I'd like to cache the API responses from this server for 24hrs.
What is the best way of going about this?
Is there a common library most folks use to do this?
Just looking for suggestions of approaches I should investigate (or great examples of how to do this).

Answer 1

I used this npm package:
https://www.npmjs.com/package/memory-cache
And then something like this:
import cacheData from "memory-cache";
async function fetchWithCache(url, options) {
const value = cacheData.get(url);
if (value) {
return value;
} else {
const hours = 24;
const res = await fetch(url, options);
const data = await res.json();
cacheData.put(url, data, hours * 1000 * 60 * 60);
return data;
}
}
Then if you want to fetch something with using the cache just call this function. Or it can be used as a midware in the requests. It checks if the data is already in the cache and returns it, or if not - it puts the data into the cache under the key. The key can be anything, I am using the url for instance.

Answer 2

In addition to Tobias Lins' answer:
At least if deploying on Vercel, you can use set Cache-Control headers in getStaticProps, getServerSideProps, API routes, etc to cache responses on Vercel's edge network. This solution does not require any additional dependencies and very minimal code.
api route example - source Vercel
// pages/api/user.js
export default function handler(req, res) {
res.setHeader('Cache-Control', 's-maxage=86400');
res.status(200).json({ name: 'John Doe' });
}
Example in getServerSideProps - Source NextJS
// This value is considered fresh for ten seconds (s-maxage=10).
// If a request is repeated within the next 10 seconds, the previously
// cached value will still be fresh. If the request is repeated before 59 seconds,
// the cached value will be stale but still render (stale-while-revalidate=59).
//
// In the background, a revalidation request will be made to populate the cache
// with a fresh value. If you refresh the page, you will see the new value.
export async function getServerSideProps({ req, res }) {
res.setHeader(
'Cache-Control',
'public, s-maxage=10, stale-while-revalidate=59'
)
return {
props: {},
}
}
I believe you'd want to use:
res.setHeader('Cache-Control', 's-maxage=1440000')
Here are some other useful links for caching on Vercel:
https://vercel.com/docs/concepts/functions/edge-caching
https://vercel.com/docs/concepts/edge-network/overview
https://vercel.com/docs/concepts/edge-network/caching
https://vercel.com/docs/concepts/edge-network/headers
For your specific case, you also may want to look into using getStaticPaths with getStaticProps. You can use fallback: true on getStaticPaths to only build pages when they're visited (you can still build your post popular pages at initial build time).
https://nextjs.org/docs/basic-features/data-fetching#the-fallback-key-required
I know this is an old post, but for others googling (at least those deploying on Vercel), these solutions should help where revalidate in getStaticProps does not.

Answer 3

You could use getStaticProps from Next.js for SSG
They currently have a revalidate property that you can return, that defines how often the content should be re-fetched.
Take a look here:
https://nextjs.org/blog/next-9-5#stable-incremental-static-regeneration

Answer 4

This is how we did it without any 3rd party libraries, as in our use-case we only had to cache a relatively smaller amount of global data(header/footer menus) which was shared across the site.
The data was coming from a CMS via GraphQL.
We ran an async method getGlobalData on each page from on getStaticProps method and then returned the cached data to the page component via props.
import fs from 'fs';
import path from 'path';
// Cache files are stored inside ./next folder
const CACHE_PATH = path.join(__dirname, 'globalData.json');
export default async function getGlobalData() {
let cachedData;
// #1 - Look for cached data first
try {
cachedData = JSON.parse(fs.readFileSync(CACHE_PATH, 'utf8'));
} catch (error) {
console.log('❌ CACHE NOT INITIALIZED');
}
// #2 - Create Cache file if it doesn't exist
if (!cachedData) {
// Call your APIs to-be-cached here
const data = await fetchGlobalData();
// Store data in cache files
// this always rewrites/overwrites the previous file
try {
await fs.writeFileSync(
CACHE_PATH,
JSON.stringify(data),
err =>throw err
);
console.log('💾 CACHE FILE WRITTEN SUCCESSFULLY');
} catch (error) {
console.log('❌ ERROR WRITING MEMBERS CACHE TO FILE\n', error);
}
cachedData = data;
}
return cachedData;
}
Call getGlobalData method from getStaticProps.
export async function getStaticProps({ preview = false }) {
const globalData = await getGlobalData();
// call other page-specific/non-shared APIs here
// ...
return { props: { globalData } };
}
References
https://flaviocopes.com/nextjs-cache-data-globally/
Note if you get an error saying fs or path is unknown or invalid, then please understand that, the above code is supposed to be running or referenced "serverside" i.e only inside getStaticProps or getServerSideProps. If you import and reference it "browser-side", say somewhere inside your components or on the page (other than methods mentioned above), then you will get an error, as there is no filesystem(fs) or path modules on browser. They are only available on node.