Let's say I have the following stricture for a Javascript monorepo
-- module a
package.json
-- module b
package.json
package.json
Module A and Module B both depend on lodash.
With pnpm, is there any way to ensure that they both use the same version of lodash? Like perhaps installing lodash in the root directory and saying "use whatever version is in the root of the workspace"?
P.S. I know that pnpm allows or workspaces, but to my understanding (which can be wrong), that's only used if the dependencies are already a module in the monorepo — not for 3rd-party dependencies.
You can use the overrides field in your package.json to achieve this.
https://pnpm.io/package_json#pnpmoverrides
{
"pnpm": {
"overrides": {
"lodash": "4.17.20"
}
}
}
A similar feature exists in Yarn, called resolutions. Apparently there are some differences, though I'm not familiar with the specifics. The feature was added here.
Update - alternative approach
You can also try syncpack if the overrides solution doesn't sit well with you. At my organization we're slowly moving towards syncpack for issues where the dependencies are only requested by our monorepo packages; the rationale for this is that too many overrides were accumulating over time.
But for ensuring the versions of dependencies of dependencies, overrides is still what you need.
e.g. if you install react-dom and it depends on scheduler and you need a specific version of scheduler for some reason, then the overrides approach is your only option - syncpack can't help there.
Related
Can all dependencies of a React-based website be declared as devDependencies instead of (production) dependencies?
Since the build folder is not versioned together with the code, each checkout of the repository will require a build operation to run the website. Thus every checkout can be considered a development environment.
dependencies is an object of dependencies that the app itself use, E.G react, react-dom.
On the other hand, devDependencies are dependencies that are only used when you're developing or building the package, E.G. webpack, babel, prettier.
Whether you need npm install in production or not, filling these fields appropriately will (at least) provide clarity for the readers.
Now answering your question, module bundlers and transpilers don't care whether the dependency is in devDependencies or dependencies or is it there in the first place. All it cares about that it exists in the node_modules folder.
When running react-scripts build (Create-React-App) in a sub folder (c:\Repos\web_app1\api_ui) with it's own package.json, node_modules folder, etc. I get the following error:
react-scripts build
There might be a problem with the project dependency tree.
It is likely not a bug in Create React App, but something you need to fix locally.
The react-scripts package provided by Create React App requires a dependency:
"babel-loader": "8.0.4"
Don't try to install it manually: your package manager does it automatically.
However, a different version of babel-loader was detected higher up in the tree:
c:\Repos\web_app1\node_modules\babel-loader (version: 7.1.4)
Updating the parent folder's (c:\Repos\web_app1) babel-loader to v8.0.4 is not an option as web_app1 depends on babel-loader v7.1.4
Deleting the node_modules in c:\Repos\web_app1 is not an option. This is the parent application and needs its own node_modules.
My fix was adding SKIP_PREFLIGHT_CHECK=true to and .env file. This seems like a hack and I would like another fix that involves building through the preflight check.
The package-lock.json in the sub folder (c:\Repos\web_app1\api_ui) has the correct babel-loader version (8.0.4), so why is it going to the parent folder?
Is there a way to ignore parent folder or higher tree node_modules when building react-scripts in a sub folder?
This puzzled me a lot too and I did some research, looking in to the source code of react-scripts version 4.0.3. This version requires babel-loader v. 8.1.0. With storybook installed (requiring version ^8.0.0), we get a different babel-loader in the top level node_modules so that we end up with:
|-src
|-node_modules
|
|-storybook 6.3.12
|-babel-loader 8.2.6
|-react-scripts 4.0.3
|
|-node_modules
|
|-babel-loader 8.1.0
I get the same error message as a lot of people have seen, and I think to myself, shouldn't any use of babel-loader in react-scripts get the babel-loader in its own node_modules, namely version 8.1.0? I can delete node_modules and package-lock.json any number of times and it seems the problem always persists.
How you describe it in the question is in fact how the package managers (at least npm) works, e.g. importing babel-loader from react-scripts WOULD give it version 8.1.0 from its own node_modules... but we forget about two things, that combined will cause problems:
Package hoisting
Transitive dependencies
Package hoisting
I won't go into details on when package hoisting occurs, but it happens. Basically, it amounts to a dependency package B of a dependency A being added to the project root node_modules (or some other parent node_modules) instead of to the node_modules of A.
So instead of
|-src
|-node_modules
|
|-A
|
|-node_modules
|
|-B
... we get ...
|-src
|-node_modules
|
|-A
|
|-node_modules (might exist anyway)
|-B
Due to how npm works (and as you correctly understood it in the question), if B cannot be found in node_modules/A/node_modules, it will look in node_modules for the package, which makes this work. Hoisting can be an optimization when many packages all need compatible dependencies so that we, instead of storing N similar versions of the dependency, can store only one. It also simplifies the folder structure of the project root node_modules which is then easier to debug. As a matter of fact, the default is to hoist any packages that can be hoisted, e.g. that don't already exist in the node_modules to hoist to.
Usually, fallback to the naive strategy, with "all dependencies for a package in its own node_modules", is used whenever we have conflicting package versions. This is what has happened in the top example with two different versions of babel-loader; storybook really consists of several packages and multiple of them uses babel-loader version ^8.0.0. When storybook is installed before react-scripts, it takes the most recent version of babel-loader which meets the constraint, and it hoists it to the top-level node_modules. This then causes the familiar problems when react-scripts is installed. I have another project with the same setup, but where react-scripts was installed before storybook. There, babel-loader version 8.1.0 is hoisted instead, and since this package also meets the requirements of storybook (^8.0.0), no more babel-loader's are needed. Here, create-react-scripts does not complain. It is of course desirable that npm could figure this order out by itself, which is also optimal from a storage perspective (only one babel-loader instead of two), but as far as I know, npm uses alphabetical order when processing dependencies.
Transitive dependencies
Even though package A is a dependency of my project, A can have dependencies of its own. These are transitive dependencies with respect to my project.
Why doesn't it work?
In the source for react-script version 4.0.3, the file verifyPackageTree is responsible for running the check hat results in the boring error message in the question. This file is no longer around in later prereleases, but in it at the top, we find the comment:
// We assume that having wrong versions of these
// in the tree will likely break your setup.
// This is a relatively low-effort way to find common issues.
A few rows down it says:
// These are packages most likely to break in practice.
// See https://github.com/facebook/create-react-app/issues/1795 for reasons why.
// I have not included Babel here because plugins typically don't import Babel (so it's not affected).
Looking at the referenced thread, there is a discussion about why create-react-app breaks when non-compatible dependencies are installed with the same question as above... WHY doesn't it work?
The final conclusion seems to be that if, for example, react-scripts imports a package X that has been hoisted, this package won't reside in the node_modules under react-scripts but instead in the root node_modules, where (with respect to the topmost example) also babel-loader version 8.2.6 resides (as opposed to version 8.1.0 in the node_modules folder of react-scripts). Now if X were to import babel-loader, it would get the wrong babel-loader, e.g. NOT version 8.1.0 as expected but instead 8.2.6. In the thread this is considered a bug in npm even though I don't exactly know why it would be a bug. Perhaps, there could be some flag indicating whether it is ok to hoist a package or not because if hoisting altogether is considered a bug, it seems like as step backwards.
Because the writers of create-react-app can't know whether some dependency is hoisted or not, they have implemented this simple check which throws a warning. For advanced users they give a chance to opt out of the check with the SKIP_PREFLIGHT_CHECK flag. Also, it is good to remember that the error comes from create-react-app, not npm, so it really doesn't say anything about how npm works, only what the creators of create-react-app considers a problem (which was indeed equally confusing initially).
I had to fork libA just to update one of its dependencies. LibA does not have typescript definitions on its own, but do have an entry in DefinitelyTyped.
How can I match the DefinitelyTyped with my fork (since my fork does not match the same name and it is now #myrepo/libA)?
Before I was using:
import libA, {libAProps} from 'libA';
One possible solution would be to keep the import unchanged but "rewrite" the module to your fork using the paths option in tsconfig.json. This way TypeScript will still use libA for looking up the type definitions, but the actual code comes from your fork.
{
"compilerOptions": {
"baseUrl": ".", // This must be specified if "paths" is.
"paths": {
"libA": ["node_modules/#myrepo/libA"] // This mapping is relative to "baseUrl"
}
}
}
This is probably a common problem, which I also encountered. My solution:
Instead of changing the package name (or scope), I publish my forks to a self-hosted npm registry:
I'm running a local Verdaccio, using Docker Compose. The official npm registry registry.npmjs.org is configured as an uplink, so Verdaccio becomes a transparent overlay and I can locally publish builds of my forks and still fetch other packages from the official registry.
Because the package name of the fork doesn't need to be changed, DefinitelyTyped packages are still working.
Be aware that the solution is not optimal:
additional maintenance cost (but there may be more benefits, e.g. Verdaccio is acting as a cache, accelerating future npm installs)
it is not quite obvious what packages or package versions are locally published forks.
Regarding 2., I established some kind of solution by manually adding a custom pre-release version to the original package version. Example:
original version: 1.0.2
version of fork: 1.0.2-sbusch.1
For further patches from my side that need to be built and published, I increment the last digit to 1.0.2-sbusch.2, 1.0.2-sbusch.3 and so on. As pre-release versions get lower priority according to semver.org, npm outdated (which I run quite often) always shows the installed forks as available upgrades. (Note: 1.0.3-sbusch.1 could work better in praxis as it indicates the "pre-release of the next official, to-be-released version". Didn't tried this yet, though)
Other solutions I tried: npm link, yarn link, symlinks in node_modules. Had a poor experience with them, because they're skipping the regular build/publish/install workflow and you're using their sources, not their (transpiled) builds.
Any feedback greatly appreciated!
Bootstrapped CRA 3.0 app and then ejected.
Inspecting my package.json.
Found multiple packages usages of which are not really clear to me.
"semver": "6.0.0"
Why do I need this? No usages found in config/* nor scripts/*. It seems like an artifact of react-scripts validation-like logic for related packages, so it looks like a piece of bloat in my application dependencies.
"react-app-polyfill": "^1.0.0"
Polyfills for IEs and etc.? OK, but why a separate package? It's frightening to me to use some unknown package on top of core-js or babel-polyfill. And again, no usages found in an initial code base.
P.S. I'm not asking what these packages are, I'm asking why do I see them being unused in ejected scripts
Update: react-app-polyfill/jsdom is used in Jest setup files
create-react-app uses a package named react-scripts which hides all the different packages it uses underneath.
When you eject an application, the dependencies used by react-scripts are copied over to your own package.json. But for some scripts like the eject, various sections are removed using babel annotation like #remove-on-eject-begin. The dependencies used in these sections persist even after you eject.
This is how you find packages like semver that are not used anywhere in your application code. In an unejected create react app, that package would be used for verifying the semantic versions before ejecting.
These are safe to remove now. But they would not be factored into the static bundle you are creating anyhow.
I started to build a component library that I want to publish on NPM (and reuse in other apps) by creating a React app using create-react-app. Unfortunately the create-react-app default configuration doesn't seem to support building such component libraries (see these issues). So it seems I have to eject.
Do you have some instructions what to configure after ejecting to make this a component library I can reuse in other apps (I guess some Webpack stuff)?
Re-framing my comments into an answer.
Don't eject! CRA hides a lot of stuff under the hood and ejecting throws it up. It's relatively easier to just clone your src into another project with the tooling setup done.
And it's not very difficult to setup one yourself! Here's the things you will need to do:
Basic babel and webpack configuration so your code compiles.
Make sure React and React-DOM are added as external dependencies in your package.json file and added as alias in your webpack.config.js. (A thorough discussion is here.) This is important to ensure you ship the smallest bundles only. Also, React doesn't play well with multiple copies.
Optionally, do the same for any other heavier libraries, e.g. Material-UI, Bootstrap, Lodash etc.
In webpack's configuration, decide how you want your library exports should be? You should be good with UMD.
Add main (and optionally module) entries in your package.json to let npm imports know where the components should be imported from.
Publish it.
Done!
Or, you can simply clone one of these super thin component projects and put in your components there -
https://github.com/wangzuo/react-progress-label
https://github.com/aaronshaf/react-toggle
There are more complete starter-kits as well, but IMO it's important to first know the details yourself before you abstract them away. Once you are familiar with the process, you should try leveraging the kits too as they go much beyond the basics, like test integration, react-storybook support and great npm publishing support.
UPDATES:
The purpose of CRA is to allow quick experimentation and on-boarding to React development, without going through the (somewhat complicated to a beginner) tooling setup. The intended use case of CRA, as denoted by the "App" in the name, is the whole app, not a component. Although developing anything built on language features like ES6 takes some tooling, an app typically takes more setup than a component. E.g. you also need a server component which hosts the generated code. CRA does all this and more for you.
And if you are working on an app, you will eject when you want to take control of the server side of it. It adds value in that case as you will get the basic hosting code auto-generated by CRA during eject.
To address test needs, CRA also has Jest integration, which is a test runner having React specific features like snapshot testing. Again, setting it up manually with your build pipeline is a handful and CRA once again hides all of this complexity from you, so you can simply focus on writing the tests.
This is super simple - you don't actually need webpack if you want to publish a package. Don't install it just for that if you don't want to.
If you use ES6:
You can use a simple script to create a dist folder that babel will create the files after transpiling:
"scripts": {
// other scripts
// .. might want to change "rm -rf" below if you're on a PC
"build:production": "rm -rf es && cross-env NODE_ENV=production node_modules/.bin/babel ./src -d es"
},
What this does is:
Remove the es folder, which is a build folder.
Runs babel for your files (in case they are located in src folder, change it to wherever your files reside) and create the es folder with the transpiled files.
In order to use the script above you need to install the following dependencies:
babel-cli / babel-core / babel-preset-es2015 / cross-env
Make sure you have a .babelrc file so that babel will work:
{
"presets": [
["es2015", { "modules": false }]
]
}
Now all you have to do (after running npm run build:production) is to run:
npm publish
And your library is published in npm. This is also the command you run if you want to update, just don't forget to update the version number in your package.json.
In case you have files you dont want published to npm, just have a file in your root called .npmignore (similar to .gitignore) and list everything you want excluded.
If you don't use ES6:
In this case you don't need to transpile anything, just go straight to the npm publish section above and run it on the root folder you want published.
create-react-app is meant for quick setting up and development of apps, not really for development of libraries. For one thing, create-react-app creates an index.html file when building, which is usually not needed by libraries. Libraries require a different set of configuration (much less).
I faced this problem myself and have written a React component boilerplate for writing and publishing React components: https://github.com/yangshun/react-component-starter, after referencing how popular ES6 libraries were written, such as Redux.
Pros:
Similar to create-react-app, batteries are included
It has got all the important areas covered: development, linting, testing, and distributing
Minimal configuration
Babel for transpilation to ES5 and CommonJS format
ESLint included and configured
Jest tests examples
Webpack configuration that compiles to UMD
React Storybook for quick development of the components
Support for stylesheet distribution alongside your component
Cons:
Highly opinionated in terms of choice of tooling; they follow the choices set by create-react-app.
Have to manually update each package dependencies in future (you'll face this issue if you did eject anyways)
There is minimal configuration included and setup is easy. The various webpack, Babel and ESLint configurations should be quite understandable.
Hope this will be helpful to you. Feel free to give suggestions and comments on how it can be improved.