Discourse core is switching to pnpm for JS package management

We’re switching Discourse core’s JS package manager from from ‘yarn classic’ to pnpm. This will bring significant improvements to install speed & storage requirements.

Production

If you use managed hosting or a standard install, then you don’t need to do anything. The change will be applied automatically on your next update.

If you have a custom production environment, you’ll need to change any yarn ... commands to pnpm ...

Development

After pulling the latest version of Discourse core, you may see an error like this when starting the server:

pnpm is not installed

or, if you run a command like yarn install, you will see

error discourse@: The engine "yarn" is incompatible with this module. Expected version "please-use-pnpm". Got "..."

To resolve this, you should:

  1. run npm install -g pnpm

  2. run pnpm install

  3. replace any uses of yarn ... in your development environment with pnpm ...

Plugins / Themes

For now, our official plugins/themes continue to use yarn for their linting dependencies. We’ll be updating them over the coming weeks.

Switching back to yarn

If you need to switch back to a yarn-based version of Discourse (e.g. developing on the stable branch), then you’ll need to manually delete all of core’s node_modules directories:

rm -rf node_modules app/assets/javascripts/*/node_modules
23 Likes

Can report a faultless smooth experience on dev, working first time, thanks very much! :rocket:

7 Likes

If your self hosted. Will we need to do anything if not using custom components & plugins?

Ie some plugins & components are not official.

3 Likes

Assuming you’re using our standard install, you don’t need to do anything. No changes are required to plugins or themes

5 Likes

Should we change this expected version to, perhaps:

please-use-pnpm-see-https://meta.discourse.org/t/324521

:smiley:

6 Likes

I am contributing to another FOSS project where I was recently introduced for the first time to PNPM. The transition was a complete no-brainer, smooth, effective, and a real pleasure.

I wrote up very brief notes in the documentation for that project for What PNPM Is, and how to use it. While the tiny details are different for every project, I hope what’s there might provide a quick intro to anyone here, who like me didn’t have a clue before an announcement was made.

https://mysql-to-zod.pages.dev/getting-started/pnpm/

5 Likes

Good resource, thanks @TonyG

It’s worth noting though - none of the “configuration notes” there are required for Discourse. Not sure why they need it… maybe because it’s a windows-based application?

Also, their note about “In an existing project, just delete the ‘node_modules’ folder” is handled automatically in Discourse by this script :sunglasses:

So for us, it really should be as simple as:

npm install -g pnpm
pnpm install
1 Like

One question here, David:

How to run linting locally?

ie what replaces, e.g.: yarn prettier --write plugins/discourse-events

I’ve tried to run

pnpm pprettier --write plugins/discourse-events

but it throws an error:

Error: File not found with singular glob: /Users/blah/dev/disc/discourse/plugins/discourse-events (if this was purposeful, use allowEmpty option)

I think you have too many p there?

So you want:

pnpm prettier --write plugins/discourse-events

pprettier is a tool for running prettier in parallel, but I guess it doesn’t support running on a single directory like that.

3 Likes

Thanks for your contribution to FOSS docs @david :lol_: It does look like the page is oriented toward Windows, and that the info there might be required knowledge for using pnpm. I will edit and clarify that neither is the case for that utility.

To be clear, the info about how pnpm works is only there for the comfort of the user/developer, including those here who might like to understand this new tool that replaces a significant, often used tool in our kit.

For Discourse devs, that info explains where things are located and how to change default locations. This is intended to answer a question like “if all of my node_modules are now in one location, where are they?” In the Discourse container, you devs might not want packages in the default location. If, for whatever reason, a plugin developer is directly referencing the node_modules folder and they find links to another location rather than files, the info on that page concisely explains how that location is determined.

So much for trying to be brief. :facepalm: :lolsob:

Anyway, good move to pnpm, and thanks.

2 Likes

Yes, pprettier was not appropriate, thanks.

Worked out my ultimate problem too.

You apparently need to run pnpm install in the plugin directory before you can run linting checks (even from the discourse directory).

2 Likes

Hmm intersting :thinking:

With our standard plugin skeleton, each plugin has its own package.json file with its linting dependencies. And for now, the skeleton still uses yarn.

So to lint a specific plugin, you would go into the plugin directory and run

yarn install
yarn prettier --write

Using core’s linting config on plugins can work sometimes. But if the version/config diverges, it can get painful because the version of eslint/prettier in core won’t match the version running in your plugin’s CI.

1 Like

Would we consider using pnpm from the plugin directory too?

I believe it is still using the local package.json, no?

3 Likes

Yes we definitely will! Just letting the dust settle on the core change before we start that adventure

(fun fact: CDCK maintain almost 600 theme/plugin repositories which will all need updating :sweat_smile:)

2 Likes

Sure, no issues!

1 Like

This seems to have worked correctly, but was confused by notice that seems to state this needs to be done manually and is not automatic for standard install site:

1 Like

Thanks @Architect. These PRs will suppress the upgrade messages:

6 Likes

Ran into an issue upgrading via the web tool — the upgrade page said the upgrade failed, with the Expected version "please-use-pnpm" error, but when I subsequently visited the Versions page again it appeared that the upgrade succeeded:

…however none of the Admin pages are working now:

Update:
I did a rebuild on the command line and that’s fixed things.

2 Likes

Thanks for reporting @alxndr.

I just pushed this fix which should stop this happening to others.

4 Likes

Maybe it’s our Communiteq environment, but TBH I don’t see any significant speed improvements during installation? Our stable test install even installs 23 seconds quicker than the tests-passed one.

1 Like