Deprecate pnpm-workspace.yaml

[hinell]

I propose to deprecate pnpm-workspace.yaml in favor of package.json#workspaces field.

Rationale

Other tools tend to use root package.json as starting point of configuration for monorepo project. At least two use workspaces field for glob patterns (as of November 4, 2020). Here is a small comparison between tools:

Package Manager	File*	Field
pnpm	pnpm-workspace.yaml	packages: ...
yarn	package.json	workspaces: [...]
npm	package.json	workspaces: [...]
lerna	lerna.json	packages : [...]
rush	rush.json	projects : [ {packageName: ... }... ]

Pros

• less to learn, quick to grasp
• Interop with existing tools enabling easy bidirectional migration (in case of tool's discontinuation)
• No need for extra file to pollute the project's file tree and
• At least two issues can be closed immediately: Allow custom name for pnpm-workspace.yaml #2614 Custom location for pnpm-workspace.yaml #1197
• No need for extra YAML dependency, no need for maintenance
  (assuming no other pnpm features are YAML-bound)

Cons

• Interop limitations prevented by possible differences in third-party tools's workspaces: definitions (highly unlikely)
• Reuse of workspaces by third-party tools may cause false-positivie "bug" reports due to false expectations from consumers
• Need for migration from pnpm-workspace.yaml to package.json#workspaces

Steps to deprecate

1. Add deprecation notices, brief guide on migration; continue to use pnpm-workspace.yaml
2. Commit as Minor version (per semver)
3. Refactor all commands to honor package.json#workspaces field instead of pnpm-workspace.yaml in first place
4. Commit as Major version: Breaking change

[zkochan]

There are some issues with Yarn's approach, which npm 7 adopted as well.

First of all, they use the noun "workspace" to call a package inside the monorepo. Which is not what most people expect, according to the polls I did on Twitter. A workspace usually means a set of projects in IDE world.

The second issue is that it is much harder to find the root of the workspace, when that info is declared in package.json. It is not enough to find the first pnpm-workspace.yaml in the parent directories, you need to read each package.json file in the parent directories. It would be fine to use workspace.json file instead, which would behave similarly to a package.json but dedicated to the root workspace project.

I cannot object if most of the users want this change but then we'll have to do a lot of renamings in the repository.

[hinell]

@zkochan Thanks for speedy answer.

A workspace usually means a set of projects in IDE world.

1. Considering that the bad naming is already established and abused by major managers there is little space for maneuver. I'm afraid that suggesting to use packages: [...] would cause general hassle in all these project's lol. Doesn't worth efforts.

[...] it is much harder to find the root of the workspace, when that info is declared in package.json [...]

2. The best approach IMO is just to use root at the folder where the pnpm was called. The private: true and zero dependencies can serve as a hint of root package.json for pnpm-cli to stop at.

[zkochan]

I am not convinced yet that Yarn's approach can be considered a standard. Lerna is still very popular and uses a separate file. pnpm uses a separate file and Rush uses a rush.json as well.

[statianzo]

Reusing the same package.json key could give the expectation that pnpm needs to behave identically to other monorepo managers.

There was a similar backtracking of resolutions in #2946

[zkochan]

Let's hear what others think

cc @pnpm/collaborators

[KSXGitHub]

• No need for extra YAML dependency, no need for maintenance

Does pnpm still require YAML for lockfiles?

[zkochan]

yes

pnpm-lock.yaml

also, Yarn uses yaml for its lockfile as well.

[ExE-Boss]

I personally prefer pnpm‑workspace.yaml, for the reasons highlighted above.

[hinell]

Yarn's approach can be considered a standard. ...

@zkochan It may not and should not be in first place. The proposal is for the sake of interoperability/coherence between tools only.

Lerna is still very popular and uses a separate file ...

@zkochan Primary maintainers may discontinue the project in favor of other tools like pnpm etc at any point as more monrepo-like features are getting added over time into other tools.

could give the expectation that pnpm needs to behave identically. [...]

@statianzo Good point. I've updated description, checkout. Such expectations are completely unwarranted. No further compliance with existing tools is required by this proposal.

[juanpicado]

My 0.50 cents. Long story short, I prefer to have a separate file, either is only for workspaces or anything else.

I'd rather prefer to see a .pnpmrc. file or .pnpm.yaml in the root of the project or something generic open for other sorts of settings and stop using .npmrc as yarn2 just did, not only workspaces.

One reason is, the package.json should contain only the relevant metadata of the package itself, instead of being polluted with third-party stuff. From my perspective (node.js registries) I see this pattern growing in many projects and makes metadata bigger and this information is useless for package managers and registries (I/O has a price). eg:
http://registry.npmjs.org/react

  "browserify": {
        "transform": [
          "loose-envify"
        ]
      },

There are other examples, as commitlint, opencollective and a long etc.

Having a separate file you have more options to scale, avoid collisions, and is way easier to read and recognize the origin.

I understand is more convenient to have everything in the package.json, but is far away to be a standard. Cannot be commented out (I find this useful for contributors or team workmates who want to know more about what X means).

Having this, you can workout nice stuff like better IDE strategy for autocomplete or stuff like that (vcode extensions), I think this is hardly possible using the package.json.

[hinell]

[...] as yarn2 just did, not only workspaces. [...]
[...] instead of being polluted with third-party stuff. [...]

@juanpicado Thanks for sharing. My proposal doesn't concern .rc files. In fact, pnpm uses two different files for two different configs pathways. I'm completely fine with .pnpmrc. The workspacesare not third party stuff anymore but rather a built-in, native to npm ones. Checkout OP's table.

[ExE-Boss]

The issue for supporting .pnpmrc is #2803.

[ematipico]

Personally, I think it would be best to keep everything inside a separate file. The package.json ultimately has been abused a lot, where various tools decided to put their own configuration. As a result, the file becomes really big and it contains sections that are not even standard.

[hinell]

@ematipico workspaces are standard already, read thread.

[tvvignesh]

@zkochan Rather than worrying about this, I guess you can have a sort of CLI option supporting both 😅 While I personally like pnpm-workspace, reading the same config from the package.json may be valid for people who wants to have an interop for which they can probably use a non-default CLI flag where you can specify the root folder path. That way, you can probably satisfy both worlds.

[jcayzac]

My opinion is that pnpm should be 'transparent' and fully compatible with npm.

IMHO no package manager ever has to care about looking like npm or using the same files as npm. I don't think any package manager has even to use package.json as its primary development metadata format. I'm all for killing it and replacing it with YAML, for all the reasons the format is better for teamwork than JSON.

The only rightful concerns are the ability to release packages to the npm registry (and that includes generating a valid npm-compatible package.json during the publish phase) and the ability to load a package from the npm registry (and so parse npm's package.json files).

[desmap]

There is no reason to change anything and why I would strongly opt against adapting the "workspace" field in package.json:

• The workspace feature in npm 7 barely works, it's nice they have this feature now but yeah it's night and day to pnpm's and it's at its beginning, wouldn't use it in production
• I never got warm with yarn's workspaces, I have a yarn 1 legacy monorepo which doesn't work anymore and still have to migrate to yarn 2 (🙄), it's just a not so nice experience, the ux in particular

=> Adopting now workspace as package.json field creates not just a negative association to aforementioned package managers but it has also to work exactly as in its counterparts, this will create tons of confusion and issues and for what? To have an even more bloated package.json?

More reasons:

• All the ones mentioned in the thread
• package.json is already too bloated and I have to maintain long READMEs to recall who is putting what in which package.json so nobody forgets an important field, it's a bad practice and we shouldn't follow
• Yes, pnpm-workspace.yaml could have been a shorter or a nicer filename but that's how it is now, no change would justify the migration, support and breaking headaches the maintainers would face, we should spend our energy on more important topics or really advanced stuff like regex support for --filter haha.

[shellscape]

I agree with @desmap on this one. Having a distinction for pnpm is an advantage imho.

[zkochan]

Probably because this was initially an issue, which I converted to a discussion.

[hinell]

@pie6k Thanks for your opinion. Seems like nobody gets the point of backward compatibility. Unique way of configuration isn't always good if it comes at cost of learning new things.

@zkochan That's right. This is proposal, not a bug report.

[jcayzac]

The proposal is for the sake of interoperability/coherence between tools only.

I don't see a need for interoperability between package managers. I don't use pnpm only because it's faster than yarn, I use it because it's safer and the DX is better. I couldn't care less about it not being interoperable with package managers I have no intention of using.

I would argue that, on the contrary, interoperability is harmful. Because people may just run npm or yarn because they can, with horrible results. Anything that would finally not require relying on hacks like pnpm-only just to try to block people from using those would make pnpm even better!

[alshdavid]

For me and my team, there is great utility is using pnpm as a direct replacement for npm. Following npm's design decisions directly reduces the friction of using pnpm.

If I can tell my team that they can just run the following and carry on, that's a good day

alias npm="pnpm"

Extending features and behaving as a superset is fine, of course, but learning tooling is what gets JavaScript the most criticism

[jcayzac]

If all you want is a more efficient npm, I think this is more relevant to npm than it is to pnpm, no?
Personally I'm interested in pnpm because of the developer experience and how it fixed npm's broken dependency management, not because of its performances. It could be twice as slow, it would still be my choice. Generating the same dependency graph as npm would be a major dealbreaker.

[hinell]

@jcayzac It's never been about efficiency, but about interop.

[liujiayii]

Will we support useYarnConfig: true or package.json#workspaces?Using the same workspaces configuration will greatly facilitate our switching from yarn to pnpm.

[zkochan]

Related PR: #5554

[zanminkian]

I am in favor of workspaces field in package.json.

1. It's the standard of npm now. Developer don't need to learn too much about the differences between npm and pnpm. It's good for pnpm.
2. Keep the root of project clean. Having too many files in the root of project looks chaotic. We should reuse the package.json fields which was defined as standard by npm. This will make the project clean.

What about There are too many fields in package.json. It looks chaotic too. ?

1. Fields like husky, jest, commitlint etc should not be put into package.json. Because they are not the standard of npm.
2. By learning the standard of npm, developer will know all the package.json fields' meaning in their project. It won't be chaotic.
3. Deprecating some fields of package.json and keeping the fields minimal are the duty of npm, not pnpm.

[hinell]

@zanminkian There are many projects (NPM, NodeJS, Webpack, etc.) that have their own vision of the package.json fields and all of the three have some specs interleaving, and some not. It would be best to follow them up to keep compatibility stable (i.e. without errors).

[zanminkian]

@hinell You are right. See https://nodejs.org/api/packages.html#nodejs-packagejson-field-definitions.

There are only 6 fields in package.json used by the Node.js runtime. Other fields in package.json may be solved in their own way in different package manager.

So there are 2 way for pnpm to choose:

1. Follow the common rules. This means pnpm should try its best to be compatible with npm.
2. Just follow Node.js standard. Don't allow pnpm project to set any fields into package.json except the 6 fields above. And pnpm provide a detail document to tell developer that what file pnpm will create and what field pnpm will record.

Obviously, I think following npm and yarn is better. I am in favor of workspaces field in package.json as what npm and yarn do.

[shellscape]

@zanminkian I'd agree with you if package.json had a schema. But it's a schemaless, standardless document that's the wild west. It's a dumpster fire of a document. I much prefer something standardized, and I prefer keeping pnpm configuration separate, and not mingling it into the other mess of config in package.json.

It'd be like taking a pristine painting and tossing it into a 10 meter high pile of fingerpaints by toddlers.

[jonaskello]

Having two separate files (root package.json and pnpm-workspace.yaml) also means you need to include one more file in your Dockerfile , CI etc. which took a while for me to realise.

So coming from yarn workspaces, then moving to npm workspaces, and now to pnpm workspaces, my preference would have been to have it the same way so I don't have to update every Dockerfile, .dockerignore , docker-compose.yaml, gitlab-ci.yamletc. to include the extra pnpm-workspace.yaml file. Also If we ever move back to yarn or npm we need to go through all those files and delete the pnpm-workspace.yaml reference everywhere.

I would say having a separate file adds a lot of friction but little value for the end-users. In general I would agree that polluting package.json with info not related to package installation is not a good idea (like prettier config etc). However workspaces is about which packages should be installed/linked, so putting it in package.json to me seems like just putting things where they belong rather then pollution :-)

[shellscape]

Tool nuance is a good thing; it's a differentiator. Having been in the Node ecosystem since v0.10 I can say with confidence that following NPM for "standards" (if we can even call them that - package.json still doesn't have a schema) is a quick path to pain.

I'd suggest that if there's references to a configuration file in four other configuration files, there's likely a bigger architecture problem in the repo in question. Been working with pnpm and Docker for a few years now and I've never had to intentionally include the workspace yaml, and with the deploy command it's pretty painless to shuffle over the correct files to Docker, even if a subsequent pnpm install is needed. If pnpm-workspace.yml ends up in a Docker container, there's probably something not right.

[jonaskello]

Even if you do a multi-stage image you would at some point need to include the workspace file even if you prune it later or am I missing something? So somewhere a extra docker command to copy it to some stage is needed?

Tool nuance is a good thing

Although I agree it is good to have choices, having developed in javascript/typescript for many years, I know the big variety of tools for is seen by many as the main pain point of the JavaScript ecosystem and I also agree on that.

Causing tool friction is not really helping anyone if it does not add a lot of extra value IMO. As usual it's all about tradeoffs. Not sure I see the value added by the extra file outweigh the friction it adds. Having a tool work different than the mainstream de facto standard is also asking for a bigger up-front buy-in to start using it. Again that might be worth it but IMO only if it adds a lot of value to the users.

[shellscape]

Should lerna, Nx, Bazel, Rome, and Moon all adopt the non-standardized NPM package.json format (which again, is just schemaless JSON)? After all, they're all build and runner tools that work with the Node/NPM ecosystem. Should NPM throw in the towel on .npmrc and use package.json? Should every tool in the Node ecosystem throw in the towel on their config files and use package.json? I would not welcome a 1000+ line package.json file.

The argument based on tool friction has been pretty often boiled down to not liking change. And there's nothing wrong with not liking change, or wanting things to be familiar. But it's also not a convincing argument for not having differences between tools.

Even if you do a multi-stage image you would at some point need to include the workspace file even if you prune it later or am I missing something?

I would likely need to see your implementations, but having done some really wild things with Docker and Node apps, I've never needed the workspace file in the container for any ops.

Another fun aspect of this is that everything that is in package.json, at the time of publishing a package, ends up in the registry. Everything. I've seen people put 2000 line READMEs as a package property and that ends up in the registry. There are packages that have larger package.json files than code in the package. Who's responsible for removing tool configuration from package.json on publish?

The point being is that while folks are grasping for familiarity and sibling-tool-compatibility, many are missing the detriments, nuances, complications, and complexity with doing so. Piggybacking on package.json for workspaces by Yarn was a poor decision, and NPM following suit was a poor decision that was only made to try and regain adoption from Yarn users.

[jonaskello]

A lot of arguments seems to revolve around having config for different tools in package.json. Tools for linting, formatting, tool runners etc. should not have config in package.json because it just creates a mess. I totally agree with that being a bad idea so no discussion needed on that point :-).

However the config file for the package manager tool (npm, yarn, pnpm) is package.json . So adding more information in that same file for the same tool seems quite logical to me instead of creating two separate files for the same tool. Yes the schema is not standardized per se, but as much other things in this industry there is a de facto standard created by common conventions by mainstream tools.

Regarding Dockerfile, here's how we do the cached docker build in one project. I'd be interesting to have input on how it can be done better using pnpm as this was originally done for yarn and then adopted to npm, and now finally to pnpm :-).

# https://stackoverflow.com/a/63142468/2761797
# On npm install in second build stage the layer cache will be reused even if any file in packages directory has changed.

# First build stage, only to get package.json files for all packages
# This is needed for docker cache of installed node_modules
FROM node:18.12.1-alpine
WORKDIR /application

# Copy all files in apps
COPY apps apps
# Copy all files in all libs
COPY libs libs
# Find and remove non-package.json files
RUN find -type f -not -name "package.json" -delete
RUN find -empty -type d -delete

# Copy root package.json and package-lock.json
COPY ["package.json", "pnpm-lock.yaml", "pnpm-workspace.yaml", ".npmrc", "./"]

# Define second build stage that gets package.json files from the first stage
FROM node:18.12.1-alpine
WORKDIR /application
# Copy package.json files from the first build stage.
COPY --from=0 /application .
# pnpm uses NODE_ENV so set it before using it
ENV NODE_ENV=production
# Enable pnpm
RUN corepack enable
RUN corepack prepare pnpm@latest-8 --activate
# Disable husky script and install dependencies, the layer cache will be reused here
RUN pnpm pkg delete scripts.prepare
RUN pnpm install --frozen-lockfile --prefer-offline

# Copy all files in apps
COPY apps apps
# Copy all files in libs
COPY libs libs
# Install again to restore workspaces symlinks
RUN pnpm install --frozen-lockfile --prefer-offline

[Mufasa]

I am fairly new to using pnpm as a package manager but absolutely love it. However I came across a recent issue when I setup a next.js app in my monorepo. The monorepo was setup as follows:

- package.json
- pnpm-workspace.yaml
- ...
- packages
  - apps
    - my-app-1
    - my-app-2
    - ...
  - libs
    - my-lib-1
    - my-lib-2
    - ...

where packages/apps held each of my next.js apps and packages/libs held shared libraries used across all my apps. The pnpm-workspace.yaml file contained:

packages:
  - 'packages/apps/*'
  - 'packages/libs/*'

Now, when I navigate into the packages/apps/my-app-1 folder and then run pnpm dev, it correctly started my app in development mode and I could browse to it in Chrome. If I changed any files within the packages/apps/my-app-1 folder, then I saw the change reflected in the browser without me having to refresh the page.

When I run pnpm --filter my-app-1 dev from the root of my monorepo project, it again correctly started my app in development and I could browse to it in Chrome. However, if I changed any files within the packages/apps/my-app-1 folder, then those changes were not being reflected in the browser.

I had to add this section to my root package.json file in order to get everything to work correctly:

  "workspaces": [
    "packages/apps/*",
    "packages/libs/*"
  ],

Assuming I have not made some fundamental mistake here then this seems like unnecessary duplication to me. So I too would vote to adopt the use of workspaces within package.json over defining these in pnpm-workspace.yaml.

[shellscape]

that's your dev tool at fault, not pnpm.

[danielbayley]

I also think it seems daft to have to pollute the root of my repo with yet another config file—in this case with a lengthy name—only to contain a couple of lines…

packages:
- packages/*

[KKonstantinov]

Moving away from pnpm for this sole reason.

There are some enterprise tools that you to just use yarn and npm when you upload a package on their systems. We would have liked to keep pnpm for local development - which would have been fine, had it supported the package.json workspace option. In that case locally we'd use pnpm, but that's not an option it seems.

[shellscape]

There's so much conjecture and reading comprehension failure in that reply that I'm not entirely sure how to address it. If you don't consider an expectation of interoperability to be entitlement, but rather completely reasonable, then you're going to have a long, hard road in software.

[purepani]

I am legitimately wondering: what else did you expect that person to do?
They had 2 options:

1. Use yarn/npm and have that work with their existing tooling.
2. Use pnpm and have to write custom software to make it work with their other tooling.

How is choosing option 1 entitlement in any way? They literally just made a choice that's less work for them.

[purepani]

I am honestly not trying to sound hostile here: I legitimately don't understand your viewpoint and I'm trying to figure out why it seems entitled to you, because you seem like a reasonable person.
The only reason I can think it seems entitled to you is that he came onto this discussion and publically stated that they switched off of pnpm because of this. That, I can see as being petty, but it doesn't make the fundamental decision of them switching off of pnpm petty.

If that's what you meant by petty then, yes, maybe I do need to read a bit better 😅

[Aankhen]

the expectation that tools will be interoperable is modern entitlement. that may sound brash, but it is true.

It doesn’t sound brash, just bewilderingly inaccurate. Interoperability is the goal that gave us the IBM PC more than four decades ago. Linux is a three-decade-old operating system built on the fundamental idea of small, interoperable tools. The Web Standards Project from the mid-2000s is what gave us cross-browser compatibility and allows you to use this very website in any modern browser instead of being locked into Internet Explorer and Microsoft FrontPage.

Let’s not rewrite history to dismiss one response.

[weyert]

Why not help these Enterprise tools to get support for pnpm? I have spend plenty of time to help get tooling to support pnpm. From Gitlab to Snyk. What kinds of Enterprise tools are you talking about?

[zanminkian]

Since catalogs configs added to pnpm-workspace.yaml, this file is becoming more and more difficult to deprecate. I really don't like this file as I don't use catalogs and I want to make my projects are packge manager insensitive. Because of this file, other developers will be confused: Am I have to develop this project by pnpm?