Migration script for expected types in EventCatalog on updates

[boyney123]

Use Case

When new packages are added to EventCatalog (lodash for example), the types are also added. Now EventCatalog will bundle the lodash package fine, but people will miss the types as they dev dependency.

We either, add them add project dependencies or write a script to add them to folks dev deps for them when they run the dev command for example.

I'm leaning towards making them a dep of the project....

Proposed Solution

No response

Implementation Notes

No response

Community Notes

• Please vote by adding a 👍 reaction to the issue to help us prioritize.
• If you are interested to work on this issue, please leave a comment.
• If this issue is labeled needs-discussion, it means the spec has not been finalized yet. Please reach out in the EventCatalog Discord.

[carlosallexandre]

This PR add these dependencies to EC and create a test emulating the package on production (npm pack) to catch errors such as missing dependency sooner.

But, i think we could go a step further.

The way i see, EC is two projects - the eventcatalog bin and astro project. The bin needs the following packages at production: @parcel/watcher, commander, concurrently, cross-env and rimraf (Now, some of them are installed by the create-eventcatalog bin as devDependencies at the user project). All other dependencies are there for the astro only. So, if we pack only the bin dependencies the final size could be reduced a lot. The other dependencies could be installed at the user directory.

But, how do you distinguish the packages needed for each one? By separating the EC into two projects with workspaces, resulting in the following structure:

.
├── astro             # All the files related to astro
│   ├── ...          
│   └── package.json                   
├── bin               # Files related with `eventcatalog` bin
│   ├── scripts          
│   └── package.json                   
├── examples          # Catalog examples
│   ├── default          
│   └── ...                   
└── ...               # Other config files

The final package would be the bin bundled by tsup along with its dependencies and the astro directory to copy/paste into the user's .eventcatalog-core directory.

[boyney123]

Yeah interesting @carlosallexandre I think you might be ontop something here. Happy to explore it in more details, what would the workspaces look like? Would we need to use turbo or anything? Or just plain workspaces?

[carlosallexandre]

@boyney123 I think just plain workspaces.

I will work on something more concrete by this weekend to show you.

[IsmaelMartinez]

It might be an overkill but maybe using https://projen.io/ instead can solve this problem? I haven't look into the details but just my 2 cents.

[carlosallexandre]

@boyney123

After working on a POC with plain workspaces, I think that it's not helpful as I thought.

In my humble opinion, the best option for now is keep the packages needed at production as dependencies.

The only valuable thing from this POC is bundling public/ and src/ into one directory (like astro/). Now, EC copies the entire package to .eventcatalog-core (copyCore here). By bundling, EC could copy just astro/ to .eventcatalog-core.

[XaaXaaX]

@boyney123 @carlosallexandre
Why not use them as peerDependencies? to me if you put them in peer dependencies they will be installed automatically when the @eventcatalog/core is installed

[carlosallexandre]

Hey @XaaXaaX,

Answer me a doubt 🙏.

Let's say that the user project has the following package:

{ 
  "devDependecies": {
    "@eventcatalog/core": "2.8.2"
  }
}

And @eventcatalog/core has the following:

{
  "dependencies": {
    "@parcel/watcher": "1.0.0",
    "commander": "1.0.0",
    "axios": "1.0.0"
  },
  "peerDependencies": {
    "astro": "1.0.0"
  }
}

As you can see, the user doesn't have a direct dependency of astro. How package managers handle that, especially pnpm?

I ask this because of this issue. For pnpm to work properly, almost all dependencies (especially astro related) should be a direct dependency of the user project. So if pnpm automatically installs this peerDependencies as direct dependency in the user project, I think peerDependencies can be a great option. But if not, there are too many dependencies to install manually. So I fall back to workspaces 🤣. Because we can create .eventcatalog-core with these package.json and install the dependencies in the user project. This process of installing dependencies in the user directory was the reason I didn't think workspaces so helpful early.

Another option to solve #575 is doesn't create the .eventcatalog-core dir and consider the package inside the node_modules as .eventcatalog-core. In my tests, all commands from eventcatalog works properly except the dev command. This option for me sounds a little strange.

[carlosallexandre]

Now, this BRANCH handles the installation of dependencies at .eventcatalog-core and turns @eventcatalog/core fully compatible with pnpm (resolves #575).