abstraction/generalization to allow using it for other ecosystems

[dominikg]

#33 focuses on using it for forks of vite (PR), but the general pattern of executing downstream ci's is interesting for a lot of ecosystems.

eg. svelte, nuxt, vue

• make the base repository configurable
• find a way to distribute this (npm package?)

[patak-dev]

I think an npm package and a set of GitHub actions could work here. Maybe it doesn't need a big API surface if a factory to create the CLI and the few utils (runInRepo, $) that are used to build the suites are exposed?

const cli = createEcosystemCI({
  name: 'vite',
  setup: () => { ...setup vite repo },
  build: () => { ...build vite repo }
  testsDir: 'tests'
})

[brillout]

Also for vite-plugin-ssr since a couple of frameworks are currently being built on top of it.

[natemoo-re]

💯 Astro may be interested in this to run tests (even just basic pass/fail smoke tests) against large repos from notable users!

[dominikg]

great, i'll try to get this going. Please post ideas/requirements here.

One thing i'm thinking about is having a js config file thats auto-discovered (think vite.config.js) and a generic cli to avoid overhead for different ecosystems.

ecosystem-ci.config.js

export default config; // describe repo and tests to run

and in package.json of consuming projects there's a script to run it "test:ecosystem":"ecosystem-ci"

[jasikpark]

+1 for a usecase of withastro/compiler testing whether anything in withastro/astro breaks in addition to @natemoo-re's Astro ecosystem argument above!

[johnsoncodehk]

I also want to support this for volar packages (such as vue-tsc), the current one is just fork vite-ecosystem-ci and modify the relevant code.

This is also of great benefit to users, because users can add tests for different ecosystems in a unified way. :D

[haoqunjiang]

So far, the fork-and-modify approach works mostly fine for https://github.com/vuejs/ecosystem-ci

I think we still need a base repository, even if npm packages and GitHub actions are abstracted out, because we would still need to write a lot of boilerplate configurations in ci.yml etc. and I can't think of a way to get rid of them.
It's also convenient to have the documentation upfront in the project README, instead of pointing to an npm package page, which can be used for more advanced topics.

The base repository could be set as a template repository like vitesse, so that we don't have to detach the fork, while still maintaining the relationship with the base repository.

[dominikg]

the problem with fork-and-modify is that updates like the one from yesterday that work around issues with package managers or other changes/improvements would have to be pulled/merged.

A published cli could be consumed by repos, maybe have a create-ecosysetm-ci that generates from a template?

Having some kind of descriptor/config inside the repositories connected by a generic ecosystem-ci would help avoid having to do config updates in the ecosystem-ci repo.

Ideally there is a way to notify each repo separately as well, maybe have them running the actual task to split the load on github runners more easily.

If we wanted to get really fancy, taking a look at custom github app and/or actions would be a step up from the current setup that is basically just a lot of helpers around linking and calling the right scripts in the right order