feat: add dev gateways with dashboards #116
Conversation
There was a problem hiding this comment.
Cool @laurentsenta ! A few things coming to mind:
- Can we add Boxo/Gateway?
- Will we add a description for how someone gets their gateway implementation or gateway instance added to the dashboard?
- Can someone click into the results to see more details, especially about what tests failed? (Similar to how can go from https://wpt.fyi/results/webtransport?label=experimental&label=master&aligned to https://wpt.fyi/results/webtransport/datagrams.https.any.worker.html?label=experimental&label=master&aligned )
- Will this be published under https://github.com/singulargarden/gateway-conformance or https://github.com/ipfs/gateway-conformance ?
- Would this make sense to be a github page like was done for pinning-service-compliance so have some durable URLs: https://ipfs-shipyard.github.io/pinning-service-compliance/api.pinata.cloud.html
- How will this get discovered? Will we link from https://specs.ipfs.tech/http-gateways/ ?
- (nit) The green orbs are of different sizes in some cases
- Add a description for how someone gets their gateway added?
- Any reason we're not adding ipfs.io gateway?
- How will this get discovered? Will we link from https://specs.ipfs.tech/http-gateways/ or https://ipfs.github.io/public-gateway-checker/ ?
| @@ -2,7 +2,9 @@ | |||
|
|
|||
| `gateway-conformance` is a tool designed to test if an IPFS Gateway implementation complies with the IPFS Gateway Specification correctly. The tool is distributed as a Docker image, as well as a GitHub Action(s). | |||
|
|
|||
| []() | |||
| [/badge.svg?branch=main)]() | |||
There was a problem hiding this comment.
Rather than "Production" and "Dev" maybe use...
Gateway Instance Conformance Dashboard
Gateway Implementation Conformance Dashboard
Dev and Prod sounds like one is "development/preproduction" and one is "production".
laurentsenta
force-pushed
the
feat/test-dev-gateways-with-dashboards
branch
from
8d00f33
to
85948b9
Compare
There was a problem hiding this comment.
Thank you @laurentsenta
Will we add a description for how someone gets their gateway implementation or gateway instance added to the dashboard? [..]
How will this get discovered?
For this repo, I'd keep the CI that runs in this repo limited to things that PLN/Stewards control or have working relationship (boxo-based impls, our gateways, and PLN-funded things).
To avoid drive-by PRs from random people, we can leverage https://ipfs.github.io/public-gateway-checker/ instead. Use CI there against gateways listed there. This way people who run public gateway learn about a problem with their gateway, and then learn gateway-conformance test suite exists, but we put some distance between checker and this repo: by leveraging public-gateway-checkerr we are not in the business of "endorsing" specific projects with being "worthy" of being on the list.
+ Some suggestions inline.
There was a problem hiding this comment.
We need either:
- error CI on missing entry (so we don't miss updating this file when we rename a func, or add new one)
- replace this JSON with some golang convention (godoc, or maybe additional strings passed to each test), so these strings and URLs live in the same file as tests (+error CI when empty)
Co-authored-by: Marcin Rataj <[email protected]>
galargh
force-pushed
the
feat/test-dev-gateways-with-dashboards
branch
from
71a3080
to
0b594e6
Compare
galargh
force-pushed
the
feat/test-dev-gateways-with-dashboards
branch
from
0b594e6
to
bda8985
Compare
Example dashboard for dev gateways
Example dashboard for production gateways
Todo
This job updates prod e2e workflow and commits id df008fa
Then this job updates after dev e2e workflow and commit id 2568390
The push overwrites the previous one. Without an error, surprisingly.
I guess we need to use the current ref and not the current sha.
find a less invasive change in called workflows
At the moment, we have to hard-code the repository in the reusable workflow, this is too invasive, see the required changes in gateways:
agree with the team on constraints
For example, we don't want to override the gateway-conformance versions used in the workflows, but it does mean if there are two gateways running conformance with different versions, refactoring a test would create a lot of noise.