Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PROPOSAL] Reduce friction in the testing framework #188

Closed
Xtansia opened this issue Jan 24, 2024 · 1 comment
Closed

[PROPOSAL] Reduce friction in the testing framework #188

Xtansia opened this issue Jan 24, 2024 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@Xtansia
Copy link
Collaborator

Xtansia commented Jan 24, 2024

What/Why

What are you proposing?

Refactor and adapt the existing testing framework to use the already generated complete OpenAPI specification rather than the manually captured individual snippets.

What problems are you trying to solve?

Currently to add a new test requires manually extracting each individual operation as a separate OpenAPI model. These then can fall out of sync with the originating Smithy model unless they're manually updated, and therefore aren't necessarily testing what is in the Smithy spec. This also adds a lot of friction for adding test cases.

Why should it be built? Any reason not to?

  • If we reduce friction in the process for adding tests we can better enforce testing of newly added specs.
  • This is (appears to be) a relatively low-hanging fruit to reduce a major friction point. However, there may be a more fit for purpose pre-existing testing framework/strategy that would be better to switch to rather than improving the existing one.
@dblock dblock added enhancement New feature or request good first issue Good for newcomers and removed untriaged labels Feb 7, 2024
@nhtruong
Copy link
Collaborator

nhtruong commented Apr 2, 2024

Closing this since Smithy is no longer used and the test framework has been removed as a result.
The new coverage workflow will also help us uncover missing endpoints. We can add a new test framework to test the spec against the server if need to. But in my opinion, it should be the other way around: the server should be tested against the spec, and every update to OpenSearch features should start with the spec.

@nhtruong nhtruong closed this as completed Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

3 participants