Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need better performance regression testing #2

Open
LilithHafner opened this issue Nov 11, 2023 · 0 comments
Open

Need better performance regression testing #2

LilithHafner opened this issue Nov 11, 2023 · 0 comments

Comments

@LilithHafner
Copy link
Owner

  • Must have very low false positivity rate
  • Must run quickly
  • Should be data efficient
  • Should provide nice visualizations on failures that paint a compelling picture that the failure is real
  • Should perform drift detection
  • Should track performance separately across operating systems or other configurable values
  • Should provide searchable, browsable visualizations of all tracked parameters
  • Should easily interoperate with other visualization and benchmarking programs and data formats
  • Should use to test this package
  • Should provide an interface for users of this package to use
  • Should provide regression testing to Base
  • Should support measuring TTFX, load time, compile time, and classic runtime.
  • Should support arbitrary real numbers
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant