Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move away from comparative speed method names #110

Closed
nirvdrum opened this issue May 9, 2016 · 2 comments
Closed

Move away from comparative speed method names #110

nirvdrum opened this issue May 9, 2016 · 2 comments
Labels

Comments

@nirvdrum
Copy link
Collaborator

nirvdrum commented May 9, 2016

This is a proposal to move away from method names like slow, fast, and fastest and instead use something named after the operation. The convention is a bit error prone when authoring the benchmark to begin with, since you can't know a priori what the relative speeds are until you've written the benchmarks. Thus, it requires a renaming of the methods after the benchmark is run. It may introduce confirmation bias as well.

The real problem, I think, is it risks the entire repository becoming bit rotted. Ruby's performance is not fixed in time. If Ruby 3 manages to hit its 3x performance gain, it may very well come at the cost of reshuffling the relative performance of methods involved in these benchmarks. Then we'd be looking at the unenviable case of the fast method being faster than the fastest method and so on.

Going along with performance being implementation-specific across MRI versions, the problem is exacerbated when running the benchmarks on alternative Ruby implementations. In that case, the relative method values do not hold in many cases.

@nateberkopec
Copy link
Collaborator

I don't think anyone would be against a PR that updated/changed all of the current benchmarks in the way that you propose. 👍

@etagwerker
Copy link
Member

Sounds good, we can continue the discussion in #215

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants