Skip to content

Commit

Permalink
⚗️ Experiment on more gems: alt_memery, memoist3
Browse files Browse the repository at this point in the history
- 📌 Ruby 3.3.5 for development
- 📌 Ruby 3.3.5 for benchmarking
- ♻️ Refactor re-namespacing of colliding gems
- ➕ gem_bench
  • Loading branch information
pboling committed Sep 18, 2024
1 parent 9900684 commit 1a1c1f9
Show file tree
Hide file tree
Showing 10 changed files with 155 additions and 90 deletions.
2 changes: 1 addition & 1 deletion .ruby-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.3.2
3.3.5
1 change: 1 addition & 0 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ruby 3.3.5
39 changes: 25 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,48 +5,59 @@ follows a format inspired by [Keep a Changelog](https://keepachangelog.com/en/1.

This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased](https://github.com/panorama-ed/memo_wise/compare/v1.9.0...HEAD)
## [Unreleased](https://github.com/panorama-ed/memo_wise/compare/v1.10.0...HEAD)

**Gem enhancements:**

_No breaking changes!_

**Project enhancements:**

## [v1.10.0](https://github.com/panorama-ed/memo_wise/compare/v1.9.0...v1.10.0)

**Gem enhancements:**

- Reduced gem size from 173 kB compressed (312 kB unpacked) to 18.5 kB (68 kB unpacked) [[#345](https://github.com/panorama-ed/memo_wise/pull/345)]

_No breaking changes!_

**Project enhancements:**

- Updated official test coverage to support Ruby 3.3 [[#335](https://github.com/panorama-ed/memo_wise/pull/335)]
- Added `alt_memery` and `memoist3` to benchmarks [[#339](https://github.com/panorama-ed/memo_wise/pull/339)]
- Updated benchmark results in `README.md` to Ruby 3.3.5 [[#339](https://github.com/panorama-ed/memo_wise/pull/339)]

## [v1.9.0](https://github.com/panorama-ed/memo_wise/compare/v1.8.0...v1.9.0)

**Gem enhancements:**

- Fixed a bug that overwrote existing self.extended method definitions. [[#324]](https://github.com/panorama-ed/memo_wise/pull/314)
- Fixed a bug that overwrote existing self.inherited method definitions. [[#325]](https://github.com/panorama-ed/memo_wise/pull/315)
- Fixed a bug that overwrote existing self.extended method definitions. [[#324](https://github.com/panorama-ed/memo_wise/pull/314)]
- Fixed a bug that overwrote existing self.inherited method definitions. [[#325](https://github.com/panorama-ed/memo_wise/pull/315)]

_Breaking changes:_
- Removed Ruby 2.4 (EOL) support to allow upgrading rexml dependency version from a version that includes a [CVE](https://www.ruby-lang.org/en/news/2024/05/16/dos-rexml-cve-2024-35176/) [[#336]](https://github.com/panorama-ed/memo_wise/pull/336)
- Removed Ruby 2.4 (EOL) support to allow upgrading rexml dependency version from a version that includes a [CVE](https://www.ruby-lang.org/en/news/2024/05/16/dos-rexml-cve-2024-35176/) [[#336](https://github.com/panorama-ed/memo_wise/pull/336)]

**Project enhancements:**

- Fixed `bundle exec yard server --reload` and related documentation [[#333]](https://github.com/panorama-ed/memo_wise/pull/333)
- Fixed Codecov rate limiting errors affecting pull requests by upgrading `codecov/codecov-action` and using a Codecov token [[#317]](https://github.com/panorama-ed/memo_wise/pull/317)
- Fixed `bundle exec yard server --reload` and related documentation [[#333](https://github.com/panorama-ed/memo_wise/pull/333)]
- Fixed Codecov rate limiting errors affecting pull requests by upgrading `codecov/codecov-action` and using a Codecov token [[#317](https://github.com/panorama-ed/memo_wise/pull/317)]

## [v1.8.0](https://github.com/panorama-ed/memo_wise/compare/v1.7.0...v1.8.0) - 2023-10-25

**Gem enhancements:**

- In Ruby3.2+, for singleton classes, use `#attached_object` instead of `ObjectSpace` [[#318]](https://github.com/panorama-ed/memo_wise/pull/318)
- In Ruby3.2+, for singleton classes, use `#attached_object` instead of `ObjectSpace` [[#318](https://github.com/panorama-ed/memo_wise/pull/318)]

_No breaking changes!_

**Project enhancements:**

- Switched RuboCop configuration from `panolint` to `panolint-ruby` [[#312]](https://github.com/panorama-ed/memo_wise/pull/312)
- Updated benchmark results in `README.md` to Ruby 3.2.2 and 2.7.8 [[#313]](https://github.com/panorama-ed/memo_wise/pull/297)
- Updated `Dry::Core` gem version to 1.0.0 in benchmarks [[#297]](https://github.com/panorama-ed/memo_wise/pull/297)
- Updated `Memery` gem version to 1.5.0 in benchmarks [[#313]](https://github.com/panorama-ed/memo_wise/pull/313)
- Updated `Memoized` gem version to 1.1.1 in benchmarks [[#288]](https://github.com/panorama-ed/memo_wise/pull/288)
- Reorganized `CHANGELOG.md` for improved clarity and completeness
[[#282](https://github.com/panorama-ed/memo_wise/pull/282)]
- Switched RuboCop configuration from `panolint` to `panolint-ruby` [[#312](https://github.com/panorama-ed/memo_wise/pull/312)]
- Updated benchmark results in `README.md` to Ruby 3.2.2 and 2.7.8 [[#313](https://github.com/panorama-ed/memo_wise/pull/297)]
- Updated `Dry::Core` gem version to 1.0.0 in benchmarks [[#297](https://github.com/panorama-ed/memo_wise/pull/297)]
- Updated `Memery` gem version to 1.5.0 in benchmarks [[#313](https://github.com/panorama-ed/memo_wise/pull/313)]
- Updated `Memoized` gem version to 1.1.1 in benchmarks [[#288](https://github.com/panorama-ed/memo_wise/pull/288)]
- Reorganized `CHANGELOG.md` for improved clarity and completeness [[#282](https://github.com/panorama-ed/memo_wise/pull/282)]

## [v1.7.0](https://github.com/panorama-ed/memo_wise/compare/v1.6.0...v1.7.0) - 2022-04-04

Expand Down
2 changes: 1 addition & 1 deletion Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ GIT
PATH
remote: .
specs:
memo_wise (1.9.0)
memo_wise (1.10.0)

GEM
remote: https://rubygems.org/
Expand Down
46 changes: 23 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,36 +114,36 @@ For more usage details, see our detailed [documentation](#documentation).

Benchmarks are run in GitHub Actions, and the tables below are updated with every code change. **Values >1.00x represent how much _slower_ each gem’s memoized value retrieval is than the latest commit of `MemoWise`**, according to [`benchmark-ips`](https://github.com/evanphx/benchmark-ips) (2.11.0).

Results using Ruby 3.3.2:

|Method arguments|`Dry::Core`\* (1.0.1)|`Memery` (1.5.0)|
|--|--|--|
|`()` (none)|0.60x|3.17x|
|`(a)`|1.01x|7.94x|
|`(a, b)`|0.85x|6.38x|
|`(a:)`|1.00x|11.78x|
|`(a:, b:)`|0.88x|9.67x|
|`(a, b:)`|0.83x|9.44x|
|`(a, *args)`|0.67x|1.45x|
|`(a:, **kwargs)`|0.68x|1.88x|
|`(a, *args, b:, **kwargs)`|0.64x|1.29x|
Results using Ruby 3.3.5:

|Method arguments|`alt_memery` (2.1.0)|`dry-core`\* (1.0.1)|`memery` (1.6.0)|`memoist3` (1.0.0)|
|--|--|--|--|--|
|`()` (none)|11.84x|0.67x|3.10x|2.58x|
|`(a)`|9.50x|1.11x|3.78x|15.21x|
|`(a, b)`|7.67x|0.93x|3.00x|12.06x|
|`(a:)`|15.99x|1.16x|7.12x|21.32x|
|`(a:, b:)`|12.83x|0.91x|5.70x|21.20x|
|`(a, b:)`|12.95x|0.94x|5.72x|17.11x|
|`(a, *args)`|1.89x|0.70x|0.74x|2.91x|
|`(a:, **kwargs)`|2.81x|0.69x|1.19x|4.65x|
|`(a, *args, b:, **kwargs)`|1.66x|0.58x|0.81x|2.80x|

\* `Dry::Core`
[may cause incorrect behavior caused by hash collisions](https://github.com/dry-rb/dry-core/issues/63).

Results using Ruby 2.7.8 (because these gems raise errors in Ruby 3.x):

|Method arguments|`DDMemoize` (1.0.0)|`Memoist` (0.16.2)|`Memoized` (1.1.1)|`Memoizer` (1.0.3)|
|Method arguments|`ddmemoize` (1.0.0)|`memoist` (0.16.2)|`memoized` (1.1.1)|`memoizer` (1.0.3)|
|--|--|--|--|--|
|`()` (none)|22.57x|2.27x|23.46x|2.63x|
|`(a)`|20.96x|14.29x|20.54x|11.97x|
|`(a, b)`|18.22x|13.21x|17.76x|11.34x|
|`(a:)`|30.66x|23.52x|25.37x|21.61x|
|`(a:, b:)`|27.31x|21.98x|23.02x|20.31x|
|`(a, b:)`|26.21x|20.85x|21.57x|19.20x|
|`(a, *args)`|3.06x|2.23x|3.10x|1.92x|
|`(a:, **kwargs)`|2.67x|2.18x|2.39x|2.02x|
|`(a, *args, b:, **kwargs)`|2.14x|1.80x|1.89x|1.70x|
|`()` (none)|24.14x|2.44x|23.84x|2.59x|
|`(a)`|22.16x|14.80x|20.70x|11.67x|
|`(a, b)`|19.39x|13.66x|18.03x|11.46x|
|`(a:)`|30.54x|23.68x|25.21x|21.20x|
|`(a:, b:)`|27.75x|22.59x|23.47x|20.65x|
|`(a, b:)`|26.72x|21.39x|21.73x|19.43x|
|`(a, *args)`|3.26x|2.31x|3.09x|1.93x|
|`(a:, **kwargs)`|2.87x|2.29x|2.51x|2.10x|
|`(a, *args, b:, **kwargs)`|2.23x|1.88x|1.97x|1.73x|

You can run benchmarks yourself with:

Expand Down
16 changes: 13 additions & 3 deletions benchmarks/Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,26 @@ git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby ">= 2.7.7"

gem "benchmark-ips", "2.13.0"
gem "benchmark-ips", "2.14.0"
gem "gem_bench", "2.0.1"

# NOTE: Regarding `require: false` below
# 1. GitHub version of MemoWise and the local source of MemoWise share a namespace
# 2. memery & alt_memery share the namespace Memery
# 3. memoist & memoist3 share the namespace Memoist, and also share a load path for their version.rb files.
# This means we must `require: false` in `benchmarks/Gemfile` all, or all but one, of each of these duplicates,
# or we take care to only load them in discrete Ruby versions,
# to avoid a namespace collision before re-namespacing duplicates
if RUBY_VERSION > "3"
gem "alt_memery", "2.1.0", require: false
gem "dry-core", "1.0.1"
gem "memery", "1.5.0"
gem "memery", "1.6.0"
gem "memoist3", "1.0.0", require: false
else
gem "ddmemoize", "1.0.0"
gem "memoist", "0.16.2"
gem "memoized", "1.1.1"
gem "memoizer", "1.0.3"
end

gem "memo_wise", github: "panorama-ed/memo_wise", branch: "main"
gem "memo_wise", github: "panorama-ed/memo_wise", branch: "main", require: false
129 changes: 89 additions & 40 deletions benchmarks/benchmarks.rb
Original file line number Diff line number Diff line change
@@ -1,41 +1,81 @@
# frozen_string_literal: true

require "benchmark/ips"

require "tempfile"
require "benchmark/ips"
require "gem_bench/jersey"

github_memo_wise_path = Gem.loaded_specs["memo_wise"].full_gem_path

# This string is both used for temp filepaths necessary to separate the GitHub
# version of MemoWise and the local version, and used for the reported results
# Constants used for temp file paths necessary to separate gem namespaces that would otherwise collide.
GITHUB_MAIN = "MemoWise_GitHubMain"

# We download a the main branch of MemoWise on GitHub into a tmp directory to
# compare against the local version when we run benchmarks
Dir.mktmpdir do |directory|
Dir["#{github_memo_wise_path}/lib/**/*.rb"].each do |file|
Tempfile.open([File.basename(file)[0..-4], ".rb"], directory) do |tempfile|
tempfile.write(File.read(file).gsub("MemoWise", GITHUB_MAIN))
tempfile.rewind
require tempfile.path
end
end
end

GITHUB_MAIN_BENCHMARK_NAME = "memo_wise-github-main"
LOCAL_BENCHMARK_NAME = "memo_wise-local"

# 1. GitHub version of MemoWise and the local source of MemoWise share a namespace
# 2. memery & alt_memery share the namespace Memery
# 3. memoist & memoist3 share the namespace Memoist, and also share a load path for their version.rb files.
# This means we must `require: false` in `benchmarks/Gemfile` all, or all but one, of each of these duplicates,
# or we take care to only load them in discrete Ruby versions,
# to avoid a namespace collision before re-namespacing duplicates
re_namespaced_gems = [
GemBench::Jersey.new(
gem_name: "memo_wise",
trades: {
"MemoWise" => GITHUB_MAIN
},
metadata: {
activation_code: "prepend #{GITHUB_MAIN}",
memoization_method: :memo_wise,
},
),
GemBench::Jersey.new(
gem_name: "alt_memery",
trades: {
"Memery" => "AltMemery"
},
metadata: {
activation_code: "include AltMemery",
memoization_method: :memoize,
},
),
GemBench::Jersey.new(
gem_name: "memoist3",
trades: {
"Memoist" => "MemoistThree"
},
metadata: {
activation_code: "extend MemoistThree",
memoization_method: :memoize,
},
),
GemBench::Jersey.new(
gem_name: "memoist",
trades: {
"Memoist" => "MemoistOne"
},
metadata: {
activation_code: "extend MemoistOne",
memoization_method: :memoize,
},
),
].each(&:doff_and_don) # Copies, re-namespaces, and requires each gem.

# We've already installed the `memo_wise` version on the `main` branch from GitHub in the
# Gemfile, and moved it into a tmp directory and re-namespaced it so it doesn't collide with
# the `MemoWise` constant. Now we require the local version of `memo_wise` to compare
# this branch against it.
require_relative "../lib/memo_wise"

# Some gems do not yet work in Ruby 3 so we only require them if they're loaded
# in the Gemfile.
%w[memery memoist memoized memoizer ddmemoize dry-core].
# in the Gemfile. Gems re-namespaced by GemBench::Jersey will have already been loaded by now.
%w[memery memoized memoizer ddmemoize dry-core].
each { |gem| require gem if Gem.loaded_specs.key?(gem) }

# Some Gems Have Modules Which Need To Be Required Manually:
require "dry/core/memoizable" if Gem.loaded_specs.key?("dry-core")

# The VERSION constant does not get loaded above for these gems.
%w[memoized memoizer].
each { |gem| require "#{gem}/version" if Gem.loaded_specs.key?(gem) }

# The Memoizable module from dry-core needs to be required manually
require "dry/core/memoizable" if Gem.loaded_specs.key?("dry-core")

class BenchmarkSuiteWithoutGC
def warming(*)
run_gc
Expand All @@ -59,26 +99,35 @@ def run_gc
end
suite = BenchmarkSuiteWithoutGC.new

BenchmarkGem = Struct.new(:klass, :activation_code, :memoization_method) do
BenchmarkGem = Struct.new(:klass, :activation_code, :memoization_method, :name) do
def benchmark_name
"#{klass} (#{klass::VERSION})"
"#{name} (#{klass::VERSION})"
end
end

# We alphabetize this list for easier readability, but shuffle the list before
# using it to minimize the chance that our benchmarks are affected by ordering.
# NOTE: Some gems do not yet work in Ruby 3 so we only test with them if they've
# been `require`d.
BENCHMARK_GEMS = [
BenchmarkGem.new(MemoWise_GitHubMain, "prepend #{GITHUB_MAIN}", :memo_wise),
BenchmarkGem.new(MemoWise, "prepend MemoWise", :memo_wise),
(BenchmarkGem.new(DDMemoize, "DDMemoize.activate(self)", :memoize) if defined?(DDMemoize)),
(BenchmarkGem.new(Dry::Core, "include Dry::Core::Memoizable", :memoize) if defined?(Dry::Core)),
(BenchmarkGem.new(Memery, "include Memery", :memoize) if defined?(Memery)),
(BenchmarkGem.new(Memoist, "extend Memoist", :memoize) if defined?(Memoist)),
(BenchmarkGem.new(Memoized, "include Memoized", :memoize) if defined?(Memoized)),
(BenchmarkGem.new(Memoizer, "include Memoizer", :memoize) if defined?(Memoizer))
].compact.shuffle
benchmarked_gems = re_namespaced_gems.select(&:required?).map do |re_namespaced_gem|
BenchmarkGem.new(
re_namespaced_gem.as_klass,
re_namespaced_gem.metadata[:activation_code],
re_namespaced_gem.metadata[:memoization_method],
re_namespaced_gem.gem_name == "memo_wise" ? GITHUB_MAIN_BENCHMARK_NAME : re_namespaced_gem.gem_name,
)
end
benchmarked_gems.push(
BenchmarkGem.new(MemoWise, "prepend MemoWise", :memo_wise, LOCAL_BENCHMARK_NAME),
(BenchmarkGem.new(DDMemoize, "DDMemoize.activate(self)", :memoize, "ddmemoize") if defined?(DDMemoize)),
(BenchmarkGem.new(Dry::Core, "include Dry::Core::Memoizable", :memoize, "dry-core") if defined?(Dry::Core)),
(BenchmarkGem.new(Memery, "include Memery", :memoize, "memery") if defined?(Memery)),
(BenchmarkGem.new(Memoized, "include Memoized", :memoize, "memoized") if defined?(Memoized)),
(BenchmarkGem.new(Memoizer, "include Memoizer", :memoize, "memoizer") if defined?(Memoizer))
)
BENCHMARK_GEMS = benchmarked_gems.compact.shuffle

puts "\nWill BENCHMARK_GEMS:\n\t#{BENCHMARK_GEMS.map(&:benchmark_name).join("\n\t")}\n"

# Use metaprogramming to ensure that each class is created in exactly the
# the same way.
Expand Down Expand Up @@ -232,10 +281,10 @@ def positional_splat_keyword_and_double_splat_args(a, *args, b:, **kwargs)

# MemoWise will not appear in the comparison table, but we will use it to
# compare against other gems' benchmarks
memo_wise = benchmark_json.find { |json| json["name"].split.first == "MemoWise" }
memo_wise = benchmark_json.find { |json| json["name"].split.first == LOCAL_BENCHMARK_NAME }
benchmark_json -= [memo_wise]

github_main = benchmark_json.find { |json| json["name"].split.first == GITHUB_MAIN }
github_main = benchmark_json.find { |json| json["name"].split.first == GITHUB_MAIN_BENCHMARK_NAME }
benchmark_json = github_comparison ? [github_main] : benchmark_json - [github_main]

# Sort benchmarks by gem name to alphabetize our final output table.
Expand All @@ -245,9 +294,9 @@ def positional_splat_keyword_and_double_splat_args(a, *args, b:, **kwargs)
if i.zero?
benchmark_headers = benchmark_json.map do |benchmark_gem|
# Gem name is of the form:
# "MemoWise (1.1.0): ()"
# "memoist (1.1.0): ()"
# We use this mapping to get a header of the form
# "`MemoWise` (1.1.0)
# "`memoist` (1.1.0)"
gem_name_parts = benchmark_gem["name"].split
"`#{gem_name_parts[0]}` #{gem_name_parts[1][...-1]}"
end.join("|")
Expand Down
2 changes: 1 addition & 1 deletion lib/memo_wise/version.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# frozen_string_literal: true

module MemoWise
VERSION = "1.9.0"
VERSION = "1.10.0"
end
Binary file modified logo/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 1 addition & 7 deletions memo_wise.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,7 @@ Gem::Specification.new do |spec|
spec.required_ruby_version = Gem::Requirement.new(">= 2.5.0")

# Specify which files should be added to the gem when it is released.
# The `git ls-files -z` loads the files in the RubyGem that have been added
# into git.
spec.files = Dir.chdir(File.expand_path(__dir__)) do
`git ls-files -z`.split("\x0").reject do |f|
f.match(%r{^(test|spec|features)/})
end
end
spec.files = Dir.glob("{CHANGELOG.md,LICENSE.txt,README.md,lib/**/*.rb}")
spec.require_paths = ["lib"]

spec.metadata = {
Expand Down

0 comments on commit 1a1c1f9

Please sign in to comment.