Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-3385: Allow updating the data type cache in the database #5285

Merged
merged 1 commit into from
Oct 1, 2024

Conversation

TimDiekmann
Copy link
Member

🌟 What is the purpose of this PR?

We change data type inheritence in quite a few PRs currently and to avoid running snapshot migrations for everything it's easier to run a command from the CLI. The functionality was borrowed from snapshots, which has to do the same functionality, but this PR also exposes the functionality to the CLI:

cargo run --bin hash-graph -- reindex-cache --data-types

@github-actions github-actions bot added area/apps > hash* Affects HASH (a `hash-*` app) type/eng > backend Owned by the @backend team area/tests New or updated tests area/tests > integration New or updated integration tests area/apps area/apps > hash-graph labels Oct 1, 2024
Copy link

codecov bot commented Oct 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 20.15%. Comparing base (b9d8189) to head (bf4050c).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5285      +/-   ##
==========================================
- Coverage   20.18%   20.15%   -0.03%     
==========================================
  Files         510      510              
  Lines       17300    17323      +23     
  Branches     2546     2546              
==========================================
  Hits         3492     3492              
- Misses      13770    13793      +23     
  Partials       38       38              
Flag Coverage Δ
apps.hash-ai-worker-ts 1.39% <ø> (ø)
apps.hash-api 1.18% <ø> (-0.01%) ⬇️
local.hash-backend-utils 8.81% <ø> (ø)
local.hash-isomorphic-utils 1.06% <ø> (ø)
local.hash-subgraph 24.32% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Oct 1, 2024

Benchmark results

@rust/graph-benches – Integrations

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$53.7 \mathrm{ms} \pm 260 \mathrm{μs}\left({\color{gray}0.494 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$49.6 \mathrm{ms} \pm 261 \mathrm{μs}\left({\color{gray}-0.239 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$43.7 \mathrm{ms} \pm 350 \mathrm{μs}\left({\color{gray}2.66 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$66.1 \mathrm{ms} \pm 426 \mathrm{μs}\left({\color{gray}-0.285 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$58.7 \mathrm{ms} \pm 287 \mathrm{μs}\left({\color{gray}1.44 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$39.9 \mathrm{ms} \pm 236 \mathrm{μs}\left({\color{gray}3.60 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$93.4 \mathrm{ms} \pm 470 \mathrm{μs}\left({\color{gray}0.859 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$89.2 \mathrm{ms} \pm 614 \mathrm{μs}\left({\color{gray}0.831 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$79.3 \mathrm{ms} \pm 493 \mathrm{μs}\left({\color{gray}-0.055 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$106 \mathrm{ms} \pm 519 \mathrm{μs}\left({\color{gray}0.451 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$98.8 \mathrm{ms} \pm 588 \mathrm{μs}\left({\color{gray}0.791 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$41.1 \mathrm{ms} \pm 250 \mathrm{μs}\left({\color{gray}1.14 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$16.2 \mathrm{ms} \pm 167 \mathrm{μs}\left({\color{gray}4.29 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$15.3 \mathrm{ms} \pm 149 \mathrm{μs}\left({\color{lightgreen}-5.236 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$15.4 \mathrm{ms} \pm 191 \mathrm{μs}\left({\color{lightgreen}-35.191 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$16.1 \mathrm{ms} \pm 190 \mathrm{μs}\left({\color{gray}3.47 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$15.6 \mathrm{ms} \pm 187 \mathrm{μs}\left({\color{lightgreen}-14.623 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$16.4 \mathrm{ms} \pm 177 \mathrm{μs}\left({\color{gray}1.18 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$16.0 \mathrm{ms} \pm 190 \mathrm{μs}\left({\color{red}20.4 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$16.2 \mathrm{ms} \pm 178 \mathrm{μs}\left({\color{gray}3.87 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$16.0 \mathrm{ms} \pm 208 \mathrm{μs}\left({\color{gray}-0.966 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.42 \mathrm{ms} \pm 9.14 \mathrm{μs}\left({\color{gray}1.24 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 10 entities $$2.08 \mathrm{ms} \pm 10.0 \mathrm{μs}\left({\color{gray}0.406 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$2.68 \mathrm{ms} \pm 73.2 \mathrm{μs}\left({\color{lightgreen}-7.349 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$1.91 \mathrm{ms} \pm 10.7 \mathrm{μs}\left({\color{gray}0.440 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$4.11 \mathrm{ms} \pm 25.4 \mathrm{μs}\left({\color{gray}0.945 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.89 \mathrm{ms} \pm 12.1 \mathrm{μs}\left({\color{gray}-0.035 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 10 entities $$53.0 \mathrm{ms} \pm 182 \mathrm{μs}\left({\color{gray}2.73 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$71.8 \mathrm{ms} \pm 404 \mathrm{μs}\left({\color{gray}-3.171 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$25.1 \mathrm{ms} \pm 149 \mathrm{μs}\left({\color{gray}1.59 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$270 \mathrm{ms} \pm 2.35 \mathrm{ms}\left({\color{lightgreen}-32.364 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.1 \mathrm{ms} \pm 103 \mathrm{μs}\left({\color{gray}0.520 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 10000 entities $$9.52 \mathrm{ms} \pm 148 \mathrm{μs}\left({\color{lightgreen}-23.922 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$1.88 \mathrm{ms} \pm 8.08 \mathrm{μs}\left({\color{gray}0.761 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$2.04 \mathrm{ms} \pm 9.09 \mathrm{μs}\left({\color{gray}0.982 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$2.76 \mathrm{ms} \pm 17.8 \mathrm{μs}\left({\color{gray}0.508 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.87 \mathrm{ms} \pm 10.5 \mathrm{μs}\left({\color{gray}1.20 \mathrm{\%}}\right) $$ Flame Graph

@TimDiekmann TimDiekmann added this pull request to the merge queue Oct 1, 2024
Merged via the queue into main with commit dbb6e16 Oct 1, 2024
82 of 83 checks passed
@TimDiekmann TimDiekmann deleted the t/h-3385-write-migration-for-h-3354 branch October 1, 2024 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-graph area/apps area/tests > integration New or updated integration tests area/tests New or updated tests type/eng > backend Owned by the @backend team
Development

Successfully merging this pull request may close these issues.

2 participants