Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Narek/bm25 bloom #343

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open

Narek/bm25 bloom #343

wants to merge 23 commits into from

Conversation

Ngalstyan4
Copy link
Contributor

No description provided.

@Ngalstyan4 Ngalstyan4 requested a review from var77 October 15, 2024 02:52
greptile-apps[bot]
greptile-apps bot reviewed Oct 15, 2024
View reviewed changes
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

Hi! Looks like you've reached your API usage limit. You can increase it from your account settings page here: app.greptile.com/settings/usage

3 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings | Greptile

@CLAassistant
Copy link

CLAassistant commented Oct 15, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

dqii
dqii reviewed Oct 15, 2024
View reviewed changes
lantern_extras/src/lib.rs Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 15, 2024 •
edited
Loading

Benchmarks

metric old new pct change
recall (after create) 0.951 0.938 -1.37%
recall (after insert) 0.000 0.000 -
select tps 11699.898 12007.805 +2.63%
select bulk(100) tps 36.385 34.569 -4.99%
select latency (ms) 1.054 ± 2.162𝜎 0.836 ± 2.230𝜎 -20.68%
select bulk(100) latency (ms) 853.805 ± 119.628𝜎 883.525 ± 152.265𝜎 +3.48%
create latency (ms) 419561.642 409382.588 -2.43%
insert tps 438.155 443.359 +1.19%
insert bulk(100) tps 4.737 4.638 -2.09%
insert latency (ms) 72.373 ± 17.948𝜎 71.567 ± 16.174𝜎 -1.11%
insert bulk(100) latency (ms) 6642.609 ± 124.848𝜎 6783.496 ± 94.631𝜎 +2.12%
disk usage (bytes) 8192008192.000 8192008192.000 -

@Ngalstyan4
Add bloom filter type and casts from postgres arrays
b4a80bf
@Ngalstyan4
Make lantern_extras prefix reserved in pgrx
e72b373
@Ngalstyan4
Make sure MarkGUCPrefix is only used in pg15 and pg16
d7cabf5
@Ngalstyan4
Rename cargo to cargo.toml in .cargo
34799d7 
.cargo/config has been deprecated in favor of config.toml since rust
1.38.
We use Rust 2021 edition, which is stable since 1.56, so it should be
fine to rename the file and avoid the warning during compilation
@Ngalstyan4
Add rust and tsvector stemmers API
9bfeb50
@Ngalstyan4
Add bm25 table creation API
a2045d0
@Ngalstyan4
Implement bm25_agg acreator
40d966a 
bm25 agg
@Ngalstyan4
Fix typo in text_to_stem_array function name
225189d
@Ngalstyan4
Explicitly order the output of unnest, since there are no guarantees …
fd80114 
…on its ordering
@Ngalstyan4
Fix cast issue
aab9d3e
@Ngalstyan4
Add debug print possibility to bm25_agg test
32736b4
@Ngalstyan4
Fix bm25/idf algo implementation bug
4e10c63
@Ngalstyan4
Add option to recreate bm25 table
b97fbd9
@Ngalstyan4
Improve error messages when set_user_stopwords fails to write to file
f1ea915
@Ngalstyan4
Add search_bm25 API
51aa4aa
@Ngalstyan4
Make bm25 aux table logged
e3b6ef6
@Ngalstyan4
Store table stats in _bm25 table to avoid expensive aggregates at que…
7e1bd38 
…ry time
@Ngalstyan4
Make content_join optional in search
3ea8793
@Ngalstyan4
Fix lifetime issues in tests with pgrx 0.12
5da19eb
@Ngalstyan4
Add bm25 approximation threshhold guc
c95199d
@Ngalstyan4
Add bm25 hyperparameter GUCs
0fbeb95
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments

@dqii dqii dqii left review comments

@var77 var77 Awaiting requested review from var77

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Ngalstyan4 @CLAassistant @dqii