Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add -outputRerankerRequests option to create input for RankLLM #2463

Merged
merged 5 commits into from
Apr 24, 2024

Conversation

ronakice
Copy link
Member

As discussed in RankLLM, we can create an intermediate JSONL file with this option.

Example Usage:
java -cp ls target/*-fatjar.jar io.anserini.search.SearchCollection -index msmarco-v2.1-doc-segmented -topics trec2021-dl -output runs/run.msmarco-v2.1-doc-segmented.dl21.txt -outputRerankerRequests runs/retrieve_results_msmarco-v2.1-doc-segmented-dl21_top20.jsonl -bm25 -hits 20 -threads 16 -format trec

We only use JSONL, since RankLLM supports it.

The results for a single query can then be visualized by head -1 runs/retrieve_results_msmarco-v2.1-doc-segmented-dl21_top20.jsonl | jq.

@ronakice ronakice requested a review from lintool April 24, 2024 10:41
dependency-reduced-pom.xml Outdated Show resolved Hide resolved
Copy link

codecov bot commented Apr 24, 2024

Codecov Report

Attention: Patch coverage is 39.21569% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 66.66%. Comparing base (da5958b) to head (650affb).

❗ Current head 650affb differs from pull request most recent head 1b198bc. Consider uploading reports for the commit 1b198bc to get more accurate results

Files Patch % Lines
.../main/java/io/anserini/search/RunOutputWriter.java 15.38% 18 Missing and 4 partials ⚠️
...main/java/io/anserini/search/SearchCollection.java 54.54% 4 Missing and 1 partial ⚠️
...ava/io/anserini/search/SearchHnswDenseVectors.java 71.42% 2 Missing ⚠️
...io/anserini/search/SearchInvertedDenseVectors.java 71.42% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2463      +/-   ##
============================================
- Coverage     66.80%   66.66%   -0.14%     
  Complexity     1418     1418              
============================================
  Files           213      213              
  Lines         12206    12244      +38     
  Branches       1488     1494       +6     
============================================
+ Hits           8154     8163       +9     
- Misses         3538     3563      +25     
- Partials        514      518       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lintool lintool self-requested a review April 24, 2024 11:08
Copy link
Member

@lintool lintool left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this merged in!

@ronakice ronakice merged commit c085c1b into master Apr 24, 2024
@ronakice ronakice deleted the rank_llm_request branch April 24, 2024 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants