Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add query and candidate #111

Merged
merged 8 commits into from
Apr 25, 2024
Merged

Add query and candidate #111

merged 8 commits into from
Apr 25, 2024

Conversation

sahel-sh
Copy link
Member

This cl:

  • adds rerank_batch, query and candidate data classes.
  • removes doc inline demo
  • retriever only supports dataset or custom index,
  • pyserini_retriever creates requests
  • updates all demos to work with rerank_batch
  • rerank (non-batch) will be added in a follow up cl
  • tests will be updated in a follow up cl
  • if we decide to keep pyserini retriever, caching will be reenabled in a followup cl.

Pull Request Checklist

Reference Issue

Please provide the reference to issue this PR is addressing (# followed by the issue number). If there is no associated issue, write "N/A".

ref: #110

Checklist Items

Before submitting your pull request, please review these items:

  • Have you followed the contributing guidelines?
  • Have you verified that there are no existing Pull Requests for the same update/change?
  • Have you updated any relevant documentation or added new tests where needed?

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Documentation content changes
  • Other...
    • Description:

Copy link
Member

@ronakice ronakice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor nits but looks good to me besides

src/rank_llm/data.py Outdated Show resolved Hide resolved
src/rank_llm/demo/rerank_inline_hits.py Show resolved Hide resolved
src/rank_llm/evaluation/trec_eval.py Outdated Show resolved Hide resolved
Copy link
Member

@ronakice ronakice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sahel-sh sahel-sh merged commit bfb1cba into main Apr 25, 2024
@sahel-sh sahel-sh deleted the add_Query_and_Candidate branch April 26, 2024 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants