Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pyserini: Port Torch Models to MLX and enable Indexing and Searching on Mac M series machines #39

Open
ToluClassics opened this issue Apr 15, 2024 · 0 comments

Comments

@ToluClassics
Copy link
Member

Summary: The goal is to extend the encode module (https://github.com/castorini/pyserini/tree/master/pyserini/encode ) of Pyserini to allow loading and inferencing retriever models in MLX similar to the way it is currently done in PyTorch. This enables us to index and search effectively on Mac M series laptops with near Cuda speed. So similar to DprDocumentEncoder we will have something like MlxDprDocumentEncoder .

What this entails is translating model architecture code from PyTorch to MLX. The interface of both frameworks are very similar except that MLX is more numpy like

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant