Slow speed on FAISS search

[m-hoseyny]

Hello there,

I want to retrieve passages for about 21 million queries on MS MARCO v1. I am using pyserini.faiss with an ANCE index. I installed pyserini step by step from the installation guidelines, and I have access to GPUs. I run the search process with this command:

      python -m pyserini.search.faiss \
            --index msmarco-v1-passage.ance  \
            --topics queries.tsv \
            --encoder castorini/ance-msmarco-passage \
            --output run.ance.queries.txt \
            --output-format trec \
            --batch-size 128 --threads 64 --device cuda:0 --hits 10

There is a process for this script on the GPU which allocates about 800MB of VRAM. However, the process is very slow, at about 3 iterations per second.

My faiss versions:
faiss-cpu==1.7.4
faiss-gpu==1.7.2
pysernin version: 0.21.1
python version: 3.8.16 (to support faiss-gpu)
GPU: RTX A6000

Is there any method to speed up the process? I did not find anything in the documentation.

[lintool]

Pyserini currently does not support faiss-gpu by default - but it's something the @jacklin64 might have experience with?

[jacklin64]

Hi @m-hoseyny, as I know, Pyserini does not support GPU search on Faiss. It seems that --device cuda:0 only generates query embeddings through GPU but still perform search on Faiss CPU.

[m-hoseyny]

Thanks for the answers.

Are there any solutions (e.g., reindexing with GPUs) for leveraging GPUs to improve retrieval performance? My current query set is so large that using Pyserini to retrieve relevant passages within an appropriate timeframe isn't feasible.

[lintool]

Unfortunately, not currently implemented in Pyserini... However, contributions are more than welcome!