Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HNSWFlat index with JensenShannon metric brings program stuck #3828

Open
2 tasks
qwevdb opened this issue Sep 4, 2024 · 2 comments
Open
2 tasks

HNSWFlat index with JensenShannon metric brings program stuck #3828

qwevdb opened this issue Sep 4, 2024 · 2 comments

Comments

@qwevdb
Copy link

qwevdb commented Sep 4, 2024

Summary

When using HNSWFlat index with JensenShannon metric and processed data > 1.0, the program will get stuck.

Platform

OS: Ubuntu 24.04 LTS

Faiss version: faiss-gpu 1.8.0

Installed from: Anaconda

Faiss compilation options:

Running on:

  • [✔] CPU
  • GPU

Interface:

  • C++
  • [✔] Python

Reproduction instructions

When using HNSWFlat index with JensenShannon metric and handling data greater than 1.0, the Python script will get stuck.
However, the Python script runs normally, when handling data between 0 and 1.0 .

import faiss
import numpy as np

d = 2
metric_type = faiss.METRIC_JensenShannon
M = 10
index = faiss.IndexHNSWFlat(d, M, metric_type)

np.random.seed(0)
insert_data = np.random.randint(0, 10, (10, d))  # stuck
# insert_data = np.random.randn(10, d)           # stuck
# insert_data = np.random.random((10, d))        # not stuck

index.train(insert_data)
index.add(insert_data)

query_data = np.random.randint(0, 10, (1, d))
# query_data = np.random.randn(1, d)         
# query_data = np.random.random((1, d))

k = 5

#program stuck
D, I = index.search(query_data, k)
print(D)
print(I)
@qwevdb
Copy link
Author

qwevdb commented Sep 26, 2024

Any results for this bug? @asadoughi @kuarora

@asadoughi
Copy link
Contributor

I am able to reproduce the issue with the code provided. However, I'm not intimately familiar with Jensen-Shannon metric and will not be able to prioritize looking into this bug at this time. Normalizing your vectors should be a sufficient workaround for now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants