index not found[collection=docs_embeddings]

[matsuo1234567]

I have created a program based on this page.
https://zilliz.com/learn/Exploring-BGE-M3-the-future-of-information-retrieval-with-milvus

zilliz.py

from FlagEmbedding import BGEM3FlagModel
import pymilvus
import numpy as np
from pymilvus import (
    connections,
    FieldSchema,
    CollectionSchema,
    DataType,
    Collection,
    utility,
)

connections.connect("default", host="127.0.0.1", port="19530")

fields = [
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=1024),
]

schema = CollectionSchema(fields, description="Document Embeddings Collection")
collection = Collection(name="docs_embeddings", schema=schema)


collection.load()

if utility.has_collection(collection_name):
    collection.release()


if collection.has_index():
    collection.drop_index()


index_params = {"index_type": "IVF_FLAT", "metric_type": "L2", "params": {"nlist": 128}}
collection.create_index("embedding", index_params)
print("New index created successfully.")


collection.load()


from pymilvus.model.hybrid import BGEM3EmbeddingFunction

bge_m3 = BGEM3EmbeddingFunction(model_name="BAAI/bge-m3", device="cpu", use_fp16=False)

documents = [
    "Climate change is a significant global issue.",
    "El cambio climático es un problema global significativo.",
    "气候变化是一个重大的全球问题。",
]


docs_embeddings = bge_m3.encode_documents(documents)
entities = [{"embedding": doc.tolist()} for doc in docs_embeddings["dense"]]
insert_result = collection.insert(entities)


collection.load()


search_params = {"metric_type": "L2", "params": {"nprobe": 10}}
query_embeddings = docs_embeddings["dense"][2:3]
result = collection.search(
    query_embeddings, "embedding", search_params, limit=6, output_fields=["id"]
)

for hits in result:
    for hit in hits:
        print(f"hit: {hit}, id: {hit.id}")

When I run it, I get the following error.

% python zilliz.py
RPC error: [load_collection], <MilvusException: (code=700, message=index not found[collection=docs_embeddings])>, <Time:{'RPC start': '2024-09-11 16:15:35.363838', 'RPC error': '2024-09-11 16:15:35.368702'}>
Traceback (most recent call last):
  File "/Users/dev/milvus-lite/QAbot-lite/zilliz.py", line 32, in <module>
    collection.load()
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/orm/collection.py", line 425, in load
    conn.load_collection(
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/decorators.py", line 148, in handler
    raise e from e
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/decorators.py", line 144, in handler
    return func(*args, **kwargs)
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/decorators.py", line 183, in handler
    return func(self, *args, **kwargs)
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/decorators.py", line 123, in handler
    raise e from e
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/decorators.py", line 87, in handler
    return func(*args, **kwargs)
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/client/grpc_handler.py", line 1160, in load_collection
    check_status(response)
  File "/Users/dev/milvus-lite/QAbot-lite/qa_venv/lib/python3.10/site-packages/pymilvus/client/utils.py", line 63, in check_status
    raise MilvusException(status.code, status.reason, status.error_code)
pymilvus.exceptions.MilvusException: <MilvusException: (code=700, message=index not found[collection=docs_embeddings])>

[yhmo]

You call collection.load() twice, one is before collection.create_index(), the other is after collection.create_index(). The error is returned by the first call of collection.load(). Remove the first call and try again.

[matsuo1234567]

Resolved. Thank you.

But not output

print(f"hit: {hit}, id: {hit.id}")

[yhmo]

Set consistency_level="Strong" then you will see the results:

result = collection.search(
    query_embeddings, "embedding", search_params, limit=6, output_fields=["id"],
    consistency_level="Strong"
)

consistency_level is to control the data visibility for search/query. Read the doc for more info: https://milvus.io/docs/consistency.md#Consistency-levels

[matsuo1234567]

Thank you very much.