do_bulk_insert requires the collection to already exist?

[thomcom]

I'm trying to create a new collection with do_bulk_insert and getting an error (that I was not getting previously).

    utility.do_bulk_insert(ingest_process.collection_id, ingest_process.filename)

RPC error: [do_bulk_insert], <MilvusException: (code=100, message=collection not found[database=default]
[collection=a552b9ad4_adb1_47e0_9e98_33d913b52757])>, <Time:{'RPC start': '2024-08-14 14:00:04.783492',
 'RPC error': '2024-08-14 14:00:04.786796'}>

Is it necessary to have created an empty collection with schema before I can begin bulk insert? I'm confused because I successfully implemented this just a couple of weeks ago, and appeared to be creating collections with bulk insert from scratch, and now I have this error.

[yhmo]

do_bulk_insert requires a collection and a list of file paths on S3:
https://milvus.io/api-reference/pymilvus/v2.4.x/ORM/utility/do_bulk_insert.md

Before you call this interface, you need to ensure the collection exists.
This API returns an ID for the bulkinsert task, then you can use get_bulk_insert_state() to check the state of this task.

state = utility.get_bulk_insert_state(task_id=id)
if state.state == BulkInsertState.ImportFailed or state.state == BulkInsertState.ImportFailedAndCleaned:
    print(state)
    print("The task", state.task_id, "failed, reason:", state.failed_reason)

Ideally, this interface returns error if the collection doesn't exist. Which version of milvus you used a couple of weeks ago?

[thomcom]

Thanks for your help - I thought I'd already followed up in here answering my own question. I think I was using a collection that had already been created when I successfully used bulk_import a couple of weeks ago, then when I cleaned up the dataset found that I wasn't creating collections anymore.