RUST-1488 BulkWriteFailure does not contain inserted_ids when insert_many fails

[decatur]

Versions/Environment

1. What version of Rust are you using? cargo 1.60.0
2. What operating system are you using? MS Windows 10
3. What versions of the driver and its dependencies are you using? mongodb:2.3.0 & bson:2.4.0
4. What version of MongoDB are you using? AtlasDB 5.0.12
5. What is your MongoDB topology (standalone, replica set, sharded cluster, serverless)? M5 (General) Replica Set - 3 nodes

Describe the bug

The documentation states:

ordered:
If true, when an insert fails, return without performing the remaining writes. If false, when a write fails, continue with the remaining writes, if any.

So when inserting documents, with some violating an index, I expect that the insert does not panic but inserts the non-violating documents.
The actual behaviour is: The driver panics, and no document is inserted:

let options = mongodb::options::InsertManyOptions::builder().ordered(false).build();
let res = col.insert_many(document, options).await.unwrap();

The panic is

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: 
Error { kind: BulkWrite(BulkWriteFailure { write_errors: 
  Some([BulkWriteError {
   index: 0, code: 11000, code_name: None, message: "E11000 duplicate key error collection: col_name index: 
   index_name dup key: { field_name: \"X1\" }", details: None }]), write_concern_error: None, inserted_ids: {} }
 ), labels: {}, wire_version: None, source: None }'

The use case is optimistic locking: Bulk insert those documents which are not yet inserted.

[kmahar]

Hi @decatur!

The panic is expected given that you call unwrap() on the Result; this method returns an error if any of the individual writes fail. To avoid the panic, you could write your code to inspect the Result and ignore duplicate key errors.

Regarding the non-violating documents not getting inserted when ordered=false, I'm not able to reproduce this.

I've written the following program:

use futures::stream::TryStreamExt;
use mongodb::{Client, Collection, bson::Document, bson::doc, options::InsertManyOptions};

#[tokio::main]
async fn main() -> mongodb::error::Result<()>{
    let client = Client::with_uri_str("mongodb://localhost:27017,localhost:27018,localhost:27019/?replicaSet=repl0").await?;
    let coll: Collection<Document> = client.database("bulk_test").collection("test");
    // clear out any existing data
    coll.drop(None).await?;

    coll.insert_one(
        doc! { "_id": 1},
        None
    ).await?;

    let mut cursor = coll.find(None, None).await?;
    println!("querying collection after insert_one");
    while let Some(result) = cursor.try_next().await? {
        println!("Found document: {}", result);
    }

    let insert_opts = InsertManyOptions::builder().ordered(false).build();
    let insert_result = coll.insert_many(
        [
            doc! { "_id": 1},
            doc! { "_id": 2},
            doc! { "_id": 3}
        ],
        insert_opts
    ).await;
    match insert_result {
        Ok(_) => { println!("insert succeeded") },
        Err(e) => { println!("insert failed with error: {}", e) },
    };

    println!("querying collection after insert_many");
    let mut cursor = coll.find(None, None).await?;
    while let Some(result) = cursor.try_next().await? {
        println!("Found document: {}", result);
    }

    Ok(())
}

Which produces the following output:

querying collection after insert_one
Found document: { "_id": 1 }
insert failed with error: An error occurred when trying to execute a write operation: BulkWriteFailure { write_errors: Some([BulkWriteError { index: 0, code: 11000, code_name: None, message: "E11000 duplicate key error collection: bulk_test.test index: _id_ dup key: { _id: 1 }", details: None }]), write_concern_error: None, inserted_ids: {} }
querying collection after insert_many
Found document: { "_id": 1 }
Found document: { "_id": 2 }
Found document: { "_id": 3 }

Given the output it appears that the documents following the violating one, with _id values 2 and 3, are still inserted despite the duplicate key error inserting the first with _id 1.

Can you give me some more information about the data you inserting (what the documents look like, what the unique index looks like)? How are you verifying whether the non-violating documents were inserted after encountering the error?

[github-actions]

There has not been any recent activity on this ticket, so we are marking it as stale. If we do not hear anything further from you, this issue will be automatically closed in one week.

[decatur]

Hello @kmahar,

you are right, the non-violating documents get inserted, and I checked that the sitting document does not get updated!
I missed that.

What threw me off though is the BulkWriteFailure response (same in your example),

BulkWriteFailure { ..., inserted_ids: {} }

whereas clearly two documents were inserted.
Furthermore, as an error is returned and not a InsertManyResult, there seems to be no way to get the ids of the inserted documents.

[kmahar]

@decatur Good catch. The omission of the inserted_ids is definitely a bug, which I have filed RUST-1488 for. This should be a straightforward fix and I will make a PR for it this week.

[decatur]

Considerung both #761 and the fact that (at least with the Node driver) the desired result (getting a list of documents not inserted) can be achieved with

db.runCommand({insert: "own_trades", documents:[{"remark": "offending doc"}, {"TradeId": "4713", "timestamp": Timestamp(0, 2)}], ordered: false})

I will close this issue. Note that we did not tested the db.runCommand in the rust driver, i.e. the run_command

[dizda]

@decatur Good catch. The omission of the inserted_ids is definitely a bug, which I have filed RUST-1488 for. This should be a straightforward fix and I will make a PR for it this week.

I'm also having the same issue here, can't get the inserted_ids that have succeeded.