Way to replace documents in a quickwit index?

[camerondavison]

I think that this has been touched on by some previous discussions, so I apologize if it is a duplicate question, but I just want to be more explicit with my question.

Is there a way to replace a document with new data in Quickwit? The old document would be marked deleted, and the new document would be the only one that is searchable.

To give some context, we have an event driven architecture (somewhat event sourced). I started using Quickwit to index all the events so that it would be easy to search and find what is happening in the system. Quickwit seems purpose built for this use case, and it works great (easy to get up and running quickly consuming from kafka). I also want to reduce these events down and provide a view of some of the documents that is a merged copy of a bunch of these events. Currently we store this view as jsonb in a postgres database so it is easy to get back out one at a time, but I want to allow quick searching over these documents. I looked into some other systems for this (such as meilisearch, which also worked well) but I think it could be too difficult for me to have to learn the eccentricities of both of these systems at the same time. I think it would be ideal for me to use quickwit for both of these cases.

[fmassot]

Hey @camerondavison, we don't support updates currently and it's not on the roadmap.

We support delete queries, but this feature is here for compliance use cases. Delete queries are expensive and you should not execute delete queries often. We explain it here in the docs.

[iampranabroy]

Well, that's a bummer. Is there a way you think we can work around it? For files uploaded in AWS S3, we can parse them using Apache Tika and write to the Quickwit index. But if a new version of the same document is uploaded, how would it work? I'd appreciate if you could share some thoughts on this.
@camerondavison - Did you find a workaround for this?

[camerondavison]

No. I used elastic search for stuff that was mutable. I use quick wit for immutable event logs.

[iampranabroy]

@fmassot -Could we use the delete API if there are only a few updates per day and the team is aware of its limitations without any performance degradation?
Also, are there any plans to upgrade the API in the future? For example, can we specify the document type during index as mutable so that the split is handled differently than the immutable ones? This is just a suggestion.:)