Indexing new documents without indexing the full collection

[Fatima-Haouari]

Hi,
I used pyserini to index a collection of documents. If my collection of documents was expanded, how can I index just the new documents without indexing the full collection again?

[lintool]

If you've modified the collection, including expansion, you'll have to re-index the entire collection.

[Fatima-Haouari]

Thanks a lot for quick response. If adding new documents to the collection, do I need re-index the entire collection as well?

[lintool]

In Pyserini, yes. For Lucene in general, no. Unfortunately, we have not exposed the hooks to Lucene that allow you to add new documents to the index.

[Fatima-Haouari]

Thanks for your help.