Bundling sqlite can cause data corruption on Android N and later

[M66B]

Please see here for the details:

https://ericsink.com/entries/sqlite_android_n.html

Since more and more people were reporting losing all data, especially on Redmi, OnePlus and Oppo devices, I stopped bundling sqlite. See also here:

https://forum.xda-developers.com/t/app-5-0-fairemail-fully-featured-open-source-privacy-oriented-email-app.3824168/post-87494245

IMHO there should be a warning about this in the description of this library.

[npurushe]

Could you share what about Android N causes an issue? this library doesn't link against the system sqlite version, instead it contains its own separate version. There is a another issue mentioned that is maybe more relevant about multiple sqlite instances linked to an app, I think that is true you should avoid using the Android OS sqlite APIs and this libraries APIs on the same db file at the same time but that is true for any Android OS version. Is that the issue you think you were seeing ?

[M66B]

Please see here for what I know, which is not much, unfortunately:

https://github.com/M66B/FairEmail/blob/master/FAQ.md#redmi

I am not 100% sure if this is the issue, though. With the data gone, debugging and error reporting become hard, not to say impossible.

I can say it is happening more often than before, possibly because the number of people using Android N or later is increasing.

I also know that, for example, the Android work manager uses sqlite as well.

[M66B]

There is some more information here:

https://ericsink.com/entries/multiple_sqlite_problem.html

[M66B]

Linking two sqlite libraries (requery's and Android's) is known to result in occasional corruption:

https://sqlite.org/howtocorrupt.html#multiple_copies_of_sqlite_linked_into_the_same_application

There is a proof of concept here:

https://github.com/jeremysheeley/Android_N_SQLite_Corruption

[npurushe]

For those cases the two instances have to be operating on the same database file. As long as you ensure you are not using both apis concurrently on the same database file there should be no issue.

[M66B]

That was my thought too, but in practice I am still seeing more database corruption than expected.

I will respond here after a while if I see changes.

[M66B]

I haven't seen any case of data loss anymore since removing this library from the app. That can't be coincidence.