OutOfMemoryError: Out of Memory

[fabioselau077]

OutOfMemoryError: Out of Memory
File "store_impl.dart", in SembastStore.currentRecords
File "store_impl.dart", in SembastStore.forEachRecords
File "store_impl.dart", in SembastStore.txnFindRecords
File "store_ref_impl.dart", in SembastStoreRefExtensionImpl.findImmutableRecords
File "store_ref_impl.dart", in SembastStoreRefExtension.find

[alextekartik]

sembast is defintely not memory friendly - see recommendations -, however some things could be done on my side to improve it. Do you have more context information (platform, size of the database, number of records, does it happen right away or after some time, flutter or dart context, even better if you can share your database to help reproducing and investigation)?. Than you!

[haashem]

I have the same issue and started to show recently. using 3.7.2

I use Sembast to cache network requests:

_store.record(url).put(_sembastDatabase.instance, response);

[alextekartik]

Indeed compact (that you don't control) is an expensive operation. I would recommend switching to sembast_sqflite, the API is the same, only the factory is different and it will store persistent information in a database instead of a big file and should avoid some memory issues.

[alextekartik]

Also I don't know how big are your responses (1K, 100K, 10Mb) but if too big, sembast might not be the best solution here.
I would:

• limit the number of cached request (by adding a timestamp field and clearing old cached response when reaching NNN requests - see recommandations.
• if response are big (more than 100Kb, you might better store them as file and only store a reference to the file in the sembast database)

[alextekartik]

(and the compact operation could also be optimized as it doubled the memory usage during the operation, I would look into that)

[haashem]

Thanks for quick reply. the responses are less than 10 KB, and I have cached around 20 network requests. they will be overwritten by new responses so the size of cache will be less than 50Kb at max.

[alextekartik]

Indeed that is not a lot. Do you confirm that you don't have more than 20 different url (exactly the same as each url will create a new entry, changing a parameter will change the url) and are you able to share a database I can look at?

[haashem]

Ohhh, in that case can be many, some urls have query parameters so can cause new record.

I try to share a local db from simulator.

[haashem]

here you are:

db.sembast.zip

the db size 36 Mb with 150 records!

[alextekartik]

Indeed this is not in the recommended usage as most of the records are more than 500KB in a json format. I have slightly improve memory usage in compact in sembast in 3.7.3+3 just published but I would recommend:

• Add a timestamp to each record added
• Remove old records when reaching more than XXX records (let say 100)
• Save the the cached content in a file and hold a reference to the file in the db (and you can listen to deleted records to clear up the files created - see https://pub.dev/documentation/sembast/latest/sembast/SembastStoreRefExtension/addOnChangesListener.html). You might also add safe guard to clean up the files regurlary as since some request have information, the number could be infinite