Why doesn't bigcache consider designing expiration time for each cache key?

[zsyu9779]

During the process of using and reading the source code, it was found that in each instance of bigcache, all key-value pairs can only be set with a unified expiration time. A time.Ticker is used to periodically check if the cache has expired based on the interval set by CleanWindow. So why wasn't a method considered to allow setting expiration time for each cache key?

[janisz]

So why wasn't a method considered to allow setting expiration time for each cache key?

While it is technically feasible to implement such a feature, it could introduce complexity to the cache system. The bigcache library is based on a circular buffer (i.e., queue.BytesQueue) and does not have a defragmentation process. If we allow every key to be removed at different times, it could result in a significant amount of unused memory. On the other hand, we do allow the removal of a single key, but it is more like marking it as obsolete rather than actually deleting it.

[zsyu9779]

So why wasn't a method considered to allow setting expiration time for each cache key?

While it is technically feasible to implement such a feature, it could introduce complexity to the cache system. The bigcache library is based on a circular buffer (i.e., queue.BytesQueue) and does not have a defragmentation process. If we allow every key to be removed at different times, it could result in a significant amount of unused memory. On the other hand, we do allow the removal of a single key, but it is more like marking it as obsolete rather than actually deleting it.

Regarding the problem of memory waste, can we consider designing a compression algorithm? That is, when operating the bytesQueue, record the number of expired keys, and when the number of expired keys reaches a certain threshold, start a compression algorithm to asynchronously create a new, expanded bytesQueue. Then move the data from the old bytesQueue to the new one. During this process, read and write requests are synchronized between the two bytesQueues.

[zsyu9779]

So why wasn't a method considered to allow setting expiration time for each cache key?

While it is technically feasible to implement such a feature, it could introduce complexity to the cache system. The bigcache library is based on a circular buffer (i.e., queue.BytesQueue) and does not have a defragmentation process. If we allow every key to be removed at different times, it could result in a significant amount of unused memory. On the other hand, we do allow the removal of a single key, but it is more like marking it as obsolete rather than actually deleting it.

Regarding the problem of memory waste, can we consider designing a compression algorithm? That is, when operating the bytesQueue, record the number of expired keys, and when the number of expired keys reaches a certain threshold, start a compression algorithm to asynchronously create a new, expanded bytesQueue. Then move the data from the old bytesQueue to the new one. During this process, read and write requests are synchronized between the two bytesQueues.

However, this approach does seem to introduce some complexity to the entire system :(

[janisz]

In my opinion, we need to carefully consider the trade-off between complexity and usability when evaluating changes to our system. While adding new features can improve functionality, it can also introduce greater maintenance complexity. It's worth exploring other solutions available in the market for in-memory caching, as they may offer better performance or better align with specific requirements. For instance, the Bigcache solution is particularly well-suited for static maps that are not frequently modified after the application startup.