Don't cache fresh copies of previously-cached content?

[marienz]

The magic cache seems to be caching many copies of the same NAR (for the same branch) differing only in random suffix (example: 1k3359wrsh92is6zq3m72p68n3nq8i7398srp9bxz7bjhxxf019h.nar.zstd-vbsU, 1k3359wrsh92is6zq3m72p68n3nq8i7398srp9bxz7bjhxxf019h.nar.zstd-p3uc and several more). These occupy a lot of space in Github's cache.

That random suffix looks like it's coming from allocate_file_with_random_suffix, which is documented as "a hack to allow for easy "overwriting" without deleting the original cache". And it looks like the cache may be using this "overwrite" for most or all uploads: in particular post_build_hook looks like it ends up calling this for the entire closure of the path just built.

I'm not sure I understand why the magic cache is attempting to overwrite cached content as frequently as it seems to be doing. If it's doing that just in case I have bugs in my derivations resulting in non-equivalent output when building the same derivation repeatedly (with the cache making sure the most recent build gets cached): can I opt out from this safeguard somehow? I'd prefer the improved cache efficiency.

(I'm happy to contribute a PR to change this behavior or make it configurable if you can clarify what the desired behavior is. If this is a safety feature that could be configurable, I can send a PR for that. But if the cache is overwriting or uploading content it shouldn't, I'd appreciate some pointers on how it's supposed to work so I can troubleshoot why it seems to be uploading duplicates.)