Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filegroup_external should set canonical_id #441

Open
wchargin opened this issue Nov 6, 2019 · 0 comments
Open

filegroup_external should set canonical_id #441

wchargin opened this issue Nov 6, 2019 · 0 comments

Comments

@wchargin
Copy link
Contributor

wchargin commented Nov 6, 2019

Using filegroup_external to download files has the downside that if
one bumps the version number of the source URL, but forgets to update
the SHA-256 checksum, then Bazel will assume that the content is
unchanged and will silently use the old version. The canonical issue
thread for this is: bazelbuild/bazel#5144

Bazel now exposes a way to mitigate this problem—ctx.download takes a
new argument canonical_id, which forms part of the cache key:
https://github.com/bazelbuild/proposals/blob/master/designs/2019-04-29-cache.md

The filegroup_external rule could set the canonical_id to (a
lossless encoding of) the whole list of URLs, such that any change to
the list would cause a re-download. This has the slight downside that
adding a new URL to an existing list purely to increase redundancy would
also cause a one-time cache miss, which seems acceptable.

This would have saved a fair amount of confusion on the following PR:
tensorflow/tensorboard#2898

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant