Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dockerTools: Allow separately specifying metadata and filesystem timestamps #327579

Merged

Conversation

the-sun-will-rise-tomorrow
Copy link
Contributor

@the-sun-will-rise-tomorrow the-sun-will-rise-tomorrow commented Jul 16, 2024

Description of changes

Setting the image creation timestamp in the image metadata to a constant date can cause problems with self-hosted container registries, that need to e.g. prune old images. This timestamp is also useful for debugging.

However, it is almost never useful to set the filesystem timestamp to a non-constant value. Doing so not only causes the image to possibly no longer be reproducible, but also removes any possibility of deduplicating layers with other images, causing unnecessary storage space usage.

Therefore, this commit introduces mtime, a new parameter to streamLayeredImage, which allows specifying the filesystem timestamps separately from created. For backwards compatibility, mtime defaults to the value of created.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@the-sun-will-rise-tomorrow
Copy link
Contributor Author

Some considerations:

  • Do we want tests (for separate created and mtime values)?
  • Is using the same mtime as created a good default? Would it be more useful to break backwards compatibility and decide on an arguably more useful default?
  • Do we want the same functionality for buildImage?
  • buildImage's behavior here currently differs from buildLayeredImage: it always uses --mtime="@$SOURCE_DATE_EPOCH". Do we want to reconcile this?

@tomberek
Copy link
Contributor

Reconciling buildImage and buildLayeredImage seems like a good idea. I'll take a look at this and review.

…stamps

Setting the image creation timestamp in the image metadata to a
constant date can cause problems with self-hosted container
registries, that need to e.g. prune old images.  This timestamp is
also useful for debugging.

However, it is almost never useful to set the filesystem timestamp to
a constant value.  Doing so not only causes the image to possibly no
longer be reproducible, but also removes any possibility of
deduplicating layers with other images, causing unnecessary storage
space usage.

Therefore, this commit introduces "mtime", a new parameter to
streamLayeredImage, which allows specifying the filesystem timestamps
separately from "created".  For backwards compatibility, "mtime"
defaults to the value of "created".
@tomberek
Copy link
Contributor

Added a change to the default such that mtime will be epoch more often, unless someone overrides it.

@tomberek tomberek merged commit d4567f6 into NixOS:master Sep 25, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants