feat: Add zstd compression to crane append and mutate #1798
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
ReillyBrogan
force-pushed
the
crane-zstd-append
branch
2 times, most recently
from
6e60934
to
188e23b
Compare
ReillyBrogan
force-pushed
the
crane-zstd-append
branch
from
188e23b
to
710efd3
Compare
| @@ -90,7 +107,11 @@ func getLayer(path string, layerType types.MediaType) (v1.Layer, error) { | |||
| return stream.NewLayer(f, stream.WithMediaType(layerType)), nil | |||
| } | |||
|
|
|||
| return tarball.LayerFromFile(path, tarball.WithMediaType(layerType)) | |||
| if layerType == types.OCILayerZStd { | |||
| return tarball.LayerFromFile(path, tarball.WithMediaType(layerType), tarball.WithCompression(compression.ZStd)) | |||
There was a problem hiding this comment.
I don't think this works properly. Imagine you gave it a .tar.gz file and asked for zstd compression. No re-compression will happen. You will have zstd in manifest, but contents of the blob will be gzip.
Note that there is a bug even before this PR: if one calls crane append with a zstd file, it will blindly append it with gzip in manifest.
| return "Compression" | ||
| } | ||
|
|
||
| var ErrZStdNonOci = errors.New("ZSTD compression can only be used with an OCI base image") |
There was a problem hiding this comment.
This statement is not true. There exists vnd.docker.image.rootfs.diff.tar.zstd type. See moby/buildkit#2344 (comment), moby/buildkit#3968 and containerd/containerd#9263
It is now possible to `crane append -f bla.tar.zstd` to upload Zstd layer, both from file and from pipe. Before this commit, crane would erroneously upload such layer with gzip mime type. While this PR does not _fully_ solve google#1798, it is very close to that. Signed-off-by: Marat Radchenko <[email protected]>
|
See #1827 for alternative implementation. |
|
This Pull Request is stale because it has been open for 90 days with |
|
No description provided. |
|
This Pull Request is stale because it has been open for 90 days with |
Support for zstd-compression was added in #1487. Add handling for this to crane append and mutate which should allow other projects that use Crane to begin creating zstd images (notably rules_oci which is used by distroless amongst other projects).