[WIP] this script allows us to compare two images #669
Conversation
|
Skipping CI for Draft Pull Request. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
| # Example usage: | ||
| # ./ci/compare-images.sh quay.io/opendatahub/workbench-images@sha256:e92bf20e127e545bdf56887903dc72ad227082b8bc23f45ff4f0fc67e6430318 ghcr.io/jiridanek/notebooks/workbench-images:base-ubi9-python-3.9-jd_ubi_base_adedd4a943977ecdcb67bc6eb9eda572d10c3ddc | ||
|
|
||
| shopt -s globstar |
There was a problem hiding this comment.
What about
set -Eeuxo pipefail
https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
to fail if something needed is missing?
I happen to not have skopeo, so
./ci/compare-images.sh quay.io/opendatahub/workbench-images@sha256:e92bf20e127e545bdf56887903dc72ad227082b8bc23f45ff4f0fc67e6430318 ghcr.io/jiridanek/notebooks/workbench-images:base-ubi9-python-3.9-jd_ubi_base_adedd4a943977ecdcb67bc6eb9eda572d10c3ddc
./ci/compare-images.sh: line 26: shopt: globstar: invalid shell option name
Gathering the metadata for the image: 'quay.io/opendatahub/workbench-images@sha256:e92bf20e127e545bdf56887903dc72ad227082b8bc23f45ff4f0fc67e6430318'
Image SHA: 'e92bf20e127e545bdf56887903dc72ad227082b8bc23f45ff4f0fc67e6430318'
./ci/compare-images.sh: line 42: skopeo: command not found
./ci/compare-images.sh: line 45: skopeo: command not found
| diff -y "${tmp_dir}"/*-global-pip.txt | ||
| diff -y "${tmp_dir}"/*-local-pip.txt | ||
| diff -y "${tmp_dir}"/*-rpms.txt |
There was a problem hiding this comment.
For these, https://man.archlinux.org/man/comm.1.en may work a little bit better?
| podman run --entrypoint /usr/bin/pip --rm -it "${image}" list > "${tmp_dir}/${image_sha}-global-pip.txt" | ||
| podman run --entrypoint /opt/app-root/bin/pip --rm -it "${image}" list > "${tmp_dir}/${image_sha}-local-pip.txt" |
There was a problem hiding this comment.
--disable-pip-version-check ?
There was a problem hiding this comment.
I like this. We should (eventually) have the results of this available everytime we update metadata in release-2.8 etc. release branch of images, also everytime we update 2024a etc., and for every PR that makes changes to images.
There was a problem hiding this comment.
yep, that is exactly my point; I'll create a tracking issue for this so I can plan this for some of next sprints then
update: https://issues.redhat.com/browse/RHOAIENG-11254
also, I realized I can't edit description of this, so once I address your concerns here, I will probably close this PR and will raise my own later on 🙂
| echo "Let's compare the image metadata now:" | ||
|
|
||
| diff -y "${tmp_dir}"/*-size.txt | ||
| diff -y "${tmp_dir}"/*-metadata.txt |
There was a problem hiding this comment.
This is essentially a json, or it can be printed as json, so the diff should use that
diff <(jq --sort-keys . A.json) <(jq --sort-keys . B.json)
some people on stackoverflow suggest jd -set, https://stackoverflow.com/questions/31930041/using-jq-or-alternative-command-line-tools-to-compare-json-files
|
Have you seen this? Maybe it could be useful. |
There was a problem hiding this comment.
Have you seen this? Maybe it could be useful.
Not bad, but the functionality seems different. This script compares packages, whereas the linked tool compares files in the images. Guess it could be added to the script, but then it would require us to have the tool preinstalled.
There was a problem hiding this comment.
@caponetto Actually, I tried that tool before I started to implement this my own thing. I didn't like the output of that script much. It's too verbose and shows complete differences of the image on the file level instead of the package level. I'm not sure whether this is what we want. That is why I went with our own way.
There was a problem hiding this comment.
Cool! I'm glad you've already considered it.
|
@jiridanek: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Description
How Has This Been Tested?
Merge criteria: