Skip to content

Commit

Permalink
build: support podman for load/pushing container images
Browse files Browse the repository at this point in the history
By default docker is tried, and then podman.
Support is simplistically verified with the exit code from --version
First choice can be overridden with DOCKER env variable.

Signed-off-by: Tiago Castro <[email protected]>
  • Loading branch information
tiagolobocastro committed Jul 29, 2024
1 parent 09f9189 commit 1983c87
Show file tree
Hide file tree
Showing 3 changed files with 112 additions and 17 deletions.
33 changes: 28 additions & 5 deletions doc/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,8 @@ $ sudo nixos-rebuild switch --update
[running][doc-run], then [testing][doc-test].
You can use a tool like [`direnv`][direnv] to automate `nix shell` entry.
If you are unable to use the Nix provided Rust for some reason, there are `norust` and
`nospdk` arguments to Nix shell. `nix-shell --arg norust true`
If you are unable to use the Nix provided Rust for some reason, there are `rust` and
`spdk` arguments to Nix shell. `nix-shell --arg rust none`
After cloning the repository don't forget to run a:
Expand Down Expand Up @@ -129,11 +129,12 @@ cargo build --release
**Want to run or hack on Mayastor?** _You need more configuration!_ See
[running][doc-running], then [testing][doc-testing].
Whilst the nix develop will allow you to build mayastor exactly as the image build, it might not have all the necessary components required for testing.
Whilst the nix develop will allow you to build mayastor exactly as the image build, it might not have all the necessary
components required for testing.
For that you might want to use the explicit shell configuration file: ci.nix:
```bash
nix-shell ci.nix
nix-shell
```
To ensure you are aware of this, we greet you with a nice cow.
Expand Down Expand Up @@ -164,7 +165,7 @@ done
Try them as if they were installed:
```rust
```bash
nix shell -f . io-engine
```
Expand Down Expand Up @@ -216,6 +217,10 @@ Build the Docker images with the CI build script:
❯ ./scripts/release.sh --registry localhost:5000 --image "mayastor-io-engine"
```
The container images are packaged and pushed using either docker or podman - whichever is run successfully with
--version cli argument.
If you want to specifically test one of these first, please set DOCKER env variable.
Build the Docker images with [`nix build`][nix-build]:
```bash
Expand All @@ -241,21 +246,39 @@ you want to run them locally:
> `spdk-rs` README on this matter.
[doc-run]: ./run.md
[doc-test]: ./test.md
[direnv]: https://direnv.net/
[nix-explore]: https://nixos.org/explore.html
[nix-install]: https://nixos.org/download.html
[nix-develop]: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-develop.html
[nix-paper]: https://edolstra.github.io/pubs/nixos-jfp-final.pdf
[nix-build]: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-build.html
[nix-bundle]: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-bundle.html
[nix-shell]: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-shell.html
[nix-channel]: https://nixos.wiki/wiki/Nix_channels
[nixos]: https://nixos.org/
[rust-lang]: https://www.rust-lang.org/
[windows-wsl2]: https://wiki.ubuntu.com/WSL#Ubuntu_on_WSL
[windows-hyperv]: https://wiki.ubuntu.com/Hyper-V
[docker-install]: https://docs.docker.com/get-docker/
[reproducible-builds]: https://reproducible-builds.org/
[cii-best-practices]: https://www.coreinfrastructure.org/programs/best-practices-program/
[direnv]: https://direnv.net/
27 changes: 26 additions & 1 deletion doc/run.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,35 +280,60 @@ Otherwise, you can follow [the production guide][running-on-a-real-kubernetes-cl
Our testing showed some common Kubernetes development environments are not sufficient for
Mayastor development.
Here are the ones known to not work:
Here are the ones known to not work by default:
- [`kind`][kind]
In order to make this one work, you need to add `/run/udev` and `/run/udev` to the kind node hostPath.
Once the node containers are running, you may need to remount `/sys` as rw.
Here is an example: https://github.com/openebs/mayastor-extensions/blob/develop/scripts/k8s/deployer.sh
## Running on a real Kubernetes cluster
We have a [production deployment manual & user manual][manual] prepared. Please refer to that for
production Mayastor deployment and operation instructions.
[running-on-a-real-kubernetes-cluster]: #running-on-a-real-kubernetes-cluster
[doc-build]: ./build.md
[doc-build-iterative-builds]: ./build.md#Iterative-Builds
[doc-build-building-docker-images]: ./build.md#Building-Docker-images
[doc-build-building-portable-nix-bundles]: ./build.md#Building-portable-Nix-bundles
[doc-test]: ./test.md
[io_uring-intro]: https://unixism.net/loti/what_is_io_uring.html
[hugepages-lwn-one]: https://lwn.net/Articles/374424/
[hugepages-lwn-two]: https://lwn.net/Articles/375096/
[hugepages-lwn-three]: https://lwn.net/Articles/376606/
[hugepages-lwn-four]: https://lwn.net/Articles/378641/
[hugepages-lwn-five]: https://lwn.net/Articles/379748/
[m-dot-2]: https://en.wikipedia.org/wiki/M.2
[nix-install]: https://nixos.org/download.html
[nix-shell]: https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-shell.html
[control-groups]: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/cgroups.html
[mozilla-rust-overlay]: https://github.com/mozilla/nixpkgs-mozilla/blob/master/rust-overlay.nix
[isa]: https://en.wikipedia.org/wiki/Instruction_set_architecture
[kind]: https://kind.sigs.k8s.io/
[manual]: https://mayastor.gitbook.io/
[lxd]: https://linuxcontainers.org/
[libvirtd]: https://libvirt.org/index.html
[terraform-readme]: ./terraform/readme.adoc
69 changes: 58 additions & 11 deletions scripts/release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,19 @@

CI=${CI-}

# Write output to error output stream.
echo_stderr() {
echo -e "${1}" >&2
}

# Write out error and exit process with specified error or 1.
die()
{
local _return="${2:-1}"
echo_stderr "$1"
exit "${_return}"
}

set -euo pipefail

# Test if the image already exists in dockerhub
Expand Down Expand Up @@ -62,6 +75,49 @@ pre_fetch_cargo_deps() {
echo "Failed to pre-fetch the cargo vendored dependencies in $maxAttempt attempts"
exit 1
}
# Setup DOCKER with the docker or podman (which is mostly cli compat with docker and thus
# we can simply use it as an alias) cli.
# If present, the env variable DOCKER is checked for the binary, with precedence.
docker_alias() {
DOCKER_CLIS=("docker" "podman")
if [ -n "${DOCKER:-}" ]; then
DOCKER_CLIS=("$DOCKER" ${DOCKER_CLIS[@]})
fi
for cli in ${DOCKER_CLIS[@]}; do
if binary_check "$cli"; then
echo "$cli"
return
fi
done
binary_missing_die "docker"
}
# Check if the binaries are present, otherwise bail out.
binaries_check() {
FAIL=
for bin in ${@}; do
if ! binary_check "$bin"; then
binary_missing "$bin"
FAIL="y"
fi
done
if [ -n "$FAIL" ]; then
exit 1
fi
}
# Check if the binary name is present, otherwise bail out.
binary_check() {
if ! "$1" --version &>/dev/null; then
return 1
fi
}
# Check if the binary name is present, otherwise bail out.
binary_missing_die() {
die "$(binary_missing "$1")"
}
# Get the binary missing error message
binary_missing() {
echo "$1 binary missing - please install it and add it to your PATH"
}

help() {
cat <<EOF
Expand All @@ -84,7 +140,7 @@ Examples:
EOF
}

DOCKER="docker"
DOCKER=$(docker_alias)
NIX_BUILD="nix-build"
NIX_EVAL="nix eval$(nix_experimental)"
RM="rm"
Expand All @@ -106,16 +162,7 @@ CARGO_VENDOR_DIR=${CARGO_VENDOR_DIR:-}
CARGO_VENDOR_ATTEMPTS=${CARGO_VENDOR_ATTEMPTS:-25}

# Check if all needed tools are installed
curl --version >/dev/null
if [ $? -ne 0 ]; then
echo "Missing curl - install it and put it to your PATH"
exit 1
fi
$DOCKER --version >/dev/null
if [ $? -ne 0 ]; then
echo "Missing docker - install it and put it to your PATH"
exit 1
fi
binaries_check "curl" "$DOCKER"

# Parse arguments
while [ "$#" -gt 0 ]; do
Expand Down

0 comments on commit 1983c87

Please sign in to comment.