Improving reusability and composability of ublue-os images

[joshua-stone]

As more ublue-os images are added, it's becoming increasingly clear that we should reorganize our images to reduce duplication of work and increase sharing of commonly used packages and patterns into standardized base images. This will give users "upstream" images that have generally useful features and enhancements that one would otherwise end up layering after a fresh Silverblue install. Such images would include:

• Enabling RPMFusion
• Enabling codecs and hardware acceleration
• Adding vim, distrobox, and other common utilities
• Adding additional udev rules for various hardware

These images in turn would be useful for consuming in specialized derivative images. Take the Nvidia image for example, which currently includes many additions that aren't related to Nvidia support. These additions could be separated into a common upstream image definition:

ARG IMAGE_NAME="${IMAGE_NAME:-silverblue}"
ARG BASE_IMAGE="quay.io/fedora-ostree-desktops/${IMAGE_NAME}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION:-37}"

FROM ${BASE_IMAGE}:${FEDORA_MAJOR_VERSION} AS builder

COPY --from=ghcr.io/ublue-os/udev-rules:latest /ublue-os-udev-rules.noarch.rpm /tmp/ublue-os-udev-rules.noarch.rpm

RUN ls -lh /etc/yum.repos.d

RUN sed -i 's@enabled=1@enabled=0@g' /etc/yum.repos.d/fedora-{cisco-openh264,*modular,updates-modular}.repo

RUN \
        rpm-ostree install \
            https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
            https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm \
            /tmp/ublue-os-udev-rules.noarch.rpm \
            fedora-repos-archive \
            vim distrobox \
    && \
        rpm-ostree override remove $(rpm -qa --queryformat='%{NAME} ' \
            mesa-va-drivers \
            libavutil-free \
            libswscale-free \
            libswresample-free \
            libavformat-free \
            libavcodec-free \
            libavfilter-free \
            libavdevice-free \
            libpostproc-free) \
            --install=kernel-devel-$(rpm -q kernel --queryformat '%{VERSION}-%{RELEASE}.%{ARCH}') \
            --install=mesa-va-drivers-freeworld \
            --install=mesa-vdpau-drivers-freeworld \
            --install=libva-intel-driver \
            --install=ffmpeg-libs \
            --install=ffmpeg \
            --install=libavcodec-freeworld \
            --install=libva-utils \
    && \
        rm -rf /tmp/* \
    && \
        ostree container commit \
    && \
        mkdir -p /var/tmp && \
        chmod -R 1777 /var/tmp

Which leaves Nvidia-specific functionality:

ARG IMAGE_NAME="${IMAGE_NAME:-silverblue}"
ARG BASE_IMAGE="ghcr.io/ublue-os/${IMAGE_NAME}-main"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION:-37}"

FROM ${BASE_IMAGE}:${FEDORA_MAJOR_VERSION}

ARG NVIDIA_MAJOR_VERSION="${NVIDIA_MAJOR_VERSION:-525}"

# nvidia 520.xxx and newer currently don't have a -$VERSIONxx suffix in their
# package names
RUN if [ "${NVIDIA_MAJOR_VERSION}" -ge 520 ]; then echo "nvidia"; else echo "nvidia-${NVIDIA_MAJOR_VERSION}xx"; fi > /tmp/nvidia-package-name.txt

RUN rpm-ostree install \
        akmods \
        mock \
        xorg-x11-drv-$(cat /tmp/nvidia-package-name.txt)-{,cuda,devel,kmodsrc,power}*:${NVIDIA_MAJOR_VERSION}.*.fc$(rpm -E '%fedora.%_arch')  \
        binutils


# alternatives cannot create symlinks on its own during a container build
RUN ln -s /usr/bin/ld.bfd /etc/alternatives/ld && ln -s /etc/alternatives/ld /usr/bin/ld

ADD certs /tmp/certs

RUN [[ -s "/tmp/certs/private_key.priv" ]] || \
    echo "WARNING: Using test signing key. Run './generate-akmods-key' for production builds." && \
    cp /tmp/certs/private_key.priv{.test,} && \
    cp /tmp/certs/public_key.der{.test,}

RUN install -Dm644 /tmp/certs/public_key.der   /etc/pki/akmods/certs/public_key.der
RUN install -Dm644 /tmp/certs/private_key.priv /etc/pki/akmods/private/private_key.priv

# Either successfully build and install the kernel modules, or fail early with debug output
RUN NVIDIA_PACKAGE_NAME="$(cat /tmp/nvidia-package-name.txt)" \
    KERNEL_VERSION="$(rpm -q kernel --queryformat '%{VERSION}-%{RELEASE}.%{ARCH}')" \
    NVIDIA_VERSION="$(basename "$(rpm -q "xorg-x11-drv-$(cat /tmp/nvidia-package-name.txt)" --queryformat '%{VERSION}-%{RELEASE}')" ".fc$(rpm -E '%fedora')")" \
    && \
        echo $NVIDIA_VERSION && akmods --force --kernels "${KERNEL_VERSION}" --kmod "${NVIDIA_PACKAGE_NAME}" \
    && \
        modinfo /usr/lib/modules/${KERNEL_VERSION}/extra/${NVIDIA_PACKAGE_NAME}/nvidia{,-drm,-modeset,-peermem,-uvm}.ko.xz > /dev/null \
    || \
        (cat /var/cache/akmods/${NVIDIA_PACKAGE_NAME}/${NVIDIA_VERSION}-for-${KERNEL_VERSION}.failed.log && exit 1)

ADD ublue-os-nvidia-addons.spec /tmp/ublue-os-nvidia-addons/ublue-os-nvidia-addons.spec

ADD https://nvidia.github.io/nvidia-docker/rhel9.0/nvidia-docker.repo \
    /tmp/ublue-os-nvidia-addons/rpmbuild/SOURCES/nvidia-container-runtime.repo

RUN sed -i "s@gpgcheck=0@gpgcheck=1@" /tmp/ublue-os-nvidia-addons/rpmbuild/SOURCES/nvidia-container-runtime.repo

ADD files/etc/nvidia-container-runtime/config-rootless.toml \
    /tmp/ublue-os-nvidia-addons/rpmbuild/SOURCES/config-rootless.toml
ADD https://raw.githubusercontent.com/NVIDIA/dgx-selinux/master/bin/RHEL9/nvidia-container.pp \
    /tmp/ublue-os-nvidia-addons/rpmbuild/SOURCES/nvidia-container.pp
ADD files/etc/sway/environment /tmp/ublue-os-nvidia-addons/rpmbuild/SOURCES/environment

RUN install -D /etc/pki/akmods/certs/public_key.der /tmp/ublue-os-nvidia-addons/rpmbuild/SOURCES/public_key.der

RUN rpmbuild -ba \
    --define '_topdir /tmp/ublue-os-nvidia-addons/rpmbuild' \
    --define '%_tmppath %{_topdir}/tmp' \
    /tmp/ublue-os-nvidia-addons/ublue-os-nvidia-addons.spec


RUN cp /tmp/nvidia-package-name.txt /var/cache/akmods/nvidia-package-name.txt
RUN echo "${NVIDIA_MAJOR_VERSION}" > /var/cache/akmods/nvidia-major-version.txt
RUN rpm -q "xorg-x11-drv-$(cat /tmp/nvidia-package-name.txt)" \
    --queryformat '%{EPOCH}:%{VERSION}-%{RELEASE}.%{ARCH}' > /var/cache/akmods/nvidia-full-version.txt

FROM ${BASE_IMAGE}:${FEDORA_MAJOR_VERSION}

ARG IMAGE_NAME="${IMAGE_NAME}"

COPY --from=builder /var/cache/akmods /tmp/akmods
COPY --from=builder /tmp/ublue-os-nvidia-addons /tmp/ublue-os-nvidia-addons

RUN sed -i 's@enabled=1@enabled=0@g' /etc/yum.repos.d/fedora-{cisco-openh264,modular,updates-modular}.repo

RUN install -D /tmp/ublue-os-nvidia-addons/rpmbuild/SOURCES/nvidia-container-runtime.repo \
    /etc/yum.repos.d/nvidia-container-runtime.repo

RUN KERNEL_VERSION="$(rpm -q kernel --queryformat '%{VERSION}-%{RELEASE}.%{ARCH}')" \
    NVIDIA_FULL_VERSION="$(cat /tmp/akmods/nvidia-full-version.txt)" \
    NVIDIA_PACKAGE_NAME="$(cat /tmp/akmods/nvidia-package-name.txt)" \
    && \
        rpm-ostree install \
            xorg-x11-drv-${NVIDIA_PACKAGE_NAME}-{,cuda-,devel-,kmodsrc-,power-}${NVIDIA_FULL_VERSION} \
            "/tmp/akmods/${NVIDIA_PACKAGE_NAME}/kmod-${NVIDIA_PACKAGE_NAME}-${KERNEL_VERSION}-${NVIDIA_FULL_VERSION#*:}.rpm" \
            /tmp/ublue-os-nvidia-addons/rpmbuild/RPMS/noarch/ublue-os-nvidia-addons-*.rpm \
            nvidia-container-toolkit nvtop nvidia-vaapi-driver \
    && \
        mv /etc/nvidia-container-runtime/config.toml{,.orig} && \
        cp /etc/nvidia-container-runtime/config{-rootless,}.toml \
    && \
        semodule --verbose --install /usr/share/selinux/packages/nvidia-container.pp \
    && \
        sed -i 's@enabled=1@enabled=0@g' /etc/yum.repos.d/rpmfusion-{,non}free{,-updates}.repo \
    && \
        ln -s /usr/bin/ld.bfd /etc/alternatives/ld && \
        ln -s /etc/alternatives/ld /usr/bin/ld \
    && \
       ([[ "${IMAGE_NAME}" == "sericea" ]] && \
       mv /etc/sway/environment{,.orig} && \
       install -Dm644 /usr/share/ublue-os/etc/sway/environment /etc/sway/environment) ||: \
    && \
        rm -rf \
            /tmp/* \
            /var/* \
    && \
        ostree container commit \
    && \
        mkdir -p /var/tmp && \
        chmod -R 1777 /var/tmp

In theory this will improve clarity for where to file feature requests and bug reports. A request to add a specific package to the base image would lead to all derivative images benefiting by automatically including it on the next rebuild.

I believe what remains the biggest hurdle is how to name these base images on our registry when numerous names have already been reserved and it's not clear how many different base images we'd want to support. Current name proposals include:

• ghcr.io/ublue-os/${FLAVOR}-main
• ghcr.io/ublue-os/${FLAVOR}-mainline
• ghcr.io/ublue-os/${FLAVOR}-common
• ghcr.io/ublue-os/${FLAVOR}-comm

[castrojo]

Ok let me write some out so we can see what the final things look like. Am I getting this right?

• Upstream Silverblue -> our changes -> ghcr.io/ublue-os/silverblue-main
• Upstream Kinoite -> our changes -> ghcr.io/ublue-os/kinoite-main

and so on ... then for nvidia:

• ghcr.io/ublue-os/silverblue-nvidia
• ghcr.io/ublue-os/kinoite-nvidia

and then for people's individual spins:

• ghcr.io/ublue-os/ubuntu
• ghcr.io/ublue-os/ubuntu-nvidia
• ghcr.io/ublue-os/ubuntu-rocm (or something, theorizing)

[Zoialord]

I think something like that would make a lot of sense, but IMO you would need a little blurb on the site/page about what they are (just for new users to understand).

[castrojo]

Yes, for sure! We need to surface the information from the tags better, etc.

[castrojo]

Ok this looks fine to me and future-proof, unless there's any objections I say we go for it, the longer we wait the more people will need to rebase.

[xynydev]

I made a simple graph based on what I understand to be the direction we’re heading! https://www.figma.com/file/XgA7qbJ5f9ZvodGipR5L0c/Ublue-structure-draft

Feel free to correct any mistakes, etc.

I can make one for the final decided upon structure too for use on the website.

[xynydev]

Notes to self for graph updatdes:

• Call main main
• Call Amd Rocm
• Clear the spacing a bit
• Color/shape coding for images with different functions?
• Add/mention the config repo?
• Github repo names (like ublue-os/main)

Feel free to add to this.

[marcoceppi]

I like this proposal - thanks for taking time to document this. I'm against -mainline too many Linux kernel implications and I can imagine a future where we have silverblue-mainline where it's the actual mainline Linux kernel.

-main seems like the best remaining. I liked -core originally - but given the implications of core in Fedora, etc it will probably be too confusing.

Would each {flavor}-main be it's own repo? Or a "main" repo with all the flavor starting images in it?

[joshua-stone]

I believe we'd have a ublue-os/main repo where all {flavor}-main images are generated from, because there's really not much difference between them aside from preinstalled deskop environment. They'd originate from one Containerfile that's parameterized, similar to the nvidia image:

https://github.com/ublue-os/nvidia/blob/c9a544b6a165a349ca7d9953f8627bf01f361ca5/.github/workflows/build.yml#L27

[marcoceppi]

The other thing I like in @EinoHR image is it very much aligns with the docker base image -> buildpackdeps -> language pipeline. Where you have a Debian base image, then the buildpack-deps for Debian (that makes the Debian container actually useable) and others OS bases, then a Python container derived from the buildpack-deps. This project effectively becomes buildpack-deps for upstream silverblue, etc.

That image of upstream silverblue -> -main -> -nvidia / -amd -> custom-image / community-image resonates with me after reading this proposal

[ribbanya]

One thing I'm curious about in terms of maintenance is how ublue/main will be layered on top of upstream Fedora images. ublue/main is based directly off Silverblue, right? So then is it as simple as swapping out the FROM on ublue/kinoite to upstream Kinoite? Same for other official Fedora spins.

[ribbanya]

Another thing is, I think there is merit to having the repos all share the same git history, as in all be a clone (not fork) of ublue/main. This way, if there is duplicated content (like the Containerfile but also simple things like the README), they can copied from upstream via a merge. Ideally the git history would mirror the graph.

[castrojo]

Quick update to everyone on this, I've made the repos. The line of thinking was that this would be a good time to split things that make more logical sense.

• https://github.com/ublue-os/main
• https://github.com/ublue-os/config

The udev-rules package really just holds config files, so it makes more sense to move all the config there. Service units, config files, etc. That way we can also have that tracked in the package manager. So this new config repo is really just a rename of the udev-rules repo. I took an initial stab at it but then noticed we gotta rename a bunch of stuff so calling it for the night on that one. 🐕

Once we're done we'll tell people to rebase and give them a nice migration period. We have 90 days of image reaping in the registry, so whenever we freeze it + 90 days. On day 91 we archive base and udev-rules. Sound good?

[RedTopper]

Just as a slight note: you could actually put nvtop in the new base image if you want. Despite the name seeming like it's nvidia associated, it works with AMD and Intel too.

[castrojo]

I have a package request template incoming, if you don't mind filing an issue after it lands it'd be awesome to test the wording on the template so that we have a documented reason for each package we put in there.

[RedTopper]

Sure, I can give it a spin. Just mention me when it's ready to try out.

[castrojo]

We ended just putting it on there, we'll save the template for the next victim!

[castrojo]

Also forgot to ask, since this is a soft-reboot should we consider linting/shellcheck-style actions from the getgo? EDIT: set up super-linter (slim) in the experimental repo if anyone wants to try it: https://github.com/ublue-os/experimental