Using a multi-stage build to add micromamba functionality to an existing image

[wholtz]

Dockerfile for tomcat with micromamba

I needed to use a tomcat image but wanted to have multiple R/Python (conda) environments within the final image. I could have copied and pasted the contents of the micromamba Dockerfile into a new Dockerfile that started with FROM tomcat, but then I'd need to copy all the scripts the micromamba Dockerfile calls into my new repo as well. If any of those scripts were later modified within the mamba-org/micromamba-docker repo, I would then have to manually update my downstream repo to keep up to date with the changes. By using a multi-stage build, I am able to copy the scripts directly from the mambaorg/micromamba image. Now if an updated version of mambaorg/micromamba is released, I only need to update the mambaorg/micromamba tag within my downstream Dockerfile and it will pickup the newer micromamba binary along with any changes to the included scripts.

The exercise has me debating with myself if the following code from the micromamba Dockerfile should be moved into a separate script file:

micromamba-docker/Dockerfile

Lines 34 to 41 in 41d5bd8

	RUN echo "source /usr/local/bin/_activate_current_env.sh" >> ~/.bashrc && \
	echo "source /usr/local/bin/_activate_current_env.sh" >> /etc/skel/.bashrc && \
	groupadd -g "${MAMBA_USER_GID}" "${MAMBA_USER}" && \
	useradd -u "${MAMBA_USER_ID}" -g "${MAMBA_USER_GID}" -ms /bin/bash "${MAMBA_USER}" && \
	echo "${MAMBA_USER}" > "/etc/arg_mamba_user" && \
	mkdir -p "$MAMBA_ROOT_PREFIX/conda-meta" && \
	chmod -R a+rwx "$MAMBA_ROOT_PREFIX" "/home" "/etc/arg_mamba_user" && \
	:

The argument for such a change, is that my above use case would then automatically track changes to this code block when I update the mambaorg/micromamba tag within my downstream Dockerfile. A reason for not making this change is that it hides details in other files which makes it hard for someone to glance at the Dockerfile and understand exactly what is being done.

The functionality in the above code block can be broken into 2 topics:

1. user account creation
2. setup of the directory $MAMBA_ROOT_PREFIX

Perhaps it makes sense to create a script for each of these two activities. This should give a reader of the mamba-org/micromamba-docker Dockerfile a better conceptual understanding of what is being done, even if the details require going into other files.

@maresb - care to offer an opinion on this possible change?

[maresb]

care to offer an opinion on this possible change?

I'm all in favor of multiple small scripts with long descriptive filenames, e.g. along the lines of _dockerfile_initialize_user_accounts.sh and _dockerfile_setup_root_prefix.sh. This seems to me like the best of both worlds: the Dockerfile remains concise and readable, and the interested reader doesn't need to go far to find the details if desired.

(In general I find the problem of "merging" Dockerfiles intriguing. It seems to be a common problem with no good solution. I would have suggested modifying tomcat to use micromamba as the base, until I noticed that tomcat uses eclipse as a base image, which is an annoying complication.)

[wholtz]

The docker developers seem to consider multi-stage builds to be their answer to merging Dockerfiles - moby/moby#24445. But it does seem better and more powerful solutions were possible.

[maresb]

Thanks for the reference! That sounds like an interesting proposal, but I imagine it could easily read to a nightmare of subtle bugs in terms of data consistency with what would effectively become asynchronous filesystem writes.

[wholtz]

The proposed additional shell script files have been implemented in 9a608bf. Supporting documentation has been added as well: https://github.com/mamba-org/micromamba-docker#adding-micromamba-to-an-existing-docker-image

[maresb]

Very nicely done, as usual! 😄