NMS-15822: Add support for etc configmaps that get copied to the overlay

[deejgregor]

This is a PoC and is likely not yet complete, except for a basic, initial solution:

• We want to think about what to do for content outside of etc -- do we add separate configmaps for each overlay? Use the global overlay?
• Haven't tested binary content yet
• Docs changes
• Should test with > 10 configmaps to make sure the ordering is correct
• Nothing is done with uncompressing content yet (or should that be a different PR? Or go into the entrypoint of the dockerfile)
• Is the ".uncompress" file the right way to trigger the uncompress logic? Should it always be on?
• Make sure we get ownership/permissions right for OpenShift in particular (it's fine for etc, but will need work in the container for other directories)
• Any scripting/Helm charts to help make configmaps for multiple directories--not needed with ZIP support
• Make sure the characters in the files we use are allowed by the configmap key regex -- nah, just toss 'em in an archive
• Should we support extracting archives? (zip, tar.gz) -- YES
• Should we do as much of this in the Docker container instead of here? -- NO, not now
• Add support for ZIP archives (and remove support for .gz--we'll only support one compression/archive format for simplicity and documentation)

future possibilities

• Support arbitrary user-specific init containers.
• Make the etc pvc optional (maybe even disabled by default eventually?)
• Git workflow for editing etc files, merging changes between versions, etc..
• We could possibly add support for files that can be updated while Horizon is running.
• Support a zip or tar.gz of files

To test

# set your namespace first
instance=whatever
configmap=foobaz

mkdir etc
date > etc/testing-configmap
kubectl create configmap -n $instance $configmap --from-file=etc
kubectl get configmap -n $instance $configmap -o yaml
helm upgrade --reuse-values $instance . --set "core.overlayConfigMaps[0].name=$configmap" --set 'core.overlayConfigMaps[0].path=etc'

Output from the init container:

Processing overlay config maps ...
  Copying files from /opennms-overlay-configmaps/0 to /opt/opennms-overlay/ ...
    etc/testing-configmap c0f178d439e947ba9d62abb519fe56af

Notes

If the pod doesn't start, check events -- maybe the configmap name is wrong and can't be mounted:

kubectl events -n $instance

Remove the configmap before creating a new one:

kubectl delete configmap -n $instance $configmap

When you update the configmap, the pod will not automatically restart. You'll have to do that:

kubectl rollout restart -n $instance statefulset/onms-core

Testing

Binary content

$ dd if=/dev/urandom of=etc/random-file bs=1k count=1
1+0 records in
1+0 records out
1024 bytes transferred in 0.000158 secs (6481013 bytes/sec)
$ md5sum etc/random-file
ad428160a7518cd5893bd2a6c3b09938  etc/random-file

Updating the configmap and restarting the pod:

kubectl delete configmap -n $instance $configmap
kubectl create configmap -n $instance $configmap --from-file=etc
kubectl get configmap -n $instance $configmap -o yaml
kubectl rollout restart -n $instance statefulset/onms-core

Inside the container after updating the configmap and restarting the pod:

opennms@onms-core-0:~$ md5sum etc/random-file
ad428160a7518cd5893bd2a6c3b09938  etc/random-file

configmap with .zip file

Make a big file of zeros and add to a .zip file

mkdir -p zip/etc
dd if=/dev/zero bs=1k count=5000 of=zip/etc/lots-of-zeros
md5sum zip/etc/lots-of-zeros
# output is: 56179b2937a6e956250ab491685a4efb  zip/etc/lots-of-zeros
( cd zip && zip -r -o ../lots-of-zeros.zip . )

Update the configmap and update the helm chart to unzip the configmap:

kubectl delete configmap -n $instance $configmap
kubectl create configmap -n $instance $configmap --from-file=lots-of-zeros.zip
kubectl get configmap -n $instance $configmap -o yaml
helm upgrade --reuse-values $instance . --set "core.overlayConfigMaps[0].name=$configmap" --set 'core.overlayConfigMaps[0].path=null' --set 'core.overlayConfigMaps[0].unzip=true'

Inside the container after updating the configmap and restarting the pod:

opennms@onms-core-0:~$ md5sum etc/lots-of-zeros
56179b2937a6e956250ab491685a4efb  etc/lots-of-zeros

Output from the init container:

Processing overlay config maps ...
  Extracting files from /opennms-overlay-configmaps/0-unzip/lots-of-zeros.zip to /opt/opennms-overlay/ ...
    Archive:  /opennms-overlay-configmaps/0-unzip/lots-of-zeros.zip
      inflating: /opt/opennms-overlay/etc/lots-of-zeros