Include the certificate management tool in docker distribution

[setiah]

Is your feature request related to a problem? Please describe.
Today, the OpenSearch docker distribution comes pre-installed with demo certificates. The intention is to replace the insecure demo certificates with self-signed certificates as discussed in opensearch-project/OpenSearch#1618 (point 2).

Describe the solution you'd like
Package the certificate management tool (provided in opensearch-project/OpenSearch#1633) in the docker distribution. While instantiating the container, check if the certificates are present in the respective mounted volume that holds the config. If missing, it should run the certificate management tool to create self-signed certificates with default arguments before bootstrapping OpenSearch. If present, skips the certificate generation and rely on user provided certificates and yml configurations.

Implementation task details

• Provide openSSL inside docker image
• replace install_demo_configurations.sh with cert-management-tool from Provide a certificate management tool for opensearch distributions OpenSearch#1633.
• Add one time certificate installation logic that installs certs when a new container starts. Subsequent starts/stops should not modify installed certs.
• Provide a way to control self-signed certs generation based on user input.
• Make changes to docker-compose.yml based on new dockerfile

Related to opensearch-project/OpenSearch#1618

[setiah]

Adding logs for OpenSearch docker run today. the demo certificates are by default enabled as seen.

docker run -it -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchproject/opensearch:1.2.0
Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin 
OpenSearch Security Demo Installer
 ** Warning: Do not use on production or public reachable systems **
Basedir: /usr/share/opensearch
OpenSearch install type: rpm/deb on NAME="Amazon Linux"
OpenSearch config dir: /usr/share/opensearch/config
OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
OpenSearch bin dir: /usr/share/opensearch/bin
OpenSearch plugins dir: /usr/share/opensearch/plugins
OpenSearch lib dir: /usr/share/opensearch/lib
Detected OpenSearch Version: x-content-1.2.0
Detected OpenSearch Security Version: 1.2.0.0

[setiah]

Added a new volume opensearch-config to docker-compose that holds the certificates and config. This ensures the certificates are only setup once - with first time docker-compose up - and re-used for later runs.

Logs that show how subsequent docker-compose up runs check for certificates and bypass certs generation if they exist.

➜  docker-compose up
Docker Compose is now in the Docker CLI, try `docker compose up`

Creating opensearch-dashboardsX ... done
Creating opensearch-node10      ... done
Creating opensearch-node20      ... done
Attaching to opensearch-node20, opensearch-dashboardsX, opensearch-node10
opensearch-node10         |
opensearch-node10         | Checking if certificates exist.
opensearch-node10         | Enabling OpenSearch Security Plugin
opensearch-node20         |
opensearch-node20         | Checking if certificates exist.
opensearch-node20         | Enabling OpenSearch Security Plugin
opensearch-node20         | [2022-02-02T10:35:52,446][INFO ][o.o.n.Node               ] [opensearch-node20] version[1.2.0], pid[13], build[tar/c459282fd67ddb17dcc545ec9bcdc805880bcbec/2021-11-22T16:57:18.360386Z], OS[Linux/5.10.25-linuxkit/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
opensearch-node20         | [2022-02-02T10:35:52,450][INFO ][o.o.n.Node               ] [opensearch-node20] JVM home [/usr/share/opensearch/jdk], using bundled JDK [true]
opensearch-node20         | [2022-02-02T10:35:52,454][INFO ][o.o.n.Node               ] [opensearch-node20] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-674237451682325214, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dopensearch.cgroups.hierarchy.override=/, -Xms512m, -Xmx512m, -XX:MaxDirectMemorySize=268435456, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]

[bbarani]

@setiah Is this issue still valid? Can we close it for now?

[bbarani]

@setiah @davidlago @peternied Is this issue still valid? Can we close it for now?

[peternied]

The docker distribution should not be treated differently from other distributions in the certificate space - lets close out this issue as not planned.