The mongodb bootstrap can not finished due to permission issue

[hchenxa]

We found that the mongodb can not started on some of environment due to permission issue like below:

Install CS3.3 on OCP4.2

icp-memcached-7cd994d94-mbh2f                             1/1     Running                      0          176m
icp-mongodb-0                                             0/2     Init:1/2                     0          12m
management-ingress-5949f9644b-jpm6m                       1/1     Running                      0          175m

[root@knolls-inf ~]# oc logs icp-mongodb-0 -c bootstrap
2020/03/16 06:48:54 Determined Domain to be ibm-common-services.svc.cluster.local
2020/03/16 06:48:54 Peer list updated
was []
now [icp-mongodb-0.icp-mongodb.ibm-common-services.svc.cluster.local]
2020/03/16 06:48:54 execing: /init/on-start.sh with stdin: icp-mongodb-0.icp-mongodb.ibm-common-services.svc.cluster.local

After investigated, stuck in this script /init/on-start.sh, hit the permission problem

Error opening CA Certificate /data/configdb/tls.crt
139970125055888:error:0200100D:system library:fopen:Permission denied:bss_file.c:402:fopen('/data/configdb/tls.crt','r')

bash-4.2$ ls -lrth
total 16K
-rw-r--r--. 1 root root  383 Mar 16 10:30 mongod.conf
-rw-------. 1 root root    7 Mar 16 10:30 key.txt
-r--------. 1 root root 1.7K Mar 16 10:30 tls.key
-r--------. 1 root root 1.2K Mar 16 10:30 tls.crt

After modified statefulset icp-mongodb to add runAsUser: 999, the pod can be startup.

@kgcarr I check the code and find that we comment the chown/chmod in install configmap and remove the securitycontext in statefulset compare with the helm chart we used before.

[kgcarr]

The securityContext is also missing a runAsNonRoot: true which is a security risk

[kgcarr]

securityContext:
    runAsUser: 999
    runAsNonRoot: true
    allowPrivilegeEscalation: false
    readOnlyRootFilesystem: true

for bootstrap, mongodb, and metrics containers

Also fixing the missed memory limits here...two were still set to 8GB but should be 5GB

[kgcarr]

create Pod icp-mongodb-0 in StatefulSet icp-mongodb failed error: pods "icp-mongodb-0" is forbidden: unable to validate against any security context constraint: [spec.initContainers[1].securityContext.securityContext.runAsUser: Invalid value: 999: must be in the ranges: [1000570000, 1000579999] spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 999: must be in the ranges: [1000570000, 1000579999] spec.containers[1].securityContext.securityContext.runAsUser: Invalid value: 999: must be in the ranges: [1000570000, 1000579999]]

ran into this issue during testing...for Openshift we'll use 1000571999

[kgcarr]

I cannot do what I used to do because I now run in openshift.io/scc: restricted

[kgcarr]

@hchenxa see my comment in #65