Running as a non root user

[constanca-m]

Issue: #27

This adds a non root user to the Dockerfile that by default assumes the root user.

You can test it works locally like this:

Use this K8s manifest.

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gubernator
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: get-endpoints
rules:
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: get-endpoints
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: get-endpoints
subjects:
  - kind: ServiceAccount
    name: gubernator
---
apiVersion: v1
kind: Pod
metadata:
  name: gubernator
  labels:
    app: gubernator
spec:
  securityContext:
    runAsNonRoot: true
  serviceAccountName: gubernator
  containers:
    - image: custom-image:latest
      imagePullPolicy: IfNotPresent
      ports:
        - name: grpc-port
          containerPort: 1081
        - name: http-port
          containerPort: 1080
      name: gubernator
      env:
      - name: GUBER_K8S_NAMESPACE
        valueFrom:
          fieldRef:
            fieldPath: metadata.namespace
      - name: GUBER_K8S_POD_IP
        valueFrom:
          fieldRef:
            fieldPath: status.podIP
      # Must set the GRPC and HTTP addresses, as gubernator
      # defaults to listening on localhost only
      - name: GUBER_GRPC_ADDRESS
        value: 0.0.0.0:1081
      - name: GUBER_HTTP_ADDRESS
        value: 0.0.0.0:1080
      # Use the k8s API for peer discovery
      - name: GUBER_PEER_DISCOVERY_TYPE
        value: "k8s"
      # This should match the port number GRPC is listening on
      # as defined by `containerPort`
      - name: GUBER_K8S_POD_PORT
        value: "1081"
      # The selector used when listing endpoints. This selector
      # should only select gubernator peers.
      - name: GUBER_K8S_ENDPOINTS_SELECTOR
        value: "app=gubernator"
      # Gubernator can watch 'endpoints' for changes to the peers
      # or it can watch 'pods' (Defaults to 'endpoints')
      # - name: GUBER_K8S_WATCH_MECHANISM
      #  value: "endpoints"
      # Enable debug for diagnosing issues
      - name: GUBER_DEBUG
        value: "true"
      # Defines the max age of a client connection
      # Default is infinity
      # - name: GUBER_GRPC_MAX_CONN_AGE_SEC
      #  value: "30"

  restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: gubernator
  labels:
    app: gubernator
spec:
  type: LoadBalancer
  ports:
  - targetPort: 1080
    port: 1080
  selector:
    app: gubernator

Use this script.

# delete local cluster if exists
kind delete cluster

# expose load balancer
cloud-provider-kind > cloud-provider-kind.log 2>&1 &
kind create cluster

docker build -t custom-image .
kind load docker-image custom-image:latest
# ensure gubernator pod is using image: custom-image:latest
kubectl apply -f k8s-deployment.yaml

# wait until gubernator is ready
kubectl wait --for=jsonpath='{.status.loadBalancer.ingress}' service/gubernator
kubectl wait --for=condition=Ready pod/gubernator --timeout=5m

# check the pod and service
kubectl get all

# make a request
IP=$(kubectl get service/gubernator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
ENDPOINT="http://$IP:1080/v1/GetRateLimits"
echo "ENDPOINT IS $ENDPOINT"
curl $ENDPOINT \
  --header 'Content-Type: application/json' \
  --data '{
    "requests": [
        {
            "name": "requests_per_sec",
            "uniqueKey": "account:12345",
            "hits": "1",
            "limit": "10",
            "duration": "1000"
        }
    ]
}'

The curl should work and output:

{"responses":[{"status":"UNDER_LIMIT","limit":"10","remaining":"9","reset_time":"1727876656143","error":"","metadata":{}}]}

[constanca-m]

This change does not work with the default ports:

time="2024-10-02T13:49:03Z" level=error msg="while spawning daemon: while starting GRPC listener: listen tcp 0.0.0.0:81: bind: permission denied" category=gubernator

[thrawn01]

Hi @constanca-m You should be able to set the environment variables GUBER_HTTP_ADDRESS and GUBER_GRPC_ADDRESS to change the listen address ports before starting the container. See https://github.com/gubernator-io/gubernator/blob/master/example.conf#L5-L9

Your change SHOULD work if we change the listen addresses, can you explain why it doesn't work?

[constanca-m]

Hi @thrawn01,

Your change SHOULD work if we change the listen addresses

This is correct. I attached to the description an example that works.

can you explain why it doesn't work?

It would only not work when not setting GUBER_HTTP_ADDRESS and GUBER_GRPC_ADDRESS, since the defaults use the ports 80 and 81. In Linux systems, these are privileged ports (all ports below 1024 are), and it would require a root user to be running the gubernator.

So if this PR was merged - we set a non root user for gubernator with it - , the defaults settings for users would no longer work and we would be seeing the error in #28 (comment).

So the workaround would be:

• Change the default ports. Maybe not ideal if you have users relying on them.
• Find a way to give this non root user capabilities to bind to those ports, but no more than that.

Edit: I found this very nice article that explains the problem well. I have tried with the default ports 80 and 81 (I am using Linux, cluster Container Runtime Version: containerd://1.7.15):

1. Adding to the K8s deployment:

  securityContext:
    runAsNonRoot: true
    sysctls:
      - name: net.ipv4.ping_group_range
        value: "0 2147483647"

It still did not work: while starting GRPC listener: listen tcp 0.0.0.0:81: bind: permission denied.

2. Adding the capability:

      securityContext:
        runAsNonRoot: true
        capabilities:
          add:
            - NET_BIND_SERVICE

It also did not work: while starting GRPC listener: listen tcp 0.0.0.0:81: bind: permission denied.

[constanca-m]

I got it to work by adding:

  securityContext:
    runAsNonRoot: true
    sysctls:
      - name: net.ipv4.ip_unprivileged_port_start
        value: "80"

I found this article Trouble Binding Ports After Switching to Containerd? that explained the problem better for containerd and contains a bit of history on privilege ports and docker.

[thrawn01]

Thank you for explaining! Yes, I agree we should change the default which will allow Gubernator to run in a non privileged container. I'm thinking we should change it to 1051 which doesn't appear to be used by anything important. AND It's a reference to the Rick and Morty snakes episode.

Do you want to make this change, or should I?

[constanca-m]

Do you want to make this change, or should I?

I have tried to update the defaults in all files. I think this workaround is best in the long term. Could you double check the changes?

[thrawn01]

Looks good, thanks for making this change!

[thrawn01]

https://github.com/gubernator-io/gubernator/releases/tag/v2.8.0