maxConcurrency with Websocket connections

[sjaakiejj]

Hi all,

We've got a system where we run a fairly CPU-heavy process as part of incoming websocket requests. To ensure the system both protects itself from running out of resources and can handle sudden high loads, we're looking into Knative Serving as a potential solution, given that the containerConcurrency property seems to do exactly what we need.

Effectively we're looking to limit each container to 10 active websocket connections and force the auto scaler to add new instances if we need more. We've setup our deployment yaml as follows:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: socket-service
  namespace: sockets

spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/metric: "concurrency"
        autoscaling.knative.dev/target-utilization-percentage: "80"
        autoscaling.knative.dev/min-scale: '1'
        autoscaling.knative.dev/max-scale: '10'
        autoscaling.knative.dev/target: '1'

      labels:
        app: socket-service
    spec:
      containerConcurrency: 10
      timeoutSeconds: 600
      containers:
      - name: socket-service
        image: my-socket-image:latest
        imagePullPolicy: Always

        ports:
        - containerPort: 8080

However, when applying a load of 40 active socket connections to this service, all of them go to the same pod. I checked netstat to get the number of active connections in the pod, and it returned 210 ESTABLISHED records on port 8080.

Displaying the list of pods, I get this:

NAME                                                  READY   STATUS    RESTARTS   AGE
pod/socket-service-00001-deployment-cf64c4bd5-npdcj   2/2     Running   0          9h

It seems no matter how much load I throw at it, it refuses to scale. There are no errors in the autoscaler logs but also no indications that it attempts to scale. I feel like I'm missing something obvious, but I can't figure it out based on the documentation.

Is this expected to work the way I think it should? What's the best way to debug issues like this?