Specifying a nonexistent plugin name via konghq.com/plugins annotation is silently ignored, configuration is applied with that plugin

[pmalek]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When users use a custom plugin in konghq.com/plugins annotation e.g. like so:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: route
  annotations:
    konghq.com/plugins: key-auth-2 # doesn't exist
spec:
  parentRefs:
  - name: kong
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /route
    backendRefs:
    - name: echo
      port: 80

then KIC emits configuration without this plugin, emits KongConfigurationSucceeded event, and the following error log:

error   Failed to fetch KongPlugin resource     {"kongplugin_name": "key-auth-2", "kongplugin_namespace": "default", "error": "no KongPlugin or KongClusterPlugin was found for default/key-auth-2"}

Expected Behavior

Expected behavior yet to be decided but I'd expect that

• KIC emits KongConfigurationTranslationFailed event
• when using fallback configuration without backfilling, the object annotated with the non-existing plugin gets removed from the applied configuration

Steps To Reproduce

1. Apply the following manifest and observe `HTTPRoute` being applied without the plugin configured (no key-auth configured)


apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-1
  labels:
    app: echo-1
spec:
  selector:
    matchLabels:
      app: echo-1
  template:
    metadata:
      labels:
        app: echo-1
    spec:
      containers:
      - name: echo-1
        image: kong/go-echo:0.3.0
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
        ports:
        - containerPort: 80
        resources:
          limits:
            memory: "64Mi"
            cpu: "250m"
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: echo-1
  name: echo-1
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 1027
  selector:
    app: echo-1
  type: ClusterIP
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: kong
  annotations:
    konghq.com/gatewayclass-unmanaged: "true"
spec:
  controllerName: konghq.com/kic-gateway-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong
spec:
  gatewayClassName: kong
  listeners:
  - name: http
    protocol: HTTP
    port: 80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: route-b
  annotations:
    konghq.com/plugins: key-auth-2
spec:
  parentRefs:
  - name: kong
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /route-b
    backendRefs:
    - name: echo-1
      port: 80

### Kong Ingress Controller version

```shell
8a88685739440391cc6c47830f2c7b73a8e58d5e

Kubernetes version

No response

Anything else?

Relevant piece of code that skips creation of failure events (through failuresCollector) :

kubernetes-ingress-controller/internal/dataplane/kongstate/kongstate.go

Lines 490 to 495 in e7b3048

	if err != nil {
	logger.Error(err, "Failed to fetch KongPlugin resource",
	"kongplugin_name", kongPluginName,
	"kongplugin_namespace", namespace)
	continue
	}

[czeslavo]

That is something that existed before introducing FallbackConfiguration. I'm not sure what the desired behavior should be here, but IMO emitting KongConfigurationTranslationFailed should be good enough in any case (fallback enabled or not) as that's similar to other translation errors (e.g. ignoring CACerts with invalid secret content).

when using fallback configuration without backfilling, the object annotated with the non-existing plugin gets removed from the applied configuration

If we'd like to make this happen, that would require another refactor that would integrate translation failures with the fallback somehow. Definitely not something easy to implement.