From cc98d989982cc40e14354fcdd0849e429c46a1a7 Mon Sep 17 00:00:00 2001 From: William Morgan Date: Sun, 29 Sep 2024 08:41:50 -0500 Subject: [PATCH] update protocol detection doc - Add NATS to the list of protocols that need opaque ports (not sure how this was dropped) - Instruct that the resource should always be set on the Service object. Signed-off-by: William Morgan --- .../2.16/features/protocol-detection.md | 65 +++++++++---------- 1 file changed, 32 insertions(+), 33 deletions(-) diff --git a/linkerd.io/content/2.16/features/protocol-detection.md b/linkerd.io/content/2.16/features/protocol-detection.md index 079f7d229d..78d9f25582 100644 --- a/linkerd.io/content/2.16/features/protocol-detection.md +++ b/linkerd.io/content/2.16/features/protocol-detection.md @@ -20,9 +20,9 @@ If Linkerd *cannot* determine that a connection is using HTTP or HTTP/2, Linkerd will proxy the connection as a plain TCP connection, applying [mTLS](../automatic-mtls/) and providing byte-level metrics as usual. -(Note that HTTPS calls to or from meshed pods are treated as TCP, not as HTTP. -Because the client initiates the TLS connection, Linkerd is not be able to -decrypt the connection to observe the HTTP transactions.) +(Note that HTTPS calls pods are treated as TCP, not as HTTP. Because the client +initiates the TLS connection, Linkerd is not be able to decrypt the connection +to observe the HTTP traffic.) ## Configuring protocol detection @@ -34,11 +34,13 @@ This section will help you understand how to fix this. {{< /note >}} In some cases, Linkerd's protocol detection will time out because it doesn't see -any bytes from the client. This situation is commonly encountered when using -protocols where the server sends data before the client does (such as SMTP) or -protocols that proactively establish connections without sending data (such as -Memcache). In this case, the connection will proceed as a TCP connection after a -10-second protocol detection delay. +any bytes from the client within the first 10 seconds of connection +establishment. This is commonly encountered when using protocols where the +server sends data before the client does (such as SMTP) or protocols that +proactively establish connections without sending data (such as Memcache). In +this case, Linkerd will wait for 10 seconds before establishing a connection to +the server. After this delay, the connection will be esablished and treated by +Linkerd as a TCP connection. To avoid this delay, you will need to provide some configuration for Linkerd. There are two basic mechanisms for configuring protocol detection: _opaque @@ -49,29 +51,28 @@ ports_ and _skip ports_: * Skip ports bypass the proxy entirely. Opaque ports are generally preferred as they allow Linkerd to provide mTLS, -TCP-level metrics, policy, etc. Skip ports circumvent Linkerd's ability to -provide security features. +TCP-level metrics, policy, etc. Skip ports, by contrast, create networking rules +that avoid the proxy entirely, circumventing Linkerd's ability to provide +security features. Linkerd maintains a default list of opaque ports that corresponds to the standard ports used by protocols that interact poorly with protocol detection. -As of the 2.12 release, that list is: **25** (SMTP), **587** (SMTP), **3306** -(MySQL), **4444** (Galera), **5432** (Postgres), **6379** (Redis), **9300** -(ElasticSearch), and **11211** (Memcache). ## Protocols that may require configuration The following table contains common protocols that may require additional configuration. -| Protocol | Standard port(s) | In default list? | Notes | +| Protocol | Standard ports | In default list? | Notes | |-----------------|------------------|------------------|-------| | SMTP | 25, 587 | Yes | | | MySQL | 3306 | Yes | | -| MySQL with Galera | 3306, 4444, 4567, 4568 | Partially | Ports 4567 and 4568 are not in Linkerd's default set of opaque ports | +| MySQL with Galera | 3306, 4444, 4567, 4568 | Partially | Ports 4567 and 4568 are not in Linkerd's default list of opaque ports | | PostgreSQL | 5432 | Yes | | | Redis | 6379 | Yes | | | ElasticSearch | 9300 | Yes | | | Memcache | 11211 | Yes | | +| NATS | 4222, 6222, 8222 | No | | If you are using one of those protocols, follow this decision tree to determine which configuration you need to apply. @@ -81,27 +82,31 @@ which configuration you need to apply. ## Marking ports as opaque You can use the `config.linkerd.io/opaque-ports` annotation to mark a port as -opaque. Note that this annotation should be set on the _destination_, not on the -source, of the traffic. +opaque. Linkerd will skip protocol detection on opaque ports, and treat +connections to them as TCP streams. -This annotation can be set in a variety of ways: +This annotation should be set on the _destination_, not on the source, of the +traffic. This is true even if the destination is unmeshed, as it controls the +behavior of meshed clients. -1. On the workload itself, e.g. on the Deployment's Pod spec receiving the traffic. +This annotation *must* be set in two places: 1. On the Service receiving the traffic. -1. On a namespace (in which it will apply to all workloads in the namespace). -1. In an [authorization policy](../server-policy/) `Server` object's - `proxyProtocol` field, in which case it will apply to all pods targeted by that - `Server`. - -When set, Linkerd will skip protocol detection both on the client side and on -the server side. Note that since this annotation informs the behavior of meshed -_clients_, it can be applied to unmeshed workloads as well as meshed ones. +2. On the workload itself (e.g. on the Deployment's Pod spec receiving the +traffic), or on enclosing namespace, in which it will apply to all workloads in +the namespace. {{< note >}} Multiple ports can be provided as a comma-delimited string. The values you provide will _replace_, not augment, the default list of opaque ports. {{< /note >}} +{{< note >}} +If you are using [authorization policies](../server-policy/), the `Server`'s +`proxyProtocol` field which can be used to control protocol detection behavior +and can be used instead of a Service annotation. Regardless, we suggest +annotating the Service object for clarity. +{{< /note >}} + ## Marking ports as skip ports Sometimes it is necessary to bypass the proxy altogether. In this case, you can @@ -128,12 +133,6 @@ Note that the default set of opaque ports can be configured at install time, e.g. by using `--set proxy.opaquePorts`. This may be helpful in conjunction with `enable-external-profiles`. -{{< note >}} -There was a bug in Linkerd 2.11.0 and 2.11.1 that prevented the opaque ports -behavior of `enable-external-profiles` from working. This was fixed in Linkerd -2.11.2. -{{< /note >}} - ## Using `NetworkPolicy` resources with opaque ports When a service has a port marked as opaque, any `NetworkPolicy` resources that