Local replay mapper basic implementation and tests (#227)

* First iteration, added basic implementation and tests

* Fixed lint

* Added docs and configuration

* Added possibility to define nested json format

* Allow to define custom content type

* Added content type to configuration

* Code review changes

docs/features/local_reply_mapper.md

@@ -0,0 +1,135 @@
+# Local reply modification configuration
+
+Envoy Control allows defining custom format for responses returned by Envoy. Thanks to [Envoy functionality](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_conn_man/local_reply) 
+it is possible to configure the custom format, status code for specific responses.
+
+## Define default response format for responses returned by Envoy
+
+It is possible to define a custom format for all responses returned by Envoy. Envoy can return response either
+in a text format or JSON format. It is possible to define only one of: `textFormat` and `jsonFormat`.
+If the format isn't specified, then default from Envoy is returned.
+
+### Configure text format response
+
+Property `envoy-control.envoy.snapshot.dynamic-listeners.local-reply-mapper.response-format.text-format` allows configuring text response format.
+Text format supports [command operators](https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#config-access-log-command-operators) that allows
+operating on request/response data.
+
+An example configuration: 
+
+```yaml
+envoy-control.envoy.snapshot.dynamic-listeners.local-reply-mapper.response-format.text-format: "my custom response with flags: %RESPONSE_FLAGS%"
+```
+
+Response:
+
+```text
+"my custom response with flags: UF"
+```
+
+### Configure JSON format response
+
+Property `envoy-control.envoy.snapshot.dynamic-listeners.local-reply-mapper.response-format.json-format` allows configuring JSON response format.
+It accepts a JSON formatted string with constant string and  [command operators](https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#config-access-log-command-operators).
+
+An example configuration:
+
+```yaml
+envoy-control:
+  envoy:
+    snapshot:
+      dynamic-listeners:
+        local-reply-mapper:
+          response-format:
+            json-format: """{
+              "myKey":"My custom body",
+              "responseFlags":"%RESPONSE_FLAGS%",
+              "host":"%REQ(:authority)%"
+            }"""
+```
+
+Response:
+
+```json
+{
+    "myKey" : "My custom body",
+    "responseFlags" : "UF",
+    "host" : "example-service"
+}
+```
+
+## Match specific response
+
+It is possible to define custom response or override status code only for specific responses. In that case you can use matchers
+which allows defining custom response status and response body for matched responses. Currently, 3 types of matchers are supported:
+- status code matcher
+- response flag matcher
+- header matcher
+
+You can choose only one of: `statusCodeMatcher`, `headerMatcher`, `responseFlagMatcher`. Also, configuration supports response body format override
+for matched responses.
+
+### Status code matcher
+
+Allows filtering only specific status codes: An expected format is: `{operator}`:`{status code}`.
+
+Allowed operators are:
+
+* `le` - lower equal
+* `eq` - equal
+* `ge` - greater equal
+
+Example:
+
+```yaml
+statusCodeMatcher: "EQ:400"
+```
+
+By default, it is an empty string which means that matcher is disabled.
+
+### Header matcher
+
+Allows filtering responses based on header presence or value. Only one of: `exactMatch`, `regexMatch` can be used. If none is used
+then presence matcher will match responses with specified header name.
+
+Example:
+
+```yaml
+  headerMatcher:
+    name: "host"
+    exactMatch: "service1"
+```
+
+By default, all fields are equals to empty string which means that matcher is disabled.
+
+### Response flags matcher
+
+Allows filtering responses based on response flags (refer to [Envoy docs](https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#config-access-log-format-response-flags)).
+
+Example:
+
+```yaml
+  responseFlagMatcher:
+      - "UF"
+      - "NR"
+```
+
+By default, it is set an empty list which means there is no filtering by response flags.
+
+### Status code override
+
+When response is matched and property `statusCodeToReturn` for this matcher is defined then Envoy will change response status code 
+to value of the property `statusCodeToReturn`. By default, it is set to 0 which means that status code won't be overridden.
+
+### Custom body
+
+When response is matched and property `bodyToReturn` for this matcher is defined then Envoy will set body to value of the property `bodyToReturn`.
+If you defined custom format then the value can be accessed by using placeholder `%LOCAL_REPLY_BODY%` (refer to [Envoy docs](https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#config-access-log-format-filter-state)).
+
+By default, it is an empty string which means that body won't be overridden.
+
+### Different response format for different matchers
+
+It is possible to configure different response formats for different matchers. If matcher configuration has `responseFormat` configuration then 
+it will be used instead of response format defined at `localReplyMapper` level. When there is no configuration, default Envoy's format will be returned.
+

docs/index.md

@@ -13,6 +13,7 @@ Data Plane that is platform agnostic.
 * [Weighted load balancing and canary support](features/load_balancing.md)
 * [Service tags routing support](features/service_tags.md)
 * [Access log filter](features/access_log_filter.md)
+* [Local reply modification](features/local_reply_mapper.md)
 
 ## Why another Control Plane?
 Our use case for Service Mesh is running 800 microservices on [Mesos](https://mesos.apache.org/) / [Marathon](https://mesosphere.github.io/marathon/) stack.

envoy-control-core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/ControlPlane.kt

@@ -30,7 +30,6 @@ import pl.allegro.tech.servicemesh.envoycontrol.snapshot.SnapshotsVersions
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.clusters.EnvoyClustersFactory
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.endpoints.EnvoyEndpointsFactory
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.EnvoyListenersFactory
-import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.filters.AccessLogFilterFactory
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.filters.EnvoyHttpFilters
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.routes.EnvoyEgressRoutesFactory
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.routes.EnvoyIngressRoutesFactory
@@ -95,11 +94,8 @@ class ControlPlane private constructor(
         var metrics: EnvoyControlMetrics = DefaultEnvoyControlMetrics(meterRegistry = meterRegistry)
         var envoyHttpFilters: EnvoyHttpFilters = EnvoyHttpFilters.emptyFilters
 
-        val accessLogFilterFactory = AccessLogFilterFactory()
-
         var nodeGroup: NodeGroup<Group> = MetadataNodeGroup(
-            properties = properties.envoy.snapshot,
-            accessLogFilterFactory = accessLogFilterFactory
+            properties = properties.envoy.snapshot
         )
 
         fun build(changes: Flux<MultiClusterState>): ControlPlane {

envoy-control-core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/groups/MetadataNodeGroup.kt

@@ -5,13 +5,11 @@ import com.google.protobuf.Value
 import io.envoyproxy.controlplane.cache.NodeGroup
 import pl.allegro.tech.servicemesh.envoycontrol.logger
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.SnapshotProperties
-import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.filters.AccessLogFilterFactory
 import io.envoyproxy.envoy.api.v2.core.Node as NodeV2
 import io.envoyproxy.envoy.config.core.v3.Node as NodeV3
 
 class MetadataNodeGroup(
-    val properties: SnapshotProperties,
-    val accessLogFilterFactory: AccessLogFilterFactory
+    val properties: SnapshotProperties
 ) : NodeGroup<Group> {
     private val logger by logger()
 
@@ -84,7 +82,7 @@ class MetadataNodeGroup(
         val egressHostValue = metadata.fieldsMap["egress_host"]
         val egressPortValue = metadata.fieldsMap["egress_port"]
         val accessLogFilterSettings = AccessLogFilterSettings(
-            metadata.fieldsMap["access_log_filter"], accessLogFilterFactory
+            metadata.fieldsMap["access_log_filter"]
         )
 
         val listenersHostPort = metadataToListenersHostPort(

envoy-control-core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/groups/NodeMetadata.kt

@@ -5,10 +5,10 @@ import com.google.protobuf.Struct
 import com.google.protobuf.Value
 import com.google.protobuf.util.Durations
 import io.envoyproxy.controlplane.server.exception.RequestException
-import io.envoyproxy.envoy.config.accesslog.v3.ComparisonFilter
 import io.grpc.Status
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.SnapshotProperties
-import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.filters.AccessLogFilterFactory
+import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.util.StatusCodeFilterParser
+import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.util.StatusCodeFilterSettings
 import java.net.URL
 import java.text.ParseException
 
@@ -25,17 +25,9 @@ class NodeMetadata(metadata: Struct, properties: SnapshotProperties) {
     val proxySettings: ProxySettings = ProxySettings(metadata.fieldsMap["proxy_settings"], properties)
 }
 
-data class AccessLogFilterSettings(
-    val statusCodeFilterSettings: StatusCodeFilterSettings?
-) {
-    constructor(proto: Value?, accessLogFilterFactory: AccessLogFilterFactory) : this(
-        statusCodeFilterSettings = proto?.field("status_code_filter").toStatusCodeFilter(accessLogFilterFactory)
-    )
-
-    data class StatusCodeFilterSettings(
-        val comparisonOperator: ComparisonFilter.Op,
-        val comparisonCode: Int
-    )
+data class AccessLogFilterSettings(val proto: Value?) {
+    val statusCodeFilterSettings: StatusCodeFilterSettings? = proto?.field("status_code_filter")
+        .toStatusCodeFilter()
 }
 
 data class ProxySettings(
@@ -67,10 +59,9 @@ private fun getCommunicationMode(proto: Value?): CommunicationMode {
     }
 }
 
-fun Value?.toStatusCodeFilter(accessLogFilterFactory: AccessLogFilterFactory):
-    AccessLogFilterSettings.StatusCodeFilterSettings? {
+fun Value?.toStatusCodeFilter(): StatusCodeFilterSettings? {
     return this?.stringValue?.let {
-        accessLogFilterFactory.parseStatusCodeFilter(it.toUpperCase())
+        StatusCodeFilterParser.parseStatusCodeFilter(it.toUpperCase())
     }
 }
 

envoy-control-core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/snapshot/SnapshotProperties.kt

@@ -36,6 +36,7 @@ class MetricsProperties {
 class ListenersFactoryProperties {
     var enabled = true
     var httpFilters = HttpFiltersProperties()
+    var localReplyMapper = LocalReplyMapperProperties()
 }
 
 class HttpFiltersProperties {
@@ -86,6 +87,7 @@ class TlsAuthenticationProperties {
     var servicesAllowedToUseWildcard: MutableSet<String> = mutableSetOf()
     var tlsContextMetadataMatchKey = "acceptMTLS"
     var protocol = TlsProtocolProperties()
+
     /** if true, a request without a cert will be rejected during handshake and will not reach RBAC filter */
     var requireClientCertificate: Boolean = false
     var validationContextSecretName: String = "validation_context"
@@ -266,3 +268,30 @@ class HostHeaderRewritingProperties {
     var enabled = false
     var customHostHeader = "x-envoy-original-host"
 }
+
+class LocalReplyMapperProperties {
+    var enabled = false
+    var responseFormat = ResponseFormat()
+    var matchers = emptyList<MatcherAndMapper>()
+}
+
+class MatcherAndMapper {
+    var statusCodeMatcher = ""
+    var headerMatcher = HeaderMatcher()
+    var responseFlagMatcher = emptyList<String>()
+    var responseFormat = ResponseFormat()
+    var statusCodeToReturn = 0
+    var bodyToReturn = ""
+}
+
+class HeaderMatcher {
+    var name = ""
+    var exactMatch: String = ""
+    var regexMatch: String = ""
+}
+
+class ResponseFormat {
+    var textFormat = ""
+    var jsonFormat = ""
+    var contentType = ""
+}

envoy-control-core/src/main/kotlin/pl/allegro/tech/servicemesh/envoycontrol/snapshot/resource/listeners/EnvoyListenersFactory.kt

@@ -42,7 +42,9 @@ import pl.allegro.tech.servicemesh.envoycontrol.groups.ListenersConfig
 import pl.allegro.tech.servicemesh.envoycontrol.groups.ResourceVersion
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.GlobalSnapshot
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.SnapshotProperties
+import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.config.LocalReplyConfigFactory
 import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.filters.EnvoyHttpFilters
+import pl.allegro.tech.servicemesh.envoycontrol.snapshot.resource.listeners.util.StatusCodeFilterSettings
 import com.google.protobuf.Any as ProtobufAny
 
 typealias HttpFilterFactory = (node: Group, snapshot: GlobalSnapshot) -> HttpFilter?
@@ -62,6 +64,9 @@ class EnvoyListenersFactory(
     private val accessLogLogger = stringValue(listenersFactoryProperties.httpFilters.accessLog.logger)
     private val egressRdsInitialFetchTimeout: Duration = durationInSeconds(20)
     private val ingressRdsInitialFetchTimeout: Duration = durationInSeconds(30)
+    private val localReplyConfig = LocalReplyConfigFactory(
+        snapshotProperties.dynamicListeners.localReplyMapper
+    ).configuration
 
     private val tlsProperties = snapshotProperties.incomingPermissions.tlsAuthentication
     private val requireClientCertificate = BoolValue.of(tlsProperties.requireClientCertificate)
@@ -251,6 +256,10 @@ class EnvoyListenersFactory(
             )
         }
 
+        if (listenersFactoryProperties.localReplyMapper.enabled) {
+            connectionManagerBuilder.localReplyConfig = localReplyConfig
+        }
+
         return Filter.newBuilder()
                 .setName("envoy.filters.network.http_connection_manager")
                 .setTypedConfig(ProtobufAny.pack(
@@ -351,7 +360,7 @@ class EnvoyListenersFactory(
         }
     }
 
-    fun AccessLog.Builder.buildFromSettings(settings: AccessLogFilterSettings.StatusCodeFilterSettings) {
+    fun AccessLog.Builder.buildFromSettings(settings: StatusCodeFilterSettings) {
         this.setFilter(
                 AccessLogFilter.newBuilder().setStatusCodeFilter(
                         StatusCodeFilter.newBuilder()