otlp: add docs for per tenant otlp config (#11849)

**What this PR does / why we need it**:
In PR #11143 we added support for per tenant otlp config. This PR adds
the relevant documentation to explain how the config looks and works.

**Checklist**
- [x] Documentation added

---------

Co-authored-by: J Stickler <[email protected]>

docs/sources/configure/_index.md

@@ -3143,14 +3143,22 @@ shard_streams:
 
 # OTLP log ingestion configurations
 otlp_config:
+  # Configuration for resource attributes to store them as index labels or
+  # Structured Metadata or drop them altogether
   resource_attributes:
-    [ignore_defaults: <boolean>]
+    # Configure whether to ignore the default list of resource attributes to be
+    # stored as index labels and only use the given resource attributes config
+    [ignore_defaults: <boolean> | default = false]
 
-    [attributes: <list of AttributesConfigs>]
+    [attributes_config: <list of attributes_configs>]
 
-  [scope_attributes: <list of AttributesConfigs>]
+  # Configuration for scope attributes to store them as Structured Metadata or
+  # drop them altogether
+  [scope_attributes: <list of attributes_configs>]
 
-  [log_attributes: <list of AttributesConfigs>]
+  # Configuration for log attributes to store them as Structured Metadata or
+  # drop them altogether
+  [log_attributes: <list of attributes_configs>]
 ```
 
 ### frontend_worker
@@ -5292,6 +5300,24 @@ Named store from this example can be used by setting object_store to store-1 in
 [cos: <map of string to cos_storage_config>]
 ```
 
+### attributes_config
+
+Define actions for matching OpenTelemetry (OTEL) attributes.
+
+```yaml
+# Configures action to take on matching attributes. It allows one of
+# [structured_metadata, drop] for all attribute types. It additionally allows
+# index_label action for resource attributes
+[action: <string> | default = ""]
+
+# List of attributes to configure how to store them or drop them altogether
+[attributes: <list of strings>]
+
+# Regex to choose attributes to configure how to store them or drop them
+# altogether
+[regex: <Regexp>]
+```
+
 ## Runtime Configuration file
 
 Loki has a concept of "runtime config" file, which is simply a file that is reloaded while Loki is running. It is used by some Loki components to allow operator to change some aspects of Loki configuration without restarting it. File is specified by using `-runtime-config.file=<filename>` flag and reload period (which defaults to 10 seconds) can be changed by `-runtime-config.reload-period=<duration>` flag. Previously this mechanism was only used by limits overrides, and flags were called `-limits.per-user-override-config=<filename>` and `-limits.per-user-override-period=10s` respectively. These are still used, if `-runtime-config.file=<filename>` is not specified.

docs/sources/send-data/otel/_index.md

@@ -69,7 +69,7 @@ service:
 
 ## Format considerations
 
-Since the OpenTelemetry protocol  differs from the Loki storage model, here is how data in the OpenTelemetry format will be mapped to the Loki data model during ingestion:
+Since the OpenTelemetry protocol differs from the Loki storage model, here is how data in the OpenTelemetry format will be mapped by default to the Loki data model during ingestion, which can be changed as explained later:
 
 - Index labels: Resource attributes map well to index labels in Loki, since both usually identify the source of the logs. Because Loki has a limit of 30 index labels, we have selected the following resource attributes to be stored as index labels, while the remaining attributes are stored as [Structured Metadata]({{< relref "../../get-started/labels/structured-metadata" >}}) with each log entry:
   - cloud.availability_zone
@@ -116,3 +116,102 @@ Things to note before ingesting OpenTelemetry logs to Loki:
 - Stringification of non-string Attribute values
 
   While converting Attribute values in OTLP to Index label values or Structured Metadata, any non-string values are converted to string using [AsString method from the OTEL collector lib](https://github.com/open-telemetry/opentelemetry-collector/blob/ab3d6c5b64701e690aaa340b0a63f443ff22c1f0/pdata/pcommon/value.go#L353).
+
+### Changing the default mapping of OTLP to Loki Format
+
+Loki supports [per tenant]({{< relref "../../configure#limits_config" >}}) OTLP config which lets you change the default mapping of OTLP to Loki format for each tenant.
+It currently only supports changing the storage of Attributes. Here is how the config looks like:
+
+```yaml
+# OTLP log ingestion configurations
+otlp_config:
+  # Configuration for Resource Attributes to store them as index labels or
+  # Structured Metadata or drop them altogether
+  resource_attributes:
+    # Configure whether to ignore the default list of Resource Attributes to be
+    # stored as Index Labels and only use the given Resource Attributes config
+    [ignore_defaults: <boolean>]
+
+    [attributes_config: <list of attributes_configs>]
+
+  # Configuration for Scope Attributes to store them as Structured Metadata or
+  # drop them altogether
+  [scope_attributes: <list of attributes_configs>]
+
+  # Configuration for Log Attributes to store them as Structured Metadata or
+  # drop them altogether
+  [log_attributes: <list of attributes_configs>]
+
+attributes_config:
+  # Configures action to take on matching Attributes. It allows one of
+  # [structured_metadata, drop] for all Attribute types. It additionally allows
+  # index_label action for Resource Attributes
+  [action: <string> | default = ""]
+
+  # List of attributes to configure how to store them or drop them altogether
+  [attributes: <list of strings>]
+
+  # Regex to choose attributes to configure how to store them or drop them
+  # altogether
+  [regex: <Regexp>]
+```
+
+Here are some example configs to change the default mapping of OTLP to Loki format:
+
+#### Example 1:
+
+```yaml
+otlp_config:
+  resource_attributes:
+    attributes_config:
+      - action: index_label
+        attributes:
+          - service.group
+```
+
+With the example config, here is how various kinds of Attributes would be stored:
+* Store all 17 Resource Attributes mentioned earlier and `service.group` Resource Attribute as index labels.
+* Store remaining Resource Attributes as Structured Metadata.
+* Store all the Scope and Log Attributes as Structured Metadata.
+
+#### Example 2:
+
+```yaml
+otlp_config:
+  resource_attributes:
+    ignore_defaults: true
+    attributes_config:
+      - action: index_label
+        regex: service.group
+```
+
+With the example config, here is how various kinds of Attributes would be stored:
+* **Only** store `service.group` Resource Attribute as index labels.
+* Store remaining Resource Attributes as Structured Metadata.
+* Store all the Scope and Log Attributes as Structured Metadata.
+
+#### Example 2:
+
+```yaml
+otlp_config:
+  resource_attributes:
+    attributes_config:
+      - action: index_label
+        regex: service.group
+  scope_attributes:
+    - action: drop
+      attributes:
+        - method.name
+  log_attributes:
+    - action: structured_metadata
+      attributes:
+        - user.id
+    - action: drop
+      regex: .*
+```
+
+With the example config, here is how various kinds of Attributes would be stored:
+* Store all 17 Resource Attributes mentioned earlier and `service.group` Resource Attribute as index labels.
+* Store remaining Resource Attributes as Structured Metadata.
+* Drop Scope Attribute named `method.name` and store all other Scope Attributes as Structured Metadata.
+* Store Log Attribute named `user.id` as Structured Metadata and drop all other Log Attributes.

pkg/loghttp/push/otlp_config.go

@@ -56,9 +56,9 @@ var DefaultOTLPConfig = OTLPConfig{
 }
 
 type OTLPConfig struct {
-	ResourceAttributes ResourceAttributesConfig `yaml:"resource_attributes,omitempty"`
-	ScopeAttributes    []AttributesConfig       `yaml:"scope_attributes,omitempty"`
-	LogAttributes      []AttributesConfig       `yaml:"log_attributes,omitempty"`
+	ResourceAttributes ResourceAttributesConfig `yaml:"resource_attributes,omitempty" doc:"description=Configuration for resource attributes to store them as index labels or Structured Metadata or drop them altogether"`
+	ScopeAttributes    []AttributesConfig       `yaml:"scope_attributes,omitempty" doc:"description=Configuration for scope attributes to store them as Structured Metadata or drop them altogether"`
+	LogAttributes      []AttributesConfig       `yaml:"log_attributes,omitempty" doc:"description=Configuration for log attributes to store them as Structured Metadata or drop them altogether"`
 }
 
 func (c *OTLPConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
@@ -115,9 +115,9 @@ func (c *OTLPConfig) Validate() error {
 }
 
 type AttributesConfig struct {
-	Action     Action         `yaml:"action,omitempty"`
-	Attributes []string       `yaml:"attributes,omitempty"`
-	Regex      relabel.Regexp `yaml:"regex,omitempty"`
+	Action     Action         `yaml:"action,omitempty" doc:"description=Configures action to take on matching attributes. It allows one of [structured_metadata, drop] for all attribute types. It additionally allows index_label action for resource attributes"`
+	Attributes []string       `yaml:"attributes,omitempty" doc:"description=List of attributes to configure how to store them or drop them altogether"`
+	Regex      relabel.Regexp `yaml:"regex,omitempty" doc:"description=Regex to choose attributes to configure how to store them or drop them altogether"`
 }
 
 func (c *AttributesConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
@@ -146,8 +146,8 @@ func (c *AttributesConfig) UnmarshalYAML(unmarshal func(interface{}) error) erro
 }
 
 type ResourceAttributesConfig struct {
-	IgnoreDefaults   bool               `yaml:"ignore_defaults,omitempty"`
-	AttributesConfig []AttributesConfig `yaml:"attributes,omitempty"`
+	IgnoreDefaults   bool               `yaml:"ignore_defaults,omitempty" doc:"default=false|description=Configure whether to ignore the default list of resource attributes to be stored as index labels and only use the given resource attributes config"`
+	AttributesConfig []AttributesConfig `yaml:"attributes_config,omitempty"`
 }
 
 func (c *ResourceAttributesConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {

pkg/loghttp/push/otlp_config_test.go

@@ -19,7 +19,7 @@ func TestUnmarshalOTLPConfig(t *testing.T) {
 			name: "only resource_attributes set",
 			yamlConfig: []byte(`
 resource_attributes:
-  attributes:
+  attributes_config:
     - action: index_label
       regex: foo`),
 			expectedCfg: OTLPConfig{
@@ -39,7 +39,7 @@ resource_attributes:
 			yamlConfig: []byte(`
 resource_attributes:
   ignore_defaults: true
-  attributes:
+  attributes_config:
     - action: index_label
       regex: foo`),
 			expectedCfg: OTLPConfig{
@@ -82,7 +82,7 @@ scope_attributes:
 			name: "all 3 set",
 			yamlConfig: []byte(`
 resource_attributes:
-  attributes:
+  attributes_config:
     - action: index_label
       regex: foo
 scope_attributes:

tools/doc-generator/parse/parser.go

@@ -462,6 +462,8 @@ func getCustomFieldType(t reflect.Type) (string, bool) {
 		return "remote_write_config...", true
 	case reflect.TypeOf(validation.OverwriteMarshalingStringMap{}).String():
 		return "headers", true
+	case reflect.TypeOf(relabel.Regexp{}).String():
+		return fieldString, true
 	default:
 		return "", false
 	}

tools/doc-generator/parse/root_blocks.go

@@ -20,6 +20,7 @@ import (
 	"github.com/grafana/loki/pkg/distributor"
 	"github.com/grafana/loki/pkg/ingester"
 	ingester_client "github.com/grafana/loki/pkg/ingester/client"
+	"github.com/grafana/loki/pkg/loghttp/push"
 	"github.com/grafana/loki/pkg/loki/common"
 	frontend "github.com/grafana/loki/pkg/lokifrontend"
 	"github.com/grafana/loki/pkg/querier"
@@ -272,6 +273,12 @@ storage_config:
       store-1:
         endpoint: s3://foo-bucket
         region: us-west1
-Named store from this example can be used by setting object_store to store-1 in period_config.`},
+Named store from this example can be used by setting object_store to store-1 in period_config.`,
+		},
+		{
+			Name:       "attributes_config",
+			StructType: []reflect.Type{reflect.TypeOf(push.AttributesConfig{})},
+			Desc:       "Define actions for matching OpenTelemetry (OTEL) attributes.",
+		},
 	}
 )