[release-v0.38] Prepare for v0.38.1 release (#5893)

* Allow x-faro-session-id header for faro receiver (#5835) (cherry picked from commit cc7cb37) * misc: follow up on #5835 (#5837) * Add missing CHANGELOG entry * Mirror fix to static mode (cherry picked from commit 7da5726) * Agent Management: Introduce support for template variables (#5788) * Agent Management: Introduce support for template variables - This change allows managing template variables for remote configuration on a per-agent basis. - Both base configurations and snippets can be interpreted as templates and evaluated at load time with the provided template variables. - Templates must follow go's `text/template` syntax. - This greatly increases the flexibility and reusability of snippets. - Template evaluation has been tested in different scenarios and seems really robust. If the variables defined in the template cannot be resolved (even nested ones), and empty string is rendered instead. - Note: templates are only evaluated when the `template_variables` field within the `agent_metadata` remote config field is non-empty. - Note: this feature only applies to static mode. * Improve naming * Check error for template execution * Add tests - Tests different scenarios, including: - Referencing non existing nested objects - Conditionals - Ranges - Character escaping * Update CHANGELOG * Always evaluate templates - This is required because certain agents might start before their labels are synced. If some of the snippets assigned to them contain template variables, loading the config will fail. * Add test for template inside a template - Templates inside templates must be escaped using backticks to avoid them being evaluated by the snippet template execution * Move feature to the next release in CHANGELOG * Document templating functionality * Fix doc (cherry picked from commit d388f94) * pyroscope.scrape: change error log level to not swallow errors (#5840) (cherry picked from commit 2242e4a) * Update windows defaults to use upstream defaults except for enabled collectors. (#5832) Fix #5831 and use the defaults from windows. (cherry picked from commit 15d3d9f) * Bump otelgrpc to fix CVE 2023 47108 (#5806) * Fix CVE-2023-47108 by updating `otelgrpc` from v0.45.0 to v0.46.0. * Stop using the deprecated trace.NewNoopTracerProvider. Using noop.NewTracerProvider from "go.opentelemetry.io/otel/trace/noop" instead. * Reorder changelog Comply with the ordering in: https://github.com/grafana/agent/blob/main/docs/developer/contributing.md#updating-the-changelog --------- Signed-off-by: hainenber <[email protected]> Co-authored-by: Paulin Todev <[email protected]> (cherry picked from commit a2348a0) * fix(otelcol/fanoutconsumer): nil check during fanout consumer creation (#5854) Signed-off-by: hainenber <[email protected]> (cherry picked from commit 84344fb) * prometheus.operator.* - Fix issue with missing targets when one monitor's name is a prefix of another (#5862) Co-authored-by: Paul Bormans <[email protected]> (cherry picked from commit f232fb4) * max_cache_size was being set to 0 (#5869) * max_cache_size was being set to 0, due to issue where it doesnt exist in static but the default wasnt carrying over to river syntax. In truth we should never write it. * Clean up from PR (cherry picked from commit 356c50c) * Add Deploy Mode to usage stats. (#5880) Co-authored-by: Clayton Cornell <[email protected]> (cherry picked from commit 7bf82ea) * Fix promtail converter: docker_sd_configs (#5881) * Fix promtail converter: docker_sd * changelog and lint * typo (cherry picked from commit c5dc968) * prepare for 0.38.1 release (#5891) (cherry picked from commit fc2fd5b) * fix misplaced CHANGELOG entry for #5869 --------- Co-authored-by: Cedric Ziel <[email protected]> Co-authored-by: Jorge Creixell <[email protected]> Co-authored-by: Tolya Korniltsev <[email protected]> Co-authored-by: mattdurham <[email protected]> Co-authored-by: Đỗ Trọng Hải <[email protected]> Co-authored-by: Craig Peterson <[email protected]> Co-authored-by: Piotr <[email protected]>
grafana · Nov 30, 2023 · 15e3e86 · 15e3e86
1 parent 643e817
commit 15e3e86
Show file tree

Hide file tree

Showing 49 changed files with 1,008 additions and 271 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,8 +7,38 @@ This document contains a historical list of changes between releases. Only
 changes that impact end-user behavior are listed; changes to documentation or
 internal API changes are not present.
 
-Main (unreleased)
------------------
+v0.38.1 (2023-11-30)
+--------------------
+
+### Security fixes
+
+- Fix CVE-2023-47108 by updating `otelgrpc` from v0.45.0 to v0.46.0. (@hainenber)
+
+### Features
+
+- Agent Management: Introduce support for templated configuration. (@jcreixell)
+
+### Bugfixes
+
+- Permit `X-Faro-Session-ID` header in CORS requests for the `faro.receiver`
+  component (flow mode) and the `app_agent_receiver` integration (static mode).
+  (@cedricziel)
+
+- Fix issue with windows_exporter defaults not being set correctly. (@mattdurham)
+
+- Fix agent crash when process null OTel's fan out consumers. (@hainenber)
+
+- Fix issue in `prometheus.operator.*` where targets would be dropped if two crds share a common prefix in their names. (@Paul424, @captncraig)
+
+- Fix issue where `convert` command would generate incorrect Flow Mode config
+  when provided `promtail` configuration that uses `docker_sd_configs` (@thampiotr)
+
+- Fix converter issue with `loki.relabel` and `max_cache_size` being set to 0
+  instead of default (10_000). (@mattdurham)
+
+### Other changes
+
+- Add Agent Deploy Mode to usage report. (@captncraig)
 
 v0.38.0 (2023-11-21)
 --------------------

diff --git a/component/faro/receiver/handler.go b/component/faro/receiver/handler.go
@@ -69,7 +69,7 @@ func (h *handler) Update(args ServerArguments) {
 	if len(args.CORSAllowedOrigins) > 0 {
 		h.cors = cors.New(cors.Options{
 			AllowedOrigins: args.CORSAllowedOrigins,
-			AllowedHeaders: []string{apiKeyHeader, "content-type"},
+			AllowedHeaders: []string{apiKeyHeader, "content-type", "x-faro-session-id"},
 		})
 	} else {
 		h.cors = nil // Disable cors.

diff --git a/component/otelcol/internal/fanoutconsumer/logs.go b/component/otelcol/internal/fanoutconsumer/logs.go
@@ -29,6 +29,10 @@ func Logs(in []otelcol.Consumer) otelconsumer.Logs {
 	for i := 0; i < len(in)-1; i++ {
 		consumer := in[i]
 
+		if consumer == nil {
+			continue
+		}
+
 		if consumer.Capabilities().MutatesData {
 			clone = append(clone, consumer)
 		} else {
@@ -40,10 +44,12 @@ func Logs(in []otelcol.Consumer) otelconsumer.Logs {
 
 	// The final consumer can be given to the passthrough list regardless of
 	// whether it mutates as long as there's no other read-only consumers.
-	if len(passthrough) == 0 || !last.Capabilities().MutatesData {
-		passthrough = append(passthrough, last)
-	} else {
-		clone = append(clone, last)
+	if last != nil {
+		if len(passthrough) == 0 || !last.Capabilities().MutatesData {
+			passthrough = append(passthrough, last)
+		} else {
+			clone = append(clone, last)
+		}
 	}
 
 	return &logsFanout{

diff --git a/component/otelcol/receiver/prometheus/prometheus.go b/component/otelcol/receiver/prometheus/prometheus.go
@@ -20,8 +20,8 @@ import (
 	"github.com/prometheus/prometheus/storage"
 	otelcomponent "go.opentelemetry.io/collector/component"
 	otelreceiver "go.opentelemetry.io/collector/receiver"
-	"go.opentelemetry.io/otel/metric/noop"
-	"go.opentelemetry.io/otel/trace"
+	metricNoop "go.opentelemetry.io/otel/metric/noop"
+	traceNoop "go.opentelemetry.io/otel/trace/noop"
 )
 
 func init() {
@@ -109,8 +109,8 @@ func (c *Component) Update(newConfig component.Arguments) error {
 			Logger: zapadapter.New(c.opts.Logger),
 
 			// TODO(tpaschalis): expose tracing and logging statistics.
-			TracerProvider: trace.NewNoopTracerProvider(),
-			MeterProvider:  noop.NewMeterProvider(),
+			TracerProvider: traceNoop.NewTracerProvider(),
+			MeterProvider:  metricNoop.NewMeterProvider(),
 
 			ReportComponentStatus: func(*otelcomponent.StatusEvent) error {
 				return nil

diff --git a/component/prometheus/exporter/windows/config.go b/component/prometheus/exporter/windows/config.go
@@ -6,72 +6,6 @@ import (
 	windows_integration "github.com/grafana/agent/pkg/integrations/windows_exporter"
 )
 
-// DefaultArguments holds non-zero default options for Arguments when it is
-// unmarshaled from YAML.
-//
-// Some defaults are populated from init functions in the github.com/grafana/agent/pkg/integrations/windows_exporter package.
-
-var DefaultArguments = Arguments{
-	EnabledCollectors: strings.Split(windows_integration.DefaultConfig.EnabledCollectors, ","),
-	Dfsr: DfsrConfig{
-		SourcesEnabled: strings.Split(windows_integration.DefaultConfig.Dfsr.SourcesEnabled, ","),
-	},
-	Exchange: ExchangeConfig{
-		EnabledList: strings.Split(windows_integration.DefaultConfig.Exchange.EnabledList, ","),
-	},
-	IIS: IISConfig{
-		AppBlackList:  windows_integration.DefaultConfig.IIS.AppBlackList,
-		AppWhiteList:  windows_integration.DefaultConfig.IIS.AppWhiteList,
-		SiteBlackList: windows_integration.DefaultConfig.IIS.SiteBlackList,
-		SiteWhiteList: windows_integration.DefaultConfig.IIS.SiteWhiteList,
-		AppInclude:    windows_integration.DefaultConfig.IIS.AppInclude,
-		AppExclude:    windows_integration.DefaultConfig.IIS.AppExclude,
-		SiteInclude:   windows_integration.DefaultConfig.IIS.SiteInclude,
-		SiteExclude:   windows_integration.DefaultConfig.IIS.SiteExclude,
-	},
-	LogicalDisk: LogicalDiskConfig{
-		BlackList: windows_integration.DefaultConfig.LogicalDisk.BlackList,
-		WhiteList: windows_integration.DefaultConfig.LogicalDisk.WhiteList,
-		Include:   windows_integration.DefaultConfig.LogicalDisk.Include,
-		Exclude:   windows_integration.DefaultConfig.LogicalDisk.Exclude,
-	},
-	MSMQ: MSMQConfig{
-		Where: windows_integration.DefaultConfig.MSMQ.Where,
-	},
-	MSSQL: MSSQLConfig{
-		EnabledClasses: strings.Split(windows_integration.DefaultConfig.MSSQL.EnabledClasses, ","),
-	},
-	Network: NetworkConfig{
-		BlackList: windows_integration.DefaultConfig.Network.BlackList,
-		WhiteList: windows_integration.DefaultConfig.Network.WhiteList,
-		Include:   windows_integration.DefaultConfig.Network.Include,
-		Exclude:   windows_integration.DefaultConfig.Network.Exclude,
-	},
-	Process: ProcessConfig{
-		BlackList: windows_integration.DefaultConfig.Process.BlackList,
-		WhiteList: windows_integration.DefaultConfig.Process.WhiteList,
-		Include:   windows_integration.DefaultConfig.Process.Include,
-		Exclude:   windows_integration.DefaultConfig.Process.Exclude,
-	},
-	ScheduledTask: ScheduledTaskConfig{
-		Include: windows_integration.DefaultConfig.ScheduledTask.Include,
-		Exclude: windows_integration.DefaultConfig.ScheduledTask.Exclude,
-	},
-	Service: ServiceConfig{
-		UseApi: windows_integration.DefaultConfig.Service.UseApi,
-		Where:  windows_integration.DefaultConfig.Service.Where,
-	},
-	SMTP: SMTPConfig{
-		BlackList: windows_integration.DefaultConfig.SMTP.BlackList,
-		WhiteList: windows_integration.DefaultConfig.SMTP.WhiteList,
-		Include:   windows_integration.DefaultConfig.SMTP.Include,
-		Exclude:   windows_integration.DefaultConfig.SMTP.Exclude,
-	},
-	TextFile: TextFileConfig{
-		TextFileDirectory: windows_integration.DefaultConfig.TextFile.TextFileDirectory,
-	},
-}
-
 // Arguments is used for controlling for this exporter.
 type Arguments struct {
 	// Collectors to mark as enabled
@@ -92,11 +26,6 @@ type Arguments struct {
 	TextFile      TextFileConfig      `river:"text_file,block,optional"`
 }
 
-// SetToDefault implements river.Defaulter.
-func (a *Arguments) SetToDefault() {
-	*a = DefaultArguments
-}
-
 // Convert converts the component's Arguments to the integration's Config.
 func (a *Arguments) Convert() *windows_integration.Config {
 	return &windows_integration.Config{

diff --git a/component/prometheus/exporter/windows/config_default_windows_test.go b/component/prometheus/exporter/windows/config_default_windows_test.go
@@ -1,10 +1,8 @@
 package windows
 
 import (
-	"strings"
 	"testing"
 
-	windows_integration "github.com/grafana/agent/pkg/integrations/windows_exporter"
 	"github.com/grafana/river"
 	"github.com/stretchr/testify/require"
 )
@@ -14,26 +12,26 @@ func TestRiverUnmarshalWithDefaultConfig(t *testing.T) {
 	err := river.Unmarshal([]byte(""), &args)
 	require.NoError(t, err)
 
-	require.Equal(t, strings.Split(windows_integration.DefaultConfig.EnabledCollectors, ","), args.EnabledCollectors)
-	require.Equal(t, strings.Split(windows_integration.DefaultConfig.Dfsr.SourcesEnabled, ","), args.Dfsr.SourcesEnabled)
-	require.Equal(t, strings.Split(windows_integration.DefaultConfig.Exchange.EnabledList, ","), args.Exchange.EnabledList)
-	require.Equal(t, windows_integration.DefaultConfig.IIS.AppExclude, args.IIS.AppExclude)
-	require.Equal(t, windows_integration.DefaultConfig.IIS.AppInclude, args.IIS.AppInclude)
-	require.Equal(t, windows_integration.DefaultConfig.IIS.SiteExclude, args.IIS.SiteExclude)
-	require.Equal(t, windows_integration.DefaultConfig.IIS.SiteInclude, args.IIS.SiteInclude)
-	require.Equal(t, windows_integration.DefaultConfig.LogicalDisk.Exclude, args.LogicalDisk.Exclude)
-	require.Equal(t, windows_integration.DefaultConfig.LogicalDisk.Include, args.LogicalDisk.Include)
-	require.Equal(t, windows_integration.DefaultConfig.MSMQ.Where, args.MSMQ.Where)
-	require.Equal(t, strings.Split(windows_integration.DefaultConfig.MSSQL.EnabledClasses, ","), args.MSSQL.EnabledClasses)
-	require.Equal(t, windows_integration.DefaultConfig.Network.Exclude, args.Network.Exclude)
-	require.Equal(t, windows_integration.DefaultConfig.Network.Include, args.Network.Include)
-	require.Equal(t, windows_integration.DefaultConfig.Process.Exclude, args.Process.Exclude)
-	require.Equal(t, windows_integration.DefaultConfig.Process.Include, args.Process.Include)
-	require.Equal(t, windows_integration.DefaultConfig.ScheduledTask.Exclude, args.ScheduledTask.Exclude)
-	require.Equal(t, windows_integration.DefaultConfig.ScheduledTask.Include, args.ScheduledTask.Include)
-	require.Equal(t, windows_integration.DefaultConfig.Service.UseApi, args.Service.UseApi)
-	require.Equal(t, windows_integration.DefaultConfig.Service.Where, args.Service.Where)
-	require.Equal(t, windows_integration.DefaultConfig.SMTP.Exclude, args.SMTP.Exclude)
-	require.Equal(t, windows_integration.DefaultConfig.SMTP.Include, args.SMTP.Include)
-	require.Equal(t, windows_integration.DefaultConfig.TextFile.TextFileDirectory, args.TextFile.TextFileDirectory)
+	require.Equal(t, DefaultArguments.EnabledCollectors, args.EnabledCollectors)
+	require.Equal(t, DefaultArguments.Dfsr.SourcesEnabled, args.Dfsr.SourcesEnabled)
+	require.Equal(t, DefaultArguments.Exchange.EnabledList, args.Exchange.EnabledList)
+	require.Equal(t, DefaultArguments.IIS.AppExclude, args.IIS.AppExclude)
+	require.Equal(t, DefaultArguments.IIS.AppInclude, args.IIS.AppInclude)
+	require.Equal(t, DefaultArguments.IIS.SiteExclude, args.IIS.SiteExclude)
+	require.Equal(t, DefaultArguments.IIS.SiteInclude, args.IIS.SiteInclude)
+	require.Equal(t, DefaultArguments.LogicalDisk.Exclude, args.LogicalDisk.Exclude)
+	require.Equal(t, DefaultArguments.LogicalDisk.Include, args.LogicalDisk.Include)
+	require.Equal(t, DefaultArguments.MSMQ.Where, args.MSMQ.Where)
+	require.Equal(t, DefaultArguments.MSSQL.EnabledClasses, args.MSSQL.EnabledClasses)
+	require.Equal(t, DefaultArguments.Network.Exclude, args.Network.Exclude)
+	require.Equal(t, DefaultArguments.Network.Include, args.Network.Include)
+	require.Equal(t, DefaultArguments.Process.Exclude, args.Process.Exclude)
+	require.Equal(t, DefaultArguments.Process.Include, args.Process.Include)
+	require.Equal(t, DefaultArguments.ScheduledTask.Exclude, args.ScheduledTask.Exclude)
+	require.Equal(t, DefaultArguments.ScheduledTask.Include, args.ScheduledTask.Include)
+	require.Equal(t, DefaultArguments.Service.UseApi, args.Service.UseApi)
+	require.Equal(t, DefaultArguments.Service.Where, args.Service.Where)
+	require.Equal(t, DefaultArguments.SMTP.Exclude, args.SMTP.Exclude)
+	require.Equal(t, DefaultArguments.SMTP.Include, args.SMTP.Include)
+	require.Equal(t, DefaultArguments.TextFile.TextFileDirectory, args.TextFile.TextFileDirectory)
 }
diff --git a/component/prometheus/exporter/windows/config_windows.go b/component/prometheus/exporter/windows/config_windows.go
@@ -0,0 +1,75 @@
+package windows
+
+import (
+	windows_integration "github.com/grafana/agent/pkg/integrations/windows_exporter"
+	col "github.com/prometheus-community/windows_exporter/pkg/collector"
+	"strings"
+)
+
+// DefaultArguments holds non-zero default options for Arguments when it is
+// unmarshaled from YAML.
+var DefaultArguments = Arguments{
+	EnabledCollectors: strings.Split(windows_integration.DefaultConfig.EnabledCollectors, ","),
+	Dfsr: DfsrConfig{
+		SourcesEnabled: strings.Split(col.ConfigDefaults.Dfsr.DfsrEnabledCollectors, ","),
+	},
+	Exchange: ExchangeConfig{
+		EnabledList: strings.Split(col.ConfigDefaults.Exchange.CollectorsEnabled, ","),
+	},
+	IIS: IISConfig{
+		AppBlackList:  col.ConfigDefaults.Iis.AppExclude,
+		AppWhiteList:  col.ConfigDefaults.Iis.AppInclude,
+		SiteBlackList: col.ConfigDefaults.Iis.SiteExclude,
+		SiteWhiteList: col.ConfigDefaults.Iis.SiteInclude,
+		AppInclude:    col.ConfigDefaults.Iis.AppInclude,
+		AppExclude:    col.ConfigDefaults.Iis.AppExclude,
+		SiteInclude:   col.ConfigDefaults.Iis.SiteInclude,
+		SiteExclude:   col.ConfigDefaults.Iis.SiteExclude,
+	},
+	LogicalDisk: LogicalDiskConfig{
+		BlackList: col.ConfigDefaults.LogicalDisk.VolumeExclude,
+		WhiteList: col.ConfigDefaults.LogicalDisk.VolumeInclude,
+		Include:   col.ConfigDefaults.LogicalDisk.VolumeInclude,
+		Exclude:   col.ConfigDefaults.LogicalDisk.VolumeExclude,
+	},
+	MSMQ: MSMQConfig{
+		Where: col.ConfigDefaults.Msmq.QueryWhereClause,
+	},
+	MSSQL: MSSQLConfig{
+		EnabledClasses: strings.Split(col.ConfigDefaults.Mssql.EnabledCollectors, ","),
+	},
+	Network: NetworkConfig{
+		BlackList: col.ConfigDefaults.Net.NicExclude,
+		WhiteList: col.ConfigDefaults.Net.NicInclude,
+		Include:   col.ConfigDefaults.Net.NicInclude,
+		Exclude:   col.ConfigDefaults.Net.NicExclude,
+	},
+	Process: ProcessConfig{
+		BlackList: col.ConfigDefaults.Process.ProcessExclude,
+		WhiteList: col.ConfigDefaults.Process.ProcessInclude,
+		Include:   col.ConfigDefaults.Process.ProcessInclude,
+		Exclude:   col.ConfigDefaults.Process.ProcessExclude,
+	},
+	ScheduledTask: ScheduledTaskConfig{
+		Include: col.ConfigDefaults.ScheduledTask.TaskInclude,
+		Exclude: col.ConfigDefaults.ScheduledTask.TaskExclude,
+	},
+	Service: ServiceConfig{
+		UseApi: "false",
+		Where:  col.ConfigDefaults.Service.ServiceWhereClause,
+	},
+	SMTP: SMTPConfig{
+		BlackList: col.ConfigDefaults.Smtp.ServerExclude,
+		WhiteList: col.ConfigDefaults.Smtp.ServerInclude,
+		Include:   col.ConfigDefaults.Smtp.ServerInclude,
+		Exclude:   col.ConfigDefaults.Smtp.ServerExclude,
+	},
+	TextFile: TextFileConfig{
+		TextFileDirectory: col.ConfigDefaults.Textfile.TextFileDirectories,
+	},
+}
+
+// SetToDefault implements river.Defaulter.
+func (a *Arguments) SetToDefault() {
+	*a = DefaultArguments
+}