Skip to content

Commit

Permalink
Merge branch 'main' into switch-testing-build-image-to-gh-action
Browse files Browse the repository at this point in the history
  • Loading branch information
hainenber authored Oct 5, 2023
2 parents 5c0bd7c + b553d06 commit 22c4678
Show file tree
Hide file tree
Showing 165 changed files with 9,278 additions and 1,015 deletions.
60 changes: 50 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ internal API changes are not present.
Main (unreleased)
-----------------

v0.37.0-rc.0 (2023-10-05)
-----------------

### Breaking changes

- Set `retry_on_http_429` to `true` by default in the `queue_config` block in static mode's `remote_write`. (@wildum)
Expand All @@ -26,16 +29,13 @@ Main (unreleased)
got replaced by the pair of `__meta_component_name` and `__meta_component_id`
labels. (@tpaschalis)

- Flow: Allow `prometheus.exporter.unix` to be specified multiple times and used in modules. This now means all
`prometheus.exporter.unix` references will need a label `prometheus.exporter.unix "example"`. (@mattdurham)

### Features

- New Grafana Agent Flow components:

- `otelcol.connector.spanlogs` creates logs from spans. It is the flow mode equivalent
to static mode's `automatic_logging` processor. (@ptodev)
- `otelcol.connector.servicegraph` creates service graph metrics from spans. It is the
flow mode equivalent to static mode's `service_graphs` processor. (@ptodev)
- `otelcol.processor.k8sattributes` adds Kubernetes metadata as resource attributes
to spans, logs, and metrics. (@acr92)
- `discovery.consulagent` discovers scrape targets from Consul Agent. (@wildum)
- `discovery.kuma` discovers scrape targets from the Kuma control plane. (@tpaschalis)
- `discovery.linode` discovers scrape targets from the Linode API. (@captncraig)
Expand All @@ -46,13 +46,26 @@ Main (unreleased)
- `discovery.serverset` discovers Serversets stored in Zookeeper. (@thampiotr)
- `discovery.scaleway` discovers scrape targets from Scaleway virtual
instances and bare-metal machines. (@rfratto)
- `faro.receiver` accepts Grafana Faro-formatted telemetry data over the
network and forwards it to other components. (@megumish, @rfratto)
- `prometheus.exporter.azure` collects metrics from Azure. (@wildum)
- `discovery.dockerswarm` discovers scrape targets from Docker Swarm. (@wildum)
- `otelcol.connector.servicegraph` creates service graph metrics from spans. It is the
flow mode equivalent to static mode's `service_graphs` processor. (@ptodev)
- `otelcol.connector.spanlogs` creates logs from spans. It is the flow mode equivalent
to static mode's `automatic_logging` processor. (@ptodev)
- `otelcol.processor.k8sattributes` adds Kubernetes metadata as resource attributes
to spans, logs, and metrics. (@acr92)
- `otelcol.processor.probabilistic_sampler` samples logs and traces based on configuration options. (@mar4uk)
- `otelcol.processor.transform` transforms OTLP telemetry data using the
OpenTelemetry Transformation Language (OTTL). It is most commonly used
for transformations on attributes.
- `remote.kubernetes.configmap` loads a configmap's data for use in other components (@captncraig)
- `remote.kubernetes.secret` loads a secret's data for use in other components (@captncraig)
- `prometheus.exporter.agent` - scrape agent's metrics. (@hainenber)
- `prometheus.exporter.vsphere` - scrape vmware vsphere metrics. (@marctc)
- `prometheus.exporter.agent` exposes the agent's internal metrics. (@hainenber)
- `prometheus.exporter.azure` collects metrics from Azure. (@wildum)
- `prometheus.exporter.cadvisor` exposes cAdvisor metrics. (@tpaschalis)
- `prometheus.exporter.vsphere` exposes vmware vsphere metrics. (@marctc)

- Flow: allow the HTTP server to be configured with TLS in the config file
using the new `http` config block. (@rfratto)
Expand All @@ -76,9 +89,13 @@ Main (unreleased)
This will load all River files in the directory as a single configuration;
component names must be unique across all loaded files. (@rfratto, @hainenber)

- Added support for `static` configuration conversion in `grafana-agent convert` and `grafana-agent run` commands. (@erikbaranowski)

- Flow: the `prometheus.scrape` component can now configure the scraping of
Prometheus native histograms. (@tpaschalis)

- Flow: the `prometheus.remote_write` component now supports SigV4 and AzureAD authentication. (@ptodev)

### Enhancements

- Clustering: allow advertise interfaces to be configurable, with the possibility to select all available interfaces. (@wildum)
Expand All @@ -99,7 +116,7 @@ Main (unreleased)

- Flow: add `randomization_factor` and `multiplier` to retry settings in
`otelcol` components. (@rfratto)

- Add support for `windows_certificate_filter` under http tls config block. (@mattdurham)

- Add `openstack` config converter to convert OpenStack yaml config (static mode) to river config (flow mode). (@wildum)
Expand Down Expand Up @@ -127,6 +144,22 @@ Main (unreleased)

- Promtail converter will now treat `global positions configuration is not supported` as a Warning instead of Error. (@erikbaranowski)

- Add new `agent_component_dependencies_wait_seconds` histogram metric and a dashboard panel
that measures how long components wait to be evaluated after their dependency is updated (@thampiotr)

- Add additional endpoint to debug scrape configs generated inside `prometheus.operator.*` components (@captncraig)

- Components evaluation is now performed in parallel, reducing the impact of
slow components potentially blocking the entire telemetry pipeline.
The `agent_component_evaluation_seconds` metric now measures evaluation time
of each node separately, instead of all the directly and indirectly
dependant nodes. (@thampiotr)

- Update Prometheus dependency to v2.46.0. (@tpaschalis)

- The `client_secret` config argument in the `otelcol.auth.oauth2` component is
now of type `secret` instead of type `string`. (@ptodev)

### Bugfixes

- Fixed `otelcol.exporter.prometheus` label names for the `otel_scope_info`
Expand All @@ -142,6 +175,8 @@ Main (unreleased)

- Fixed a bug where the BackOffLimit for the kubernetes tailer was always set to zero. (@anderssonw)

- Fixed a bug where Flow agent fails to load `comment` statement in `argument` block. (@hainenber)

### Other changes

- Use Go 1.21.1 for builds. (@rfratto)
Expand All @@ -161,7 +196,7 @@ Main (unreleased)

- Documentation updated to link discovery.http and prometheus.scrape advanced configs (@proffalken)

- Bump SNMP exporter version to v0.23 (@marctc)
- Bump SNMP exporter version to v0.24.1 (@marctc)

- Switch to `IBM/sarama` module. (@hainenber)

Expand All @@ -171,6 +206,11 @@ Main (unreleased)

- Migrate `Check Linux/Windows build image` to GitHub Actions. (@hainenber)

- Documentation updated to correct default path from `prometheus.exporter.windows` `text_file` block (@timo1707)

- Bump `redis_exporter` to v1.54.0 (@spartan0x117)


v0.36.2 (2023-09-22)
--------------------

Expand Down
2 changes: 1 addition & 1 deletion cmd/internal/flowmode/cmd_run.go
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ depending on the nature of the reload error.
StringVar(&r.clusterName, "cluster.name", r.clusterName, "The name of the cluster to join")
cmd.Flags().
BoolVar(&r.disableReporting, "disable-reporting", r.disableReporting, "Disable reporting of enabled components to Grafana.")
cmd.Flags().StringVar(&r.configFormat, "config.format", r.configFormat, "The format of the source file. Supported formats: 'flow', 'prometheus'.")
cmd.Flags().StringVar(&r.configFormat, "config.format", r.configFormat, fmt.Sprintf("The format of the source file. Supported formats: %s.", supportedFormatsList()))
cmd.Flags().BoolVar(&r.configBypassConversionErrors, "config.bypass-conversion-errors", r.configBypassConversionErrors, "Enable bypassing errors when converting")
return cmd
}
Expand Down
3 changes: 3 additions & 0 deletions component/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import (
_ "github.com/grafana/agent/component/discovery/serverset" // Import discovery.serverset
_ "github.com/grafana/agent/component/discovery/triton" // Import discovery.triton
_ "github.com/grafana/agent/component/discovery/uyuni" // Import discovery.uyuni
_ "github.com/grafana/agent/component/faro/receiver" // Import faro.receiver
_ "github.com/grafana/agent/component/local/file" // Import local.file
_ "github.com/grafana/agent/component/local/file_match" // Import local.file_match
_ "github.com/grafana/agent/component/loki/echo" // Import loki.echo
Expand Down Expand Up @@ -81,6 +82,7 @@ import (
_ "github.com/grafana/agent/component/otelcol/processor/probabilistic_sampler" // Import otelcol.processor.probabilistic_sampler
_ "github.com/grafana/agent/component/otelcol/processor/span" // Import otelcol.processor.span
_ "github.com/grafana/agent/component/otelcol/processor/tail_sampling" // Import otelcol.processor.tail_sampling
_ "github.com/grafana/agent/component/otelcol/processor/transform" // Import otelcol.processor.transform
_ "github.com/grafana/agent/component/otelcol/receiver/jaeger" // Import otelcol.receiver.jaeger
_ "github.com/grafana/agent/component/otelcol/receiver/kafka" // Import otelcol.receiver.kafka
_ "github.com/grafana/agent/component/otelcol/receiver/loki" // Import otelcol.receiver.loki
Expand All @@ -92,6 +94,7 @@ import (
_ "github.com/grafana/agent/component/prometheus/exporter/apache" // Import prometheus.exporter.apache
_ "github.com/grafana/agent/component/prometheus/exporter/azure" // Import prometheus.exporter.azure
_ "github.com/grafana/agent/component/prometheus/exporter/blackbox" // Import prometheus.exporter.blackbox
_ "github.com/grafana/agent/component/prometheus/exporter/cadvisor" // Import prometheus.exporter.cadvisor
_ "github.com/grafana/agent/component/prometheus/exporter/cloudwatch" // Import prometheus.exporter.cloudwatch
_ "github.com/grafana/agent/component/prometheus/exporter/consul" // Import prometheus.exporter.consul
_ "github.com/grafana/agent/component/prometheus/exporter/dnsmasq" // Import prometheus.exporter.dnsmasq
Expand Down
2 changes: 1 addition & 1 deletion component/common/loki/wal/wal.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ type wrapper struct {
func New(cfg Config, log log.Logger, registerer prometheus.Registerer) (WAL, error) {
// TODO: We should fine-tune the WAL instantiated here to allow some buffering of written entries, but not written to disk
// yet. This will attest for the lack of buffering in the channel Writer exposes.
tsdbWAL, err := wlog.NewSize(log, registerer, cfg.Dir, wlog.DefaultSegmentSize, false)
tsdbWAL, err := wlog.NewSize(log, registerer, cfg.Dir, wlog.DefaultSegmentSize, wlog.CompressionNone)
if err != nil {
return nil, fmt.Errorf("failde to create tsdb WAL: %w", err)
}
Expand Down
93 changes: 93 additions & 0 deletions component/faro/receiver/arguments.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
package receiver

import (
"time"

"github.com/alecthomas/units"
"github.com/grafana/agent/component/common/loki"
"github.com/grafana/agent/component/otelcol"
"github.com/grafana/river"
"github.com/grafana/river/rivertypes"
)

// Defaults for various arguments.
var (
DefaultArguments = Arguments{
Server: DefaultServerArguments,
SourceMaps: DefaultSourceMapsArguments,
}

DefaultServerArguments = ServerArguments{
Host: "127.0.0.1",
Port: 12347,
RateLimiting: DefaultRateLimitingArguments,
MaxAllowedPayloadSize: 5 * units.MiB,
}

DefaultRateLimitingArguments = RateLimitingArguments{
Enabled: true,
Rate: 50,
BurstSize: 100,
}

DefaultSourceMapsArguments = SourceMapsArguments{
Download: true,
DownloadFromOrigins: []string{"*"},
DownloadTimeout: time.Second,
}
)

// Arguments configures the app_agent_receiver component.
type Arguments struct {
LogLabels map[string]string `river:"extra_log_labels,attr,optional"`

Server ServerArguments `river:"server,block,optional"`
SourceMaps SourceMapsArguments `river:"sourcemaps,block,optional"`
Output OutputArguments `river:"output,block"`
}

var _ river.Defaulter = (*Arguments)(nil)

// SetToDefault applies default settings.
func (args *Arguments) SetToDefault() { *args = DefaultArguments }

// ServerArguments configures the HTTP server where telemetry information will
// be sent from Faro clients.
type ServerArguments struct {
Host string `river:"listen_address,attr,optional"`
Port int `river:"listen_port,attr,optional"`
CORSAllowedOrigins []string `river:"cors_allowed_origins,attr,optional"`
APIKey rivertypes.Secret `river:"api_key,attr,optional"`
MaxAllowedPayloadSize units.Base2Bytes `river:"max_allowed_payload_size,attr,optional"`

RateLimiting RateLimitingArguments `river:"rate_limiting,block,optional"`
}

// RateLimitingArguments configures rate limiting for the HTTP server.
type RateLimitingArguments struct {
Enabled bool `river:"enabled,attr,optional"`
Rate float64 `river:"rate,attr,optional"`
BurstSize float64 `river:"burst_size,attr,optional"`
}

// SourceMapsArguments configures how app_agent_receiver will retrieve source
// maps for transforming stack traces.
type SourceMapsArguments struct {
Download bool `river:"download,attr,optional"`
DownloadFromOrigins []string `river:"download_from_origins,attr,optional"`
DownloadTimeout time.Duration `river:"download_timeout,attr,optional"`
Locations []LocationArguments `river:"location,block,optional"`
}

// LocationArguments specifies an individual location where source maps will be loaded.
type LocationArguments struct {
Path string `river:"path,attr"`
MinifiedPathPrefix string `river:"minified_path_prefix,attr"`
}

// OutputArguments configures where to send emitted logs and traces. Metrics
// emitted by app_agent_receiver are exported as targets to be scraped.
type OutputArguments struct {
Logs []loki.LogsReceiver `river:"logs,attr,optional"`
Traces []otelcol.Consumer `river:"traces,attr,optional"`
}
Loading

0 comments on commit 22c4678

Please sign in to comment.