diff --git a/.github/depcheck.yml b/.github/depcheck.yml index cc72fb1f6fc7..b3eeb7c9946b 100644 --- a/.github/depcheck.yml +++ b/.github/depcheck.yml @@ -23,7 +23,7 @@ github_repos: - github.com/google/dnsmasq_exporter v0.2.0 - github.com/ncabatoff/process-exporter v0.7.5 - github.com/prometheus/mysqld_exporter v0.13.0 - - github.com/prometheus-community/postgres_exporter v0.10.0 + - github.com/prometheus-community/postgres_exporter v0.15.0 - github.com/prometheus-community/windows_exporter v0.16.0 - github.com/percona/mongodb_exporter v0.20.7 - project: github.com/prometheus/prometheus diff --git a/.github/workflows/trivy.yml b/.github/workflows/trivy.yml index ceea4dbcbefd..cc32f0a2474f 100644 --- a/.github/workflows/trivy.yml +++ b/.github/workflows/trivy.yml @@ -26,7 +26,7 @@ jobs: - name: Checkout code uses: actions/checkout@v4 - name: Run Trivy vulnerability scanner - uses: aquasecurity/trivy-action@062f2592684a31eb3aa050cc61e7ca1451cecd3d + uses: aquasecurity/trivy-action@d710430a6722f083d3b36b8339ff66b32f22ee55 with: image-ref: 'grafana/agent:main' format: 'template' diff --git a/CHANGELOG.md b/CHANGELOG.md index 84f1f7a53862..cdb515335bd2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,7 +16,21 @@ Main (unreleased) 4317 to 55678 to align with upstream. (@rfratto) - The default sync interval for `mimir.rules.kubernetes` has changed from `30s` - to `5m` to reduce load on Mimir. (@56quarters) + to `5m` to reduce load on Mimir. (@56quarters) + +- `prometheus.exporter.postgres` has been updated to the latest upstream + version which changes the set of exported metrics. The following metrics were + removed: `pg_stat_database_session_time`, `pg_stat_database_sessions`, + `pg_stat_database_sessions_abandoned`, `pg_stat_database_sessions_fatal`, + `pg_stat_database_sessions_killed`, `pg_stat_database_idle_in_transaction_time`, + `pg_stat_database_checksum_failures`, `pg_stat_database_checksum_last_failure`, + `pg_stat_database_active_time`. The following metrics were + renamed: `pg_stat_bgwriter_buffers_alloc`, `pg_stat_bgwriter_buffers_backend`, + `pg_stat_bgwriter_buffers_backend_fsync`, `pg_stat_bgwriter_buffers_checkpoint`, + `pg_stat_bgwriter_buffers_clean`, `pg_stat_bgwriter_checkpoint_sync_time`, + `pg_stat_bgwriter_checkpoint_write_time`, `pg_stat_bgwriter_checkpoints_req`, + `pg_stat_bgwriter_checkpoints_timed`, `pg_stat_bgwriter_maxwritten_clean`, + `pg_stat_bgwriter_stats_reset` - the new names include the `_total` suffix. (@thampiotr) ### Enhancements @@ -45,23 +59,46 @@ Main (unreleased) - In `prometheus.exporter.kafka`, the interpolation table used to compute estimated lag metrics is now pruned on `metadata_refresh_interval` instead of `prune_interval_seconds`. (@wildum) -### Features +- Add ability to convert static mode positions file to `loki.source.file` compatible via `legacy_positions_file` argument. (@mattdurham) + +- Added support for `otelcol` configuration conversion in `grafana-agent convert` and `grafana-agent run` commands. (@rfratto, @erikbaranowski, @tpaschalis, @hainenber) + +- Added support for `static` configuration conversion of the `traces` subsystem. (@erikbaranowski, @wildum) + +- Add automatic conversion for `legacy_positions_file` in component `loki.source.file`. (@mattdurham) -- Added a new CLI flag `--stability.level` which defines the minimum stability - level required for the features that the agent is allowed to use. Default is `experimental`. (@thampiotr) +- Propagate request metadata for `faro.receiver` to downstream components. (@hainenber) + +### Features - A new `loki.rules.kubernetes` component that discovers `PrometheusRule` Kubernetes resources and loads them into a Loki Ruler instance. (@EStork09) +- Add `beyla.ebpf` component to automatically instrument services with eBPF. (@marctc) + ### Bugfixes - Fix an issue where JSON string array elements were not parsed correctly in `loki.source.cloudflare`. (@thampiotr) + +- Fix SSRF vulnerability in `faro.receiver` by disabling source map download. (@hainenber) + +- Fix an issue where the azure exporter was not correctly gathering subscription scoped metrics when only one region was configured (@kgeckhart) + - Update gcp_exporter to a newer version with a patch for incorrect delta histograms (@kgeckhart) - Fix an issue where the default values of some component's arguments change whenever that argument is explicitly configured. This issue only affected a small subset of arguments across 15 components. (@erikbaranowski, @rfratto) +- Fix panic when fanning out to invalid receivers. (@hainenber) + +- Fix a bug where a panic could occur when reloading custom components. (@wildum) + +- The `import.git` config block did not work with branches or tags this now fixes that behavior. (@mattdurham) + +- Fixed an issue where creating a `prometheus.exporter.postgres` component with + multiple `data_source_names` would result in an error. (@thampiotr) + ### Other changes - Clustering for Grafana Agent in Flow mode has graduated from beta to stable. @@ -84,9 +121,9 @@ v0.40.3 (2024-03-14) - Upgrade to Go 1.22.1 (@thampiotr) - Upgrade from OpenTelemetry Collector v0.87.0 to v0.96.0: - * [ottl]: Fix bug where named parameters needed a space after the equal sign (`=`) + * [ottl]: Fix bug where named parameters needed a space after the equal sign (`=`) https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/28511 - * [exporters] Additional enqueue_failed metrics + * [exporters] Additional enqueue_failed metrics https://github.com/open-telemetry/opentelemetry-collector/issues/8673 * [otelcol.receiver.kafka]: Fix issue where counting number of logs emitted could cause panic * [otelcol.processor.k8sattributes]: The time format of k8s.pod.start_time attribute value migrated to RFC3339: @@ -99,9 +136,9 @@ https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/26115 https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27451 * [otelcol.connector.spanmetrics] A new `max_per_data_point` argument for exemplar generation. * https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29242 - * [ottl] Add IsBool Converter + * [ottl] Add IsBool Converter https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27897 - * [otelcol.processor.tail_sampling] Optimize memory performance of tailsamplingprocessor + * [otelcol.processor.tail_sampling] Optimize memory performance of tailsamplingprocessor https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/27889 * [otelcol.connector.servicegraph] Add a `metrics_flush_interval` argument. https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27679 @@ -121,7 +158,7 @@ https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30162 https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30274 * [ottl] Add Hour converter https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/29468 - * [otelcol.connector.spanmetrics] A new `resource_metrics_key_attributes` argument to fix broken spanmetrics counters + * [otelcol.connector.spanmetrics] A new `resource_metrics_key_attributes` argument to fix broken spanmetrics counters after a span producing service restart, when resource attributes contain dynamic/ephemeral values (e.g. process id). https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/29711 * [ottl] Issue with the hash value of a match group in the replace_pattern editors diff --git a/CODEOWNERS b/CODEOWNERS index 4f1541f12dbd..b48a1816109b 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -18,4 +18,5 @@ /docs/sources/ @clayton-cornell # Components: -/component/pyroscope/ @grafana/grafana-agent-profiling-maintainers +/internal/component/pyroscope/ @grafana/grafana-agent-profiling-maintainers +/internal/component/beyla/ @marctc \ No newline at end of file diff --git a/cmd/grafana-agent/Dockerfile b/cmd/grafana-agent/Dockerfile index 09b38ea7d6e9..d855e2f54d8e 100644 --- a/cmd/grafana-agent/Dockerfile +++ b/cmd/grafana-agent/Dockerfile @@ -41,7 +41,7 @@ LABEL org.opencontainers.image.source="https://github.com/grafana/agent" # Install dependencies needed at runtime. RUN <}} configuration. + +Many OpenTelemetry Collector components are supported. +Review the `otelcol.*` component information in the [Component Reference][] for more information about `otelcol` components that you can convert. +If a source configuration has unsupported features, you will receive [errors] when you convert it to a flow configuration. +The converter raises warnings for configuration options that may require your attention. + +Refer to [Migrate from OpenTelemetry Collector to {{< param "PRODUCT_NAME" >}}][migrate-otelcol] for a detailed migration guide. + ### Prometheus Using the `--source-format=prometheus` will convert the source configuration from @@ -86,7 +98,7 @@ This includes Prometheus features such as and many supported *_sd_configs. Unsupported features in a source configuration result in [errors]. -Refer to [Migrate from Prometheus to {{< param "PRODUCT_NAME" >}}]({{< relref "../../tasks/migrate/from-prometheus/" >}}) for a detailed migration guide. +Refer to [Migrate from Prometheus to {{< param "PRODUCT_NAME" >}}][migrate-prometheus] for a detailed migration guide. ### Promtail @@ -100,21 +112,28 @@ are supported and can be converted to {{< param "PRODUCT_NAME" >}} configuration If you have unsupported features in a source configuration, you will receive [errors] when you convert to a flow configuration. The converter will also raise warnings for configuration options that may require your attention. -Refer to [Migrate from Promtail to {{< param "PRODUCT_NAME" >}}]({{< relref "../../tasks/migrate/from-promtail/" >}}) for a detailed migration guide. +Refer to [Migrate from Promtail to {{< param "PRODUCT_NAME" >}}][migrate-promtail] for a detailed migration guide. ### Static Using the `--source-format=static` will convert the source configuration from a -[Grafana Agent Static]({{< relref "../../../static" >}}) configuration to a {{< param "PRODUCT_NAME" >}} configuration. +[Grafana Agent Static][] configuration to a {{< param "PRODUCT_NAME" >}} configuration. Include `--extra-args` for passing additional command line flags from the original format. For example, `--extra-args="-enable-features=integrations-next"` will convert a Grafana Agent Static -[integrations-next]({{< relref "../../../static/configuration/integrations/integrations-next/" >}}) -configuration to a {{< param "PRODUCT_NAME" >}} configuration. You can also +[integrations-next][] configuration to a {{< param "PRODUCT_NAME" >}} configuration. You can also expand environment variables with `--extra-args="-config.expand-env"`. You can combine multiple command line flags with a space between each flag, for example `--extra-args="-enable-features=integrations-next -config.expand-env"`. If you have unsupported features in a Static mode source configuration, you will receive [errors][] when you convert to a Flow mode configuration. The converter will also raise warnings for configuration options that may require your attention. -Refer to [Migrate from Grafana Agent Static to {{< param "PRODUCT_NAME" >}}]({{< relref "../../tasks/migrate/from-static/" >}}) for a detailed migration guide. +Refer to [Migrate from Grafana Agent Static to {{< param "PRODUCT_NAME" >}}][migrate-static] for a detailed migration guide. + +[Component Reference]: ../../components/ +[migrate-otelcol]: ../../../tasks/migrate/from-otelcol/ +[migrate-prometheus]: ../../../tasks/migrate/from-prometheus/ +[migrate-promtail]: ../../../tasks/migrate/from-promtail/ +[migrate-static]: ../../../tasks/migrate/from-static/ +[Grafana Agent Static]: ../../../../static/ +[integrations-next]: ../../../../static/configuration/integrations/integrations-next/ \ No newline at end of file diff --git a/docs/sources/flow/reference/cli/run.md b/docs/sources/flow/reference/cli/run.md index c81c13f7374d..71c17837eb46 100644 --- a/docs/sources/flow/reference/cli/run.md +++ b/docs/sources/flow/reference/cli/run.md @@ -61,7 +61,7 @@ The following flags are supported: * `--cluster.advertise-interfaces`: List of interfaces used to infer an address to advertise. Set to `all` to use all available network interfaces on the system. (default `"eth0,en0"`). * `--cluster.max-join-peers`: Number of peers to join from the discovered set (default `5`). * `--cluster.name`: Name to prevent nodes without this identifier from joining the cluster (default `""`). -* `--config.format`: The format of the source file. Supported formats: `flow`, `prometheus`, `promtail`, `static` (default `"flow"`). +* `--config.format`: The format of the source file. Supported formats: `flow`, `otelcol`, `prometheus`, `promtail`, `static` (default `"flow"`). * `--config.bypass-conversion-errors`: Enable bypassing errors when converting (default `false`). * `--config.extra-args`: Extra arguments from the original format used by the converter. diff --git a/docs/sources/flow/reference/compatibility/_index.md b/docs/sources/flow/reference/compatibility/_index.md index 61775bcf26b5..30825a901f4b 100644 --- a/docs/sources/flow/reference/compatibility/_index.md +++ b/docs/sources/flow/reference/compatibility/_index.md @@ -44,6 +44,10 @@ The following components, grouped by namespace, _export_ Targets. +{{< collapse title="beyla" >}} +- [beyla.ebpf](../components/beyla.ebpf) +{{< /collapse >}} + {{< collapse title="discovery" >}} - [discovery.azure](../components/discovery.azure) - [discovery.consul](../components/discovery.consul) @@ -315,6 +319,10 @@ The following components, grouped by namespace, _consume_ OpenTelemetry `otelcol +{{< collapse title="beyla" >}} +- [beyla.ebpf](../components/beyla.ebpf) +{{< /collapse >}} + {{< collapse title="faro" >}} - [faro.receiver](../components/faro.receiver) {{< /collapse >}} diff --git a/docs/sources/flow/reference/components/beyla.ebpf.md b/docs/sources/flow/reference/components/beyla.ebpf.md new file mode 100644 index 000000000000..ad35b9ccba45 --- /dev/null +++ b/docs/sources/flow/reference/components/beyla.ebpf.md @@ -0,0 +1,276 @@ +--- +canonical: https://grafana.com/docs/agent/latest/flow/reference/components/beyla.ebpf/ +description: Learn about beyla.ebpf +title: beyla.ebpf +labels: + stage: beta +--- + +# beyla.ebpf + +{{< docs/shared lookup="flow/stability/beta.md" source="agent" version="" >}} + +The `beyla.ebpf` component is used as a wrapper for [Grafana Beyla][] which uses [eBPF][] to automatically inspect application executables and the OS networking layer, and capture trace spans related to web transactions and Rate Errors Duration (RED) metrics for Linux HTTP/S and gRPC services. +You can configure the component to collect telemetry data from a specific port or executable path, and other criteria from Kubernetes metadata. +The component exposes metrics that can be collected by a Prometheus scrape component, and traces that can be forwarded to an OTEL exporter component. + +{{< admonition type="note" >}} +To run this component, {{< param "PRODUCT_NAME" >}} requires administrative (`sudo`) privileges, or at least it needs to be granted the `CAP_SYS_ADMIN` and `CAP_SYS_PTRACE` capability. In Kubernetes environments, app armour must be disabled for the Deployment or DaemonSet running {{< param "PRODUCT_NAME" >}}. +{{< /admonition >}} + +[Grafana Beyla]: https://github.com/grafana/beyla +[eBPF]: https://ebpf.io/ + +## Usage + +```river +beyla.ebpf "