From 1caa3ba158bb21fa99fa3f0ebe86f76cae1c4aed Mon Sep 17 00:00:00 2001 From: Robert Fratto Date: Wed, 24 Jan 2024 13:44:41 -0500 Subject: [PATCH] docs/rfcs: remove Status line from RFCs (#6231) We haven't been updating the status line from RFCs to reflect their current status. This commit removes the status line from every RFC, with the exception of RFC-0003 which has been explicitly abandoned in favor of flow mode. --- docs/rfcs/0000-template.md | 1 - docs/rfcs/0001-designing-in-the-open.md | 1 - docs/rfcs/0002-integrations-in-operator.md | 1 - docs/rfcs/0003-new-metrics-subsystem.md | 2 +- docs/rfcs/0004-agent-flow.md | 91 +++++++++++----------- docs/rfcs/0005-river.md | 1 - docs/rfcs/0006-clustering.md | 13 ++-- docs/rfcs/0006-future-of-agent-operator.md | 3 +- docs/rfcs/0007-flow-modules.md | 11 ++- docs/rfcs/0008-backwards-compatibility.md | 1 - 10 files changed, 58 insertions(+), 67 deletions(-) diff --git a/docs/rfcs/0000-template.md b/docs/rfcs/0000-template.md index bbc01019c3ab..c565ea04e584 100644 --- a/docs/rfcs/0000-template.md +++ b/docs/rfcs/0000-template.md @@ -3,4 +3,3 @@ * Date: YYYY-MM-DD * Author: Full Name (@github_username) * PR: [grafana/agent#XXXX](https://github.com/grafana/agent/pull/XXXX) -* Status: Draft diff --git a/docs/rfcs/0001-designing-in-the-open.md b/docs/rfcs/0001-designing-in-the-open.md index 8f73f5d7a8d2..7419b060a375 100644 --- a/docs/rfcs/0001-designing-in-the-open.md +++ b/docs/rfcs/0001-designing-in-the-open.md @@ -3,7 +3,6 @@ * Date: 2021-11-02 * Author: Robert Fratto (@rfratto) * PR: [grafana/agent#1055](https://github.com/grafana/agent/pull/1055) -* Status: Implemented ## Summary diff --git a/docs/rfcs/0002-integrations-in-operator.md b/docs/rfcs/0002-integrations-in-operator.md index 8003606d3d46..ed54d40de6bd 100644 --- a/docs/rfcs/0002-integrations-in-operator.md +++ b/docs/rfcs/0002-integrations-in-operator.md @@ -3,7 +3,6 @@ * Date: 2022-01-04 * Author: Robert Fratto (@rfratto) * PR: [grafana/agent#1224](https://github.com/grafana/agent/pull/1224) -* Status: Draft ## Background diff --git a/docs/rfcs/0003-new-metrics-subsystem.md b/docs/rfcs/0003-new-metrics-subsystem.md index 961cd983e14a..336c0e4cc475 100644 --- a/docs/rfcs/0003-new-metrics-subsystem.md +++ b/docs/rfcs/0003-new-metrics-subsystem.md @@ -3,7 +3,7 @@ * Date: 2021-11-29 * Author: Robert Fratto (@rfratto) * PR: [grafana/agent#1140](https://github.com/grafana/agent/pull/1140) -* Status: Draft +* Status: Abandoned ## Background diff --git a/docs/rfcs/0004-agent-flow.md b/docs/rfcs/0004-agent-flow.md index db061af7a16b..3c1052e926ad 100644 --- a/docs/rfcs/0004-agent-flow.md +++ b/docs/rfcs/0004-agent-flow.md @@ -1,14 +1,13 @@ -# This provided the basis for Agent Flow, and though not all the concepts/ideas will make it into flow, it is good to have the historical context for why we started down this path. +# This provided the basis for Agent Flow, and though not all the concepts/ideas will make it into flow, it is good to have the historical context for why we started down this path. -# Agent Flow - Agent Utilizing Components +# Agent Flow - Agent Utilizing Components * Date: 2022-03-30 * Author: Matt Durham (@mattdurham) -* PRs: - * [grafana/agent#1538](https://github.com/grafana/agent/pull/1538) - Problem Statement +* PRs: + * [grafana/agent#1538](https://github.com/grafana/agent/pull/1538) - Problem Statement * [grafana/agent#1546](https://github.com/grafana/agent/pull/1546) - Messages and Expressions -* Status: Draft ## Overarching Problem Statement @@ -17,7 +16,7 @@ The Agents configuration and onboarding is difficult to use. Viewing the effect ## Description -Agent Flow is intended to solve real world needs that the Grafana Agent team have identified in conversations with users and developers. +Agent Flow is intended to solve real world needs that the Grafana Agent team have identified in conversations with users and developers. These broadly include: @@ -32,13 +31,13 @@ These broadly include: - Lack of understanding how telemetry data moves through agent - Other systems use pipeline/extensions to allow users to understand how data moves through the system -# 1. Introduction and Goals +# 1. Introduction and Goals -This design document outlines Agent Flow, a system for describing a programmable pipeline for telemetry data. +This design document outlines Agent Flow, a system for describing a programmable pipeline for telemetry data. Agent Flow refers to both the execution, configuration and visual configurator of data flow. -### Goals +### Goals * Allow users to more easily understand the impact of their configuration * Allow users to collect integration metrics across a set of agents @@ -55,43 +54,43 @@ Agent Flow refers to both the execution, configuration and visual configurator o At a high level, Agent Flow: -* Breaks apart the existing hierarchical configuration file into reusable components +* Breaks apart the existing hierarchical configuration file into reusable components * Allows components to be connected, resulting in a programmable pipeline of telemetry data -This document considers three potential approaches to allow users to connect components together: +This document considers three potential approaches to allow users to connect components together: -1. Message passing (i.e., an actor model) +1. Message passing (i.e., an actor model) 2. Expressions (i.e., directly referencing the output of another component) -3. A hybrid of both messages and expressions +3. A hybrid of both messages and expressions -The Flow Should in general resemble a flowchart or node graph. The data flow diagram would conceptually look like the below, with each node being composable and connecting with other nodes. +The Flow Should in general resemble a flowchart or node graph. The data flow diagram would conceptually look like the below, with each node being composable and connecting with other nodes. ``` -┌─────────────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ ┌───────────────────┐ -│ │ ┌─────▶│ Target Filter │─────────▶│ Redis Integration │──────▶│ Metric Filter │──┐ -│ │ │ └──────────────────┘ └─────────────────────┘ └───────────────────┘ │ -│ Service Discovery │──────┤ │ -│ │ │ │ -│ │ │ │ -└─────────────────────────┘ │ ┌─────────────────┐ ┌──────────────────────┐ ┌────────┘ - ├─────▶│ Target Filter │──────────▶│ MySQL Integrations │───────────┐ │ - │ └─────────────────┘ └──────────────────────┘ │ │ - │ │ │ - │ ┌─────────────────┐ ┌─────────────┐ │ │ +┌─────────────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ ┌───────────────────┐ +│ │ ┌─────▶│ Target Filter │─────────▶│ Redis Integration │──────▶│ Metric Filter │──┐ +│ │ │ └──────────────────┘ └─────────────────────┘ └───────────────────┘ │ +│ Service Discovery │──────┤ │ +│ │ │ │ +│ │ │ │ +└─────────────────────────┘ │ ┌─────────────────┐ ┌──────────────────────┐ ┌────────┘ + ├─────▶│ Target Filter │──────────▶│ MySQL Integrations │───────────┐ │ + │ └─────────────────┘ └──────────────────────┘ │ │ + │ │ │ + │ ┌─────────────────┐ ┌─────────────┐ │ │ └──────▶│ Target Filter │─────────────▶│ Scraper │─────────────┐ │ │ ┌────────────────┐ └─────────────────┘ └─────────────┘ └──┴┬───────┴─▶│ Remote Write │ │ └────────────────┘ - │ - │ -┌──────────────────────────┐ │ -│ Remote Write Receiver │─────┐ ┌───────────────────────┐ │ -└──────────────────────────┘ │ ┌────▶│ Metric Transformer │─────────┘ - │ │ └───────────────────────┘ - │ │ -┌─────────────────────────┐ │ ┌────────────────────┐ │ -│ HTTP Receiver │──────┴─────▶│ Metric Filter │────┘ ┌──────────────────────────────────┐ -└─────────────────────────┘ └────────────────────┘ │ Global and Server Settings │ - └──────────────────────────────────┘ + │ + │ +┌──────────────────────────┐ │ +│ Remote Write Receiver │─────┐ ┌───────────────────────┐ │ +└──────────────────────────┘ │ ┌────▶│ Metric Transformer │─────────┘ + │ │ └───────────────────────┘ + │ │ +┌─────────────────────────┐ │ ┌────────────────────┐ │ +│ HTTP Receiver │──────┴─────▶│ Metric Filter │────┘ ┌──────────────────────────────────┐ +└─────────────────────────┘ └────────────────────┘ │ Global and Server Settings │ + └──────────────────────────────────┘ ``` **Note: Consider all examples pseudoconfig** @@ -107,14 +106,14 @@ Expression based is writing expressions that allow referencing other components **Cons** * Harder for users to wire things together - * References to components are more complex, which may be harder to understand + * References to components are more complex, which may be harder to understand * Harder to build a GUI for * Every field of a component is potentially dynamic, making it harder to represent visually ## 2.2 Message Based -Message based is where components have no knowledge of other components and information is passed strictly via input and output streams. +Message based is where components have no knowledge of other components and information is passed strictly via input and output streams. **Pros** @@ -122,7 +121,7 @@ Message based is where components have no knowledge of other components and info * Easier to build a GUI for * Inputs and Outputs are well defined and less granular * Connections are made by connecting two components directly, compared to expressions which connect subsets of a component's output -* References between components are no more than strings, making the text-based representation language agnostic (e.g., it could be YAML, JSON, or any language) +* References between components are no more than strings, making the text-based representation language agnostic (e.g., it could be YAML, JSON, or any language) **Cons** @@ -130,16 +129,16 @@ Message based is where components have no knowledge of other components and info * Larger type system needed * More structured to keep the amount of types down -Messages require a more rigid type structure to minimize the number of total components. +Messages require a more rigid type structure to minimize the number of total components. For example, it would be preferable to have a single `Credential` type that can be emitted by an s3, Vault, or Consul component. These components would then need to set a field that marks their output as a specific kind of Credential (such as Basic Auth or Bearer Auth). If, instead, you had multiple Credential types, like `MySQLCredentials` and `RedisCredentials`, you would have the following components: -* Vault component for MySQL credentials -* Vault component for Redis credentials -* S3 component for MySQL credentials -* S3 component for Redis credentials +* Vault component for MySQL credentials +* Vault component for Redis credentials +* S3 component for MySQL credentials +* S3 component for Redis credentials * (and so on) ## 2.3 Hybrid @@ -157,10 +156,10 @@ discovery "mysql_pods" { integration "mysql" { - # Create one mysql integration for every element in the array here + # Create one mysql integration for every element in the array here for_each = discovery.mysql_pods.targets - # Each spawned mysql integration has its data_source_name derived from + # Each spawned mysql integration has its data_source_name derived from # the address label of the input target. data_source_name = "root@(${each.labels["__address__"]})" } diff --git a/docs/rfcs/0005-river.md b/docs/rfcs/0005-river.md index 8f4e3e12299b..3fa82a5f7eb2 100644 --- a/docs/rfcs/0005-river.md +++ b/docs/rfcs/0005-river.md @@ -3,7 +3,6 @@ * Date: 2022-06-27 * Author: Robert Fratto (@rfratto), Matt Durham (@mattdurham) * PR: [grafana/agent#1839](https://github.com/grafana/agent/pull/1839) -* Status: Draft ## Summary diff --git a/docs/rfcs/0006-clustering.md b/docs/rfcs/0006-clustering.md index b6c08b2bc210..b29070410e47 100644 --- a/docs/rfcs/0006-clustering.md +++ b/docs/rfcs/0006-clustering.md @@ -3,7 +3,6 @@ * Date: 2023-03-02 * Author: Paschalis Tsilias (@tpaschalis) * PR: [grafana/agent#3151](https://github.com/grafana/agent/pull/3151) -* Status: Draft ## Summary - Background We routinely run agents with 1-10 million active series; we regularly see @@ -98,7 +97,7 @@ presented in the next section. ## Use cases In the first iteration of agent clustering, we would like to start with the following use-cases. These two are distinct in the way that they make use of -scheduling. +scheduling. The first one makes sure that we have a way of notifying components of cluster changes and calling their Update method and continuously re-evaluate ownership @@ -112,9 +111,9 @@ it is scraping/reading logs from. Components that use the Flow concept of a “target” as their Arguments should be able to distribute the target load between themselves. To do that we can introduce a layer of abstraction over the Targets definition that can interact with the Sharder provided by the -clusterer and provide a simple API, for example: +clusterer and provide a simple API, for example: ```go -type Targets interface { +type Targets interface { Get() []Target } ``` @@ -136,9 +135,9 @@ I propose that we start with the following set of components that make use of this functionality: prometheus.scrape, loki.source.file, loki.source.kubernetes, and pyroscope.scrape. -Here’s how the configuration for a component could look like: +Here’s how the configuration for a component could look like: ```river -prometheus.scrape "pods" { +prometheus.scrape "pods" { clustering { node_updates = true } @@ -200,7 +199,7 @@ information. On a more practical note, we’ll have to choose how components might use to opt-in to the component scheduling. -For example, we could implement either: +For example, we could implement either: * Implicitly adding a new Argument block that is implicitly present by default on _all_ components: ``` diff --git a/docs/rfcs/0006-future-of-agent-operator.md b/docs/rfcs/0006-future-of-agent-operator.md index e0ed4bef9304..3a5c3d2e5611 100644 --- a/docs/rfcs/0006-future-of-agent-operator.md +++ b/docs/rfcs/0006-future-of-agent-operator.md @@ -3,7 +3,6 @@ * Date: 2022-08-17 * Author: Craig Peterson (@captncraig) * PR: [grafana/agent#2046](https://github.com/grafana/agent/pull/2046) -* Status: Draft ## Summary @@ -31,6 +30,6 @@ The operator is a fairly complex piece of code, and has been slower than some ot ## Beta status -The Grafana Agent Operator is still considered beta software. It has received a better reception than anticipated, and is now an important part of the Agent project. We are committed to supporting the Operator into the future, but are going to leave the beta designation in place while making larger refactorings as described above. We make efforts to avoid breaking changes, and hope that custom resource definitions will remain compatible, but it is possible some changes will be necessary. We will make every effort to justify and communicate such scenarios as they arise. +The Grafana Agent Operator is still considered beta software. It has received a better reception than anticipated, and is now an important part of the Agent project. We are committed to supporting the Operator into the future, but are going to leave the beta designation in place while making larger refactorings as described above. We make efforts to avoid breaking changes, and hope that custom resource definitions will remain compatible, but it is possible some changes will be necessary. We will make every effort to justify and communicate such scenarios as they arise. Once we are confident we have an Operator we are happy with and that the resource definitions are stable, we will revisit the beta status as soon as we can. diff --git a/docs/rfcs/0007-flow-modules.md b/docs/rfcs/0007-flow-modules.md index f08fb74c0f3b..5058663dd3ba 100644 --- a/docs/rfcs/0007-flow-modules.md +++ b/docs/rfcs/0007-flow-modules.md @@ -3,7 +3,6 @@ * Date: 2023-01-27 * Author: Matt Durham @mattdurham * PR: [grafana/agent#2898](https://github.com/grafana/agent/pull/2898) -* Status: Draft [Formatted Link for ease of user](https://github.com/grafana/agent/blob/rfc_modules/docs/rfcs/0007-flow-modules.md) @@ -30,7 +29,7 @@ During this time the Agent team saw a lot of potential in the idea of "modules." ### Enable re-use of common patterns -Common functionality can be wrapped in a set of common components that form a module. These shared modules can then be used instead of reinventing use cases. +Common functionality can be wrapped in a set of common components that form a module. These shared modules can then be used instead of reinventing use cases. ### Allow loading a module from a string @@ -42,11 +41,11 @@ Modules will be able to load other modules, with reasonable safe guards. There w ### Modules should be sandboxed except via arguments and exports -Modules cannot directly access children or parent modules except through predefined arguments and exports. +Modules cannot directly access children or parent modules except through predefined arguments and exports. ## Non Goals -Non goals represent capabilities that are not going to be done in the initial release of modules but may come in later versions. +Non goals represent capabilities that are not going to be done in the initial release of modules but may come in later versions. * Add additional capabilities to load strings * Any type of versioning @@ -66,7 +65,7 @@ Modules will not contain any sort of versioning nor will check for compatibility ### Any user interface work beyond ensuring it works as the UI currently does -Users will not be able to drill into modules, they will be represented as any other normal component. +Users will not be able to drill into modules, they will be represented as any other normal component. ## Example @@ -122,7 +121,7 @@ prometheus.scrape "scraper" { * A module cannot directly or indirectly load itself, this will not be enforced by the system * Singleton components are not supported at this time. Example [node_exporter](https://grafana.com/docs/agent/latest/flow/reference/components/prometheus.integration.node_exporter/). * Modules will not prevent competing resources, such as starting a server on the same port -* [Configuration blocks](https://grafana.com/docs/agent/latest/flow/reference/config-blocks/#configuration-blocks) will not be supported. +* [Configuration blocks](https://grafana.com/docs/agent/latest/flow/reference/config-blocks/#configuration-blocks) will not be supported. * Names of arguments and exports within a module must be unique across that module. ## Proposal diff --git a/docs/rfcs/0008-backwards-compatibility.md b/docs/rfcs/0008-backwards-compatibility.md index 147490f41e40..56d4bac647a3 100644 --- a/docs/rfcs/0008-backwards-compatibility.md +++ b/docs/rfcs/0008-backwards-compatibility.md @@ -3,7 +3,6 @@ * Date: 2023-05-25 * Author: Robert Fratto (@rfratto) * PR: [grafana/agent#3981](https://github.com/grafana/agent/pull/3981) -* Status: Draft Grafana Agent has been following [semantic versioning](https://semver.org/) since its inception. After three years of development and 33 minor releases, the project is on trajectory to have a 1.0 release.