Skip to content

Commit

Permalink
docs/rfcs: remove Status line from RFCs (#6231)
Browse files Browse the repository at this point in the history
We haven't been updating the status line from RFCs to reflect their
current status. This commit removes the status line from every RFC, with
the exception of RFC-0003 which has been explicitly abandoned in favor
of flow mode.
  • Loading branch information
rfratto authored Jan 24, 2024
1 parent 78a2632 commit 1caa3ba
Show file tree
Hide file tree
Showing 10 changed files with 58 additions and 67 deletions.
1 change: 0 additions & 1 deletion docs/rfcs/0000-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,3 @@
* Date: YYYY-MM-DD
* Author: Full Name (@github_username)
* PR: [grafana/agent#XXXX](https://github.com/grafana/agent/pull/XXXX)
* Status: Draft
1 change: 0 additions & 1 deletion docs/rfcs/0001-designing-in-the-open.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
* Date: 2021-11-02
* Author: Robert Fratto (@rfratto)
* PR: [grafana/agent#1055](https://github.com/grafana/agent/pull/1055)
* Status: Implemented

## Summary

Expand Down
1 change: 0 additions & 1 deletion docs/rfcs/0002-integrations-in-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
* Date: 2022-01-04
* Author: Robert Fratto (@rfratto)
* PR: [grafana/agent#1224](https://github.com/grafana/agent/pull/1224)
* Status: Draft

## Background

Expand Down
2 changes: 1 addition & 1 deletion docs/rfcs/0003-new-metrics-subsystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* Date: 2021-11-29
* Author: Robert Fratto (@rfratto)
* PR: [grafana/agent#1140](https://github.com/grafana/agent/pull/1140)
* Status: Draft
* Status: Abandoned

## Background

Expand Down
91 changes: 45 additions & 46 deletions docs/rfcs/0004-agent-flow.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
# This provided the basis for Agent Flow, and though not all the concepts/ideas will make it into flow, it is good to have the historical context for why we started down this path.
# This provided the basis for Agent Flow, and though not all the concepts/ideas will make it into flow, it is good to have the historical context for why we started down this path.

# Agent Flow - Agent Utilizing Components
# Agent Flow - Agent Utilizing Components


* Date: 2022-03-30
* Author: Matt Durham (@mattdurham)
* PRs:
* [grafana/agent#1538](https://github.com/grafana/agent/pull/1538) - Problem Statement
* PRs:
* [grafana/agent#1538](https://github.com/grafana/agent/pull/1538) - Problem Statement
* [grafana/agent#1546](https://github.com/grafana/agent/pull/1546) - Messages and Expressions
* Status: Draft

## Overarching Problem Statement

Expand All @@ -17,7 +16,7 @@ The Agents configuration and onboarding is difficult to use. Viewing the effect

## Description

Agent Flow is intended to solve real world needs that the Grafana Agent team have identified in conversations with users and developers.
Agent Flow is intended to solve real world needs that the Grafana Agent team have identified in conversations with users and developers.

These broadly include:

Expand All @@ -32,13 +31,13 @@ These broadly include:
- Lack of understanding how telemetry data moves through agent
- Other systems use pipeline/extensions to allow users to understand how data moves through the system

# 1. Introduction and Goals
# 1. Introduction and Goals

This design document outlines Agent Flow, a system for describing a programmable pipeline for telemetry data.
This design document outlines Agent Flow, a system for describing a programmable pipeline for telemetry data.

Agent Flow refers to both the execution, configuration and visual configurator of data flow.

### Goals
### Goals

* Allow users to more easily understand the impact of their configuration
* Allow users to collect integration metrics across a set of agents
Expand All @@ -55,43 +54,43 @@ Agent Flow refers to both the execution, configuration and visual configurator o

At a high level, Agent Flow:

* Breaks apart the existing hierarchical configuration file into reusable components
* Breaks apart the existing hierarchical configuration file into reusable components
* Allows components to be connected, resulting in a programmable pipeline of telemetry data

This document considers three potential approaches to allow users to connect components together:
This document considers three potential approaches to allow users to connect components together:

1. Message passing (i.e., an actor model)
1. Message passing (i.e., an actor model)
2. Expressions (i.e., directly referencing the output of another component)
3. A hybrid of both messages and expressions
3. A hybrid of both messages and expressions

The Flow Should in general resemble a flowchart or node graph. The data flow diagram would conceptually look like the below, with each node being composable and connecting with other nodes.
The Flow Should in general resemble a flowchart or node graph. The data flow diagram would conceptually look like the below, with each node being composable and connecting with other nodes.

```
┌─────────────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ ┌───────────────────┐
│ │ ┌─────▶│ Target Filter │─────────▶│ Redis Integration │──────▶│ Metric Filter │──┐
│ │ │ └──────────────────┘ └─────────────────────┘ └───────────────────┘ │
│ Service Discovery │──────┤ │
│ │ │ │
│ │ │ │
└─────────────────────────┘ │ ┌─────────────────┐ ┌──────────────────────┐ ┌────────┘
├─────▶│ Target Filter │──────────▶│ MySQL Integrations │───────────┐ │
│ └─────────────────┘ └──────────────────────┘ │ │
│ │ │
│ ┌─────────────────┐ ┌─────────────┐ │ │
┌─────────────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ ┌───────────────────┐
│ │ ┌─────▶│ Target Filter │─────────▶│ Redis Integration │──────▶│ Metric Filter │──┐
│ │ │ └──────────────────┘ └─────────────────────┘ └───────────────────┘ │
│ Service Discovery │──────┤ │
│ │ │ │
│ │ │ │
└─────────────────────────┘ │ ┌─────────────────┐ ┌──────────────────────┐ ┌────────┘
├─────▶│ Target Filter │──────────▶│ MySQL Integrations │───────────┐ │
│ └─────────────────┘ └──────────────────────┘ │ │
│ │ │
│ ┌─────────────────┐ ┌─────────────┐ │ │
└──────▶│ Target Filter │─────────────▶│ Scraper │─────────────┐ │ │ ┌────────────────┐
└─────────────────┘ └─────────────┘ └──┴┬───────┴─▶│ Remote Write │
│ └────────────────┘
┌──────────────────────────┐ │
│ Remote Write Receiver │─────┐ ┌───────────────────────┐ │
└──────────────────────────┘ │ ┌────▶│ Metric Transformer │─────────┘
│ │ └───────────────────────┘
│ │
┌─────────────────────────┐ │ ┌────────────────────┐ │
│ HTTP Receiver │──────┴─────▶│ Metric Filter │────┘ ┌──────────────────────────────────┐
└─────────────────────────┘ └────────────────────┘ │ Global and Server Settings │
└──────────────────────────────────┘
┌──────────────────────────┐ │
│ Remote Write Receiver │─────┐ ┌───────────────────────┐ │
└──────────────────────────┘ │ ┌────▶│ Metric Transformer │─────────┘
│ │ └───────────────────────┘
│ │
┌─────────────────────────┐ │ ┌────────────────────┐ │
│ HTTP Receiver │──────┴─────▶│ Metric Filter │────┘ ┌──────────────────────────────────┐
└─────────────────────────┘ └────────────────────┘ │ Global and Server Settings │
└──────────────────────────────────┘
```

**Note: Consider all examples pseudoconfig**
Expand All @@ -107,39 +106,39 @@ Expression based is writing expressions that allow referencing other components

**Cons**
* Harder for users to wire things together
* References to components are more complex, which may be harder to understand
* References to components are more complex, which may be harder to understand
* Harder to build a GUI for
* Every field of a component is potentially dynamic, making it harder to represent visually


## 2.2 Message Based

Message based is where components have no knowledge of other components and information is passed strictly via input and output streams.
Message based is where components have no knowledge of other components and information is passed strictly via input and output streams.

**Pros**

* Easier for users to understand the dependencies between components
* Easier to build a GUI for
* Inputs and Outputs are well defined and less granular
* Connections are made by connecting two components directly, compared to expressions which connect subsets of a component's output
* References between components are no more than strings, making the text-based representation language agnostic (e.g., it could be YAML, JSON, or any language)
* References between components are no more than strings, making the text-based representation language agnostic (e.g., it could be YAML, JSON, or any language)

**Cons**

* More time consuming to implement, existing integrations/items would need to be componentized
* Larger type system needed
* More structured to keep the amount of types down

Messages require a more rigid type structure to minimize the number of total components.
Messages require a more rigid type structure to minimize the number of total components.

For example, it would be preferable to have a single `Credential` type that can be emitted by an s3, Vault, or Consul component. These components would then need to set a field that marks their output as a specific kind of Credential (such as Basic Auth or Bearer Auth).

If, instead, you had multiple Credential types, like `MySQLCredentials` and `RedisCredentials`, you would have the following components:

* Vault component for MySQL credentials
* Vault component for Redis credentials
* S3 component for MySQL credentials
* S3 component for Redis credentials
* Vault component for MySQL credentials
* Vault component for Redis credentials
* S3 component for MySQL credentials
* S3 component for Redis credentials
* (and so on)

## 2.3 Hybrid
Expand All @@ -157,10 +156,10 @@ discovery "mysql_pods" {
integration "mysql" {
# Create one mysql integration for every element in the array here
# Create one mysql integration for every element in the array here
for_each = discovery.mysql_pods.targets
# Each spawned mysql integration has its data_source_name derived from
# Each spawned mysql integration has its data_source_name derived from
# the address label of the input target.
data_source_name = "root@(${each.labels["__address__"]})"
}
Expand Down
1 change: 0 additions & 1 deletion docs/rfcs/0005-river.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
* Date: 2022-06-27
* Author: Robert Fratto (@rfratto), Matt Durham (@mattdurham)
* PR: [grafana/agent#1839](https://github.com/grafana/agent/pull/1839)
* Status: Draft

## Summary

Expand Down
13 changes: 6 additions & 7 deletions docs/rfcs/0006-clustering.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
* Date: 2023-03-02
* Author: Paschalis Tsilias (@tpaschalis)
* PR: [grafana/agent#3151](https://github.com/grafana/agent/pull/3151)
* Status: Draft

## Summary - Background
We routinely run agents with 1-10 million active series; we regularly see
Expand Down Expand Up @@ -98,7 +97,7 @@ presented in the next section.
## Use cases
In the first iteration of agent clustering, we would like to start with the
following use-cases. These two are distinct in the way that they make use of
scheduling.
scheduling.

The first one makes sure that we have a way of notifying components of cluster
changes and calling their Update method and continuously re-evaluate ownership
Expand All @@ -112,9 +111,9 @@ it is scraping/reading logs from. Components that use the Flow concept of a
“target” as their Arguments should be able to distribute the target load
between themselves. To do that we can introduce a layer of abstraction over
the Targets definition that can interact with the Sharder provided by the
clusterer and provide a simple API, for example:
clusterer and provide a simple API, for example:
```go
type Targets interface {
type Targets interface {
Get() []Target
}
```
Expand All @@ -136,9 +135,9 @@ I propose that we start with the following set of components that make use of
this functionality: prometheus.scrape, loki.source.file,
loki.source.kubernetes, and pyroscope.scrape.

Here’s how the configuration for a component could look like:
Here’s how the configuration for a component could look like:
```river
prometheus.scrape "pods" {
prometheus.scrape "pods" {
clustering {
node_updates = true
}
Expand Down Expand Up @@ -200,7 +199,7 @@ information.
On a more practical note, we’ll have to choose how components might use to
opt-in to the component scheduling.

For example, we could implement either:
For example, we could implement either:
* Implicitly adding a new Argument block that is implicitly present by default on
_all_ components:
```
Expand Down
3 changes: 1 addition & 2 deletions docs/rfcs/0006-future-of-agent-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
* Date: 2022-08-17
* Author: Craig Peterson (@captncraig)
* PR: [grafana/agent#2046](https://github.com/grafana/agent/pull/2046)
* Status: Draft

## Summary

Expand Down Expand Up @@ -31,6 +30,6 @@ The operator is a fairly complex piece of code, and has been slower than some ot

## Beta status

The Grafana Agent Operator is still considered beta software. It has received a better reception than anticipated, and is now an important part of the Agent project. We are committed to supporting the Operator into the future, but are going to leave the beta designation in place while making larger refactorings as described above. We make efforts to avoid breaking changes, and hope that custom resource definitions will remain compatible, but it is possible some changes will be necessary. We will make every effort to justify and communicate such scenarios as they arise.
The Grafana Agent Operator is still considered beta software. It has received a better reception than anticipated, and is now an important part of the Agent project. We are committed to supporting the Operator into the future, but are going to leave the beta designation in place while making larger refactorings as described above. We make efforts to avoid breaking changes, and hope that custom resource definitions will remain compatible, but it is possible some changes will be necessary. We will make every effort to justify and communicate such scenarios as they arise.

Once we are confident we have an Operator we are happy with and that the resource definitions are stable, we will revisit the beta status as soon as we can.
Loading

0 comments on commit 1caa3ba

Please sign in to comment.