Skip to content

Commit

Permalink
docs: Manual backport of #12493 (#12557)
Browse files Browse the repository at this point in the history
  • Loading branch information
JStickler authored Apr 10, 2024
1 parent 6ea1e0d commit 414bc41
Show file tree
Hide file tree
Showing 16 changed files with 34 additions and 59 deletions.
2 changes: 1 addition & 1 deletion docs/sources/alert/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ ruler:

The Ruler supports the following types of storage: `azure`, `gcs`, `s3`, `swift`, `cos` and `local`. Most kinds of storage work with the sharded Ruler configuration in an obvious way, that is, configure all Rulers to use the same backend.

The local implementation reads the rule files off of the local filesystem. This is a read-only backend that does not support the creation and deletion of rules through the [Ruler API]({{< relref "../reference/api#ruler" >}}). Despite the fact that it reads the local filesystem this method can still be used in a sharded Ruler configuration if the operator takes care to load the same rules to every Ruler. For instance, this could be accomplished by mounting a [Kubernetes ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) onto every Ruler pod.
The local implementation reads the rule files off of the local filesystem. This is a read-only backend that does not support the creation and deletion of rules through the [Ruler API](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#ruler). Despite the fact that it reads the local filesystem this method can still be used in a sharded Ruler configuration if the operator takes care to load the same rules to every Ruler. For instance, this could be accomplished by mounting a [Kubernetes ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) onto every Ruler pod.

A typical local configuration might look something like:
```
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/get-started/labels/bp-labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Loki has several client options: [Promtail]({{< relref "../../send-data/promtail

Each of these come with ways to configure what labels are applied to create log streams. But be aware of what dynamic labels might be applied.
Use the Loki series API to get an idea of what your log streams look like and see if there might be ways to reduce streams and cardinality.
Series information can be queried through the [Series API]({{< relref "../../reference/api" >}}), or you can use [logcli]({{< relref "../../query" >}}).
Series information can be queried through the [Series API](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api/), or you can use [logcli]({{< relref "../../query" >}}).

In Loki 1.6.0 and newer the logcli series command added the `--analyze-labels` flag specifically for debugging high cardinality labels:

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/operations/storage/logs-deletion.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Log entries that fall within a specified time window and match an optional line

Log entry deletion is supported _only_ when TSDB or BoltDB shipper is configured as the index store.

The compactor component exposes REST [endpoints]({{< relref "../../reference/api#compactor" >}}) that process delete requests.
The compactor component exposes REST [endpoints](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#compactor) that process delete requests.
Hitting the endpoint specifies the streams and the time window.
The deletion of the log entries takes place after a configurable cancellation time period expires.

Expand Down
5 changes: 2 additions & 3 deletions docs/sources/operations/storage/retention.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,10 +196,9 @@ can be parsed using the Prometheus common model [ParseDuration](https://pkg.go.d

{{% admonition type="warning" %}}
The retention period must be a multiple of the index and chunks table
`period`, configured in the [`period_config`](https://grafana.com/docs/loki/latest/configure/#period_config) block.
See the [Table Manager](https://grafana.com/docs/loki/latest/operations/storage/table-manager/#retention) documentation for
`period`, configured in the [`period_config`](https://grafana.com/docs/loki/<LOKI_VERSION>/configure/#period_config) block.
See the [Table Manager](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/table-manager/#retention) documentation for
more information.
{{% /admonition %}}

{{% admonition type="note" %}}
To avoid querying of data beyond the retention period,
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/operations/storage/table-manager/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ weight:
# Table manager

{{% admonition type="note" %}}
Table manager is only needed if you are using a multi-store [backend]({{< relref "../../storage" >}}). If you are using either TSDB (recommended), or BoltDB (deprecated) you do not need the Table Manager.
Table manager is only needed if you are using a multi-store [backend](https://grafana.com/docs/loki/<LOKI_VERSION>/configure/storage/). If you are using either TSDB (recommended), or BoltDB (deprecated) you do not need the Table Manager.
{{% /admonition %}}

Grafana Loki supports storing indexes and chunks in table-based data storages. When
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/operations/storage/wal.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ When scaling down, we must ensure existing data on the leaving ingesters are flu

Consider you have 4 ingesters `ingester-0 ingester-1 ingester-2 ingester-3` and you want to scale down to 2 ingesters, the ingesters which will be shut down according to StatefulSet rules are `ingester-3` and then `ingester-2`.

Hence before actually scaling down in Kubernetes, port forward those ingesters and hit the [`/ingester/shutdown?flush=true`]({{< relref "../../reference/api#flush-in-memory-chunks-and-shut-down" >}}) endpoint. This will flush the chunks and remove itself from the ring, after which it will register as unready and may be deleted.
Hence before actually scaling down in Kubernetes, port forward those ingesters and hit the [`/ingester/shutdown?flush=true`](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#flush-in-memory-chunks-and-shut-down) endpoint. This will flush the chunks and remove itself from the ring, after which it will register as unready and may be deleted.

After hitting the endpoint for `ingester-2 ingester-3`, scale down the ingesters to 2.

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/reference/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ weight: 1000

This section provides reference material for Loki.

1. [Loki HTTP API]({{< relref "./api" >}})
- [Loki HTTP API](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api/)
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ menuTitle: HTTP API
description: Provides a reference page for the Loki HTTP API endpoints for data ingestion, data retrieval, and cluster management.
aliases:
- ../api/
weight: 100
weight: 500
---

# Loki HTTP API
Expand Down Expand Up @@ -112,7 +112,7 @@ These HTTP endpoints are exposed by all individual components:
### Deprecated endpoints

{{% admonition type="note" %}}
The following endpoints are deprecated. While they still exist and work, they should not be used for new deployments.
The following endpoints are deprecated. While they still exist and work, they should not be used for new deployments.
Existing deployments should upgrade to use the supported endpoints.
{{% /admonition %}}

Expand All @@ -132,17 +132,17 @@ Existing deployments should upgrade to use the supported endpoints.
Some Loki API endpoints return a result of a matrix, a vector, or a stream:

- **Matrix**: a table of values where each row represents a different label set
and the columns are each sample values for that row over the queried time.
Matrix types are only returned when running a query that computes some value.
and the columns are each sample values for that row over the queried time.
Matrix types are only returned when running a query that computes some value.

- **Instant Vector**: denoted in the type as just `vector`, an Instant Vector
represents the latest value of a calculation for a given labelset. Instant
Vectors are only returned when doing a query against a single point in
time.
represents the latest value of a calculation for a given labelset. Instant
Vectors are only returned when doing a query against a single point in
time.

- **Stream**: a Stream is a set of all values (logs) for a given label set over the
queried time range. Streams are the only type that will result in log lines
being returned.
queried time range. Streams are the only type that will result in log lines
being returned.

### Timestamps

Expand Down Expand Up @@ -270,7 +270,7 @@ GET /loki/api/v1/query
This type of query is often referred to as an instant query. Instant queries are mostly used for metric type LogQL queries.
It accepts the following query parameters in the URL:

- `query`: The [LogQL]({{< relref "../query" >}}) query to perform. Requests that do not use valid LogQL syntax will return errors.
- `query`: The [LogQL]({{< relref "../query" >}}) query to perform. Requests that do not use valid LogQL syntax will return errors.
- `limit`: The max number of entries to return. It defaults to `100`. Only applies to query types which produce a stream (log lines) response.
- `time`: The evaluation time for the query as a nanosecond Unix epoch or another [supported format](#timestamps). Defaults to now.
- `direction`: Determines the sort order of logs. Supported values are `forward` or `backward`. Defaults to `backward`.
Expand Down
6 changes: 3 additions & 3 deletions docs/sources/release-notes/v3-0.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ Key features in Loki 3.0.0 include the following:

- **Lambda/Promtail:** support dropping labels ([#10755](https://github.com/grafana/loki/issues/10755)) ([ec54c72](https://github.com/grafana/loki/commit/ec54c723ebbeeda88000dde188d539ecfe05dad8)).

- **Docs improvements**: All the Getting Started topics have been revised, including a new [Quickstart](https://grafana.com/docs/loki/latest/get-started/quick-start/) to help new users get up and running with Loki faster.The Storage, Configuration Reference, and API documentation have been updated to reflect deprecated and removed code, configuration options, and API endpoints.
- **Docs improvements**: All the Getting Started topics have been revised, including a new [Quickstart](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/quick-start/) to help new users get up and running with Loki faster.The Storage, Configuration Reference, and API documentation have been updated to reflect deprecated and removed code, configuration options, and API endpoints.

## Deprecations

One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removes a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:

- [Deprecated storage options](https://grafana.com/docs/loki/latest/storage/) including the deprecation of the BoltDB store.
- [Deprecated storage options](https://grafana.com/docs/loki/<LOKI_VERSION>/storage/) including the deprecation of the BoltDB store.

- [Deprecated configuration options](https://grafana.com/docs/loki/next/configure/).

Expand All @@ -38,7 +38,7 @@ To learn more about breaking changes in this release, refer to the [Upgrade guid

## Upgrade Considerations

The path from 2.9 to 3.0 includes several breaking changes. For important upgrade guideance, refer to the [Upgrade Guide](https://grafana.com/docs/loki/latest/setup/upgrade/) and the separate [Helm Upgrade Guide](https://grafana.com/docs/loki/latest/setup/upgrade/upgrade-to-6x/).
The path from 2.9 to 3.0 includes several breaking changes. For important upgrade guidance, refer to the [Upgrade Guide](https://grafana.com/docs/loki/<LOKI_VERSION>/setup/upgrade/) and the separate [Helm Upgrade Guide](https://grafana.com/docs/loki/<LOKI_VERSION>/setup/upgrade/upgrade-to-6x/).

## Bug fixes

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/send-data/fluentd/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ This plugin automatically adds a `fluentd_thread` label with the name of the buf

### `url`

The URL of the Loki server to send logs to. When sending data, the publish path (`../reference/api/loki/v1/push`) will automatically be appended.
The URL of the Loki server to send logs to. When sending data, the publish path (`../reference/api/loki-http-api/v1/push/`) will automatically be appended.
By default the url is set to `https://logs-prod-us-central1.grafana.net`, the url of the Grafana Labs [hosted Loki](/products/cloud/) service.

#### Proxy Support
Expand Down
12 changes: 6 additions & 6 deletions docs/sources/send-data/k6/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,12 +84,12 @@ The `Client` class exposes the following instance methods:
| method | description |
| ------ | ----------- |
| `push()` | shortcut for `pushParameterized(5, 800*1024, 1024*1024)` |
| `pushParameterized(streams, minSize, maxSize)` | execute push request ([POST /loki/api/v1/push]({{< relref "../../reference/api#ingest-logs" >}})) |
| `instantQuery(query, limit)` | execute instant query ([GET /loki/api/v1/query]({{< relref "../../reference/api#query-logs-at-a-single-point-in-time" >}})) |
| `client.rangeQuery(query, duration, limit)` | execute range query ([GET /loki/api/v1/query_range]({{< relref "../../reference/api#query-logs-within-a-range-of-time" >}})) |
| `client.labelsQuery(duration)` | execute labels query ([GET /loki/api/v1/labels]({{< relref "../../reference/api#query-labels" >}})) |
| `client.labelValuesQuery(label, duration)` | execute label values query ([GET /loki/api/v1/label/\<name\>/values]({{< relref "../../reference/api#query-label-values" >}})) |
| `client.seriesQuery(matchers, duration)` | execute series query ([GET /loki/api/v1/series]({{< relref "../../reference/api#query-streams" >}})) |
| `pushParameterized(streams, minSize, maxSize)` | execute push request ([POST /loki/api/v1/push](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#ingest-logs) |
| `instantQuery(query, limit)` | execute instant query ([GET /loki/api/v1/query](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#query-logs-at-a-single-point-in-time) |
| `client.rangeQuery(query, duration, limit)` | execute range query ([GET /loki/api/v1/query_range](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#query-logs-within-a-range-of-time) |
| `client.labelsQuery(duration)` | execute labels query ([GET /loki/api/v1/labels](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#query-labels) |
| `client.labelValuesQuery(label, duration)` | execute label values query ([GET /loki/api/v1/label/\<name\>/values](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#query-label-values) |
| `client.seriesQuery(matchers, duration)` | execute series query ([GET /loki/api/v1/series](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#query-streams) |
**Javascript load test example:**
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/send-data/k6/query-scenario.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Loki has 5 types of queries:

In a real-world use-case, such as querying Loki using it as a Grafana
data source, all of these queries are used. Each of them has a different
[API]({{< relref "../../reference/api.md" >}}) endpoint. The xk6-loki extension
[API](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api/) endpoint. The xk6-loki extension
provides a [Javascript API](https://github.com/grafana/xk6-loki#javascript-api)
for all these query types.

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/send-data/promtail/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ Important details are:

## Loki Push API

Promtail can also be configured to receive logs from another Promtail or any Loki client by exposing the [Loki Push API]({{< relref "../../reference/api#ingest-logs" >}}) with the [loki_push_api]({{< relref "./configuration#loki_push_api" >}}) scrape config.
Promtail can also be configured to receive logs from another Promtail or any Loki client by exposing the [Loki Push API](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#ingest-logs) with the [loki_push_api](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#loki_push_api) scrape config.

There are a few instances where this might be helpful:

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/send-data/promtail/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -926,7 +926,7 @@ max_message_length: <int>

### loki_push_api

The `loki_push_api` block configures Promtail to expose a [Loki push API]({{< relref "../../reference/api#ingest-logs" >}}) server.
The `loki_push_api` block configures Promtail to expose a [Loki push API](https://grafana.com/docs/loki/<LOKI_VERSION>/reference/loki-http-api#ingest-logs) server.

Each job configured with a `loki_push_api` will expose this API and will require a separate port.

Expand Down
28 changes: 2 additions & 26 deletions docs/sources/setup/upgrade/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ Here is the shortlist of things we think most people may encounter:
* The `shared_store` config is removed. Refer to [Removed `shared_store` and `shared_store_key_prefix` from shipper configuration](#removed-shared_store-and-shared_store_key_prefix-from-shipper-configuration).
* Loki now enforces a max line size of 256KB by default (you can disable this or increase this but this is how Grafana Labs runs Loki). Refer to [Changes to default configure values](#changes-to-default-configuration-values-in-30).
* Loki now enforces a max label limit of 15 labels per series, down from 30. Extra labels inflate the size of the index and reduce performance, you should almost never need more than 15 labels. Refer to [Changes to default configure values](#changes-to-default-configuration-values-in-30).
* Loki will automatically attempt to populate a `service_name` label on ingestion. Refer to [`service_name` label](#service_name-label).
* There are many metric name changes. Refer to [Distributor metric changes](#distributor-metric-changes), [Embedded cache metric changes](#embedded-cache-metric-changes), and [Metrics namespace](#metrics-namespace).

If you would like to see if your existing configuration will work with Loki 3.0:
Expand Down Expand Up @@ -83,29 +82,6 @@ Structured Metadata is enabled by default in Loki 3.0, however, it requires your
* Upgrade your index version and schema version before updating to 3.0, see [schema config upgrade](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/schema/).
* Disable Structured Metadata (and therefor OTLP support) and upgrade to 3.0 and perform the schema migration after. This can be done by setting `allow_structured_metadata: false` in the `limits_config` section or set the command line argument `-validation.allow-structured-metadata=false`.
#### `service_name` label
Loki 3.0 will automatically assign a `service_name` label to all ingested logs by default. A service name is something required by Open Telemetry semantic conventions and is something Grafana Labs is building into our future user interface and query experiences.
Loki will attempt to create the `service_name` label by looking for the following labels in this order:
- service
- app
- application
- name
- app_kubernetes_io_name
- container
- container_name
- component
- workload
- job
If no label is found matching the list, a value of `unknown_service` is applied.
You can change this list by providing a list of labels to `discover_service_name` in the [limits_config](/docs/loki/<LOKI_VERSION>/configure/#limits_config) block.
**You can disable this by providing an empty value for `discover_service_name`.**
#### Removed `shared_store` and `shared_store_key_prefix` from shipper configuration
The following CLI flags and the corresponding YAML settings to configure shared store for TSDB and BoltDB shippers are now removed:
Expand Down Expand Up @@ -699,7 +675,7 @@ Alerting rules previously could be specified in two formats: 1.x format (legacy
We decided to drop support for format `1.x` as it is fairly old and keeping support for it required a lot of code.
In case you're still using the legacy format, take a look at
[Alerting Rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/) for instructions
[Alerting Rules](https://prometheus.io/docs/prometheus/<LOKI_VERSION>/configuration/alerting_rules/) for instructions
on how to write alerting rules in the new format.

For reference, the newer format follows a structure similar to the one below:
Expand Down Expand Up @@ -1055,7 +1031,7 @@ In Loki 2.2 we changed the internal version of our chunk format from v2 to v3, t
This makes it important to first upgrade to 2.0, 2.0.1, or 2.1 **before** upgrading to 2.2 so that if you need to rollback for any reason you can do so easily.
{{% admonition type="note" %}}
2.0 and 2.0.1 are identical in every aspect except 2.0.1 contains the code necessary to read the v3 chunk format. Therefor if you are on 2.0 and ugrade to 2.2, if you want to rollback, you must rollback to 2.0.1.
2.0 and 2.0.1 are identical in every aspect except 2.0.1 contains the code necessary to read the v3 chunk format. Therefor if you are on 2.0 and upgrade to 2.2, if you want to rollback, you must rollback to 2.0.1.
{{% /admonition %}}
### Loki Config
Expand Down
Loading

0 comments on commit 414bc41

Please sign in to comment.