diff --git a/docs/sources/alert/_index.md b/docs/sources/alert/_index.md index 81a2671c2101..47bf991a565c 100644 --- a/docs/sources/alert/_index.md +++ b/docs/sources/alert/_index.md @@ -360,7 +360,7 @@ ruler: The Ruler supports the following types of storage: `azure`, `gcs`, `s3`, `swift`, `cos` and `local`. Most kinds of storage work with the sharded Ruler configuration in an obvious way, that is, configure all Rulers to use the same backend. -The local implementation reads the rule files off of the local filesystem. This is a read-only backend that does not support the creation and deletion of rules through the [Ruler API]({{< relref "../reference/api#ruler" >}}). Despite the fact that it reads the local filesystem this method can still be used in a sharded Ruler configuration if the operator takes care to load the same rules to every Ruler. For instance, this could be accomplished by mounting a [Kubernetes ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) onto every Ruler pod. +The local implementation reads the rule files off of the local filesystem. This is a read-only backend that does not support the creation and deletion of rules through the [Ruler API](https://grafana.com/docs/loki//reference/loki-http-api#ruler). Despite the fact that it reads the local filesystem this method can still be used in a sharded Ruler configuration if the operator takes care to load the same rules to every Ruler. For instance, this could be accomplished by mounting a [Kubernetes ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) onto every Ruler pod. A typical local configuration might look something like: ``` diff --git a/docs/sources/get-started/labels/bp-labels.md b/docs/sources/get-started/labels/bp-labels.md index c2bc2e925c23..7800345684ce 100644 --- a/docs/sources/get-started/labels/bp-labels.md +++ b/docs/sources/get-started/labels/bp-labels.md @@ -45,7 +45,7 @@ Loki has several client options: [Promtail]({{< relref "../../send-data/promtail Each of these come with ways to configure what labels are applied to create log streams. But be aware of what dynamic labels might be applied. Use the Loki series API to get an idea of what your log streams look like and see if there might be ways to reduce streams and cardinality. -Series information can be queried through the [Series API]({{< relref "../../reference/api" >}}), or you can use [logcli]({{< relref "../../query" >}}). +Series information can be queried through the [Series API](https://grafana.com/docs/loki//reference/loki-http-api/), or you can use [logcli]({{< relref "../../query" >}}). In Loki 1.6.0 and newer the logcli series command added the `--analyze-labels` flag specifically for debugging high cardinality labels: diff --git a/docs/sources/operations/storage/logs-deletion.md b/docs/sources/operations/storage/logs-deletion.md index cb0337c7bc90..eef99609ffd3 100644 --- a/docs/sources/operations/storage/logs-deletion.md +++ b/docs/sources/operations/storage/logs-deletion.md @@ -11,7 +11,7 @@ Log entries that fall within a specified time window and match an optional line Log entry deletion is supported _only_ when TSDB or BoltDB shipper is configured as the index store. -The compactor component exposes REST [endpoints]({{< relref "../../reference/api#compactor" >}}) that process delete requests. +The compactor component exposes REST [endpoints](https://grafana.com/docs/loki//reference/loki-http-api#compactor) that process delete requests. Hitting the endpoint specifies the streams and the time window. The deletion of the log entries takes place after a configurable cancellation time period expires. diff --git a/docs/sources/operations/storage/retention.md b/docs/sources/operations/storage/retention.md index f301dce87345..d21110978638 100644 --- a/docs/sources/operations/storage/retention.md +++ b/docs/sources/operations/storage/retention.md @@ -196,8 +196,8 @@ can be parsed using the Prometheus common model [ParseDuration](https://pkg.go.d {{% admonition type="warning" %}} The retention period must be a multiple of the index and chunks table -`period`, configured in the [`period_config`](https://grafana.com/docs/loki/latest/configure/#period_config) block. -See the [Table Manager](https://grafana.com/docs/loki/latest/operations/storage/table-manager/#retention) documentation for +`period`, configured in the [`period_config`](https://grafana.com/docs/loki//configure/#period_config) block. +See the [Table Manager](https://grafana.com/docs/loki//operations/storage/table-manager/#retention) documentation for more information. {{% /admonition %}} diff --git a/docs/sources/operations/storage/table-manager/_index.md b/docs/sources/operations/storage/table-manager/_index.md index 0e6ba42cc71f..625123f0bb0f 100644 --- a/docs/sources/operations/storage/table-manager/_index.md +++ b/docs/sources/operations/storage/table-manager/_index.md @@ -7,7 +7,7 @@ weight: # Table manager {{% admonition type="note" %}} -Table manager is only needed if you are using a multi-store [backend]({{< relref "../../storage" >}}). If you are using either TSDB (recommended), or BoltDB (deprecated) you do not need the Table Manager. +Table manager is only needed if you are using a multi-store [backend](https://grafana.com/docs/loki//configure/storage/). If you are using either TSDB (recommended), or BoltDB (deprecated) you do not need the Table Manager. {{% /admonition %}} Grafana Loki supports storing indexes and chunks in table-based data storages. When diff --git a/docs/sources/operations/storage/wal.md b/docs/sources/operations/storage/wal.md index 73034d3a76ac..2bf9010c948b 100644 --- a/docs/sources/operations/storage/wal.md +++ b/docs/sources/operations/storage/wal.md @@ -87,7 +87,7 @@ When scaling down, we must ensure existing data on the leaving ingesters are flu Consider you have 4 ingesters `ingester-0 ingester-1 ingester-2 ingester-3` and you want to scale down to 2 ingesters, the ingesters which will be shut down according to StatefulSet rules are `ingester-3` and then `ingester-2`. -Hence before actually scaling down in Kubernetes, port forward those ingesters and hit the [`/ingester/shutdown?flush=true`]({{< relref "../../reference/api#flush-in-memory-chunks-and-shut-down" >}}) endpoint. This will flush the chunks and remove itself from the ring, after which it will register as unready and may be deleted. +Hence before actually scaling down in Kubernetes, port forward those ingesters and hit the [`/ingester/shutdown?flush=true`](https://grafana.com/docs/loki//reference/loki-http-api#flush-in-memory-chunks-and-shut-down) endpoint. This will flush the chunks and remove itself from the ring, after which it will register as unready and may be deleted. After hitting the endpoint for `ingester-2 ingester-3`, scale down the ingesters to 2. diff --git a/docs/sources/reference/_index.md b/docs/sources/reference/_index.md index 89e8e38f7172..96c67836a5a1 100644 --- a/docs/sources/reference/_index.md +++ b/docs/sources/reference/_index.md @@ -9,4 +9,4 @@ weight: 1000 This section provides reference material for Loki. -1. [Loki HTTP API]({{< relref "./api" >}}) +1. [Loki HTTP API](https://grafana.com/docs/loki//reference/loki-http-api/) diff --git a/docs/sources/reference/api.md b/docs/sources/reference/loki-http-api.md similarity index 98% rename from docs/sources/reference/api.md rename to docs/sources/reference/loki-http-api.md index 5682510e4d86..96e88dd513e5 100644 --- a/docs/sources/reference/api.md +++ b/docs/sources/reference/loki-http-api.md @@ -1,10 +1,9 @@ --- title: Loki HTTP API -menuTitle: HTTP API description: Provides a reference page for the Loki HTTP API endpoints for data ingestion, data retrieval, and cluster management. aliases: - ../api/ -weight: 100 +weight: 500 --- # Loki HTTP API @@ -112,7 +111,7 @@ These HTTP endpoints are exposed by all individual components: ### Deprecated endpoints {{% admonition type="note" %}} -The following endpoints are deprecated. While they still exist and work, they should not be used for new deployments. +The following endpoints are deprecated.While they still exist and work, they should not be used for new deployments. Existing deployments should upgrade to use the supported endpoints. {{% /admonition %}} @@ -132,17 +131,17 @@ Existing deployments should upgrade to use the supported endpoints. Some Loki API endpoints return a result of a matrix, a vector, or a stream: - **Matrix**: a table of values where each row represents a different label set - and the columns are each sample values for that row over the queried time. - Matrix types are only returned when running a query that computes some value. +and the columns are each sample values for that row over the queried time. +Matrix types are only returned when running a query that computes some value. - **Instant Vector**: denoted in the type as just `vector`, an Instant Vector - represents the latest value of a calculation for a given labelset. Instant - Vectors are only returned when doing a query against a single point in - time. +represents the latest value of a calculation for a given labelset. Instant +Vectors are only returned when doing a query against a single point in +time. - **Stream**: a Stream is a set of all values (logs) for a given label set over the - queried time range. Streams are the only type that will result in log lines - being returned. +queried time range. Streams are the only type that will result in log lines +being returned. ### Timestamps @@ -270,7 +269,7 @@ GET /loki/api/v1/query This type of query is often referred to as an instant query. Instant queries are mostly used for metric type LogQL queries. It accepts the following query parameters in the URL: -- `query`: The [LogQL]({{< relref "../query" >}}) query to perform. Requests that do not use valid LogQL syntax will return errors. +- `query`: The [LogQL]({{< relref "../query" >}}) query to perform. Requests that do not use valid LogQL syntax will return errors. - `limit`: The max number of entries to return. It defaults to `100`. Only applies to query types which produce a stream (log lines) response. - `time`: The evaluation time for the query as a nanosecond Unix epoch or another [supported format](#timestamps). Defaults to now. - `direction`: Determines the sort order of logs. Supported values are `forward` or `backward`. Defaults to `backward`. diff --git a/docs/sources/release-notes/v3.0.md b/docs/sources/release-notes/v3.0.md index 7f0cc4e42b55..b89a68a5af1a 100644 --- a/docs/sources/release-notes/v3.0.md +++ b/docs/sources/release-notes/v3.0.md @@ -22,23 +22,23 @@ Key features in Loki 3.0.0 include the following: - **Lambda/Promtail:** support dropping labels ([#10755](https://github.com/grafana/loki/issues/10755)) ([ec54c72](https://github.com/grafana/loki/commit/ec54c723ebbeeda88000dde188d539ecfe05dad8)). -- **Docs improvements**: All the Getting Started topics have been revised, including a new [Quickstart](https://grafana.com/docs/loki/latest/get-started/quick-start/) to help new users get up and running with Loki faster.The Storage, Configuration Reference, and API documentation have been updated to reflect deprecated and removed code, configuration options, and API endpoints. +- **Docs improvements**: All the Getting Started topics have been revised, including a new [Quickstart](https://grafana.com/docs/loki//get-started/quick-start/) to help new users get up and running with Loki faster.The Storage, Configuration Reference, and API documentation have been updated to reflect deprecated and removed code, configuration options, and API endpoints. ## Deprecations One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removes a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include: -- [Deprecated storage options](https://grafana.com/docs/loki/latest/storage/) including the deprecation of the BoltDB store. +- [Deprecated storage options](https://grafana.com/docs/loki//storage/) including the deprecation of the BoltDB store. -- [Deprecated configuration options](https://grafana.com/docs/loki/next/configure/). +- [Deprecated configuration options](https://grafana.com/docs/loki//configure/). -- [API endpoint deprecations](https://grafana.com/docs/loki/next/reference/api/#deprecated-endpoints). +- [API endpoint deprecations](https://grafana.com/docs/loki//reference/api/#deprecated-endpoints). -To learn more about breaking changes in this release, refer to the [Upgrade guide](https://grafana.com/docs/loki/next/setup/upgrade/). +To learn more about breaking changes in this release, refer to the [Upgrade guide](https://grafana.com/docs/loki//setup/upgrade/). ## Upgrade Considerations -The path from 2.9 to 3.0 includes several breaking changes. For important upgrade guideance, refer to the [Upgrade Guide](https://grafana.com/docs/loki/latest/setup/upgrade/) and the separate [Helm Upgrade Guide](https://grafana.com/docs/loki/latest/setup/upgrade/upgrade-to-6x/). +The path from 2.9 to 3.0 includes several breaking changes. For important upgrade guidance, refer to the [Upgrade Guide](https://grafana.com/docs/loki//setup/upgrade/) and the separate [Helm Upgrade Guide](https://grafana.com/docs/loki//setup/upgrade/upgrade-to-6x/). ## Bug fixes diff --git a/docs/sources/send-data/fluentd/_index.md b/docs/sources/send-data/fluentd/_index.md index a2dffcac2dc3..61195d04dda2 100644 --- a/docs/sources/send-data/fluentd/_index.md +++ b/docs/sources/send-data/fluentd/_index.md @@ -189,7 +189,7 @@ This plugin automatically adds a `fluentd_thread` label with the name of the buf ### `url` -The URL of the Loki server to send logs to. When sending data, the publish path (`../reference/api/loki/v1/push`) will automatically be appended. +The URL of the Loki server to send logs to. When sending data, the publish path (`../reference/api/loki-http-api/v1/push/`) will automatically be appended. By default the url is set to `https://logs-prod-us-central1.grafana.net`, the url of the Grafana Labs [hosted Loki](/products/cloud/) service. #### Proxy Support diff --git a/docs/sources/send-data/k6/_index.md b/docs/sources/send-data/k6/_index.md index 0bd3f6c995ff..44e2f15ce17a 100644 --- a/docs/sources/send-data/k6/_index.md +++ b/docs/sources/send-data/k6/_index.md @@ -84,12 +84,12 @@ The `Client` class exposes the following instance methods: | method | description | | ------ | ----------- | | `push()` | shortcut for `pushParameterized(5, 800*1024, 1024*1024)` | -| `pushParameterized(streams, minSize, maxSize)` | execute push request ([POST /loki/api/v1/push]({{< relref "../../reference/api#ingest-logs" >}})) | -| `instantQuery(query, limit)` | execute instant query ([GET /loki/api/v1/query]({{< relref "../../reference/api#query-logs-at-a-single-point-in-time" >}})) | -| `client.rangeQuery(query, duration, limit)` | execute range query ([GET /loki/api/v1/query_range]({{< relref "../../reference/api#query-logs-within-a-range-of-time" >}})) | -| `client.labelsQuery(duration)` | execute labels query ([GET /loki/api/v1/labels]({{< relref "../../reference/api#query-labels" >}})) | -| `client.labelValuesQuery(label, duration)` | execute label values query ([GET /loki/api/v1/label/\/values]({{< relref "../../reference/api#query-label-values" >}})) | -| `client.seriesQuery(matchers, duration)` | execute series query ([GET /loki/api/v1/series]({{< relref "../../reference/api#query-streams" >}})) | +| `pushParameterized(streams, minSize, maxSize)` | execute push request ([POST /loki/api/v1/push](https://grafana.com/docs/loki//reference/loki-http-api#ingest-logs) | +| `instantQuery(query, limit)` | execute instant query ([GET /loki/api/v1/query](https://grafana.com/docs/loki//reference/loki-http-api#query-logs-at-a-single-point-in-time) | +| `client.rangeQuery(query, duration, limit)` | execute range query ([GET /loki/api/v1/query_range](https://grafana.com/docs/loki//reference/loki-http-api#query-logs-within-a-range-of-time) | +| `client.labelsQuery(duration)` | execute labels query ([GET /loki/api/v1/labels](https://grafana.com/docs/loki//reference/loki-http-api#query-labels) | +| `client.labelValuesQuery(label, duration)` | execute label values query ([GET /loki/api/v1/label/\/values](https://grafana.com/docs/loki//reference/loki-http-api#query-label-values) | +| `client.seriesQuery(matchers, duration)` | execute series query ([GET /loki/api/v1/series](https://grafana.com/docs/loki//reference/loki-http-api#query-streams) | **Javascript load test example:** diff --git a/docs/sources/send-data/k6/query-scenario.md b/docs/sources/send-data/k6/query-scenario.md index 40a914079ae4..d5b8d38275d4 100644 --- a/docs/sources/send-data/k6/query-scenario.md +++ b/docs/sources/send-data/k6/query-scenario.md @@ -23,7 +23,7 @@ Loki has 5 types of queries: In a real-world use-case, such as querying Loki using it as a Grafana data source, all of these queries are used. Each of them has a different -[API]({{< relref "../../reference/api.md" >}}) endpoint. The xk6-loki extension +[API](https://grafana.com/docs/loki//reference/loki-http-api/) endpoint. The xk6-loki extension provides a [Javascript API](https://github.com/grafana/xk6-loki#javascript-api) for all these query types. diff --git a/docs/sources/send-data/promtail/_index.md b/docs/sources/send-data/promtail/_index.md index 720718ea1a2d..dd12165face6 100644 --- a/docs/sources/send-data/promtail/_index.md +++ b/docs/sources/send-data/promtail/_index.md @@ -106,7 +106,7 @@ Important details are: ## Loki Push API -Promtail can also be configured to receive logs from another Promtail or any Loki client by exposing the [Loki Push API]({{< relref "../../reference/api#ingest-logs" >}}) with the [loki_push_api]({{< relref "./configuration#loki_push_api" >}}) scrape config. +Promtail can also be configured to receive logs from another Promtail or any Loki client by exposing the [Loki Push API](https://grafana.com/docs/loki//reference/loki-http-api#ingest-logs) with the [loki_push_api](https://grafana.com/docs/loki//reference/loki-http-api#loki_push_api) scrape config. There are a few instances where this might be helpful: diff --git a/docs/sources/send-data/promtail/configuration.md b/docs/sources/send-data/promtail/configuration.md index cbd5c7fc717e..ce1e329c7ea0 100644 --- a/docs/sources/send-data/promtail/configuration.md +++ b/docs/sources/send-data/promtail/configuration.md @@ -926,7 +926,7 @@ max_message_length: ### loki_push_api -The `loki_push_api` block configures Promtail to expose a [Loki push API]({{< relref "../../reference/api#ingest-logs" >}}) server. +The `loki_push_api` block configures Promtail to expose a [Loki push API](https://grafana.com/docs/loki//reference/loki-http-api#ingest-logs) server. Each job configured with a `loki_push_api` will expose this API and will require a separate port. diff --git a/docs/sources/setup/upgrade/_index.md b/docs/sources/setup/upgrade/_index.md index fd5e2f470f3c..a877f141ddcf 100644 --- a/docs/sources/setup/upgrade/_index.md +++ b/docs/sources/setup/upgrade/_index.md @@ -222,6 +222,18 @@ The previous default value `false` is applied. 1. `boltdb.shipper.compactor.deletion-mode` CLI flag and the corresponding YAML setting are removed. You can instead configure the `compactor.deletion-mode` CLI flag or `deletion_mode` YAML setting in [Limits Config](/docs/loki//configuration/#limits_config). 1. Compactor CLI flags that use the prefix `boltdb.shipper.compactor.` are removed. You can instead use CLI flags with the `compactor.` prefix. +#### Legacy ingester shutdown handler is removed + +The already deprecated handler `/ingester/flush_shutdown` is removed in favor of `/ingester/shutdown?flush=true`. + +#### Ingester configuration `max_transfer_retries` is removed. + +The setting `max_transfer_retries` (`-ingester.max-transfer-retries`) is removed in favor of the Write Ahead log (WAL). +It was used to allow transferring chunks to new ingesters when the old ingester was shutting down during a rolling restart. +Alternatives to this setting are: +- **A. (Preferred)** Enable the WAL and rely on the new ingester to replay the WAL. + - Optionally, you can enable `flush_on_shutdown` (`-ingester.flush-on-shutdown`) to flush to long-term storage on shutdowns. +- **B.** Manually flush during shutdowns via [the ingester `/shutdown?flush=true` endpoint](https://grafana.com/docs/loki//reference/loki-http-api#flush-in-memory-chunks-and-shut-down). #### Distributor metric changes diff --git a/docs/sources/setup/upgrade/upgrade-to-6x/index.md b/docs/sources/setup/upgrade/upgrade-to-6x/index.md index 48f4fde89062..ac57907e1f35 100644 --- a/docs/sources/setup/upgrade/upgrade-to-6x/index.md +++ b/docs/sources/setup/upgrade/upgrade-to-6x/index.md @@ -86,4 +86,4 @@ With these caches disabled, Loki will return to defaults which enables an in-mem #### Distributed mode -This chart introduces the ability to run Loki in distributed, or [microservices mode](https://grafana.com/docs/loki/latest/get-started/deployment-modes/#microservices-mode). Separate instructions on how to enable this as well as how to migrate from the existing community chart will be coming shortly. +This chart introduces the ability to run Loki in distributed, or [microservices mode](https://grafana.com/docs/loki//get-started/deployment-modes/#microservices-mode). Separate instructions on how to enable this as well as how to migrate from the existing community chart will be coming shortly.