Merge branch 'main' into dependabot/cargo/quickwit/tantivy-53f2fe1

docs/get-started/tutorials/trace-analytics-with-grafana.md

@@ -0,0 +1,96 @@
+---
+title: Trace analytics with Grafana
+description: A simple tutorial to use Grafana with Quickwit's datasource plugin.
+icon_url: /img/tutorials/quickwit-logo.png
+tags: [grafana, integration]
+sidebar_position: 2
+---
+
+In this tutorial, we will set up a Grafana Dashboard showing Quickwit traces using Docker Compose.
+
+You only need a few minutes to get Grafana working with Quickwit and build meaningful dashboards.
+
+## Create a Docker Compose recipe
+
+Let's add a [Quickwit instance](../installation.md) with the OTLP service enabled.
+
+```yaml
+version: '3.0'
+services:
+  quickwit:
+    image: quickwit/quickwit
+    environment:
+      QW_ENABLE_OPENTELEMETRY_OTLP_EXPORTER: "true"
+      OTEL_EXPORTER_OTLP_ENDPOINT: "http://localhost:7281"
+    ports:
+      - 7280:7280
+    command: ["run"]
+```
+
+Then we create a [Grafana](https://grafana.com/docs/grafana/latest/setup-grafana/installation/docker/#run-grafana-via-docker-compose) service with the [Quickwit Datasource](https://github.com/quickwit-oss/quickwit-datasource) plugin.
+
+```yaml
+  grafana:
+    image: grafana/grafana-oss
+    container_name: grafana
+    ports:
+      - "${MAP_HOST_GRAFANA:-127.0.0.1}:3000:3000"
+    environment:
+      GF_INSTALL_PLUGINS: https://github.com/quickwit-oss/quickwit-datasource/releases/download/v0.3.0-beta.2/quickwit-quickwit-datasource-0.3.0-beta.2.zip;quickwit-quickwit-datasource
+      GF_AUTH_DISABLE_LOGIN_FORM: "true"
+      GF_AUTH_ANONYMOUS_ENABLED: "true"
+      GF_AUTH_ANONYMOUS_ORG_ROLE: Admin
+```
+
+Save and run the recipe:
+
+```bash
+$ docker compose up
+```
+
+You should be able to access Quickwit's UI on `http://localhost:7280/` and Grafana's UI on `http://localhost:3000/`.
+
+## Setting up the datasource
+
+In Grafana, head to [Data Sources](http://localhost:3000/connections/datasources). If the plugin is installed correctly you should be able to find Quickwit in the list.
+
+We're going to set up a new Quickwit data source lookig at Quickwit's own OpenTelemetry traces, let's configure the datasource with the following parameters:
+
+- URL : `http://quickwit:7280/api/v1/` _This uses the docker service name as the host_
+- Index ID : `otel-traces-v0_6`
+- Timestamp field : `span_start_timestamp_nanos`
+- Timestamp output format : `unix_timestamp_nanos`
+
+Save and test, you should obtain a confirmation that the datasource is correctly set up.
+
+
+![Quickwit Plugin configuration success](../../assets/images/grafana-ui-quickwit-datasource-plugin-success.png)
+
+## Creating a dashboard
+
+You can then [create a new dashboard](http://localhost:3000/dashboard/new) and add a visualization : you should be able to choose our new quickwit datasource here.
+
+Quickwit sends itself its own traces, so you should already have data to display. Let's configure some panels !
+
+- a Table counting span_names 
+  - **Panel type** : Table
+  - **Query**: _empty_
+  - **Metric** : Count
+  - **Group by** : Terms : `span_name` : order by Count
+- a Bar Chart showing the amount of tantivy searches per hour :
+  - **Panel type**: Time Series
+  - **Query** : "span_name:tantivy_search"
+  - **Metric**: Count
+  - **Group by** : Date Histogram : `span_start_timestamp_nanos` : Interval 1h
+- a Bar Chart showing the amount of ERROR logs per hour for the last 6 hours :
+  - **Panel type**: Bar Chart
+  - **Query**: "service_name:quickwit AND events.event_attributes.level:ERROR"
+  - **Metric**: Count
+  - **Group by** : Terms : `span_start_timestamp_nanos` : Interval 1h
+- another query on the same Bar Chart for WARN logs
+
+## The result
+
+Here's what your first dashboard can look like :
+
+![Quickwit Panel in Grafana Dashboard](../../assets/images/screenshot-grafana-tutorial-dashboard.png)

docs/get-started/tutorials/tutorial-hdfs-logs-distributed-search-aws-s3.md

@@ -3,7 +3,7 @@ title: Distributed search on AWS S3
 description: Index log entries on AWS S3 using an EC2 instance and launch a distributed cluster.
 tags: [aws, integration]
 icon_url: /img/tutorials/aws-logo.png
-sidebar_position: 2
+sidebar_position: 4
 ---
 
 In this guide, we will index about 40 million log entries (13 GB decompressed) on AWS S3 using an EC2 instance and launch a three-node distributed search cluster.

docs/get-started/tutorials/tutorial-hdfs-logs.md

@@ -3,7 +3,7 @@ title: Index a logging dataset locally
 description: Index log entries on a local machine.
 tags: [self-hosted, setup]
 icon_url: /img/quickwit-icon.svg
-sidebar_position: 1
+sidebar_position: 3
 ---
 
 import Tabs from '@theme/Tabs';

docs/get-started/tutorials/tutorial-jaeger.md

@@ -0,0 +1,80 @@
+---
+title: Tracing with Jaeger
+sidebar_position: 1
+---
+
+In this quick start guide, we will set up a Quickwit instance and analyze its own traces with Jaeger using Docker Compose.
+
+You only need a minute to get Jaeger working with Quickwit storage backend.
+
+## Start Quickwit and Jaeger
+
+Let's use `docker compose` with the following configuration:
+
+```yaml title="docker-compose.yaml"
+version: "3"
+
+services:
+  quickwit:
+    image: quickwit/quickwit:${QW_VERSION:-0.6.5}
+    volumes:
+      - ./qwdata:/quickwit/qwdata
+    ports:
+      - 7280:7280
+    environment:
+      - QW_ENABLE_OPENTELEMETRY_OTLP_EXPORTER=true
+      - OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:7281
+    command: ["run"]
+
+  jaeger-query:
+    image: jaegertracing/jaeger-query:1.52
+    ports:
+      - 16686:16686
+    environment:
+      - SPAN_STORAGE_TYPE=grpc-plugin
+      - GRPC_STORAGE_SERVER=quickwit:7281
+      - GRPC_STORAGE_TLS=false
+```
+
+As you can see in the docker compose file, Quickwit is configured to send its own traces `OTEL_EXPORTER_OTLP_ENDPOINT` to itself `http://localhost:7281`.
+On the other side, Jaeger is configured to use a gRPC storage server `quickwit:7281`.
+
+Save and run the recipe:
+
+```bash
+$ docker compose up
+```
+
+You should be able to access Quickwit's UI on `http://localhost:7280/` and Jager's UI on `http://localhost:16686/`.
+
+
+## Searching and view traces in Jaeger
+
+Quickwit generates many traces, let's take a look at some of them:
+- `find_traces`: generated by the "Find traces" Jaeger button.
+- `get_operations`: generated by Jaeger when it is fetching the list of operations.
+- `get_services`: generated by Jaeger when it is fetching the list of services.
+- `ingest-spans`: generated when Quickwit receives spans on the gRPC OTLP API.
+- ...
+
+Here are the screenshots of the search and trace view:
+
+![Jaeger search view](../../assets/images/jaeger-ui-quickwit-search-traces.png)
+![Jaeger trace view](../../assets/images/jaeger-ui-quickwit-trace-view.png)
+
+## Searching traces with Quickwit UI
+
+You can also use the Quickwit UI at [http://localhost:7280](http://localhost:7280) to search traces.
+
+Here are a couple of query examples:
+- `service_name:quickwit AND events.event_attributes.level:INFO`
+- `span_duration_millis:>100`
+- `resource_attributes.service.version:v0.6.5`
+- `service_name:quickwit`
+
+That's it! You can level up with the following tutorials to discover all Quickwit features.
+
+## Next tutorials
+
+- [Send traces using an OTEL collector](/docs/distributed-tracing/send-traces/using-otel-collector.md)
+- [Send traces from a python web server](/docs/distributed-tracing/send-traces/using-otel-sdk-python.md)

docs/log-management/send-logs/send-docker-logs.md

@@ -8,7 +8,7 @@ sidebar_position: 5
 
 To send docker container logs into Quickwit, you just need to setup an OpenTelemetry Collector with the file logs receiver. In this tutorial, we will use `docker compose` to start the collector and Quickwit.
 
-You only needs a minute to get you Quickwit log UI!
+You only need a minute to get your Quickwit log UI!
 
 ![Quickwit UI Logs](../../assets/images/screenshot-quickwit-ui-docker-compose-logs.png)
 

docs/overview/introduction.md

@@ -27,7 +27,7 @@ Check out our guides to see how you can use Quickwit:
 
 - [Log management](../log-management/overview.md)
 - [Distributed Tracing](../distributed-tracing/overview.md)
-- Adding full-text search capabilities to [OLAP databases such as ClickHouse](../get-started/tutorials/add-full-text-search-to-your-olap-db).
+- Adding full-text search capabilities to [OLAP databases such as ClickHouse](../guides/add-full-text-search-to-your-olap-db).
 
 
 ## Key features