chore(release-3.1.x): release 3.0.0-alpha.1

[loki-gh-app]

🤖 I have created a release beep boop

3.0.0-alpha.1

3.0.0-alpha.1 (2024-10-18)

⚠ BREAKING CHANGES

• update helm chart to support distributed mode and 3.0 (#12067)
• remove async cache writeback on chunk fetcher (#12456)
• enable structure metadata by default. (#12453)
• refactor how we do defaults for runtime overrides (#12448)

Features

• Add a version of the mixin dashboards for meta monitoring (#12700) (ec1a057)
• Add backoff to flush op (#13140) (9767807)
• add detected-fields command to logcli (#12739) (210ea93)
• Add ingester_chunks_flush_failures_total (#12925) (cc3694e)
• add log patterns ingestion and query API. (#12403) (491d251)
• add lokitool (#12166) (7b7d3d4)
• Add metrics for number of patterns detected & evicted (#12918) (bc53b33)
• Add new Drain tokenizer that splits on most punctuation (#13143) (6a0fdd0)
• Add pattern ingester support in SSD mode (#12685) (19bfef4)
• add pattern match line filter (#12398) (36c703d)
• add profile tagging to ingester (#13068) (00d3c7a)
• add recalculateOwnedStreams to check stream ownership if the ring is changed (#13103) (e7689b2)
• Add step param to Patterns Query API (#12703) (7b8533e)
• add support for discovering and adding log levels as structured metadata (#12428) (b66c343)
• Add tokenizer interface for Drain Training (#13069) (797bb64)
• add toleration for bloom components (#12653) (fcb2b0a)
• Add utf8 support to Pattern Lexer to support utf8 chars (#13085) (f6f8bab)
• add warnings to metadata context directly (#12579) (c4ac8cc)
• Added getting started video (#12975) (8442dca)
• Added Interactive Sandbox to Quickstart tutorial (#12701) (97212ea)
• Added video and updated Grafana Agent -> Alloy (#13032) (1432a3e)
• Allows to wrap distributor Tee (#12391) (ca190a3)
• API: Expose optional label matcher for label names API (#11982) (8084259)
• area/promtail: Added support to install wget on promtail docker image to support docker healthcheck (#11711) (ffe684c)
• bloom-gw: Add metrics.go style log line to bloom gateway FilterChunks call (#12354) (9af191f)
• Blooms retention (#12258) (86c768c)
• blooms: Add counter metric for blocks that are not available at query time (#12968) (d6374bc)
• blooms: Add in-memory LRU cache for meta files (#12862) (fcd544c)
• blooms: bloom integration in query planning (#12208) (a36483b)
• blooms: Blooms/v2 encoding multipart series (#13093) (fbe7c55)
• blooms: compute chunks once (#12664) (bc78d13)
• blooms: ignore individual bloom-gw failures (#12863) (4c9b22f)
• blooms: ingester aware bounded impl (#12840) (7bbd8b5)
• bloom: Skip attempts to filter chunks for which blooms have not been built (#12961) (a1b1eeb)
• blooms: limit bloom size during creation (#12796) (eac5622)
• blooms: record time spent resolving shards (#12636) (9c25985)
• blooms: Separate page buffer pools for series pages and bloom pages (#12992) (75ccf21)
• bloomstore: Support for sharding blocks across multiple different directories (#12375) (15dc2ba)
• Boilerplate for new bloom build planner and worker components. (#12989) (8978ecf)
• Bring back bloom components to backend target in SSD (#12400) (d972ce0)
• cache: Add Cache-Control: no-cache support for Loki instant queries. (#12896) (88e545f)
• canary: Add test to check query results with and without cache. (#13104) (71507a2)
• Detect level as structured metadata by default (#12479) (9e6e6d2)
• Detected labels - get and filter labels from ingesters (#12432) (643f353)
• Detected labels from store (#12441) (587a6d2)
• detected_fields: initial plumbing of new endpoint (#12385) (5b4bfa5)
• detected_labels: Add pod as a detected label to demo (#12422) (a331746)
• detected_labels: Initial skeleton for the API (#12390) (5190dda)
• detected-labels: include labels with cardinality > 1 (#13128) (8be8364)
• detectedFields: add parser to response (#12872) (2b3ae48)
• detectedFields: Support multiple parsers to be returned for a single field (#12899) (19fef93)
• discovery of name of services emitting the logs (#12392) (288c006)
• Enable automatic stream sharding by default (#12451) (24f4e45)
• enable caching of index stats results, volume results, series results and label results by default (#12452) (bc26198)
• Enable log volume endpoint by default (#12628) (397aa56)
• Enable log volume endpoint by default in helm (#12690) (e39677f)
• enable structure metadata by default. (#12453) (7607f04)
• Enable warnings in Loki query responses (#12425) (cf04ec1)
• Generic logline placeholder replacement and tokenization (#12799) (4047902)
• helm: add support for additional PodLogs pipeline stages in selfMonitoring (#12389) (111c19b)
• helm: Allow extraObject items as multiline strings (#12397) (af5be90)
• helm: Support for PVC Annotations for Non-Distributed Modes (#12023) (efdae3d)
• improve performance of first_over_time and last_over_time queries by sharding them (#11605) (f66172e)
• improve syntax parser for pattern (#12489) (48dae44)
• include the stream we failed to create in the stream limit error message (#12437) (ec81991)
• Increase drain max depth from 8 -> 30 (#13063) (d0a2859)
• Introduce index audit to lokitool (#13008) (47f0236)
• Introduce special header that tells Loki not to modify query results (#12327) (c8d2c32)
• loki/main.go: Log which config file path is used on startup (#12985) (7a3338e)
• Memcached: Add TLS support (#12318) (51c54ad)
• new stream count limiter (#13006) (1111595)
• Optimize log parsing performance by using unsafe package (#13223) (9f31b25)
• parameterise the MaximumEventAgeInSeconds, LogGroupName, and IAMRoleName for lambda-promtail CloudFormation template (#12728) (8892dc8)
• prepare 3.0.0 release candidate (#12348) (664e569)
• prepare 3.0.0 release candidate again (#12379) (d484968)
• promtail: Support of RFC3164 aka BSD Syslog (#12810) (be41525)
• Querier: Split gRPC client into two. (#12726) (7b6f057)
• reporting: Report cpu usage (#12970) (87288d3)
• return real data in detected fields endpoint (#12421) (a53a0cc)
• split detected fields queries (#12491) (6c33809)
• Support negative numbers in LogQL (#13091) (6df81db)
• Tune Patterns query drain instance (#13137) (30df31e)
• Update getting started demo to Loki 3.0 (#12723) (282e385)
• update helm chart to support distributed mode and 3.0 (#12067) (79b876b)
• Update Loki monitoring docs to new meta monitoring helm (#13176) (b4d44f8)
• Updated best practises for labels (#12749) (6ebfbe6)
• Updated SS and microservices deployment docs (#13083) (1b80458)

Bug Fixes

• (Bug) correct resultType when storing instant query results in cache (#12312) (7480468)
• codec not initialized in downstream roundtripper (#12873) (b6049f6)
• Add a missing continue in fuse which may cause incorrect bloom test result (#12650) (0d1ebeb)
• Add copyString function to symbolizer to avoid retaining memory (#13146) (86b119a)
• add detected_level info when the info word appears on log message (#13218) (c9bfa3e)
• Add missing Helm helper loki.hpa.apiVersion (#12755) (3070ea7)
• Add missing OTLP endpoint to nginx config (#12709) (8096748)
• add missing parentheses in meta monitoring dashboards (#12802) (151d0a5)
• add retry middleware to the "limited" query roundtripper (#13161) (bb864b3)
• allow cluster label override in bloom dashboards (#13012) (987e551)
• attempt non-string label filtering when errors are present (#12378) (2b8db8b)
• bloom-gateway: Close all block queriers in case of error (#12338) (c74d0eb)
• blooms: bloomshipper no longer returns empty metas on fetch (#13130) (ad279e5)
• blooms: Clean block directories recursively on startup (#12895) (7b77e31)
• blooms: Correctly return unfiltered chunks for series that are not mapped to any block (#12774) (c36b114)
• blooms: Deduplicate filtered series and chunks (#12791) (3bf2d1f)
• blooms: Disable metas cache on bloom gateway (#12959) (00bdd2f)
• blooms: Do not fail requests when fetching metas from cache fails (#12838) (667076d)
• blooms: dont break iterator conventions (#12808) (1665e85)
• blooms: Extract only line filters before line format expressions (#12334) (e4a5733)
• blooms: Fix partitionSeriesByDay function (#12900) (738c274)
• blooms: Fix a regression introduced with #12774 (#12776) (ecefb49)
• blooms: Fix findGaps when ownership goes to MaxUInt64 and that is covered by existing meta (#12558) (0ee2a61)
• blooms: fixup bloom testing (#12494) (712a2b6)
• blooms: Fully deduplicate chunks from FilterChunkRef responses (#12807) (a0f358f)
• blooms: Handle not found metas gracefully (#12853) (37c8822)
• blooms: logql shard embeds tsdb (#12396) (6e1547f)
• blooms: Not filters should always match (#12358) (5c6a031)
• blooms: Remove blocks not matching any series in task (#12401) (cf71ac7)
• blooms: Reset error on LazyBloomIter.Seek (#12806) (76ba24e)
• bloomstore: Fix race conditions (#12320) (07fab4b)
• Bugfixes for detected labels API (#12466) (2c878c8)
• change log level since this is a known case (#13029) (ca030a5)
• ci: updated release workflow to use github app token (#14527) (be131eb)
• close res body (#12444) (616977a)
• codec: inject disable wrappers in ctx (#12352) (64c7812)
• config: Copy Alibaba and IBM object storage configuration from common (#14316) (7184d45)
• Correctly encode step when translating proto to http internally (#13171) (740551b)
• crrect initialization of a few slices (#12674) (0eba448)
• Defer closing blocks iter after checking error from loadWorkForGap (#12934) (cb1f5d9)
• deps: bumped dependencies versions to resolve CVEs (#13789) (34206cd)
• deps: update golang.org/x/exp digest to a685a6e (main) (#11155) (6665685)
• detected_fields: fix issues with frontend integration (#12406) (246623f)
• Do not filter out chunks for store when From==Through and From==start (#13117) (d9cc513)
• docs: broken link in getting started readme (#12736) (425a2d6)
• docs: Move promtail configuration to the correct doc (#12737) (1161846)
• don't shard quantile_over_time if it's not the top level aggregation for a query (#12326) (ab17a50)
• Ensure Drain patterns are valid for LogQL pattern match filter (#12815) (fd2301f)
• errors reported by the race detector (#13174) (2b19dac), closes #8586
• explicitly serialize and deserialize noop label filters (#12124) (1d99b06)
• Fix bloom deleter PR after merge (#13167) (c996349)
• Fix compactor matcher in the loki-deletion dashboard (#12790) (a03846b)
• Fix duplicate enqueue item problem in bloom download queue when do sync download (#13114) (f98ff7f)
• Fix for how the loop sync is done (#12941) (5cd850e)
• Fix incorrect sorting of chunks in bloom-filtered response since ChunkRef.Cmp method is used in reverse (#12999) (670cd89)
• Fix indentation of query range values in helm (#12577) (9823f20)
• Fix log level detection (#12651) (6904a65)
• Fix multi_tenant querier for detected labels (#12447) (9994f56)
• Fix panic on requesting out-of-order Pattern samples (#13010) (2171f64)
• fix parsing of default per tenant otlp config (#12836) (7cc9a93)
• fix setting of info log level when trying to detect level from log lines (#12635) (0831802)
• fix some comments (#12417) (420169a)
• fix span logging based on changes to request types timestamps (#12393) (018856c)
• Fix the lokitool imports (#12673) (6dce988)
• Fixes detected labels API wiring (#12442) (fbb8686)
• Fixes read & backend replicas settings (#12828) (d751134)
• helm: Set compactor addr for distributed mode. (#12748) (521d40a)
• helm: Fix GEL image tag, bucket name and proxy URLs (#12878) (67ed2f7)
• helm: fix query-frontend and ruler targetPort 'http-metrics' in Service template (#13024) (1ab9d27)
• helm: fix queryScheduler servicemonitor (#12753) (8101e21)
• helm: fixed ingress paths mapping (#12932) (5ada92b)
• helm: increase default value of backend minReplicas (#12317) (da2a2c6)
• helm: only default bucket names when using minio (#12548) (2e32ec5)
• helm: Removed duplicate bucketNames from documentation and fixed key name deploymentMode (#12641) (0d8ff9e)
• incorrect compactor matcher in loki-deletion dashboard mixin (#12567) (006f88c)
• indexstats: do not collect stats from "IndexStats" lookups for other query types (#12978) (1f5291a)
• Ingester zoneAwareReplication (#12659) (9edb0ce)
• Introduce feature flag for [last|first]_over_time sharding. (#13067) (6e45550)
• Invalidate caches when pipeline wrappers are disabled (#12903) (a772ed7)
• ksonnet: Do not generate rbac for consul if you are using memberlist (#12688) (2d62fca)
• lambda-promtail, update s3 filename regex to allow finding of log files from AWS GovCloud regions (#12482) (7a81d26)
• log state of 'disable pipeline wrappers' (#12345) (04398a1)
• logql: Linefilter behaviour with chained or (#12463) (e9b6ce9)
• logql: updated JSONExpressionParser not to unescape extracted values if it is JSON object. (backport release-3.1.x) (#14503) (759f9c8)
• loki version prefix in Makefile (#12514) (dff72d2)
• loki-operational.libsonnet (#12789) (51a841f)
• loki/production/docker-compose: upgrade loki and grafana production image tags to 3.1.1 (#14025) (36fe29e)
• make detected fields work for both json and proto (#12682) (f68d1f7)
• make the tsdb filenames correctly reproducible from the identifier (#12536) (ec888ec)
• Missing password for Loki-Canary when loki.auth_enabled is true (#12411) (68b23dc)
• mixin generation when cluster label is changed (#12613) (1ba7a30)
• mixin: dashboards $__auto fix (#12707) (91ef72f)
• Mixins - Add missing log datasource on loki-deletion (#13011) (1948899)
• mixins: Align loki-writes mixins with loki-reads (#13022) (757b776)
• nix: lambda-promtail vendor hash (#12763) (ae180d6)
• not owned stream count (#13030) (4901a5c)
• operator: add alertmanager client config to ruler template (#13182) (6148c37)
• operator: Bump golang builder to 1.21.9 (#12503) (f680ee0)
• operator: Configure Loki to use virtual-host-style URLs for S3 AWS endpoints (#12469) (0084262)
• operator: Improve API documentation for schema version (#13122) (3a9f50f)
• operator: Improve validation of provided S3 storage configuration (#12181) (f9350d6)
• operator: Use a minimum value for replay memory ceiling (#13066) (4f3ed77)
• Optimize regular initialization (#12926) (a46d14f)
• orFilters: fix multiple or filters would get wrong filtertype (#13169) (9981e9e)
• otel: Map 500 errors to 503 (#13173) (b31e04e)
• packaging: Require online network in systemd unit file for Loki and Promtail (#12741) (57f78b5)
• panics when ingester response is nil (#12946) (3cc28aa)
• Pass query to pipeline wrappers from the AST (#12305) (fed7566)
• promtail race fixes (#12656) (4e04d07)
• promtail; clean up metrics generated from logs after a config reload. (#11882) (39a7181)
• promtail: Fix bug with Promtail config reloading getting stuck indefinitely (#12795) (4d761ac)
• promtail: Fix UDP receiver on syslog transport (#10708) (a00f1f1)
• promtail: Handle docker logs when a log is split in multiple frames (#12374) (c0113db)
• properly return http status codes from ingester to querier for RPC function calls (#13134) (691b174)
• quantile sharding; fix bug where sharding would always happen (#12283) (611cb01)
• query sharding: Generalize avg -> sum/count sharding using existing binop mapper (#12599) (11e7687)
• regression: reverts grafana/loki#13039 to prevent use-after-free corruptions (#13162) (41c5ee2)
• Remove Hardcoded Bucket Name from EventBridge Example CloudFormation Template (#12609) (8c18463)
• remove unneccessary disk panels for ssd read path (#13014) (8d9fb68)
• remove unused parameter causing lint error (#12801) (33e82ec)
• Revert build image to Debian Bullseye to fix libc version issue in Promtail (#14387) (05b6a65)
• ruler: pass noop analyseRules to rules manager (#12353) (774b01d)
• spans: corrects early-close for a few spans (#12887) (93aaf29)
• storage/chunk/client/aws: have GetObject check for canceled context (backport release-3.1.x) (#14421) (f3d69ff)
• temporarily moving from alloy -> alloy dev (#13062) (7ffe0fb)
• Track all OTLP metadata bytes in usage tracker. (#12376) (016daa5)
• Track bytes discarded by ingester. (#12981) (88c6711)
• tsdb: correctly use bit prefix calculation in tsdb shard matching (#12394) (3922d38)
• Update deprecated angular graph panel in dashboards mixin (#12430) (9fe31c1)
• Update expected patterns when pruning (#13079) (2923a7d)
• update to build image 0.33.2, fixes bug with promtail windows DNS resolution (#12732) (759f42d)
• updated all dockerfiles go1.22 (#12708) (71a8f2c)
• Updated Loki Otlp Ingest Configuration (#12648) (ff88f3c)
• upgrade old plugin for the loki-operational dashboard. (#13016) (d3c9cec)
• Use an intermediate env variable in GH workflow (#12905) (772616c)
• use nanosecond precision for timestamp in compacted tsdb index file names (#12502) (be501c2)
• Use to the proper config names in warning messages (#12114) (4a05964)
• workflows: don't run metric collector on forks (#12687) (7253444)

Performance Improvements

• blooms: Optimize memory usage for blocks that have no compression (#12316) (a54f2db)
• blooms: Resolve bloom blocks on index gateway and shard by block address (#12720) (5540c92)
• bloomstore: Cache metas LIST operation (#12414) (bf0dd9d)
• blooms: Use jumpmap for client side sharding of the filter requests (#12470) (58c8c61)
• Improve Detected labels API (#12816) (e7fdeb9)
• Introduce fixed size memory pool for bloom querier (#13039) (fc26431)
• Replace channel check with atomic bool in tailer.send() (#12976) (4a5edf1)
• TSDB: Add fast-path to inversePostingsForMatcher (#12679) (402d1d7)

Code Refactoring

• refactor how we do defaults for runtime overrides (#12448) (d39dc09)
• remove async cache writeback on chunk fetcher (#12456) (3ece2ea)

Continuous Integration

• prepare 3.0 release workflow (#12344) (40f695e)

---

Merging this PR will release the artifacts of be131eb