Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure parsing a serialized LogQL AST results in the same AST. #11227

Closed
wants to merge 27 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
91cdb6c
Ensure parsing a serialized LogQL AST results in the same AST.
jeschkies Nov 14, 2023
90638b4
Test duration as well
jeschkies Nov 14, 2023
eaac0fb
Merge remote-tracking branch 'grafana/main' into karsten/roundtrip-se…
jeschkies Nov 15, 2023
259a4d5
Represent bytes only as B
jeschkies Nov 20, 2023
0db5a7a
Increase test coverage for AST serialization to >94%. (#11230)
jeschkies Nov 15, 2023
5c5983f
[bloom-compactor] Move meta.json creation at the end of compaction cy…
poyzannur Nov 15, 2023
e5a439c
[bloom-compactor] Add configs to enable compactor per tenant (#11235)
poyzannur Nov 15, 2023
c30f339
lambda-promtail: fix IAM policy for clouddwatch log stream (#10909)
keyolk Nov 16, 2023
7666c0d
Update nix configuration (#8452)
angaz Nov 16, 2023
8a8d153
Do not run snyk pr comment workflow on forks (#11240)
trevorwhitney Nov 16, 2023
da80818
Fix per-pod panel unit (from 'ms' to 's') (#11245)
DylanGuedes Nov 16, 2023
51f90eb
operator: Update dependencies and dev tools (#11232)
periklis Nov 17, 2023
9292780
ksonnet: generate tsdb_shipper storage_config even if using_boltdb_sh…
Canuteson Nov 17, 2023
b6f6ae7
iterable tokenizer w/ comparison benching (#11255)
owen-d Nov 17, 2023
b209492
inverts the logic when testing a block against a list of chunks (#11248)
owen-d Nov 17, 2023
ea08e30
boundscheck & partitioning fingerprints between blocks (#11237)
owen-d Nov 17, 2023
9e179f2
compactor: do not block compation when retention is taking too long (…
sandeepsukhani Nov 20, 2023
3b70dba
inflight-logging: Add extra metadata to inflight requests logging (#1…
kavirajk Nov 20, 2023
8d2cb04
operator: Adds new value v13 to schema (#10932)
JoaoBraveCoding Nov 20, 2023
dab7099
operator: Remove outdated BoltDB dashboards (#11022)
JoaoBraveCoding Nov 20, 2023
12106e2
compaction: Separate metrics for tracking retention and compaction (#…
sandeepsukhani Nov 20, 2023
7ef65b5
Propagate trace ID with HTTP gRPC request. (#11251)
jeschkies Nov 20, 2023
0c94702
tokenizer v1 cleanup (#11272)
paul1r Nov 20, 2023
b08319e
Fixed the grammatical mistake in an _index.md file (#11226)
ibilalkayy Nov 20, 2023
8c65803
Fix Typo in the _index.md (#11242)
Pavaningithub Nov 20, 2023
eb8ad93
use nanosecond precision for timestamp in compacted boltdb-shipper in…
sandeepsukhani Nov 21, 2023
31fc35b
Merge remote-tracking branch 'grafana/main' into karsten/roundtrip-se…
jeschkies Nov 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions pkg/logql/syntax/parser_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2007,6 +2007,30 @@ var ParseTestCases = []struct {
OpRangeTypeQuantile, &Grouping{Without: false, Groups: []string{"namespace", "instance"}}, NewStringLabelFilter("0.99998"),
),
},
{
in: `{app="foo"} | json | size >= 2.56GiB `,
exp: &PipelineExpr{
Left: newMatcherExpr([]*labels.Matcher{{Type: labels.MatchEqual, Name: "app", Value: "foo"}}),
MultiStages: MultiStageExpr{
newLabelParserExpr(OpParserTypeJSON, ""),
&LabelFilterExpr{
LabelFilterer: log.NewBytesLabelFilter(log.LabelFilterGreaterThanOrEqual, "size", 2_748_779_070),
},
},
},
},
{
in: `{app="foo"} | json | duration >= 3s15ms`,
exp: &PipelineExpr{
Left: newMatcherExpr([]*labels.Matcher{{Type: labels.MatchEqual, Name: "app", Value: "foo"}}),
MultiStages: MultiStageExpr{
newLabelParserExpr(OpParserTypeJSON, ""),
&LabelFilterExpr{
LabelFilterer: log.NewDurationLabelFilter(log.LabelFilterGreaterThanOrEqual, "duration", 3_015_000_000),
},
},
},
},
{
in: `sum without (foo) (
quantile_over_time(0.99998,{app="foo"} |= "bar" | json | latency >= 250ms or ( status_code < 500 and status_code > 200)
Expand Down Expand Up @@ -3146,6 +3170,12 @@ func TestParse(t *testing.T) {
ast, err := ParseExpr(tc.in)
require.Equal(t, tc.err, err)
require.Equal(t, tc.exp, ast)

if err == nil {
roundtrip, err := ParseExpr(ast.String())
require.NoError(t, err)
require.Equal(t, ast, roundtrip)
}
})
}
}
Expand Down
4 changes: 0 additions & 4 deletions pkg/logql/syntax/serialize_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ package syntax

import (
"bytes"
"strings"
"testing"

"github.com/stretchr/testify/require"
Expand Down Expand Up @@ -74,9 +73,6 @@ func TestJSONSerializationParseTestCases(t *testing.T) {
t.Run(tc.in, func(t *testing.T) {
ast, err := ParseExpr(tc.in)
require.NoError(t, err)
if strings.Contains(tc.in, "KiB") {
t.Skipf("Byte roundtrip conversion is broken. '%s' vs '%s'", tc.in, ast.String())
}

var buf bytes.Buffer
err = EncodeJSON(ast, &buf)
Expand Down
Loading