Found some broken tests

[angaz]

Describe the bug
I was working on #8452 and I ran into a problem where fixing the linter errors broke the tests.

It seems that these two tests are not correct, and are only passing because the variables are being captured incorrectly, so they all end up running the same test.

To Reproduce

I saw that the variables were not being captured properly, so I tried to fix it:

diff --git a/pkg/chunkenc/memchunk_test.go b/pkg/chunkenc/memchunk_test.go
index 1485b462d..5a6642c92 100644
--- a/pkg/chunkenc/memchunk_test.go
+++ b/pkg/chunkenc/memchunk_test.go
@@ -908,12 +908,13 @@ func TestMemChunk_IteratorBounds(t *testing.T) {
                {time.Unix(0, 2), time.Unix(0, 3), logproto.BACKWARD, []bool{true, false}},
                {time.Unix(0, 3), time.Unix(0, 3), logproto.BACKWARD, []bool{false}},
        } {
+               tt := tt
+
                t.Run(
                        fmt.Sprintf("mint:%d,maxt:%d,direction:%s", tt.mint.UnixNano(), tt.maxt.UnixNano(), tt.direction),
                        func(t *testing.T) {
                                t.Parallel()

-                               tt := tt
                                c := createChunk()

                                // testing headchunk

But then some of the tests start to fail:

--- FAIL: TestMemChunk_IteratorBounds (0.00s)
    --- FAIL: TestMemChunk_IteratorBounds/mint:2,maxt:2,direction:FORWARD (0.00s)
        memchunk_test.go:924:
                Error Trace:    loki/pkg/chunkenc/memchunk_test.go:924
                Error:          Not equal:
                                expected: true
                                actual  : false
                Test:           TestMemChunk_IteratorBounds/mint:2,maxt:2,direction:FORWARD
    --- FAIL: TestMemChunk_IteratorBounds/mint:2,maxt:2,direction:BACKWARD (0.00s)
        memchunk_test.go:924:
                Error Trace:    loki/pkg/chunkenc/memchunk_test.go:924
                Error:          Not equal:
                                expected: true
                                actual  : false
                Test:           TestMemChunk_IteratorBounds/mint:2,maxt:2,direction:BACKWARD
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:1,direction:BACKWARD (0.00s)
        memchunk_test.go:924:
                Error Trace:    loki/pkg/chunkenc/memchunk_test.go:924
                Error:          Not equal:
                                expected: true
                                actual  : false
                Test:           TestMemChunk_IteratorBounds/mint:1,maxt:1,direction:BACKWARD
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:1,direction:FORWARD (0.00s)
        memchunk_test.go:924:
                Error Trace:    loki/pkg/chunkenc/memchunk_test.go:924
                Error:          Not equal:
                                expected: true
                                actual  : false
                Test:           TestMemChunk_IteratorBounds/mint:1,maxt:1,direction:FORWARD

Going back to the original code, if I check the pointer value of the captured test case:

diff --git a/pkg/chunkenc/memchunk_test.go b/pkg/chunkenc/memchunk_test.go
index 1485b462d..d76db1dff 100644
--- a/pkg/chunkenc/memchunk_test.go
+++ b/pkg/chunkenc/memchunk_test.go
@@ -913,6 +913,9 @@ func TestMemChunk_IteratorBounds(t *testing.T) {
                        func(t *testing.T) {
                                t.Parallel()

+                               t.Fail()
+                               t.Logf("%p\n", &tt)
+
                                tt := tt
                                c := createChunk()

You can see the memory address for all the captured functions is the same, so they are all running the same test:

--- FAIL: TestMemChunk_IteratorBounds (0.00s)
    --- FAIL: TestMemChunk_IteratorBounds/mint:0,maxt:1,direction:FORWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:0,maxt:1,direction:BACKWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:3,maxt:3,direction:FORWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:3,direction:BACKWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:3,maxt:3,direction:BACKWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:2,direction:FORWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:2,maxt:3,direction:BACKWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:2,direction:BACKWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:1,direction:FORWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:1,direction:BACKWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:1,maxt:3,direction:FORWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:2,maxt:3,direction:FORWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:2,maxt:2,direction:BACKWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680
    --- FAIL: TestMemChunk_IteratorBounds/mint:2,maxt:2,direction:FORWARD (0.00s)
        memchunk_test.go:917: 0xc0098d9680

This is also the case for the test:

    engine_test.go:2373: 
        	Error Trace:	loki/pkg/logql/engine_test.go:2373
        	Error:      	Not equal:
        	            	expected: *errors.errorString(&errors.errorString{s:"mock error"})
        	            	actual  : <nil>(<nil>)
        	Test:       	TestStepEvaluator_Error/stream
    --- FAIL: TestStepEvaluator_Error/stream (0.00s)

So the thing is, I have tried to understand what these tests are doing, but I do not understand any of them, so IDK if someone wants to look at these tests to see if it's fixable, or to remove the broken tests.

[DylanGuedes]

Hey thank you for your report ❤️
I recently fixed them in the main branch so I'm closing this one. And yeah, they were bugged 😅 (variable shadowing+parallel execution).