Errors found without any errors on changed files

[kylemellander]

I've experienced an issue where Balto is showing a check as failed with a lot of errors, even though it does not list any errors. I have also manually verified locally that the files in question do not contain errors.

The issue can be seen on this PR: https://github.com/ministrycentered/check-ins/pull/3548

[danielma]

I'm pretty sure what's happening is that eslint is reporting 33 errors (which I can verify locally with git diff --name-only --diff-filter AM d66335d9070eb693f8c7d422b70ce279d09d34da | grep '\(jsx\|js\)$' | xargs eslint) but since none of those errors are on lines you actually changed in that PR, we're not creating annotations.

This is a bug worth fixing. If we're not creating annotations, we shouldn't be counting those errors towards the final result

[danott]

I believe I've encountered the same error on ChurchCenterApp: https://github.com/ministrycentered/ChurchCenterApp/pull/892/checks?check_run_id=319865064

[sheck]

I think I found the source of this bug while looking into something for @alexandervalencia

balto-eslint/src/main.js

Line 119 in 611cd6d

const { results, errorCount, warningCount } = report

On that line we are grabbing the error and warning count directly from the eslint output, before filtering out errors and warnings from lines that were not changed in the PR.

Those constants currently determine the conclusion level of the check, which could result in failures being described in this issue if you make a "clean" line change in an already "dirty" file.

https://app.asana.com/0/1203194086773994/1204631278549623/f