Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace aho-corasick library #1538

Merged
merged 1 commit into from
Jul 25, 2023
Merged

Replace aho-corasick library #1538

merged 1 commit into from
Jul 25, 2023

Conversation

zricethezav
Copy link
Collaborator

A bug was discovered in the community slack channel when a npm token was not being detected. This was due to the fact that the existing ahocorasick library does not properly implement overlapping matches (see related issue here: petar-dambovaliev/aho-corasick#12).

To test this bug try running an unverified scan on the follow content:

npm_token = "00000000-0000-0000-0000-000000000000"

You'll see it does not detect a secret. This is because npm_ is currently taking precedence over the npm keyword due to

MatchKind: ahocorasick.LeftMostLongestMatch,
. However, changing this to StandardMatch does not fix it even though the library says it should support overlapping matches.

Testing this on a few example repos yields pretty similar performance results

@zricethezav zricethezav requested a review from a team as a code owner July 24, 2023 19:42
@zricethezav zricethezav merged commit 85f363f into main Jul 25, 2023
9 checks passed
@zricethezav zricethezav deleted the new-ac-lib branch July 25, 2023 00:09
@semvis123
Copy link

petar-dambovaliev/aho-corasick#13
The previous library should be fixed with the above mentioned pull request.
From my testing this library will consume way less memory, while the other library is slightly faster on larger inputs. If you want to prioritize memory usage you might want to consider switching back. But I recommend to run some benchmarks first.

@zricethezav
Copy link
Collaborator Author

@semvis123 sounds good with me. Would you mind opening a PR for this change on our fork so you can get credit for your work? https://github.com/trufflesecurity/aho-corasick

Thanks!

@semvis123
Copy link

Done 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

5 participants