Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace current regex engine with PCRE2 #4185

Merged
merged 3 commits into from
Feb 5, 2024
Merged

Replace current regex engine with PCRE2 #4185

merged 3 commits into from
Feb 5, 2024

Conversation

Rot127
Copy link
Member

@Rot127 Rot127 commented Feb 4, 2024

Your checklist for this pull request

  • I've read the guidelines for contributing to this repository
  • I made sure to follow the project's coding style
  • I've documented or updated the documentation of every function and struct this PR changes. If not so I've explained why.
  • I've added tests that prove my fix is effective or that my feature works (if possible)
  • I've updated the rizin book with the relevant information (if needed)

Detailed description

Replace the current regex engine with PCRE2.

Test plan

All green.

Closing issues

closes #3730

Partially addresses #4055

Todo

  • Add a rz_vector_pop_ptr() function which doesn't use memcpy() and use it in match_all_flat(). Or add a rz_vector_concat() function with the same functionality.
  • Terminology for groups and matches is weird.
  • docs
  • Open bug report for tcc and pcre2 compilation and __asm
  • Test rz_regex_get_match_name

PCRE2 has way better performance than the OpenBSD
library (something around 20 times faster).

The following flags are enabled for every pattern:

- PCRE2_UTF
- PCRE2_MATCH_INVALID_UTF
- PCRE2_NO_UTF_CHECK

All the others are optional.

Changes made:

- Adds PCRE2 as subproject.
- Changes the API away from POSIX to PCRE2.
- Edits many regex patterns because:
 - ' ' is skipped in patterns, if the EXTENDED flag is set for matching. '\s' must be set now.
 - '.' doesn't match newlines by default.
- Changes the API so matches and their groups are bundled into PVectors.
- Moves the regex component to rz_util.
Copy link
Member

@XVilka XVilka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if all green

@Rot127 Rot127 changed the title Dist fuzz pcre2 Replace current regex engine with PCRE2 Feb 4, 2024
XVilka

This comment was marked as resolved.

Meson currently doesn't support subprojects to be native and non-native at the same time.
See: mesonbuild/meson#10947
Unfortunately, sdb depends on rz_util which in turn depends on PCRE2.
Excluding PCRE2 from the native build makes linking of rz_util not possible anymore.
Adding it, will make Meson complain that the dependencies cannot be mixed.

Hence, we compile a copy of PCRE2 for the native build if required.
@wargio wargio merged commit 5afc51f into dev Feb 5, 2024
50 of 52 checks passed
@wargio wargio deleted the dist-fuzz-pcre2 branch February 5, 2024 04:51
@arrowd
Copy link
Contributor

arrowd commented Feb 25, 2024

Why use static linking here? I'm getting

ld: error: relocation R_X86_64_32S cannot be used against symbol '_pcre2_ucd_records_8'; recompile with -fPIC

on FreeBSD when building against system pcre2.

@wargio
Copy link
Member

wargio commented Feb 26, 2024

@Rot127

@Rot127

This comment was marked as outdated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regex is slow
4 participants