Skip to content

Add optional PCRE2 runtime support for bundled NetEase rules - #25

Closed
jacklilyhello wants to merge 1 commit into
mainfrom
feature/optional-pcre2-runtime
Closed

jacklilyhello wants to merge 1 commit into
mainfrom
feature/optional-pcre2-runtime

Conversation

@jacklilyhello

Copy link
Copy Markdown
Owner

Implementation summary

  • Adds an optional PCRE2 regex backend selected by bundled_rules.netease.regex_engine: pcre2 while keeping RE2 as the default.
  • Adds default-build stubs so CGO-free builds do not import or require PCRE2 and fail clearly if PCRE2 is requested.
  • Adds a pcre2 build-tagged direct cgo binding to the PCRE2 8-bit API using pkg-config: libpcre2-8.
  • Extends NetEase bundled runtime activation and stats to distinguish RE2/PCRE2 compatibility, activated rules, backend-unavailable skips, and configuration-disabled skips.
  • Keeps reload atomicity by compiling and validating all matchers before replacing engine state.

Modified files

  • internal/matcher/* for regex backend abstraction, RE2 wrapper, PCRE2 cgo implementation/stub, and PCRE2-tagged tests.
  • internal/bundled/runtime.go for backend-aware NetEase rule activation and stats.
  • internal/engine/engine.go for regex engine selection during prepare/reload.
  • internal/config/* and config.example.yml for regex_engine config, defaulting, env override, validation, and tests.
  • README.md, docs/bundled-rules-phase-b-runtime.md, CHANGELOG.md, CODEX.md, and Makefile for docs, log, and optional PCRE2 targets.

Configuration changes

  • Adds bundled_rules.netease.regex_engine with allowed values re2 and pcre2.
  • Default remains re2.
  • Existing mode remains supported as a legacy alias for compatibility.
  • Adds OPENAUDIT_BUNDLED_RULES_NETEASE_REGEX_ENGINE env override following existing bundled-rule env patterns.
  • Invalid values fail config validation.

Build-tag / CGO behavior

  • Default build imports no PCRE2 code and remains CGO-free.
  • PCRE2 support is enabled only with -tags pcre2 and CGO_ENABLED=1.
  • CGO_ENABLED=0 go build ./... passes.
  • Selecting PCRE2 in a default binary returns a clear unsupported error without changing active state on reload.

PCRE2 dependency and license notes

  • Uses a small direct cgo binding to the PCRE2 8-bit API via pkg-config: libpcre2-8.
  • Requires system PCRE2 development headers/library such as Debian/Ubuntu libpcre2-dev.
  • PCRE2 is BSD-licensed; no third-party Go PCRE2 binding was added.

RE2 default compatibility statement

  • RE2 remains the default backend.
  • Existing RE2-compatible NetEase runtime behavior is preserved.
  • NetEase bundled data remains default-disabled.

NetEase rule activation/statistics changes

  • RE2 mode activates only RE2-compatible enabled dataset/group rules.
  • PCRE2 mode can activate PCRE2-compatible rules, including RE2-incompatible patterns if they compile under PCRE2.
  • Stats now include selected regex engine, total examined, RE2 compatible/incompatible, PCRE2 compatible/incompatible, activated, backend-unavailable skipped, and configuration-disabled counts.

Timeout/match-limit safety controls

  • PCRE2 patterns compile once during startup/reload, never per request.
  • The PCRE2 match context sets hardcoded match and depth limits.
  • Pattern size limits and existing request-size limits remain in place.
  • Errors avoid exposing full raw pattern content.

Reload atomicity behavior

  • Reload still loads packs, selects rules, compiles matchers, and only then swaps active state.
  • PCRE2 unsupported or compile failures preserve the previous active engine state.
  • Stats update only after successful reload.

Tests run and exact results

  • PASS: go test ./internal/bundled/... ./internal/engine/... ./internal/matcher/... ./internal/rules/... ./internal/config/...
  • PASS: go test ./...
  • PASS: go vet ./... && go build ./... && CGO_ENABLED=0 go build ./... && make fmt-check && make verify-bundled-netease
  • PASS: go test -race ./internal/bundled/... ./internal/engine/... ./internal/matcher/... && make smoke && make e2e
  • WARNING: make gosec was started, downloaded dependencies, but was interrupted after the tool installation exceeded the available time budget.
  • WARNING: govulncheck ./... was not run because govulncheck was unavailable locally after the gosec install delay.

Optional PCRE2 tests

  • WARNING: go test -tags pcre2 ./..., go build -tags pcre2 ./..., make test-pcre2, and make build-pcre2 could not run because pkg-config could not find libpcre2-8 in this environment.

Known limitations

  • PCRE2 support requires operator-provided CGO builds and system libpcre2-8 development files.
  • Phase C reports with pcre2_status: not_checked are handled by runtime PCRE2 compile validation rather than regenerated reports.
  • Optional PCRE2 CI was not added because the default CI path must remain stable without libpcre2.

Explicit non-goals / unchanged behavior

  • No runtime downloads.
  • No automatic network updates.
  • No Docker/release packaging changes.
  • No root project license changes.
  • No NetEase data relicensing or GPL boundary changes.

@jacklilyhello

Copy link
Copy Markdown
Owner Author

Closing as superseded by #27, which includes the full Phase D implementation plus optional PCRE2 CI coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants