Add optional PCRE2 runtime support for bundled NetEase rules - #26
Closed
jacklilyhello wants to merge 1 commit into
Closed
jacklilyhello wants to merge 1 commit into
jacklilyhello wants to merge 1 commit into
Conversation
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Description
internal/matcher(regex_backend.go, updatedregex.go,pcre2.go,pcre2_stub.go, PCRE2-tagged tests), plus compile-time selectionCompileRegexRulesWithEngine.internal/bundled/runtime.goto select the effectiveregex_engine, evaluate PCRE2 compatibility when available, and surface new stats (selected engine, total examined, RE2/PCRE2 compatible/incompatible, backend-unavailable skipped, configuration-disabled, activated counts).bundled_rules.netease.regex_enginewith env overrideOPENAUDIT_BUNDLED_RULES_NETEASE_REGEX_ENGINE, preserve legacymodeas an alias, and validate allowed values; updates ininternal/config/*andconfig.example.yml.pcre2build tag that adds a direct cgo binding to libpcre2-8 (PCRE2 8-bit API) requiringCGO_ENABLED=1and systemlibpcre2-8development files; default builds include a stub that returns a clear unsupported error; documentation and Makefile targets fortest-pcre2/build-pcre2added; GPL/provenance and reload atomicity preserved.Modified notable files (high level):
internal/matcher/*,internal/bundled/runtime.go,internal/engine/engine.go,internal/config/*,config.example.yml,Makefile,README.md,docs/bundled-rules-phase-b-runtime.md,CHANGELOG.md,CODEX.md.PCRE2 dependency and license note: the implementation uses direct cgo to the PCRE2 8-bit API via
pkg-config: libpcre2-8(requires systemlibpcre2-dev); PCRE2 upstream license is documented and no third-party Go binding was added.Safety and reload semantics: patterns are compiled once during prepare/reload, PCRE2 match/depth limits are set in the match context, compile/match errors are deterministic and sanitized, and failed reloads or unsupported PCRE2 requests preserve previous active engine state and stats.
Testing
go test ./internal/bundled/... ./internal/engine/... ./internal/matcher/... ./internal/rules/... ./internal/config/...— PASS;go test ./...— PASS.go vet ./... && go build ./... && CGO_ENABLED=0 go build ./... && make fmt-check && make verify-bundled-netease— PASS.go test -race ./internal/bundled/... ./internal/engine/... ./internal/matcher/... && make smoke && make e2e— PASS.go test -tags pcre2 ./...andgo build -tags pcre2 ./.../make test-pcre2/make build-pcre2could not complete in this environment becausepkg-configcould not findlibpcre2-8(system dependency missing); PCRE2 code and tag are present and tests are added but were skipped/failed for lack of libpcre2 in CI environment.make gosecstarted installinggosecand began dependency fetch; installation was interrupted after tool install exceeded time budget in this environment andgovulncheckwas not run here for the same reason.Branch and PR
feature/optional-pcre2-runtime.982d3d5622e9c61ddcc9bb717f4032d4991f2591.982d3d5622e9c61ddcc9bb717f4032d4991f2591.Confirmations
CGO_ENABLED=0 go build ./...passes: yes.-tags pcre2andCGO_ENABLED=1: implemented, but PCRE2-tagged tests/builds were skipped in this environment due to missinglibpcre2-8(pkg-config).If you want, I can: open an optional CI job guidance for running
-tags pcre2withlibpcre2-devinstalled, or refine the PCRE2 cgo wrapper (timeouts, error messages) further before merge.Codex Task