Skip to content

Implement NetEase bundled rules runtime - #23

Merged
jacklilyhello merged 3 commits into
mainfrom
codex/neteaseb-2026-06-22-14-00-30
Jun 22, 2026
Merged

jacklilyhello merged 3 commits into
mainfrom
codex/neteaseb-2026-06-22-14-00-30

Conversation

@jacklilyhello

Copy link
Copy Markdown
Owner

Motivation

  • Enable Phase B runtime activation of operator-supplied NetEase Phase A Packs so RE2-compatible bundled rules participate in audits and hot reloads while keeping runtime safe and default-disabled.
  • Preserve existing engine/matcher semantics and atomic reload guarantees while avoiding PCRE2 native dependencies, network access, or bundling upstream G79/X19 data.

Description

  • Add a dedicated bundled runtime loader that reads deterministic Pack filenames netease-g79.json.gz and netease-x19.json.gz under bundled_rules.data_dir, performs bounded reads through safepath, calls ReadPackGzip and ValidatePack, converts selected compatible PackRule to rules.Rule, and merges them with the local rules.Set (internal/bundled/runtime.go).
  • Integrate into the engine with an options-based constructor (engine.NewWithOptions) while keeping engine.New(root) compatible, perform bundled merge and matcher compilation in PrepareWithOptions, and atomically swap engine state only after successful validation and compilation (internal/engine/engine.go).
  • Enforce RE2-only runtime behavior: in re2 mode only RE2Compatible==true rules are activated and incompatible rules are skipped and counted (sanitized hints exposed); pcre2 mode returns a clear unsupported-mode error only when NetEase is effectively enabled (internal/bundled/runtime.go).
  • Preserve security and reliability by using safepath root resolution, rejecting traversal/symlink escape/unexpected file types, enforcing compressed and decompressed size limits, rejecting concatenated or trailing gzip streams, validating provider/dataset and detecting duplicate IDs, and refusing to publish statistics for failed reloads; also wire server to pass cfg.BundledRules into engine (cmd/server/main.go).
  • Add backward-compatible bundled runtime statistics embedded into rules stats, synthetic Pack-focused unit tests for runtime behaviors (selection, group enablement, compatibility skipping, collisions, symlink rejection, failed reload preservation), and Phase B runtime documentation and example config updates (docs/bundled-rules-phase-b-runtime.md, README, CHANGELOG, config examples).

Testing

  • Ran formatting and static checks: gofmt/make fmt-check passed and go vet ./... passed.
  • Unit and package tests: go test ./internal/bundled/... ./internal/engine/... ./internal/config/..., go test ./..., and race tests go test -race ./internal/bundled/... ./internal/engine/... all passed; bundled runtime unit tests exercise disabled/no-read, dataset/group combos, RE2 incompatibilities, pcre2-mode behavior, missing/wrong/unsafe pack handling, ID collisions, and reload preservation.
  • Build and security checks: go build ./... and CGO_ENABLED=0 go build ./... passed; make smoke, make gosec, and make e2e passed in this environment.
  • Vulnerability index check: make govulncheck failed due to environment/network access returning Forbidden when fetching https://vuln.go.dev/index/modules.json.gz (environment limitation, not code failure).

Codex Task

@jacklilyhello
jacklilyhello merged commit 781ac73 into main Jun 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant