Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/release-dry-run.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Release dry run

on:
pull_request:

permissions:
contents: read

jobs:
release-dry-run:
name: Release dry run
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.25.11"
cache: true

- name: Run release validation
run: make release-check VERSION=${{ github.event.pull_request.head.sha }} COMMIT=${{ github.event.pull_request.head.sha }} DATE=1970-01-01T00:00:00Z

- name: Create local release snapshot
run: make snapshot VERSION=${{ github.event.pull_request.head.sha }} COMMIT=${{ github.event.pull_request.head.sha }} DATE=1970-01-01T00:00:00Z

- name: Verify default Docker image build
run: make docker-build

- name: Verify optional PCRE2 Docker image build
run: make docker-build-pcre2
80 changes: 55 additions & 25 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,22 +1,18 @@
name: Release build
name: Release

on:
workflow_dispatch:
push:
tags:
- "v*"

permissions:
contents: read
contents: write
packages: write

jobs:
build:
name: Build Linux artifacts
release:
name: Publish release
runs-on: ubuntu-latest
strategy:
matrix:
include:
- goos: linux
goarch: amd64
- goos: linux
goarch: arm64

steps:
- name: Checkout repository
Expand All @@ -28,17 +24,51 @@ jobs:
go-version: "1.25.11"
cache: true

- name: Build binary
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
CGO_ENABLED: "0"
run: |
mkdir -p dist
go build -ldflags "-s -w -X github.com/openaudit/openaudit/internal/api.Version=${GITHUB_REF_NAME:-manual} -X github.com/openaudit/openaudit/internal/api.Commit=${GITHUB_SHA} -X github.com/openaudit/openaudit/internal/api.BuildTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" -o dist/openaudit-${GOOS}-${GOARCH} ./cmd/server

- name: Upload artifact
uses: actions/upload-artifact@v4
- name: Build release artifacts
run: make snapshot VERSION=${GITHUB_REF_NAME} COMMIT=${GITHUB_SHA} DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ)

- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
files: |
dist/snapshot/*.tar.gz
dist/snapshot/SHA256SUMS
generate_release_notes: true

- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build and push default RE2 image
uses: docker/build-push-action@v6
with:
context: .
target: default
push: true
build-args: |
VERSION=${{ github.ref_name }}
COMMIT=${{ github.sha }}
BUILD_TIME=${{ github.event.repository.updated_at }}
tags: |
ghcr.io/jacklilyhello/openaudit:${{ github.ref_name }}
ghcr.io/jacklilyhello/openaudit:latest

- name: Build and push optional PCRE2 image
uses: docker/build-push-action@v6
with:
name: openaudit-${{ matrix.goos }}-${{ matrix.goarch }}
path: dist/openaudit-${{ matrix.goos }}-${{ matrix.goarch }}
context: .
target: pcre2
push: true
build-args: |
VERSION=${{ github.ref_name }}
COMMIT=${{ github.sha }}
BUILD_TIME=${{ github.event.repository.updated_at }}
tags: |
ghcr.io/jacklilyhello/openaudit:${{ github.ref_name }}-pcre2
ghcr.io/jacklilyhello/openaudit:pcre2
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Release-note style changes for OpenAudit. For detailed historical implementation
- Production hardening for environment modes, management API protection, admin exposure safeguards, production-safe logging defaults, and Cloudflare Access/Tunnel deployment guidance.
- Security scanning baseline with CI format/vet/test/build/smoke checks, blocking gosec release gate, govulncheck, CodeQL, and documented safepath/SQL invariants.
- Deterministic E2E validation through `scripts/e2e.sh` and `make e2e` for manual release validation.
- Release-readiness infrastructure with build-time version metadata, `--version`, local `build-all`/`release-check`/`snapshot` targets, pull-request release dry runs, and tag-triggered GitHub Release/GHCR publishing workflows.

### Security

Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ ARG VERSION=dev
ARG COMMIT=unknown
ARG BUILD_TIME=unknown
RUN CGO_ENABLED=0 GOOS=linux go build \
-ldflags "-s -w -X github.com/openaudit/openaudit/internal/api.Version=${VERSION} -X github.com/openaudit/openaudit/internal/api.Commit=${COMMIT} -X github.com/openaudit/openaudit/internal/api.BuildTime=${BUILD_TIME}" \
-ldflags "-s -w -X main.version=${VERSION} -X main.commit=${COMMIT} -X main.date=${BUILD_TIME}" \
-o /out/openaudit ./cmd/server

FROM golang:1.25.11-alpine AS build-pcre2
Expand All @@ -25,7 +25,7 @@ ARG VERSION=dev
ARG COMMIT=unknown
ARG BUILD_TIME=unknown
RUN CGO_ENABLED=1 GOOS=linux go build -tags pcre2 \
-ldflags "-s -w -X github.com/openaudit/openaudit/internal/api.Version=${VERSION} -X github.com/openaudit/openaudit/internal/api.Commit=${COMMIT} -X github.com/openaudit/openaudit/internal/api.BuildTime=${BUILD_TIME}" \
-ldflags "-s -w -X main.version=${VERSION} -X main.commit=${COMMIT} -X main.date=${BUILD_TIME}" \
-o /out/openaudit ./cmd/server

FROM alpine:3.20 AS default
Expand Down
39 changes: 36 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,17 @@ SHELL := /bin/sh
APP_NAME := openaudit
BIN_DIR := bin
BIN := $(BIN_DIR)/$(APP_NAME)
DIST_DIR := dist
IMAGE := openaudit:local
IMAGE_PCRE2 := openaudit:pcre2-local
GO_FILES := $(shell find . -name '*.go' -not -path './vendor/*')
VERSION ?= dev
COMMIT ?= $(shell git rev-parse --short=12 HEAD 2>/dev/null || echo unknown)
DATE ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
LDFLAGS := -s -w -X main.version=$(VERSION) -X main.commit=$(COMMIT) -X main.date=$(DATE)
PLATFORMS := linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64

.PHONY: help fmt fmt-check vet test test-pcre2 build build-pcre2 run clean ci govulncheck gosec docker-build docker-build-pcre2 docker-run docker-smoke docker-smoke-pcre2 smoke e2e verify-bundled-netease regenerate-bundled-netease
.PHONY: help fmt fmt-check vet test test-pcre2 build build-pcre2 build-all release-check snapshot run clean ci govulncheck gosec docker-build docker-build-pcre2 docker-run docker-smoke docker-smoke-pcre2 smoke e2e verify-bundled-netease regenerate-bundled-netease

help: ## Show available targets
@awk 'BEGIN {FS = ":.*##"; printf "OpenAudit development targets:\n"} /^[a-zA-Z0-9_-]+:.*##/ {printf " %-18s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
Expand All @@ -26,14 +32,41 @@ test: ## Run tests

build: ## Build OpenAudit binary into ./bin/openaudit
mkdir -p $(BIN_DIR)
go build -o $(BIN) ./cmd/server
CGO_ENABLED=0 go build -trimpath -ldflags "$(LDFLAGS)" -o $(BIN) ./cmd/server

build-all: ## Build default RE2/CGO-free release binaries for common platforms
rm -rf $(DIST_DIR)/build
mkdir -p $(DIST_DIR)/build
@set -eu; for platform in $(PLATFORMS); do \
goos=$${platform%/*}; goarch=$${platform#*/}; ext=""; \
if [ "$$goos" = "windows" ]; then ext=".exe"; fi; \
out="$(DIST_DIR)/build/$(APP_NAME)-$(VERSION)-$$goos-$$goarch$$ext"; \
echo "building $$out"; \
CGO_ENABLED=0 GOOS=$$goos GOARCH=$$goarch go build -trimpath -ldflags "$(LDFLAGS)" -o "$$out" ./cmd/server; \
done
cd $(DIST_DIR)/build && find . -type f -name '$(APP_NAME)-*' -print | LC_ALL=C sort | xargs sha256sum > SHA256SUMS

release-check: fmt-check vet test build build-all verify-bundled-netease ## Run local release validation without publishing
$(BIN) --version
cd $(DIST_DIR)/build && sha256sum -c SHA256SUMS

snapshot: build-all ## Create local compressed release snapshot artifacts and SHA256SUMS
rm -rf $(DIST_DIR)/snapshot
mkdir -p $(DIST_DIR)/snapshot
@set -eu; for f in $(DIST_DIR)/build/$(APP_NAME)-$(VERSION)-*; do \
base=$$(basename "$$f"); \
case "$$base" in SHA256SUMS) continue ;; esac; \
cp "$$f" "$(DIST_DIR)/snapshot/$$base"; \
(cd $(DIST_DIR)/snapshot && tar --sort=name --mtime='UTC 1970-01-01' --owner=0 --group=0 --numeric-owner -czf "$$base.tar.gz" "$$base" && rm "$$base"); \
done
cd $(DIST_DIR)/snapshot && find . -type f -name '*.tar.gz' -print | LC_ALL=C sort | xargs sha256sum > SHA256SUMS

test-pcre2: ## Run optional PCRE2-tagged tests (requires CGO and libpcre2-8 development files)
CGO_ENABLED=1 go test -tags pcre2 ./...

build-pcre2: ## Build optional PCRE2 binary (requires CGO and libpcre2-8 development files)
mkdir -p $(BIN_DIR)
CGO_ENABLED=1 go build -tags pcre2 -o $(BIN)-pcre2 ./cmd/server
CGO_ENABLED=1 go build -tags pcre2 -trimpath -ldflags "$(LDFLAGS)" -o $(BIN)-pcre2 ./cmd/server

run: ## Run OpenAudit locally
go run ./cmd/server
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,11 @@ Run release-oriented local checks:
```bash
make smoke
make e2e
make release-check
```

`make smoke` starts the service and performs a basic API smoke test. `make e2e` runs deterministic end-to-end release validation with `scripts/e2e.sh`.
Build metadata is available with `go run ./cmd/server --version`.

## Configuration

Expand Down Expand Up @@ -79,6 +81,7 @@ Use Cloudflare Access, Cloudflare Tunnel, and a localhost origin. Do not point a
- [DEVELOPMENT_LOG.md](DEVELOPMENT_LOG.md) — phase-by-phase implementation history.
- [CHANGELOG.md](CHANGELOG.md) — release-note style summary of completed user-facing changes.
- [ROADMAP.md](ROADMAP.md) — future-facing roadmap.
- [docs/release.md](docs/release.md) — release tags, binary artifacts, SHA256SUMS, GHCR images, and PCRE2 distribution notes.

## Security and CI summary

Expand Down Expand Up @@ -123,3 +126,7 @@ curl -H "X-API-Key: $OPENAUDIT_ADMIN_API_KEY" http://127.0.0.1:8080/rules/stats
```

Bundled runtime stats avoid raw regex patterns and offensive rule content while reporting provider/dataset enablement, selected regex engine, backend availability, compatibility counts, activated/skipped counts, safe pack hashes, and successful reload timestamps. See `docs/production-runtime-ops.md` for Docker, Compose, PCRE2, GPL/MIT data-boundary, and security guidance.

## Release distribution

Versioned releases are tag-triggered from `v*` tags. Default binary artifacts use RE2/Go regexp and are built for Linux, macOS, and Windows without requiring CGO. Optional PCRE2 distribution is Docker-first because native cross-compilation requires CGO, libpcre2, and platform toolchains. See [docs/release.md](docs/release.md) for release creation, SHA256 verification, GHCR pull/run commands, Docker tag strategy, and the bundled NetEase GPL/MIT data boundary.
18 changes: 18 additions & 0 deletions cmd/server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ import (
"context"
"encoding/json"
"flag"
"fmt"
"log"
"net/http"
"os"
"runtime"
"time"

"github.com/gin-gonic/gin"
Expand All @@ -25,11 +27,27 @@ import (
storagesqlite "github.com/openaudit/openaudit/internal/storage/sqlite"
)

var version = "dev"
var commit = "unknown"
var date = "unknown"

func printVersion() {
fmt.Fprintf(os.Stdout, "OpenAudit version=%s commit=%s date=%s go=%s regex_backends=re2:available,pcre2:%t\n", version, commit, date, runtime.Version(), matcher.PCRE2Available())
}

func main() {
configPath := flag.String("config", "", "config file path")
validateConfig := flag.Bool("validate-config", false, "validate configuration and bundled-rule runtime compatibility, then exit")
printBundledSummary := flag.Bool("print-bundled-summary", false, "print safe bundled-rule runtime summary, then exit")
versionFlag := flag.Bool("version", false, "print build version metadata and exit")
flag.Parse()
api.Version = version
api.Commit = commit
api.BuildTime = date
if *versionFlag {
printVersion()
return
}
cfg, err := config.Load(*configPath)
if err != nil {
log.Fatalf("load config: %v", err)
Expand Down
89 changes: 89 additions & 0 deletions docs/release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Release readiness and versioned distribution

OpenAudit releases are tag-driven and publish versioned binary artifacts plus Docker images. Pull-request validation uses a dry run only: it builds artifacts and images locally in CI, generates checksums, and never creates a GitHub Release, logs in to a registry, or pushes packages.

## Creating a release

1. Ensure `main` is green and the changelog is ready.
2. Create and push a signed or reviewed tag such as `v0.1.0-alpha.1`:
```sh
git checkout main
git pull --ff-only
git tag v0.1.0-alpha.1
git push origin v0.1.0-alpha.1
```
3. The tag-triggered release workflow creates the GitHub Release, uploads binaries and `SHA256SUMS`, and publishes GHCR images using only `GITHUB_TOKEN`.

Do not create release tags from pull requests. Normal branch pushes and pull requests do not publish.

## Version metadata

Binaries support:

```sh
openaudit --version
# or
go run ./cmd/server --version
```

The output includes the OpenAudit version, commit, build date, Go runtime version, and a regex backend summary. RE2 is always available. PCRE2 is reported as available only for binaries built with `CGO_ENABLED=1 -tags pcre2` and linked against libpcre2.

Local development builds default to `version=dev`, `commit=unknown`, and `date=unknown` unless overridden with ldflags or Makefile variables.

## Binary artifacts

Default release binaries are RE2/Go-regexp builds and are intended to be CGO-free. `make build-all` builds:

- `linux/amd64`
- `linux/arm64`
- `darwin/amd64`
- `darwin/arm64`
- `windows/amd64`

`make snapshot` writes compressed local artifacts under `dist/snapshot/`; `dist/` is ignored and must not be committed. `SHA256SUMS` files are generated from sorted artifact paths for deterministic verification.

Verify downloaded artifacts with:

```sh
sha256sum -c SHA256SUMS
```

On macOS, use `shasum -a 256 -c SHA256SUMS` if GNU `sha256sum` is unavailable.

## PCRE2 distribution

PCRE2 remains optional and is not the default. Native PCRE2 cross-compilation requires CGO, libpcre2 development headers, and platform-specific toolchains, so versioned native PCRE2 binary artifacts may be Linux-only or deferred. The supported release distribution for PCRE2 is the Docker image built from the `pcre2` Dockerfile target.

## Docker and GHCR tags

For tag `v0.1.0-alpha.1`, the release workflow publishes:

Default RE2 image:

- `ghcr.io/jacklilyhello/openaudit:v0.1.0-alpha.1`
- `ghcr.io/jacklilyhello/openaudit:latest`

Optional PCRE2 image:

- `ghcr.io/jacklilyhello/openaudit:v0.1.0-alpha.1-pcre2`
- `ghcr.io/jacklilyhello/openaudit:pcre2`

Pull and run the default image:

```sh
docker pull ghcr.io/jacklilyhello/openaudit:v0.1.0-alpha.1
docker run --rm -p 8080:8080 ghcr.io/jacklilyhello/openaudit:v0.1.0-alpha.1 --config /app/config.yml
```

Pull and run the PCRE2 image:

```sh
docker pull ghcr.io/jacklilyhello/openaudit:v0.1.0-alpha.1-pcre2
docker run --rm -p 8080:8080 ghcr.io/jacklilyhello/openaudit:v0.1.0-alpha.1-pcre2 --config /app/config.yml
```

## NetEase data and license boundary

OpenAudit code is MIT licensed. Bundled NetEase source snapshots and generated packs/reports are third-party GPL-3.0-only data. The root MIT license does not relicense that data. Notices and exact hashes are maintained in `THIRD_PARTY_NOTICES.md` and `data/bundled/NETEASE-NOTICE.md`.

Bundled NetEase data is default-disabled. OpenAudit does not download complete upstream data at runtime, does not automatically synchronize NetEase data on startup, and does not print raw NetEase regex patterns in release logs. Operators choose whether to enable the local bundled data and must evaluate licensing and moderation impact for their deployment.
Loading