Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 30 additions & 8 deletions .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@ jobs:
timeout-minutes: 20
permissions:
contents: read
issues: write # notify-scheduled-failure
strategy:
fail-fast: false
matrix:
Expand All @@ -92,29 +91,52 @@ jobs:
run: |
set -euo pipefail
case "${IMAGE}" in
kates) tag=$(awk '/repository: ghcr.io\/bmscomp\/kates$/ {getline; print $2}' "${REF}" | tr -d '"'); ref="ghcr.io/bmscomp/kates:${tag}" ;;
kates) tag=$(awk '/repository: ghcr.io\/bmscomp\/kates$/ {found=1; next} found && /^[[:space:]]*tag:/ {print $2; exit}' "${REF}" | tr -d '"'); ref="ghcr.io/bmscomp/kates:${tag}" ;;
connect) ref=$(grep -m1 '^image:' "${REF}" | awk '{print $2}' | tr -d '"') ;;
kates-tester) ref=$(grep -m1 'ghcr.io/bmscomp/kates-tester:' "${REF}" | sed -E 's/.*(ghcr.io\/bmscomp\/kates-tester:[^"]+).*/\1/') ;;
esac
if [ -z "${ref}" ] || [[ "${ref}" != *:* ]]; then
echo "::error::could not resolve the ${IMAGE} image from ${REF}"
tag_part="${ref##*:}"
if [ -z "${ref}" ] || [[ "${ref}" != *:* ]] || [ -z "${tag_part}" ] || [[ "${tag_part}" == -* ]]; then
echo "::error::could not resolve a valid ${IMAGE} image tag from ${REF} (got '${ref}')"
exit 1
fi
echo "ref=${ref}" | tee -a "$GITHUB_OUTPUT"

- name: Scan (Trivy)
# Same policy as ci.yml and the publish-*.yml workflows: HIGH is reported
# in the log, CRITICAL fails the weekly rescan.
- name: Report HIGH and CRITICAL vulnerabilities (Trivy)
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
with:
scan-type: image
image-ref: ${{ steps.pin.outputs.ref }}
scanners: vuln
severity: CRITICAL,HIGH
exit-code: "0"
ignore-unfixed: true
trivyignores: .trivyignore.yaml

- name: Gate on CRITICAL vulnerabilities (Trivy)
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
with:
scan-type: image
image-ref: ${{ steps.pin.outputs.ref }}
scanners: vuln
severity: CRITICAL
exit-code: "1"
ignore-unfixed: true
trivyignores: .trivyignore.yaml

- name: Open an issue for a failed weekly scan
if: failure() && github.event_name == 'schedule'
uses: ./.github/actions/notify-scheduled-failure
notify-failure:
name: Open an issue for a failed weekly scan
needs: [rescan-released]
if: always() && needs.rescan-released.result == 'failure' && github.event_name == 'schedule'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: ./.github/actions/notify-scheduled-failure
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
8 changes: 5 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -493,9 +493,11 @@ cli-install: ## Build and install CLI on this machine (CLI_INSTALL_DIR to choos
fi; \
fi
sudo cp cli/dist/kates $(CLI_INSTALL_PATH)
sudo xattr -dr com.apple.provenance $(CLI_INSTALL_PATH) 2>/dev/null || true
sudo xattr -dr com.apple.quarantine $(CLI_INSTALL_PATH) 2>/dev/null || true
sudo codesign -f -s - $(CLI_INSTALL_PATH)
@if [ "$$(uname -s)" = "Darwin" ]; then \
sudo xattr -dr com.apple.provenance $(CLI_INSTALL_PATH) 2>/dev/null || true; \
sudo xattr -dr com.apple.quarantine $(CLI_INSTALL_PATH) 2>/dev/null || true; \
sudo codesign -f -s - $(CLI_INSTALL_PATH); \
fi
@echo "✅ Installed:"
@# Run the file just written, by absolute path. Asking `kates` would
@# report whatever PATH resolves to, which is exactly the thing this
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
<a href="https://github.com/bmscomp/kates/releases/latest"><img src="https://img.shields.io/github/v/release/bmscomp/kates?label=release&logo=github" alt="Latest release" /></a>
<a href="https://github.com/marketplace/actions/kates-quality-gate"><img src="https://img.shields.io/badge/marketplace-kates--quality--gate-blue?logo=githubactions&logoColor=white" alt="GitHub Marketplace" /></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/bmscomp/kates" alt="License" /></a>
<a href="https://katesplatform.io/"><img src="https://img.shields.io/badge/site-katesplatform.io-0F172A" alt="Kates website" /></a>
<a href="https://bmscomp.github.io/kates/"><img src="https://img.shields.io/badge/docs-the%20book-1A5BCC" alt="The Kates book" /></a>
</p>
</div>
Expand All @@ -22,7 +23,7 @@ You find out how a Kafka cluster handles a broker failure on the day a broker fa

Performance tests work against any Kafka cluster the Kates API can reach; fault injection targets Strimzi-managed brokers, by default the cluster `krafter` in namespace `kafka`. Kates has three parts: the `kates` CLI (Go), the Kates API (Quarkus) that runs the tests and faults and keeps the results in PostgreSQL, and the signed Helm charts that install Kafka and everything around it.

**[The book](https://bmscomp.github.io/kates/)** · [Why Kates](#why-kates) · [Quick start](#quick-start) · [CLI at a glance](#cli-at-a-glance) · [GitHub Action](#github-action) · [Architecture](#architecture) · [Documentation](#documentation) · [Helm charts](#helm-charts) · [Building from source](#building-from-source) · [Contributing](#contributing)
**[Website](https://katesplatform.io/)** · **[The book](https://bmscomp.github.io/kates/)** · [Why Kates](#why-kates) · [Quick start](#quick-start) · [CLI at a glance](#cli-at-a-glance) · [GitHub Action](#github-action) · [Architecture](#architecture) · [Documentation](#documentation) · [Helm charts](#helm-charts) · [Building from source](#building-from-source) · [Contributing](#contributing)

<p align="center">
<img src="docs/assets/demo.gif" width="900" alt="Animated terminal demo of kates health, kates test create --type LOAD --records 100000 --wait, kates gate --min-grade C, and kates security audit running against a live Kafka cluster" />
Expand All @@ -31,7 +32,7 @@ Performance tests work against any Kafka cluster the Kates API can reach; fault
## What it does

- **Performance tests.** Eight workload types (`LOAD`, `STRESS`, `SPIKE`, `ENDURANCE`, `VOLUME`, `CAPACITY`, `ROUND_TRIP`, `INTEGRITY`); `kates test types` lists every type the Kates API accepts. The default `native` benchmark backend runs Kafka producers and consumers inside the Kates API on virtual threads; `--backend trogdor` hands the workload to a Trogdor coordinator you run yourself. Reports give throughput, latency percentiles (`P50`, `P95`, `P99`) and error rate, and export as CSV, JUnit XML, Markdown, HTML or latency heatmaps.
- **Fault injection.** Thirteen disruption types, from broker pod kills and network partitions to CPU, memory and I/O stress, DNS errors, disk fill and node drain (`kates disruption types`). The default install runs eleven of them through LitmusChaos. Network latency and disk fill time out, because the `kates-chaos` chart ships no Litmus experiment for them (see `experiments.definitions` in its values). The direct Kubernetes chaos provider covers eight.
- **Fault injection.** Thirteen disruption types, from broker pod kills and network partitions to CPU, memory and I/O stress, DNS errors, disk fill and node drain (`kates disruption types`), run through LitmusChaos (eleven out of the box) or directly through the Kubernetes API (eight); [Chaos in practice](docs/book/07-chaos-practice.md) covers provider support and adding custom Litmus experiments.
- **Resilience runs and disruption plans.** `kates resilience run` pairs one load test with one fault and reports the before/after change and the recovery time. `kates disruption run` executes multi-step disruption plans; `kates disruption playbook list` shows the built-in playbooks. Plans get blast-radius checks, a dry run, ISR and consumer-lag tracking, and an A–F SLA grade against absolute latency, throughput and recovery-time thresholds.
- **Security audit.** `kates security audit` grades broker configuration A–F. Related commands map the checks to CIS, SOC 2 and PCI DSS, save a baseline and report drift from it.
- **CI gates.** `kates gate --min-grade B` runs a load test and exits non-zero below the performance grade, `kates test apply -f <scenario> --wait` fails when a scenario violates its SLA gates, and `kates security gate` does the same for the security grade. Finished tests can notify webhooks (`kates webhook add`).
Expand Down Expand Up @@ -281,7 +282,7 @@ make kates-local # JVM image kates:local (quicker to build), loade
```

- `make` on its own lists every target.
- On Linux, `make cli-install` ends with a `codesign` error, a macOS-only step, after the binary is already installed. The binary works, but the target stops before checking whether another `kates` comes earlier on your `PATH`.
- On macOS, `make cli-install` also strips quarantine attributes and ad-hoc signs the installed binary so Gatekeeper does not block it.
- On Apple Silicon, local `./mvnw` builds need Rosetta 2 (`softwareupdate --install-rosetta`): the gRPC code generator this Quarkus version uses ships only an x86-64 macOS binary. Docker builds are not affected.
- `(cd kates && ./mvnw quarkus:dev)` runs the backend with API-key authentication off, but it still needs a reachable PostgreSQL (`QUARKUS_DATASOURCE_JDBC_URL`) and Kafka (`KATES_KAFKA_BOOTSTRAP_SERVERS`).
- Both Dockerfiles take the repository root as build context, as in `docker build -f kates/Dockerfile.native -t kates:native-local .`.
Expand Down
Loading