Skip to content

feat: audit feedback writes from the apiserver - #30

Open
mattdjenkinson wants to merge 1 commit into
feat/authzfrom
feat/audit
Open

mattdjenkinson wants to merge 1 commit into
feat/authzfrom
feat/audit

Conversation

@mattdjenkinson

Copy link
Copy Markdown
Contributor

Feedback writes now reach the audit log with their bodies, so activity has a full record and the search indexer can index posts. Private content (submissions, security reports, uploads, draft feedback) is kept to metadata. Closes #11.

Stacked on #29 (authz), which is on #28 down to #16.

Why the apiserver audits itself

Milo proxies requests to feedback.miloapis.com without their bodies, so its events carry only metadata, and the search indexer skips events without a responseObject. As the design says, the feedback apiserver runs its own audit backend with the generic apiserver's --audit-policy-file and --audit-webhook-config-file and sends to the same Vector webhook as Milo.

What's recorded

Requests Level
create, update, patch, delete on posts, releases, changelog entries (and their status), votes, subscriptions, feedback sources RequestResponse
anything on submissions, security reports (and every subresource), attachment uploads, PostQuery Metadata
get, list, watch; health, discovery, OpenAPI not logged here; Milo's events cover reads

Side effects are written in the same transaction as the request that caused them (#7 to #9), so they're recorded as annotations on that request's event without exposing content:

  • feedback.miloapis.com/subscribed: a vote or a review followed a post for the user;
  • feedback.miloapis.com/votes-moved-to: a merged post's votes moved;
  • feedback.miloapis.com/vote-granted: a merge or a linked approval gave the submitter a vote (user/post);
  • feedback.miloapis.com/security-report: the report a submission became, so the submission's metadata event links to it.

Config

config/components/audit-logging adds the policy and the webhook ConfigMaps, plus the flags (--audit-webhook-mode=batch, 10s initial backoff) and mounts. The core-control-plane overlay includes it. The webhook points at https://vector-sidecar.activity-system.svc.cluster.local:8080/events, matching Milo's.

For #13

  • Supply the webhook's CA as ConfigMap feedback-audit-webhook-ca (key ca.crt), from the control-plane trust bundle Milo uses.
  • Add a rule to Milo's audit policy, before its catch-all, setting level: None for create, update, patch, delete and deletecollection on group: feedback.miloapis.com. Otherwise each write is recorded twice, because activity's dedupe key includes the timestamp. Milo keeps logging reads.
  • In staging, check that the events reach activity and that the search indexer picks up posts. That needs real activity and search, so it can't be checked here.

Testing

  • TestAuditPolicy loads the shipped policy into the apiserver's own evaluator and checks the level for 21 requests: every write path, every private kind and subresource, reads, and health endpoints.
  • Kind cluster, with the audit component and a sink standing in for Vector:
    • sync's post creates, votes, unvotes, a post merge and a staff hide arrived at RequestResponse with request and response objects;
    • submissions, the review, to-security-report, a security report and its status, and PostQuery arrived at Metadata with no bodies;
    • the annotations appeared on the vote, the review, the merge and the move;
    • the vote's audit ID matched the Audit-Id header kubectl got back, so Milo's ID carries through;
    • user-context events carry the parent-type and parent-name extras that activity scopes by;
    • no submission, report or draft text appeared anywhere in the received events.

Milo proxies requests to the feedback API without their bodies, so the
apiserver now runs its own audit backend and is the record for writes to
its API.

- An audit-logging component adds the policy, a webhook config for
  activity's Vector in activity-system, and the apiserver flags and mounts.
  The core-control-plane overlay includes it.
- Writes to posts, releases, changelog entries, votes, subscriptions and
  feedback sources are logged at RequestResponse, which the search indexer
  needs. Submissions, security reports, uploads and PostQuery are logged at
  Metadata. Reads are left to Milo.
- Side effects are recorded as annotations on the request that caused
  them: subscribed, votes-moved-to, vote-granted and security-report.
- A test evaluates the shipped policy with the apiserver's own evaluator.

Part of datum-cloud/enhancements#943. Closes #11.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant