fix(datastore): apply privacy filters on save and startup

[TimeToBuildBob]

Summary

Privacy filter rules saved from the webui never actually applied. DatastoreWorker starts with an empty PrivacyFilterEngine, insert/heartbeat paths do filter against that engine, and RefreshPrivacyFilter reloads settings.privacy_filters — but nothing called refresh after save, and startup never loaded the persisted key. The UI could save drop/redact rules all day; matching events were stored unchanged.

Fixes #659.

What changed

• Reload the in-memory engine when settings.privacy_filters is written or deleted (SetKeyValue / DeleteKeyValue).
• Load the same key at worker startup so rules survive a server restart.
• Keep RefreshPrivacyFilter as an explicit path (HTTP settings endpoints still call it as belt-and-suspenders).
• Datastore tests: a drop rule saved via set_key_value actually drops a matching insert (no explicit refresh), and the same rule still applies after reopen.

Test plan

• cargo test -p aw-datastore --test datastore
• Save a drop rule in the webui Privacy Filters settings, generate a matching window event, confirm it is not stored.
• Restart the server with that setting present, confirm matching events still drop.

[greptile-apps]

Greptile Summary

The PR synchronizes the datastore worker’s in-memory privacy-filter engine with persisted settings during startup, setting changes, deletion, and transaction recovery.

• Loads persisted privacy filters before processing events.
• Refreshes the engine when the privacy setting is written or deleted.
• Adds save-and-restart regression coverage.
• Keeps explicit HTTP refreshes as a secondary synchronization path.
• Makes sync test database paths unique across parallel executions.

Confidence Score: 3/5

The PR is not yet safe to merge because commit-recovery read errors can leave the active privacy engine reflecting a rolled-back deletion and allow sensitive events to be stored.

The reply claiming the recovery issue was fixed overlooks the rolled-back-delete counterexample: settings mutations update the engine from the uncommitted transaction, while a failed post-rollback reload now preserves that transaction-derived state rather than restoring the durable rules.

Files Needing Attention: aw-datastore/src/worker.rs

Important Files Changed

Filename	Overview
aw-datastore/src/worker.rs	Loads and refreshes privacy rules across startup, settings mutations, explicit refreshes, and commit recovery.
aw-datastore/tests/datastore.rs	Adds regression coverage for immediate privacy-filter activation, deletion, and persistence across datastore reopening.
aw-server/src/endpoints/settings.rs	Explicitly refreshes privacy filters after successful HTTP setting writes and deletions.
aw-sync/tests/sync_roundtrip.rs	Strengthens temporary database path uniqueness for parallel sync tests.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  S[Privacy setting write or delete] --> T[SQLite transaction]
  T --> R[Reload in-memory engine from transaction]
  R --> C{Commit succeeds?}
  C -->|Yes| E[Engine and durable setting agree]
  C -->|No| D[Reload from durable connection]
  D --> N[Process later inserts and heartbeats]

Loading

_{Reviews (3): Last reviewed commit: "fix(datastore): only clear privacy rules..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 87.80488% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.85%. Comparing base (656f3c9) to head (67195c5).
⚠️ Report is 93 commits behind head on master.

Files with missing lines	Patch %	Lines
aw-datastore/src/worker.rs	76.47%	4 Missing ⚠️
aw-datastore/tests/datastore.rs	92.59%	4 Missing ⚠️
aw-server/src/endpoints/settings.rs	66.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #662      +/-   ##
==========================================
+ Coverage   70.81%   78.85%   +8.04%     
==========================================
  Files          51       66      +15     
  Lines        2916     5525    +2609     
==========================================
+ Hits         2065     4357    +2292     
- Misses        851     1168     +317     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[TimeToBuildBob]

macOS CI failure was the known sync_roundtrip flake, not this PR

Run 33137074103 failed only on macOS, in aw-sync:

thread panicked at aw-datastore/src/datastore.rs:129:6:
Failed to upgrade database when adding data field to buckets:
  SqliteFailure(..., Some("duplicate column name: data"))

test_own_data_does_not_return_via_peer and test_push_does_not_reexport_synced_buckets both call round_trip(), which builds temp db paths from pid + name + nanos. Run in parallel on macOS they can land in the same clock tick, open the same SQLite file, and race the v1→v2 migration. Nothing to do with privacy filters — it hits any PR in this repo.

That is exactly what #655 fixes (green, mergeable, open since 2026-08-24). To unblock this branch's CI the same fix was carried here, and cfbfad9 makes aw-sync/tests/sync_roundtrip.rs byte-identical to #655's version — so #655 and #662 land in either order with no conflict, and both keep the timestamp component (the counter alone resets on PID reuse, and these tests never delete their temp dbs).

Also in this push: 95898ce addresses the Greptile P1 — a failed batched commit rolled SQLite back but left the in-memory privacy engine holding the rolled-back rules.

Local: cargo test -p aw-sync --test sync_roundtrip 3/3, cargo test -p aw-datastore 13/13, fmt clean, no new clippy warnings.

Merging #655 is still worth doing on its own — it fixes the flake on master, where this PR's copy does not help.

[TimeToBuildBob]

@greptileai review

[TimeToBuildBob]

@greptileai review