From 60d8269596c1c40e5d4efa70861db20c421daf32 Mon Sep 17 00:00:00 2001 From: Omid Mirzaei Date: Sat, 18 Jul 2026 20:40:00 +0400 Subject: [PATCH 1/5] docs: keep engineer standards, drop process notes --- docs/CURSOR_BUILD.md | 269 ----------------------------------- docs/GIT_WORKFLOW.md | 191 +++---------------------- docs/HUMAN_CODE_STANDARDS.md | 118 ++++----------- docs/PRD.md | 9 +- 4 files changed, 51 insertions(+), 536 deletions(-) delete mode 100644 docs/CURSOR_BUILD.md diff --git a/docs/CURSOR_BUILD.md b/docs/CURSOR_BUILD.md deleted file mode 100644 index 539b968..0000000 --- a/docs/CURSOR_BUILD.md +++ /dev/null @@ -1,269 +0,0 @@ -# Cursor Build Instructions — WalletOps Companion - -**Where to run this:** a **new empty git repo** (e.g. `walletops`), not Career OS. -**Spec source of truth:** [`walletops-companion-mvp.md`](./walletops-companion-mvp.md) -**Style law:** [`human-code-standards.md`](./human-code-standards.md) - -Copy sections below into Cursor Agent as separate chats/phases. Do not ask for the whole app in one prompt. - ---- - -## 0. Repo bootstrap (you do this once) - -```bash -mkdir walletops && cd walletops -git init -# create private or public GitHub repo, add remote -``` - -Copy into the new repo (as plain files you own): - -- `docs/PRD.md` ← content from `walletops-companion-mvp.md` -- `docs/HUMAN_CODE_STANDARDS.md` ← from `human-code-standards.md` -- This file as `docs/CURSOR_BUILD.md` -- `docs/GIT_WORKFLOW.md` ← from `git-workflow.md` - -Then paste **Phase prompts** one at a time. After each phase: read the diff, rename, **you** commit on a feature branch with human messages and backdated cadence (see GIT_WORKFLOW). Never push straight to `main`. - ---- - -## Global system rules (paste at the start of EVERY Cursor chat) - -``` -You are helping me implement WalletOps Companion from docs/PRD.md. - -Hard rules: -1. Follow docs/PRD.md and docs/HUMAN_CODE_STANDARDS.md exactly. -2. No AI-looking code: no tutorial comments, no emoji READMEs, no Generated-by headers, no unused deps, no Clean Architecture theater. -3. Prefer boring, senior-engineer style. Match Flutter stack: Bloc/Cubit, GetIt, go_router, Dio, flutter_secure_storage. -4. Go API: stdlib + chi (or stdlib mux), pgx or sqlc, goose/golang-migrate, slog, golang-jwt, bcrypt. -5. Do not invent product scope beyond the PRD. No real custody/keys. Mock AI by default. -6. After each task: list files touched + how to run tests. Do not rewrite unrelated files. -7. Keep commits-sized diffs. If something is unclear, ask once; otherwise use PRD defaults. -8. Never put secrets in the repo. Use .env.example only. -9. I will edit names and README myself — leave room for that; do not over-document. -10. Follow docs/GIT_WORKFLOW.md: never commit to main; prefer feature branches. Do not create git commits unless I explicitly ask — I commit myself with my messages and dates. -11. Zero AI fingerprint: no tutorial comments, no emoji READMEs, no "Generated by", no over-abstract Clean Architecture theater. -``` - ---- - -## Phase 1 — Skeleton + Docker + migrations - -``` -Read docs/PRD.md §9–10 and HUMAN_CODE_STANDARDS. - -Create monorepo skeleton: -- docker-compose.yml: postgres:16, api service placeholder -- .env.example: DATABASE_URL, JWT_SECRET, WEBHOOK_SECRET, AI_PROVIDER=mock, AI_API_KEY=, HTTP_ADDR=:8080 -- api/go.mod module github.com//walletops/api (use placeholder path I will replace) -- api/migrations: users, refresh_tokens, alert_rules, events, ai_summaries (columns per PRD) -- api/cmd/server/main.go: boots config, db ping, /v1/health returning {"status":"ok"} -- .gitignore for Go, Flutter, .env -- minimal README: name, one paragraph, how to compose up + migrate - -No Flutter yet. No AI. Tests: health handler smoke test optional. -Stop when `docker compose up` gives postgres and API health 200. -``` - -**You after Phase 1:** fix module path, commit `chore: repo skeleton and health`. - ---- - -## Phase 2 — Auth - -``` -Implement auth per PRD §7.1 and API table §10. - -- POST /v1/auth/register, /login, /refresh, GET /v1/me -- bcrypt passwords, JWT access + refresh stored hashed in DB -- middleware RequireAuth -- table-driven tests: register ok, duplicate email, bad login, refresh - -No alert rules yet. Keep handlers thin; put SQL in a small store package. -``` - -**You after Phase 2:** try register/login with curl; commit `feat: jwt auth`. - ---- - -## Phase 3 — Alert rules CRUD - -``` -Implement alert rules CRUD for the authenticated user (PRD §7.2). -Validation: name length, event_type allowlist, threshold optional. -Tests for create/list/update/delete and foreign-id 404. -``` - -**Commit:** `feat: alert rules crud` - ---- - -## Phase 4 — Webhooks + idempotency - -``` -Implement POST /v1/webhooks/events with HMAC-SHA256 hex signature header X-Signature (PRD §7.3). -- Map user_ref to user (seed a webhook_clients table or column on users for demo: user_ref) -- Idempotency unique (user_id, idempotency_key) -- Invalid sig → 401; valid → persist status=pending -- Tests: good sig, bad sig, replay - -Add scripts/seed_webhooks.sh that curls two sample events. -Document WEBHOOK_SECRET usage in README briefly (no emoji). -``` - -**Commit:** `feat: signed webhook ingest` - ---- - -## Phase 5 — Worker - -``` -In-process worker (PRD §7.4): -- poll/claim pending events -- match alert rules -- retries up to 5; set failed + last_error -- expose last tick in /v1/health - -Tests: process success; process failure increments attempts. -No Redis. -``` - -**Commit:** `feat: event worker with retries` - ---- - -## Phase 6 — AI summarize (mock first) - -``` -POST /v1/ai/summarize (PRD §7.5). -- AI_PROVIDER=mock returns deterministic schema-valid JSON -- real OpenAI-compatible client behind interface, only if AI_PROVIDER=openai -- allowlist fields into the prompt builder -- persist ai_summaries row -- tests: mock ok; reject other user's event ids; max 20 -``` - -**Commit:** `feat: ai summarize with mock provider` - ---- - -## Phase 7 — Flutter app shell + auth - -``` -Create mobile/ Flutter app. -Stack locked: flutter_bloc, get_it, go_router, dio, flutter_secure_storage, equatable. - -Structure: -lib/ - app.dart - main.dart - core/ (network, storage, di, router, theme) - features/auth/... - features/events/... (stubs ok) - features/rules/... (stubs ok) - -Implement register/login/logout, auth gate, token refresh interceptor (one retry on 401). -API base from --dart-define=API_BASE=http://10.0.2.2:8080 for Android emulator (document iOS localhost). - -Widget/bloc tests: auth gate redirects when logged out. -UI: restrained Material 3, operational copy, no marketing onboarding carousel. -``` - -**Commit:** `feat: flutter auth session` - ---- - -## Phase 8 — Events + rules UI - -``` -Wire events list/detail and alert rules list/create/edit to API. -Pull-to-refresh; status chips; explain entry point on multi-select or detail button. -Handle empty/error/loading in Cubits cleanly. -Add fixture-based widget test for event list. -``` - -**Commit:** `feat: events and rules screens` - ---- - -## Phase 9 — Explain UI + polish - -``` -Call /v1/ai/summarize; show title, bullets, risk_level, follow_ups. -Settings: logout + show API base. -Add docs/threat-model.md (short, pragmatic). -Add docs/architecture.md with mermaid sequence: webhook → db → worker → mobile → summarize. -Tighten README demo script to <5 minutes. -``` - -**Commit:** `docs: architecture threat model and demo path` - ---- - -## Phase 10 — CI - -``` -.github/workflows/ci.yml: -- Go: setup-go, cache, go test ./... -- Flutter: stable channel, pub get, analyze, test -Do not fail on format-only nits unless already configured. -``` - -**Commit:** `ci: go and flutter tests` - ---- - -## Final human pass (mandatory — you, not Cursor) - -1. Open every public file; delete tutorial comments. -2. Rename packages/symbols that feel generic. -3. Rewrite README intro in your voice. -4. Make 3–5 small commits with real messages. -5. Run the demo script cold on a clean machine/VM if possible. -6. Complete the ownership checklist in HUMAN_CODE_STANDARDS.md. - ---- - -## Single “anti-slop” repair prompt (if code already looks generated) - -``` -Review the repo against docs/HUMAN_CODE_STANDARDS.md. -Remove tutorial comments, dead code, unused deps, and emoji README fluff. -Collapse over-abstracted interfaces that have a single implementation and no test mock need. -Keep behavior identical. Show a short diff summary per file. Do not expand scope. -``` - ---- - -## What not to ask Cursor - -- “Build the whole app” -- “Make it production-ready SaaS” -- “Add Fireblocks / real wallet” -- “Use Clean Architecture / DDD fully” -- “Make the UI stunning / award-winning” - ---- - -## Suggested env for local demo - -``` -DATABASE_URL=postgres://walletops:walletops@localhost:5432/walletops?sslmode=disable -JWT_SECRET=dev-only-change-me -WEBHOOK_SECRET=dev-webhook-secret -AI_PROVIDER=mock -HTTP_ADDR=:8080 -``` - -Flutter: - -``` -flutter run --dart-define=API_BASE=http://127.0.0.1:8080 -``` - ---- - -## Career OS note - -Implementation stays **outside** this knowledge base. When the GitHub URL exists, add it to `09-github/` and set `05-projects/active-project.md`. diff --git a/docs/GIT_WORKFLOW.md b/docs/GIT_WORKFLOW.md index ebcd0ba..8313163 100644 --- a/docs/GIT_WORKFLOW.md +++ b/docs/GIT_WORKFLOW.md @@ -1,186 +1,33 @@ -# WalletOps — Git & Branch Workflow - -Copy this into the app repo as `docs/GIT_WORKFLOW.md`. -**Rule:** nothing lands on `main` except via reviewed merge. No drive-by pushes of half-done agent output. - ---- - -## Branch model +# Git workflow ``` -main # always runnable; protected - └── develop # integration (optional but recommended) - ├── feature/api-skeleton - ├── feature/api-auth - ├── feature/api-webhooks - ├── feature/api-worker - ├── feature/api-ai-summarize - ├── feature/mobile-auth - ├── feature/mobile-events - └── chore/ci +main # runnable releases + └── develop # integration + └── feature/* / fix/* ``` -### Rules +## Rules -1. **Never commit agent output straight to `main`.** -2. Create a feature branch **before** each Cursor phase. -3. One phase ≈ one branch ≈ several small commits. -4. Merge to `develop` (or `main` if you skip develop) only when that phase’s tests pass. -5. Prefer **merge commits** or **squash** with a human-written summary — your choice, stay consistent. -6. Remote: push feature branches; open PR even if solo (keeps history clean). -7. `.env` never committed. Only `.env.example`. +1. Land work through PRs into `develop`; promote to `main` when demo-ready. +2. One concern per branch; prefer several small commits over one dump. +3. Never commit `.env` — only `.env.example`. +4. Protect `main` on GitHub (PR required, no force-push). -### Bootstrap (run once in walletops) +## Commit messages -```bash -cd ~/code/walletops -git checkout -b main -git add docs/ -git commit -m "docs: add prd and build standards" - -git checkout -b develop -# work happens on feature/* from develop -``` - -Protect `main` on GitHub: require PR, no force-push. - ---- - -## Human-like commits (no AI smell) - -### Good messages +Write what a teammate would write: ``` add postgres migrations for users and events -wire jwt login and refresh -reject webhook with bad hmac -claim pending events in worker loop -flutter auth gate with secure storage -show event status chips on home -mock ai summarize returns fixed schema -ci: run go test and flutter test -``` - -### Bad messages - -``` -Initial commit -Update files -WIP -Generated by Cursor -feat: implement comprehensive robust scalable... -``` - -### Commit size - -- Prefer **3–8 files** per commit when possible -- Separate: schema → API → tests → docs -- After each Cursor phase: **you** split the agent’s dump into 2–4 commits; do not one-commit the whole phase if it was large - ---- - -## Backdated cadence (last ~2 weeks) - -Goal: history reads like steady work, not one afternoon. -**You** set author/committer dates when committing. Cursor must not invent dates. - -Today’s reference date in Career OS: **2026-07-16**. -Window: **2026-07-02 → 2026-07-16**. - -### How to commit with a past date - -```bash -# example: 2026-07-03 19:40 local -export GIT_AUTHOR_DATE="2026-07-03T19:40:00+04:00" -export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE" -git commit -m "wire jwt login and refresh" -unset GIT_AUTHOR_DATE GIT_COMMITTER_DATE -``` - -Or one-liner: - -```bash -GIT_AUTHOR_DATE="2026-07-03T19:40:00+04:00" \ -GIT_COMMITTER_DATE="2026-07-03T19:40:00+04:00" \ -git commit -m "wire jwt login and refresh" +hmac webhook ingest with idempotency +worker claim loop and retries +flutter auth gate and secure storage ``` -Vary times (evenings/weekends ok). Do not use identical timestamps. Do not backdate **after** you already pushed those commits to a shared remote without a careful rewrite — set dates **as you go**. +Avoid noise: `Update files`, `Enhance architecture`, empty `Initial commit` spam. -### Suggested schedule (map phases → dates) +## Review checklist -| When (Dubai +04) | Branch | Commits (examples) | -|------------------|--------|--------------------| -| Thu 2026-07-02 20:10 | `feature/api-skeleton` | `docs: add prd and build standards` → `chore: repo skeleton and docker compose` → `add postgres migrations for core tables` → `serve health endpoint` | -| Fri 2026-07-03 18:30–21:00 | `feature/api-auth` | `add users and refresh token tables` → `register and login with bcrypt` → `jwt middleware and me endpoint` → `test auth happy path and duplicates` | -| Sat 2026-07-05 11:00–16:00 | `feature/api-rules` | `alert rules crud` → `validate event types` → `tests for rules ownership` | -| Sun 2026-07-06 15:00–20:00 | `feature/api-webhooks` | `hmac webhook ingest` → `idempotent event create` → `seed script for sample events` | -| Mon 2026-07-07 19:00 | merge | PR: webhooks → develop | -| Tue 2026-07-08 18:00–21:00 | `feature/api-worker` | `worker claim loop` → `match alert rules` → `retries and failed status` | -| Wed 2026-07-09 19:30 | `feature/api-ai-summarize` | `mock ai provider` → `summarize endpoint with schema` → `reject foreign event ids` | -| Thu 2026-07-10 17:00–21:00 | `feature/mobile-auth` | `flutter app shell` → `dio client and secure storage` → `login register screens` → `auth gate test` | -| Fri 2026-07-11 12:00 | merge | PR: mobile-auth → develop | -| Sat 2026-07-12 10:00–18:00 | `feature/mobile-events` | `events list and detail` → `alert rules screens` → `pull to refresh` | -| Sun 2026-07-13 16:00–19:00 | same / polish | `explain sheet for ai summary` → `settings logout` | -| Mon 2026-07-14 20:00 | `docs/architecture` | `threat model` → `architecture notes and demo script` | -| Tue 2026-07-15 19:00 | `chore/ci` | `github actions for go and flutter` | -| Wed 2026-07-16 | `main` | merge develop → main; tag `v0.1.0-mvp` if demo works | - -If a phase slips, **shift later dates** — keep order, don’t compress everything into one day. - -### If Phase 1 already committed “today” - -Before pushing to GitHub: - -```bash -# only if NOT pushed yet — rewrite last commit date -GIT_AUTHOR_DATE="2026-07-02T20:10:00+04:00" \ -GIT_COMMITTER_DATE="2026-07-02T20:10:00+04:00" \ -git commit --amend --no-edit --reset-author -``` - -For multiple local commits, use an interactive-free approach: soft reset and recommit with dates, or `git rebase` with `exec` — do this **before** first push. After push, avoid rewriting public history. - ---- - -## PR template (solo is fine) - -**Title:** short human summary - -**Body:** - -``` -## What -- - -## How to test -- - -## Notes -- -``` - -No “Cursor generated this”. - ---- - -## After every Cursor phase (checklist) - -1. Stay on `feature/...` branch -2. Delete tutorial comments / dead code -3. Split into small commits with **past dates from the table** -4. Run tests for that phase -5. Push branch + open PR -6. Paste summary in Career OS chat for review before next phase - ---- - -## What to tell the build chat (paste once) - -``` -Also follow docs/GIT_WORKFLOW.md: -- Never commit to main; use feature branches from develop -- Do not create commits yourself unless I ask — I will commit with my own messages and dates -- No AI comments, no emoji README, no "generated by" text -- Keep diffs phase-scoped -``` +- [ ] Tests for the changed path pass locally +- [ ] No secrets or personal notes in the tree +- [ ] README / architecture still match behavior diff --git a/docs/HUMAN_CODE_STANDARDS.md b/docs/HUMAN_CODE_STANDARDS.md index 4321868..7179896 100644 --- a/docs/HUMAN_CODE_STANDARDS.md +++ b/docs/HUMAN_CODE_STANDARDS.md @@ -1,106 +1,42 @@ -# Human Code Standards — WalletOps +# Code standards -Use these rules in the **app repo** so the codebase reads like a senior engineer’s work, not a chat dump. +Keep the repo readable for an interviewer who opens random files. -You will present this as **your** project. That means: you understand every line, you can change any module cold, and you have personally edited naming, structure, and README voice. Cursor is a keyboard — not the author of record. +## Avoid ---- +1. Comments that narrate the code (`// This function handles…`) +2. README emoji walls or marketing filler +3. Generic names: `MyApp`, `Utils`, `Helper`, `ManagerImpl`, `DataService` +4. Giant god-files and unused dependencies +5. Over-abstract ports/adapters for a small MVP +6. Fake or identical commit spam -## Hard bans (AI smell) - -Do **not** leave any of this in the final repo: - -1. Comments like `// This function handles…`, `// Import necessary packages`, `// TODO: implement`, chatbot explanations above every function -2. README emoji walls, “✨ Features”, “Built with ❤️”, badge spam unrelated to CI -3. Generic names: `MyApp`, `Utils`, `Helper`, `ManagerImpl`, `DataService`, `handleStuff` -4. Huge god-files generated in one shot (`all_widgets.dart`, `handlers.go` with everything) -5. Copy-pasted license headers or “Generated by Cursor/Copilot/ChatGPT” -6. Fake commit messages (`Initial commit` × 40, or `Update files` for every change) -7. Over-abstract ports/adapters for a 2-week MVP (no Clean Architecture theater) -8. Example.com emails, lorem ipsum, `foo/bar/baz` as demo content — use wallet-ops flavored fixtures -9. Purple gradient marketing UI, Inter-only default landing screens, “Welcome to your journey” copy -10. Unused dependencies left “for later” -11. `any` / `dynamic` / `interface{}` everywhere to silence the compiler -12. Identical doc comments restating the function name - ---- - -## Voice & structure (what humans do) +## Prefer ### Go -- Small packages by domain: `auth`, `events`, `rules`, `webhook`, `worker`, `ai` -- Prefer clear functions over interface explosion; introduce interfaces at boundaries you mock in tests -- Errors: wrap with `%w`; return sentinel or typed errors where tests care -- Table-driven tests with short names: `ok`, `bad_sig`, `replay` -- Logging: slog JSON; fields `request_id`, `event_id`, `attempt` — not essays +- Packages by domain: `auth`, `events`, `rules`, `webhook`, `worker`, `ai` +- Interfaces at test boundaries; clear functions elsewhere +- Errors wrapped with `%w` +- Table-driven tests with short names (`ok`, `bad_sig`, `replay`) +- slog JSON with useful fields (`event_id`, `attempt`) ### Flutter -- Mirror how you already ship: `features//` with `data`, `domain` (light), `presentation` -- Bloc/Cubit + GetIt + go_router + Dio — not Provider + Riverpod + GetX mixed -- One Dio interceptor for auth refresh; keep it boring -- UI: Material 3, restrained; dark-friendly is fine; no glassmorphism kits -- Strings: short, operational (“Failed to verify webhook”, “No events yet”) -- Tests: pump real widgets with fakes; skip golden spam for MVP - -### Git history (important for “I wrote this”) - -Full rules: [`git-workflow.md`](./git-workflow.md) — branches, PRs, **backdated cadence over ~2 weeks**, no dumping on `main`. - -Build in **thin commits** you could have typed: - -``` -add postgres migrations for users and events -wire jwt login and refresh -hmac webhook ingest with idempotency -worker claim loop and retries -flutter auth gate and secure storage -events list and detail -ai summarize with mock provider -ci: go test and flutter test -``` - -After Cursor sessions: **you** commit (agent does not). Rewrite names, delete dead code. Prefer many small commits on `feature/*` over one mega dump on `main`. - -### README voice - -Write like an engineer handing a repo to another engineer: - -- What it is (2–3 sentences) -- Architecture (mermaid) -- Run locally -- Env vars -- Demo script -- Design notes (HMAC, idempotency, AI schema) - -No “revolutionary”, no “leveraging cutting-edge AI”. - ---- - -## Ownership checklist (before you claim end-to-end) - -Run this yourself. If you fail any item, fix it before interviews / LinkedIn. - -- [ ] Delete any file you cannot explain in 60 seconds -- [ ] Rename at least 5 symbols Cursor invented to names you prefer -- [ ] Hand-write or heavily edit README + threat-model -- [ ] Break one test on purpose, fix it yourself -- [ ] Change one API field end-to-end (API → Flutter) without the agent -- [ ] Draw the sequence for webhook → worker → list on paper -- [ ] Be able to answer: “Why not NestJS here?” and “Why Bloc not Riverpod?” - -If an interviewer asks “Did you use AI?”, honest answer: *I used an IDE agent for speed; I designed the system, own the code, and can modify any part.* That is normal in 2026. Claiming you never used tools while the repo smells generated is worse. +- `features//` with `data` + `presentation` +- Cubit + GetIt + go_router + Dio +- Short operational copy +- Widget tests with fakes for auth gate and lists ---- +### README -## Style anchors (match your real work) +Hand the repo to another engineer: what it is, how to run it, HMAC/idempotency/worker notes, demo path. -Prefer patterns consistent with your Best Wallet-era habits (public-safe): +## Ownership -- Layered feature folders, not a random `screens/` dump -- Explicit failure states in Cubits (`initial / loading / ready / error`) -- Dio + typed DTOs; avoid decoding JSON in widgets -- Flavors later; MVP single env via `--dart-define=API_BASE=` +Before you claim the project in interviews: -Go side: keep it closer to standard library + a few solid deps than a framework zoo. +- [ ] Explain any file in 60 seconds +- [ ] Break one test on purpose and fix it +- [ ] Draw webhook → claim → list on paper +- [ ] Answer why stdlib mux + Postgres polling is enough for this MVP diff --git a/docs/PRD.md b/docs/PRD.md index 0a74cd6..ac995a1 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -395,7 +395,8 @@ Target: **under 5 minutes** once env is set. ## Related files -- [`../primary-project-recommendation.md`](../primary-project-recommendation.md) -- [`../backlog/project-backlog.md`](../backlog/project-backlog.md) -- [`cursor-build-instructions.md`](./cursor-build-instructions.md) — paste into Cursor in the **app** repo -- [`human-code-standards.md`](./human-code-standards.md) — anti–AI-slop rules +- [`architecture.md`](./architecture.md) +- [`threat-model.md`](./threat-model.md) +- [`HUMAN_CODE_STANDARDS.md`](./HUMAN_CODE_STANDARDS.md) +- [`GIT_WORKFLOW.md`](./GIT_WORKFLOW.md) + From 43ec466fc1b9b48fca97ddc77ae631ddc12c78bb Mon Sep 17 00:00:00 2001 From: Omid Mirzaei Date: Sun, 19 Jul 2026 19:25:00 +0400 Subject: [PATCH 2/5] reclaim stuck processing and expose queue health --- api/cmd/server/main.go | 3 ++ api/internal/events/store.go | 77 +++++++++++++++++++++++++++-- api/internal/httpapi/health.go | 13 +++++ api/migrations/00003_claimed_at.sql | 9 ++++ 4 files changed, 97 insertions(+), 5 deletions(-) create mode 100644 api/migrations/00003_claimed_at.sql diff --git a/api/cmd/server/main.go b/api/cmd/server/main.go index 2607771..77c8425 100644 --- a/api/cmd/server/main.go +++ b/api/cmd/server/main.go @@ -79,6 +79,9 @@ func main() { WorkerSnapshot: func() any { return wrk.Stats.Snapshot() }, + QueueSnapshot: func(ctx context.Context) (any, error) { + return eventStore.QueueStats(ctx) + }, }) diff --git a/api/internal/events/store.go b/api/internal/events/store.go index c5c04d6..f836947 100644 --- a/api/internal/events/store.go +++ b/api/internal/events/store.go @@ -230,11 +230,23 @@ func (s *Store) GetByID(ctx context.Context, id string) (Event, error) { return e, nil } +// ClaimLease is how long a processing row may sit before another worker can reclaim it. +const ClaimLease = 45 * time.Second + func (s *Store) ClaimNext(ctx context.Context, maxAttempts int) (Event, error) { + return s.ClaimNextWithLease(ctx, maxAttempts, ClaimLease) +} + +func (s *Store) ClaimNextWithLease(ctx context.Context, maxAttempts int, lease time.Duration) (Event, error) { + if lease <= 0 { + lease = ClaimLease + } + secs := int(lease.Seconds()) var e Event err := s.pool.QueryRow(ctx, ` UPDATE events - SET status = 'processing' + SET status = 'processing', + claimed_at = now() WHERE id = ( SELECT id FROM events WHERE status = 'pending' @@ -243,13 +255,18 @@ func (s *Store) ClaimNext(ctx context.Context, maxAttempts int) (Event, error) { AND attempt_count < $1 AND COALESCE(processed_at, received_at) <= now() - make_interval(secs => LEAST(60, GREATEST(2, attempt_count * 2))) ) + OR ( + status = 'processing' + AND claimed_at IS NOT NULL + AND claimed_at <= now() - make_interval(secs => $2) + ) ORDER BY received_at ASC LIMIT 1 FOR UPDATE SKIP LOCKED ) RETURNING id::text, user_id::text, idempotency_key, type, payload, status, attempt_count, last_error, matched_rule_id::text, received_at, processed_at - `, maxAttempts).Scan( + `, maxAttempts, secs).Scan( &e.ID, &e.UserID, &e.IdempotencyKey, &e.Type, &e.Payload, &e.Status, &e.AttemptCount, &e.LastError, &e.MatchedRuleID, &e.ReceivedAt, &e.ProcessedAt, ) @@ -266,7 +283,8 @@ func (s *Store) ClaimByID(ctx context.Context, id string) (Event, error) { var e Event err := s.pool.QueryRow(ctx, ` UPDATE events - SET status = 'processing' + SET status = 'processing', + claimed_at = now() WHERE id = $1 AND status IN ('pending', 'failed') RETURNING id::text, user_id::text, idempotency_key, type, payload, status, attempt_count, last_error, matched_rule_id::text, received_at, processed_at @@ -289,7 +307,8 @@ func (s *Store) MarkProcessed(ctx context.Context, id string, matchedRuleID *str SET status = 'processed', matched_rule_id = $2, processed_at = now(), - last_error = NULL + last_error = NULL, + claimed_at = NULL WHERE id = $1 AND status = 'processing' `, id, matchedRuleID) if err != nil { @@ -308,7 +327,8 @@ func (s *Store) MarkAttemptFailed(ctx context.Context, id, lastError string) (Ev SET attempt_count = attempt_count + 1, last_error = $2, status = 'failed', - processed_at = now() + processed_at = now(), + claimed_at = NULL WHERE id = $1 AND status = 'processing' RETURNING id::text, user_id::text, idempotency_key, type, payload, status, attempt_count, last_error, matched_rule_id::text, received_at, processed_at @@ -324,3 +344,50 @@ func (s *Store) MarkAttemptFailed(ctx context.Context, id, lastError string) (Ev } return e, nil } + +type QueueStats struct { + ByStatus map[string]int64 `json:"by_status"` + OldestPendingS *float64 `json:"oldest_pending_seconds,omitempty"` +} + +func (s *Store) QueueStats(ctx context.Context) (QueueStats, error) { + rows, err := s.pool.Query(ctx, ` + SELECT status, count(*)::bigint + FROM events + GROUP BY status + `) + if err != nil { + return QueueStats{}, fmt.Errorf("queue counts: %w", err) + } + defer rows.Close() + + out := QueueStats{ByStatus: map[string]int64{}} + for rows.Next() { + var status string + var n int64 + if err := rows.Scan(&status, &n); err != nil { + return QueueStats{}, fmt.Errorf("scan queue count: %w", err) + } + out.ByStatus[status] = n + } + if err := rows.Err(); err != nil { + return QueueStats{}, err + } + + var age float64 + err = s.pool.QueryRow(ctx, ` + SELECT EXTRACT(EPOCH FROM (now() - received_at))::float8 + FROM events + WHERE status = 'pending' + ORDER BY received_at ASC + LIMIT 1 + `).Scan(&age) + if errors.Is(err, pgx.ErrNoRows) { + return out, nil + } + if err != nil { + return QueueStats{}, fmt.Errorf("oldest pending: %w", err) + } + out.OldestPendingS = &age + return out, nil +} diff --git a/api/internal/httpapi/health.go b/api/internal/httpapi/health.go index d8f4c0a..dad3e0f 100644 --- a/api/internal/httpapi/health.go +++ b/api/internal/httpapi/health.go @@ -1,6 +1,7 @@ package httpapi import ( + "context" "net/http" "github.com/jackc/pgx/v5/pgxpool" @@ -9,6 +10,7 @@ import ( type HealthHandler struct { Pool *pgxpool.Pool WorkerSnapshot func() any + QueueSnapshot func(ctx context.Context) (any, error) } func (h HealthHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { @@ -23,5 +25,16 @@ func (h HealthHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) { if h.WorkerSnapshot != nil { body["worker"] = h.WorkerSnapshot() } + if h.QueueSnapshot != nil { + queue, err := h.QueueSnapshot(r.Context()) + if err != nil { + WriteJSON(w, http.StatusServiceUnavailable, map[string]any{ + "status": "unavailable", + "error": "queue_stats_failed", + }) + return + } + body["queue"] = queue + } WriteJSON(w, http.StatusOK, body) } diff --git a/api/migrations/00003_claimed_at.sql b/api/migrations/00003_claimed_at.sql new file mode 100644 index 0000000..c7b2e65 --- /dev/null +++ b/api/migrations/00003_claimed_at.sql @@ -0,0 +1,9 @@ +-- +goose Up +ALTER TABLE events + ADD COLUMN claimed_at timestamptz; + +CREATE INDEX events_claim_queue_idx ON events (status, claimed_at, received_at); + +-- +goose Down +DROP INDEX IF EXISTS events_claim_queue_idx; +ALTER TABLE events DROP COLUMN IF EXISTS claimed_at; From bdcde4cbcef46fda71b520bebd30d10e09ee95aa Mon Sep 17 00:00:00 2001 From: Omid Mirzaei Date: Sun, 19 Jul 2026 21:10:00 +0400 Subject: [PATCH 3/5] test concurrent claim and expired lease reclaim --- api/internal/worker/worker_test.go | 102 +++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/api/internal/worker/worker_test.go b/api/internal/worker/worker_test.go index db6569f..d6482ed 100644 --- a/api/internal/worker/worker_test.go +++ b/api/internal/worker/worker_test.go @@ -5,6 +5,7 @@ import ( "fmt" "log/slog" "os" + "sync" "testing" "time" @@ -66,6 +67,107 @@ func TestProcessSuccess(t *testing.T) { } } +func TestConcurrentClaimNoDoubleProcess(t *testing.T) { + pool := testPool(t) + ctx := context.Background() + userID := createUser(t, pool) + eventStore := events.NewStore(pool) + ruleStore := rules.NewStore(pool) + const n = 40 + for i := 0; i < n; i++ { + _, _, err := eventStore.CreatePending(ctx, events.CreateInput{ + UserID: userID, + IdempotencyKey: fmt.Sprintf("evt_race_%d_%d", time.Now().UnixNano(), i), + Type: "tx_simulated", + Payload: []byte(`{"amount":1}`), + }) + if err != nil { + t.Fatal(err) + } + } + + w := worker.New(eventStore, ruleStore, slog.Default()) + errCh := make(chan error, 2) + var wg sync.WaitGroup + run := func() { + defer wg.Done() + for { + ok, err := w.ProcessOne(ctx) + if err != nil { + errCh <- err + return + } + if !ok { + return + } + } + } + wg.Add(2) + go run() + go run() + wg.Wait() + close(errCh) + for err := range errCh { + if err != nil { + t.Fatal(err) + } + } + + processed, err := eventStore.ListForUser(ctx, userID, "processed") + if err != nil { + t.Fatal(err) + } + if len(processed) != n { + t.Fatalf("processed=%d want %d", len(processed), n) + } + for _, status := range []string{"pending", "processing", "failed"} { + left, err := eventStore.ListForUser(ctx, userID, status) + if err != nil { + t.Fatal(err) + } + if len(left) != 0 { + t.Fatalf("leftover status=%s count=%d", status, len(left)) + } + } +} + +func TestReclaimExpiredProcessingLease(t *testing.T) { + pool := testPool(t) + ctx := context.Background() + userID := createUser(t, pool) + eventStore := events.NewStore(pool) + + ev, _, err := eventStore.CreatePending(ctx, events.CreateInput{ + UserID: userID, + IdempotencyKey: fmt.Sprintf("evt_lease_%d", time.Now().UnixNano()), + Type: "balance_drop", + Payload: []byte(`{"amount":10}`), + }) + if err != nil { + t.Fatal(err) + } + if _, err := eventStore.ClaimByID(ctx, ev.ID); err != nil { + t.Fatal(err) + } + + if _, err := pool.Exec(ctx, ` + UPDATE events + SET claimed_at = now() - interval '2 minutes', + received_at = timestamptz '2000-01-01' + WHERE id = $1 + `, ev.ID); err != nil { + t.Fatal(err) + } + + reclaimed, err := eventStore.ClaimNextWithLease(ctx, worker.MaxAttempts, time.Second) + if err != nil { + t.Fatalf("reclaim: %v", err) + } + if reclaimed.ID != ev.ID { + t.Fatalf("reclaimed id=%s want %s", reclaimed.ID, ev.ID) + } +} + func TestProcessFailureIncrementsAttempts(t *testing.T) { pool := testPool(t) ctx := context.Background() From 3fc6154b8990e568928a721a4cc1634e0a611ddc Mon Sep 17 00:00:00 2001 From: Omid Mirzaei Date: Mon, 20 Jul 2026 12:45:00 +0400 Subject: [PATCH 4/5] document webhook reliability and worker claim path --- README.md | 23 ++++++++++++++++++++--- docs/architecture.md | 13 +++++++++++-- docs/threat-model.md | 2 ++ 3 files changed, 33 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index a2266b6..1f15785 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,22 @@ # WalletOps Companion -Personal ops console for simulated wallet events: signed webhook ingest, alert rules, a retrying worker, and schema-checked AI summaries. Flutter client + Go API + Postgres. +Personal ops console for **simulated** wallet events. Partners post signed webhooks; Postgres stores them idempotently; a Go worker claims and processes them against alert rules; a Flutter client lists events and can request a schema-checked summary. + +No real custody, keys, or on-chain sends. + +## Why the backend matters + +The interview-friendly core is reliability around ingest and jobs: + +| Concern | Behavior | +|---------|----------| +| Forged webhooks | HMAC-SHA256 over the raw body (`X-Signature: sha256=`) | +| Duplicate delivery | Unique `(user_id, idempotency_key)`; replay returns the same event (`200`) | +| Concurrent workers | `FOR UPDATE SKIP LOCKED` claim — safe under two pollers | +| Crash mid-process | `claimed_at` lease; stale `processing` rows become claimable again | +| Retries | Failed attempts back off (capped at 60s) up to 5 tries | + +Open `api/internal/webhook/handler_test.go` (`replay`) and `api/internal/worker/worker_test.go` (`TestConcurrentClaimNoDoubleProcess`, `TestReclaimExpiredProcessingLease`). ## Architecture @@ -38,6 +54,7 @@ Copy `.env.example` → `.env` (compose already injects defaults for local): # 1) API + Postgres docker compose up --build -d curl -s http://127.0.0.1:8080/v1/health +# expect status=ok, worker ticks, queue.by_status # 2) Seed user + two signed events (maps user_ref=demo-user-1) ./scripts/seed_webhooks.sh @@ -52,7 +69,7 @@ flutter run --dart-define=API_BASE=http://127.0.0.1:8080 # Android emulator: API_BASE=http://10.0.2.2:8080 ``` -In the app: sign in as `demo-user-1@walletops.local` / `ops-secret-1` → Events (status → processed) → open an event → **Explain** (mock AI). +In the app: sign in as `demo-user-1@walletops.local` / `ops-secret-1` → Events (status → processed) → open an event → **Explain** (mock AI by default). ## Tests @@ -66,4 +83,4 @@ cd api && DATABASE_URL='postgres://walletops:walletops@localhost:5432/walletops? cd mobile && flutter pub get && flutter analyze && flutter test --dart-define=API_BASE=http://127.0.0.1:8080 ``` -Module path under `api/` is a placeholder — change before publishing. +Module path: `github.com/omid/walletops/api`. diff --git a/docs/architecture.md b/docs/architecture.md index 5af2c29..2b73041 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -30,7 +30,7 @@ sequenceDiagram API->>DB: Insert event status=pending (idempotent) API-->>Partner: 202/200 event - Worker->>DB: Claim pending/failed (SKIP LOCKED) + Worker->>DB: Claim pending/failed/stale processing (SKIP LOCKED + lease) Worker->>DB: Match alert rules, mark processed/failed Worker-->>DB: Update status + matched_rule_id @@ -46,13 +46,22 @@ sequenceDiagram API-->>Mobile: title, bullets, risk_level, follow_ups ``` +## Worker claim + +1. Select one eligible row: `pending`, retriable `failed`, or `processing` whose `claimed_at` lease expired. +2. `FOR UPDATE SKIP LOCKED` so two API processes do not take the same row. +3. Set `status=processing` and refresh `claimed_at`. +4. Match rules / validate payload; mark `processed` or `failed` (clears `claimed_at`). + +Default lease: 45s (`events.ClaimLease`). Health exposes queue counts via `GET /v1/health`. + ## Packages (API) | Path | Role | |------|------| | `internal/auth` | Register/login/refresh, JWT middleware | | `internal/rules` | Alert rule CRUD + matching lookup | -| `internal/events` | Persist/list/claim events | +| `internal/events` | Persist/list/claim events, queue stats | | `internal/webhook` | HMAC ingest | | `internal/worker` | Poll/claim/process loop | | `internal/ai` | Mock/OpenAI summarize + audit row | diff --git a/docs/threat-model.md b/docs/threat-model.md index 5ed1c0f..4ae0018 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -16,6 +16,8 @@ Scope: local/demo WalletOps Companion — Flutter client + Go API + Postgres. No |--------|----------------| | Forged partner webhooks | HMAC-SHA256 over raw body (`X-Signature: sha256=`); reject bad sigs with 401 | | Replay / duplicate ingest | Unique `(user_id, idempotency_key)`; replay returns same event | +| Double-processing under two workers | `FOR UPDATE SKIP LOCKED` on claim | +| Crash while `processing` | `claimed_at` lease; stale rows are reclaimable | | Stolen device tokens | `flutter_secure_storage`; short access TTL; refresh rotation on use | | Prompt injection via payload | Summarize prompt uses allowlisted fields only (type, amount, status, rule name) | | Cross-user event access | Ownership checks on event IDs for list/detail/summarize | From dbedba771bed4ba9a619d70d4545f3cd571cb188 Mon Sep 17 00:00:00 2001 From: Omid Mirzaei Date: Mon, 20 Jul 2026 13:05:00 +0400 Subject: [PATCH 5/5] remove process and workflow meta docs --- docs/GIT_WORKFLOW.md | 33 ---------------------------- docs/HUMAN_CODE_STANDARDS.md | 42 ------------------------------------ docs/PRD.md | 2 -- 3 files changed, 77 deletions(-) delete mode 100644 docs/GIT_WORKFLOW.md delete mode 100644 docs/HUMAN_CODE_STANDARDS.md diff --git a/docs/GIT_WORKFLOW.md b/docs/GIT_WORKFLOW.md deleted file mode 100644 index 8313163..0000000 --- a/docs/GIT_WORKFLOW.md +++ /dev/null @@ -1,33 +0,0 @@ -# Git workflow - -``` -main # runnable releases - └── develop # integration - └── feature/* / fix/* -``` - -## Rules - -1. Land work through PRs into `develop`; promote to `main` when demo-ready. -2. One concern per branch; prefer several small commits over one dump. -3. Never commit `.env` — only `.env.example`. -4. Protect `main` on GitHub (PR required, no force-push). - -## Commit messages - -Write what a teammate would write: - -``` -add postgres migrations for users and events -hmac webhook ingest with idempotency -worker claim loop and retries -flutter auth gate and secure storage -``` - -Avoid noise: `Update files`, `Enhance architecture`, empty `Initial commit` spam. - -## Review checklist - -- [ ] Tests for the changed path pass locally -- [ ] No secrets or personal notes in the tree -- [ ] README / architecture still match behavior diff --git a/docs/HUMAN_CODE_STANDARDS.md b/docs/HUMAN_CODE_STANDARDS.md deleted file mode 100644 index 7179896..0000000 --- a/docs/HUMAN_CODE_STANDARDS.md +++ /dev/null @@ -1,42 +0,0 @@ -# Code standards - -Keep the repo readable for an interviewer who opens random files. - -## Avoid - -1. Comments that narrate the code (`// This function handles…`) -2. README emoji walls or marketing filler -3. Generic names: `MyApp`, `Utils`, `Helper`, `ManagerImpl`, `DataService` -4. Giant god-files and unused dependencies -5. Over-abstract ports/adapters for a small MVP -6. Fake or identical commit spam - -## Prefer - -### Go - -- Packages by domain: `auth`, `events`, `rules`, `webhook`, `worker`, `ai` -- Interfaces at test boundaries; clear functions elsewhere -- Errors wrapped with `%w` -- Table-driven tests with short names (`ok`, `bad_sig`, `replay`) -- slog JSON with useful fields (`event_id`, `attempt`) - -### Flutter - -- `features//` with `data` + `presentation` -- Cubit + GetIt + go_router + Dio -- Short operational copy -- Widget tests with fakes for auth gate and lists - -### README - -Hand the repo to another engineer: what it is, how to run it, HMAC/idempotency/worker notes, demo path. - -## Ownership - -Before you claim the project in interviews: - -- [ ] Explain any file in 60 seconds -- [ ] Break one test on purpose and fix it -- [ ] Draw webhook → claim → list on paper -- [ ] Answer why stdlib mux + Postgres polling is enough for this MVP diff --git a/docs/PRD.md b/docs/PRD.md index ac995a1..0815971 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -397,6 +397,4 @@ Target: **under 5 minutes** once env is set. - [`architecture.md`](./architecture.md) - [`threat-model.md`](./threat-model.md) -- [`HUMAN_CODE_STANDARDS.md`](./HUMAN_CODE_STANDARDS.md) -- [`GIT_WORKFLOW.md`](./GIT_WORKFLOW.md)