Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/daily/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Update outcomes ONLY after upstream merge/close, never on open. Stop here if `--
Group by rule family; auto-flag tests/sample/examples/demos/fixtures as candidate-FP. Inspect top 5 real candidates in-repo via `gh api .../contents/<path>`. Write per-finding verdicts. Evaluate the quality gate boolean.

## Phase 5 β€” Publish (gated)
Always: write `docs/scans/<slug>.md` + prepend `docs/index.md`. If gate TRUE and not strict-norm: file focused de-branded courtesy issue (+ PR for clean one-line fixes). If strict-norm: post-only or one-issue-per-critical. If gate FALSE: post-only. Open `docs:` PR on `elfrost/ai-patchlab`, merge, verify publication via `gh run list --workflow=pages-build-deployment --limit 1` = `success` AND the post returns 200 serving the new text. Do NOT gate on `pages/builds/latest` β€” on this Actions-published site it reports `errored` even when the deploy succeeded (2026-08-26).
Always: write `docs/scans/<slug>.md` + prepend a row to `docs/index.md` + a bullet to `docs/scan-log.md` (bump both scan counts). If gate TRUE and not strict-norm: file focused de-branded courtesy issue (+ PR for clean one-line fixes). If strict-norm: post-only or one-issue-per-critical. If gate FALSE: post-only. Open `docs:` PR on `elfrost/ai-patchlab`, merge, verify publication via `gh run list --workflow=pages-build-deployment --limit 1` = `success` AND the post returns 200 serving the new text. Do NOT gate on `pages/builds/latest` β€” on this Actions-published site it reports `errored` even when the deploy succeeded (2026-08-26).

## Phase 6 β€” Record
Update `reports/.daily_state.json`; update memory on resolutions/lessons; print a 3-line summary.
Expand Down
2 changes: 1 addition & 1 deletion .claude/commands/daily.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Use `--ignore-file` if the repo has obvious sample/example/demo subtrees (until
4. **Evaluate the quality gate:** is there β‰₯1 real, exploitability-shaped, high-confidence item? Record the boolean β€” it decides Phase 5 filing.

## Phase 5 β€” Publish (gated)
1. **Always:** write `docs/scans/<slug>.md` from `docs/templates/scan-post.md`; prepend a new line to the Scans list in `docs/index.md`.
1. **Always:** write `docs/scans/<slug>.md` from `docs/templates/scan-post.md`; prepend a new row to the Scans table in `docs/index.md` **and** a new bullet to `docs/scan-log.md` (the full prose archive), and bump the scan counts in both headers. Three files, every time β€” on 2026-09-06 the log was found eight entries behind the index because this step only named `index.md`.
2. **If quality gate TRUE and repo not strict-norm:** file a focused courtesy issue on the target (de-branded, with code-path note + concrete fix). If a finding has a clean one-line/one-file fix, also fork β†’ branch β†’ PR referencing the issue.
3. **If repo strict-norm:** post-only, or one issue per critical finding β€” no grouped issue.
4. **If quality gate FALSE:** post-only (clean-scan write-up). File nothing upstream.
Expand Down
5 changes: 3 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ remediation and confidence rules to normalize the findings.

> **Want this run privately against your own codebase?** I do independent
> security review of AI agents, MCP servers, and LLM apps β€”
> [**work with me β†’**]({{ '/work-with-me' | relative_url }}). 95 scans, 21 confirmed fixes, methodology in the open.
> [**work with me β†’**]({{ '/work-with-me' | relative_url }}). 96 scans, 21 confirmed fixes, methodology in the open.

> **OpenAI just launched [Daybreak](https://openai.com/index/daybreak-securing-the-world/) and Patch the Planet.**
> Same remediation loop, opposite trade-off: their path is a cloud frontier model;
Expand Down Expand Up @@ -94,12 +94,13 @@ login and static assets. Fifty-two flagged, none reported.

## All scans

94 scans, newest first. **Findings** is the raw count the tools produced;
96 scans, newest first. **Findings** is the raw count the tools produced;
**Real** is what survived curation. The gap between those two columns is the
entire job.

| Date | Repository | Findings | Real | Outcome |
|---|---|---:|---|---|
| 2026-09-06 | [ApodexAI/FrontierAgent](scans/apodexai-frontieragent.html) | 58 | 1 real β€” withheld | private |
| 2026-09-05 | [doobidoo/mcp-memory-service](scans/doobidoo-mcp-memory-service.html) | 176 | 0 real | β€” |
| 2026-09-04 | [basicmachines-co/basic-memory](scans/basicmachines-co-basic-memory.html) | 270 | 0 real | β€” |
| 2026-09-03 | [samuelgursky/davinci-resolve-mcp](scans/samuelgursky-davinci-resolve-mcp.html) | 97 | 1 real β€” withheld | private |
Expand Down
11 changes: 10 additions & 1 deletion docs/scan-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,17 @@ description: "The complete AI PatchLab scan log: every public repository scanned

# Full scan log

Every scan in the series, newest first, with the summary written on the day of the scan. 87 scans. For the compact index, see the [scan log home]({{ '/' | relative_url }}).
Every scan in the series, newest first, with the summary written on the day of the scan. 96 scans. For the compact index, see the [scan log home]({{ '/' | relative_url }}).

- **2026-09-06** β€” [ApodexAI/FrontierAgent](scans/apodexai-frontieragent.html) β€” 58 findings (58 above the medium floor), **1 real β€” withheld** β€” an **agent runtime, terminal product and evaluation suite** (1.8kβ˜… two weeks after going public, Apache-2.0, Apodex AI): ReAct and coordinator-plus-sub-agents workflows over a task-scoped sandbox, a fifty-module plugin tool tree, and an optional Gradio demo for a Hugging Face Space, with a real `SECURITY.md` (named mailbox, 48h ack, public vuln issues forbidden) and private vulnerability reporting enabled. **Every one of the 58 collapsed on inspection** β€” loader imports keyed on a runtime class, an execution epoch logged as a "token" (seven-for-seven false), `sk-test-…` placeholders inside the project's own *leak-guard test suite*, operator-configured endpoints behind the `urllib` rule, the sandbox shell itself behind `shell=True`, a tar extractor that both resolves members and passes `filter="data"`. **The real finding is the composite class no rule sees:** a download allow-list one directory too wide, in a component whose README records β€” as *verified* β€” that the sibling directory it exposes is never served to a browser; the [docstring-as-oracle](scans/hkuds-openopc.html) diff between that paragraph and one launch argument is the whole finding. Verified by execution both ways: the artifact served, then the one-line remedy keeping deliverables downloadable while the artifact returns 403. Same-session only, filed Low; class only here Β· reported privately via GitHub PVR, post-only
- **2026-09-05** β€” [doobidoo/mcp-memory-service](scans/doobidoo-mcp-memory-service.html) β€” 176 findings, **zero real after curation; thirtieth clean scan** β€” a persistent memory server for AI agents (1.9kβ˜…) that published **four critical advisories on the exact day of the scan**, all fixed at HEAD, and the tools surfaced none of them: every one was an absence or a parity gap (an auth check missing on one transport that its sibling had). The dominant cluster was the parameterized-SQL identifier FP, ninth appearance in the series Β· post-only, strict-norm target
- **2026-09-04** β€” [basicmachines-co/basic-memory](scans/basicmachines-co-basic-memory.html) β€” 270 findings, **zero real after curation; twenty-ninth clean scan** β€” a local-first Markdown knowledge base with an MCP server; the volume was fixtures, identifier-only SQL and by-design file tooling on a project whose purpose is reading and writing the user's own notes Β· post-only, strict-norm target
- **2026-09-03** β€” [samuelgursky/davinci-resolve-mcp](scans/samuelgursky-davinci-resolve-mcp.html) β€” 97 findings, **1 real β€” withheld** β€” an MCP server for DaVinci Resolve (desktop, stdio by default, opt-in networked transport). The class: a credential-handling weakness confined to the opt-in networked transport, local exposure only, a one-line fix following a pattern the repository already contains elsewhere (the sibling differential). The policy's first-named channel (GHSA) turned out switched off β€” `enabled:false`, `403` on the create call β€” so the report went to the mailbox the README names, and the dead channel is reported alongside the finding Β· reported privately, post-only
- **2026-09-02** β€” [HKUDS/OpenOPC](scans/hkuds-openopc.html) β€” 56 findings, **1 real β€” withheld** β€” a self-hosted "AI-native company" platform (1.6kβ˜…) with a genuinely careful permission model and a WebSocket control plane that drives it while authenticating nobody, bound to `0.0.0.0` by default: the [loopback-inversion](scans/liaohch3-claude-tap.html) class, and the cleanest instance in the series because the security engineering around it is good. Verified against the project's own permission resolver (ask β†’ allow with no prompt). Filed High via GitHub PVR, `GHSA-fp48-59rc-43qg`, in triage Β· reported privately, post-only
- **2026-08-31** β€” [shy3130/tick-stock-panel](scans/shy3130-tick-stock-panel.html) β€” 77 findings, **1 real** β€” an LLM-driven A-share stock screener (4.4kβ˜…): screener conditions and ordering interpolated raw into a DuckDB query on an engine with external access enabled β€” the capability switch that turns "injection into a sandbox" into host access, invisible to SAST and SCA alike Β· βœ… **Resolved 2026-09-03 in 3 days (21st fix in the series)** β€” the maintainer shipped a regression test with a positive *and* a negative control
- **2026-08-30** β€” [SenteLabsAI/OpenExecutive](scans/sentelabsai-openexecutive.html) β€” 124 findings, **1 real β€” withheld** β€” an AI "virtual executive team" (3.8kβ˜…): a webhook whose signature verification is *skipped* rather than failed when its optional secret is unset, the documented default, while the sibling integration verifies unconditionally β€” the majority sibling is the contract. Plus a `SECURITY.md` that mandates GitHub PVR and deprecates email while PVR returns `enabled:false` Β· drafted for private delivery, post-only
- **2026-08-28** β€” [Ontos-AI/knowhere](scans/ontos-ai-knowhere.html) β€” 129 findings, **1 real β€” withheld** β€” a document-ingestion pipeline sold as the memory layer between dirty documents and AI agents (2.7kβ˜…): one missing-authentication gap on an internal storage-event webhook whose verifiers are stubs, verified by execution (wrong token rejected, absent header accepted), and a 68-CVE dependency table that all resolves to a single lockfile Β· drafted for private delivery, post-only
- **2026-08-27** β€” [Zleap-AI/SAG](scans/zleap-ai-sag.html) β€” 60 findings, **1 real** β€” an agent framework whose login did not verify the password: any name signed in as the owner account Β· βœ… **Resolved 2026-08-30 in 3 days (19th fix in the series)** β€” posing the product question beat proposing a patch
- **2026-09-01** β€” [future-agi/future-agi](scans/future-agi-future-agi.html) β€” 1227 findings (1224 above the medium floor), **1 real β€” withheld** β€” an **end-to-end LLM/agent observability and evaluation platform** (1.9kβ˜…, Apache-2.0 + EE): OTLP trace ingestion into a ClickHouse store of prompts/completions/tokens/costs, an evaluation engine, a simulation runner, a Go **AI gateway** (`agentcc-gateway`, an OpenAI/Anthropic/Gemini-compatible proxy with per-org keys, rotation and RBAC) and a Django backend, with a real `SECURITY.md` (named email, SLA table, safe harbor, in-scope list, public vuln issues forbidden). **Largest raw count in the series β€” and one real finding no rule represented.** The class: **one service in a multi-service deployment is published on all interfaces while every service beside it, including every datastore, is correctly loopback-bound, and the production overlay inherits the exposure unchanged** β€” the exposed service is the most dangerous one in the file to leave open, and reaching it hands an unauthenticated caller a path to the trace store. Three reportability properties, each a series pattern: **the maintainer's own compose comment is the oracle** (*"bound to 127.0.0.1 so only the host can reach them"*, applied to Postgres/ClickHouse/Redis/MinIO/Temporal/collector β€” the finding is the outlier), the [contract-vs-artifact](scans/vexa-ai-vexa.html) footing in a deployment file; **N deployment paths, and the question is whether they agree** β€” the production overlay re-binds every secret with `${VAR:?must be set for production}` (genuinely good) but names only `environment:`/`restart:`, not `ports:`, so the documented production command inherits the exposure; and **I proved it with the vendor's own merge engine** β€” rendered the exact `deploy/README.md` command through `docker compose config` and read the published bindings out of the result (exposed service on `0.0.0.0`, every datastore on `127.0.0.1`), [running the primitive](scans/nottelabs-notte.html) rather than reasoning about override semantics. **Scope discipline kept it to one**: two other `0.0.0.0` services are the intended public surface (frontend + reverse-proxied API), and two management UIs published without a loopback bind sit behind Compose `profiles:` β€” opt-in, absent from the default and documented-production deployments, and the rendered config confirmed they don't appear. **Two auth hypotheses returned NO and that is part of the record**: the gateway mounts a large admin plane *outside* its `/v1/` auth prefix (mint keys, set org configs, rotate creds) β€” the [claude-tap inversion](scans/liaohch3-claude-tap.html) shape β€” but all **24** admin handlers across four files call an in-handler `requireAdmin`/`checkAdminAuth` with a constant-time compare that **fails closed on an unset token**; and the `/v1/` middleware short-circuits on a valid license token (a [caller-influenced auth branch](scans/liaohch3-claude-tap.html)) but the verifier pins RS256, resolves `kid` from a **local** key map, and validates the full claim set, so branch-then-serve is safe. **Noise**: 100 mutable-action-tag GHA rows, 94+44 raw/formatted-SQL that resolve to the [#1 identifier FP](scans/mnemosyne-oss-mnemosyne.html) (ClickHouse builders interpolate regex-validated keys and an allowlisted bucket function, bind every value via `%(param)s` β€” the four taint-tracked hits were audit-log f-strings and the allowlisted bucket fn), 3 gitleaks hits all covered by the repo's own `.gitleaks.toml`. **The 26 criticals are a coverage story**: `scan_dependency` found *no root manifest* because this is a **monorepo** (manifests under `futureagi/`), so the ChromaDB/LiteLLM/Authlib/Django/langchain criticals came from Trivy's lockfile parse and want the *version-match β†’ reachable β†’ mitigated* gate β€” several are Proxy-only/transport-only in libs used as clients. **Monorepo root-only coverage gap, re-confirmed as the top backlog item.** Strict-norm Β· reported **privately** to `security@futureagi.com` with a full dossier + `docker compose config` reproduction Β· post-only, finding **withheld** (class only) until remediated Β· the email send is the operator's manual step
- **2026-08-29** β€” [ginlix-ai/LangAlpha](scans/ginlix-ai-langalpha.html) β€” 372 findings above the medium floor, **1 real** β€” a **financial-market agent platform** (1.7kβ˜…, Apache-2.0, daily merges, FastAPI + React + a Daytona/Docker code sandbox), and one of the most carefully defended codebases in the series. Semgrep's top finding β€” `pull_request_target` checking out untrusted fork code β€” **retired on evidence outside the repository tree**: the GitHub environments API shows a `fork-ci` environment with required reviewers and `prevent_self_review`, and the workflow pins the immutable head SHA, sets `persist-credentials: false`, and drops the token to `contents: read`. The author even documented that `uv sync` runs fork build hooks before pytest β€” a better threat model than the rule that flagged it. All 30 gitleaks hits are fixtures in the project's **own** secret-redaction and leak-detection tests; the 102-finding SQL cluster binds every value with `%s` and interpolates only a constant column list. What survived is a **composite**: `GET /api/v1/preview/{workspace_id}/{port}` is unauthenticated and its docstring promises it "does NOT start stopped sandboxes (to prevent denial-of-wallet)" β€” but it enforces that with a DB `status == "running"` check and then calls the *acquisition* path, which reaches `PTCSandbox.reconnect`, documented as "it **starts a stopped sandbox**." **Four sibling unauthenticated routes in the same codebase refuse exactly that call**, citing "a stale 'running' DB row with no warm session," and use the no-wake `get_session_if_ready` accessor whose fence parameter was made mandatory "so a new caller cannot omit the fence." The majority sibling is the contract; the fifth route never inherited it. The lead came from the tool's **coverage warning**, not its findings: Semgrep could not parse `claude.yml`, `release.yml`, or `sandbox-integration.yml`, which were precisely the workflows worth reading.
- **2026-08-26** β€” [ascending-llc/jarvis-registry](scans/ascending-llc-jarvis-registry.html) β€” 235 findings above the medium floor, **1 real β€” withheld** β€” an **enterprise MCP/A2A gateway** that brokers per-user OAuth credentials to downstream tool servers (2.8kβ˜…, Apache-2.0, commercial backing, **7 distinct human PR authors in 60 days**), and the cleanest demonstration yet of why the mitigation gate is not optional. All four criticals retired: the **LiteLLM CVEs** (SQL injection, Host-header auth bypass, MCP command execution, key-gen privilege escalation) are Proxy-server-only, and this repo declares `litellm>=1.50.0` then **never imports it** β€” one grep outside the lockfile returns the dependency declaration and nothing else. The largest family, **21Γ— GitHub Actions shell injection**, retired on a single fact: there is no `pull_request_target` anywhere in the repository, so every flagged workflow runs either from a fork with a read-only token and no secrets, or from a context that already requires write access. **16 gitleaks "secrets"** are docs example output and MongoDB seed fixtures, against a `tartufo.toml` that carries a written reason on every exclusion; **12Γ— `logger-credential-leak`** logs usernames and source ids, never a credential; the lone **`unverified-jwt-decode`** is by design and its two callers both verify-then-branch, over a token layer that pins RS256, pins the `kid`, enforces issuer and audience, and blocks token-class confusion with positive equality checks. **The near-miss is the story**: the registry allows CORS origins by a regex whose `.*\.compute.*\.amazonaws\.com` arm admits the public DNS name of *every EC2 instance on the internet*, with `allow_credentials=True` β€” confirmed allowed against real Starlette 1.6.0, with the controls correctly denied. It is still **inert**, because every cookie the app sets is `SameSite=Lax` and `amazonaws.com` compute domains are on the Public Suffix List, so the attacker's `fetch()` carries no session at all. Four verification steps said "real"; the fifth β€” grep for the mitigation before flagging β€” retired it, and this series has filed that same class as a genuine finding against five other projects. **The one real defect is object-level authorization no scanner can see**: two sibling handlers in one router return per-user state from the same deterministic identifier, one enforces ownership against the caller and the other takes no caller identity at all, behind a permission every role holds. The intra-repo differential is the proof β€” the guarded sibling *is* the project's own contract. Routed privately to `security@ascendingdc.com` per the project's SECURITY.md, with a patch; public detail withheld.
Expand Down
Loading
Loading