Skip to content

fix: reject truncated receive batches - #366

Merged
NikolayS merged 5 commits into
mainfrom
fix-receive-overflow
Oct 1, 2026
Merged

NikolayS merged 5 commits into
mainfrom
fix-receive-overflow

Conversation

@NikolayS

@NikolayS NikolayS commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Summary

Refs #365. Make convenience receives return the complete batch or raise on N+1. Exactly N remains valid; INT_MAX does not require overflowing arithmetic. Covers plain, cooperative and partitioned receives. Errors include actionable recovery hints, and statement rollback preserves consumer state and ownership.

Correct reference/examples/tutorial/client guidance: ticker thresholds are not batch-size caps; repeated receive calls are not pagination; never acknowledge a failed receive. Real pagination is tracked separately in #364.

Frozen sql/pgque.sql and sql/pgque-tle.sql contain the minimal v0.2.1 backport generated from v0.2.0, not alpha features. Development remains 0.3.0-devel. A separate maintenance PR will provide the stable source/tag.

Verification

On fresh local Postgres 18.3 databases:

  • Red negative controls: each receive API returned 2 of 3 events on baseline.
  • bash build/transform.sh
  • psql -X -v ON_ERROR_STOP=1 -d "$db_name" -f devel/sql/pgque.sql
  • psql -X -v ON_ERROR_STOP=1 -d "$db_name" -f tests/test_receive_overflow.sql
  • psql -X -v ON_ERROR_STOP=1 -d "$db_name" -f tests/run_all.sql
  • psql -X -v ON_ERROR_STOP=1 -d "$db_name" -f tests/acceptance/run_acceptance.sql
  • git diff --check

All local suites pass. Regression covers N-1/N/N+1, NULL compatibility, INT_MAX, allocated/unallocated batch rollback, payload completeness, and failed/successful cooperative takeover. CI and SamoRev must pass on the exact head before merge; review and post-review real-test evidence will be posted here.

Intentional scope and review context

The owner explicitly selected the minimal fail-closed overflow solution, not a new pagination protocol or an unbounded default. Keep existing default ceilings/signatures. Stalling an undersized consumer with a visible, monitorable error is intentional; increasing the default to INT_MAX or silently escalating a caller's limit would replace a resource safety ceiling with potential memory pressure. Docs and release notes require rollback/retry with an operator-chosen ceiling. Ticker thresholds cannot guarantee successful reads.

receive_partitioned() passes its hash-slot predicate into get_batch_cursor(..., extra_where); the cursor itself is filtered before the N+1 probe. Review the complete devel/sql/pgque-api/partition_keys.sql if the provider diff is truncated. The multi-slot regression checks matching rows separately from other slots.

Release ordering: merge the maintenance source PR #367 and publish its reviewed v0.2.1 tag before merging this PR. The frozen stable installers here must match those tagged artifacts byte-for-byte; they are not an independent development build. Source and generated devel SQL remain separately versioned.

pg_tle update verification

The v0.2.0→v0.2.1 wrapper now registers a function-only update edge via the documented pg_tle install_update_path API and sets the new default version. PostgreSQL can create the default version through the existing base install plus that edge; a redundant standalone install_extension_version_sql registration is not required. This was executed on pg_tle v1.5.2: both an in-place populated ALTER EXTENSION UPDATE and a fresh CREATE EXTENSION after registration pass. The regression also reruns wrapper registration, preserves active subscription state, and verifies both plain/cooperative overflow return SQLSTATE54000 followed by successful full recovery. The same test is wired into CI, with tagged v0.2.0 fetched via full checkout history.

The overflow SQLSTATE change was red/green tested (old P0001 fails the new assertion). The multi-slot partition test establishes that the probe sees only matching-slot rows; other-slot rows do not cause overflow. INT_MAX now explicitly checks zero returned rows and no active batch.

Release publication gate

The complete v0.2.1 GitHub release notes, including the default-100 ceiling compatibility warning, are included in maintenance PR #367. Before merging this PR, the release operator must verify the published tag exists and run git diff --exit-code v0.2.1 -- sql/pgque.sql sql/pgque-tle.sql sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql, posting the exact tag/head and successful result. This is an explicit release-operator gate, not an assertion that CI already enforces it.

Refs #365. Backport frozen SQL to 0.2.1; preserve complete-batch semantics across plain, cooperative and partitioned receives.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Refs #365. Replace truncation expectations with complete retry and acknowledgment checks.
@NikolayS

NikolayS commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner Author

samorev Code Review Report

Reviewed head: 6e87070c23b8d00de5f40aec2a04fe914f27a0eb (round 1).

Pipeline Coverage
PASS Not reported

BLOCKING ISSUES (5)

HIGH [bugs] The default max_return is still 100, but the default ticker_max_count is 500, and the ticker threshold does not cap batch size anyway. Batches over 100 events are common. With this change, a bare pgque.receive(queue, consumer) call (or any caller passing 100, as the reference example does) will raise on every poll for such a batch, where it used to truncate. A consumer that keeps the default stalls on that batch and keeps erroring until someone changes its configuration.

pgque.receive(queue text, consumer text, max_return int default 100) in docs/reference.md; the new guard if cnt = i_max_return then raise exception ... in receive.sql and receive_coop.
Fix: Raise the SQL default for receive/receive_coop/receive_partitioned to a full-batch ceiling (e.g. 2147483647, matching the client defaults), or at least to at least ticker_max_count. Also update the receive(..., 100) examples, and call out the change in the release notes as a behaviour break.

MEDIUM [bugs] The overflow error reuses the generic SQLSTATE P0001, so clients can only tell it apart from other raise exception errors by matching message text. The Go test does exactly that: it checks for P0001 plus the string "batch exceeds max_return". This is fragile, and it gets in the way of the client-side handling described in the previous finding.

raise exception 'pgque.receive: batch exceeds max_return of %', i_max_return using hint = ... has no errcode. The Go test checks sqlErr.SQLSTATE != "P0001" and calls strings.Contains(err.Error(), "batch exceeds max_return of 10").
Fix: Add using errcode = '...', either a custom class or a standard one such as 54000 (program_limit_exceeded), to all three receive variants, and assert that code in the tests.

MEDIUM [bugs] The frozen TLE artifact moves from 0.2.0 to 0.2.1, but there is no pg_tle upgrade path (install_update_path 0.2.0→0.2.1). For an existing 0.2.0 registration, the script hits the "different version registered … run sql/pgque-tle-uninstall.sql first" branch. Users on pg_tle would have to uninstall the extension, which can mean dropping the pgque schema and its queue data, to get the data-loss fix.

if existing_version = '0.2.1' then ... skipping / otherwise raise exception ... Run sql/pgque-tle-uninstall.sql first, with pgtle.install_extension('pgque', '0.2.1', ...) only.
Fix: Ship a pgtle.install_update_path('pgque', '0.2.0', '0.2.1', ...) containing the create or replace function bodies for the three receive functions and version(), or document a non-destructive upgrade procedure.

MEDIUM [bugs] receive_partitioned detects overflow by fetching one more row from the batch cursor after the loop. That is only correct if the cursor itself is already filtered to the slot. If slot filtering happens in PL/pgSQL inside the loop, the probe row may belong to another slot or be skipped, causing a false overflow or a missed one. The loop body was trimmed from the diff (content omitted for length), so I could not confirm which case applies.

execute 'fetch next from ' || quote_ident(v_cname) into ev; get diagnostics v_probe_count = row_count; if v_probe_count = 1 then raise ...
Fix: Confirm that get_batch_cursor applies the slot predicate in its extra_where. If it does not, apply the same filter to the probe (keep fetching until a matching row is found or the cursor is exhausted). Add a test with a multi-slot batch where other slots' rows follow exactly N matching rows.

LOW [guidelines] This feature PR edits the frozen release artifacts sql/pgque.sql and sql/pgque-tle.sql, including the version bump, and says a separate maintenance PR will provide the stable source/tag. Shipping a released artifact without a matching tag and source breaks the convention that frozen files equal tagged releases, and the two can drift apart.

-- Version: 0.2.1 and return '0.2.1'; in sql/pgque.sql. The PR description says "A separate maintenance PR will provide the stable source/tag."
Fix: Move the frozen-file backport into the maintenance/release PR that creates the v0.2.1 tag, or land the tag and source together with this change.


Summary

Area Findings Potential Filtered
CI/Pipeline 0 0 0
Security 0 0 0
Bugs 0 4 0
Tests 0 0 0
Guidelines 0 1 0
Docs 0 0 0
Metadata 0 0 0

Note:

  • Findings: High-confidence issues (8-10/10) - blocking or non-blocking per severity
  • Potential: Medium-confidence issues (4-7/10) - review manually
  • Filtered: Low-confidence issues (0-3/10) - excluded as likely false positives
Review metadata
provider=github
kind=pr
project=NikolayS/PgQue
number=366
target=github:NikolayS/PgQue#366
state=OPEN
draft=false
diff_lines=1208
diff_added=628
diff_removed=71
diff_bytes=52716
comments_count=1
commits_count=2
ci_status=success
ci_summary=total=18 success=18 failure=0 pending=0 other=0
prompt=.claude/commands/review-mr.md
blocking=true
samorev_review=1
review_round=1/10
escalated=false
posted_by=local
no_comment=true
live_posting=not-run
defender_dropped=1

Contested / dropped

  • [bugs] The Go, Python and Ruby Consumer loops were not changed to handle the new overflow error. If max_messages is set below the real batch size, the consumer used to make progress (with silent loss). Now it retries the same oversized batch forever, raising the same P0001 each time, and lag grows without bound. The diff only updates the READMEs, which say to monitor errors. There is no automatic retry with a larger ceiling and no distinct error type to alert on. — The Consumer default (MaxInt32) never hits the guard, and a too-low setting now fails loudly instead of silently losing data, which is the intended trade-off the READMEs document; a dedicated SQLSTATE or retry escalation is a separate design improvement, not a defect in this change.

samorev-assisted review (AI analysis by Tanya301/samorev)

Refs #365. Classify overflow as SQLSTATE54000, extend boundary and slot-filter tests, and provide a non-destructive stable pg_tle update.
@NikolayS

NikolayS commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

samorev Code Review Report — full AI review

Reviewed head: 6544e3552dc6b953f98c94173cc273949823d81f (base 95d5c3c12bccab8f917167659cfe0ba4d64e38ca). Any later commit voids this review; re-review the delta before merge.

Pipeline Coverage
✅ success: 18/18 checks on 6544e35 (CI run) Not reported

Verdict: ❌ CHANGES REQUESTED. There are 2 blocking findings, both documentation and guideline issues. The SQL fix, rollback semantics, partition probe and pg_tle update wrapper held up against full-source review. Separately, reviewer coverage was incomplete (see below), so this review would not report PASS even with zero findings.


BLOCKING ISSUES (2)

MEDIUM clients/go/options.go:18-26 (also options.go:126-133, clients/go/pgque.go:153, clients/typescript/README.md:77, clients/typescript/src/client.ts:152-157, clients/typescript/src/types.ts:59-69, clients/python/pgque/client.py:229-238, clients/go/concurrency_test.go:48). SDK API docs still describe the removed truncation semantics. (Docs + Bugs agents; validated, confidence 9/10)

WithMaxMessages: "If you set maxMessages below the real batch size, unreturned rows are skipped after ack." WithCoopMaxMessages: "a low limit can therefore drop rows. Match this to ticker_max_count (or larger)". The TS README says "Fetch up to max (default 100) messages … including rows beyond max". Go Receive: "fetches up to maxMessages". Issue #365's scope includes correcting batch-size guidance, and the PR says client guidance is corrected. Only the Go/Python/Ruby READMEs were updated. The godoc, JSDoc and Python docstrings that users read in their IDEs still recommend the >= ticker_max_count guidance this PR removes, and they never mention the SQLSTATE 54000 error.
Fix: Reword these comments, docstrings and the TS README to say: complete batch or SQLSTATE 54000; roll back and retry with a larger ceiling; never ack after a receive error. Drop "match to ticker_max_count".

MEDIUM clients/python/README.md:79, clients/ruby/README.md:94, clients/go/README.md:96. Migration framing in READMEs violates PgQue CLAUDE.md. (Guidelines + Docs agents; validated, confidence 8/10)

"On servers with the overflow guard…", "Older servers can truncate results, so upgrade the server before relying on this protection." CLAUDE.md says: "Do not include 'before X' or 'previously did Y, now does Z' framing in the README or under docs/. Migration notes belong in release notes … full stop."
Fix: State only current behavior: a batch larger than the ceiling raises SQLSTATE 54000. Move the "older servers truncate, upgrade first" note to the v0.2.1 release notes. While editing, the Ruby README's ack() (line 96) should be ack.


NON-BLOCKING (2)

LOW sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql:2, sql/pgque-tle.sql:2, sql/pgque-tle.sql:7123. Headers say "Auto-generated by build/transform.sh", but build/transform.sh at this head has no install_update_path, set_default_version or pgque_update_body generation. These are hand-maintained artifacts. The generated devel/sql/pgque-tle.sql still uses the old "different version registered → raise" wrapper, so the next release promoted from devel/ will not carry a managed update path forward. (Guidelines + Bugs; confidence 8/10)

Suggestion: Either generate the update path in transform.sh (preferably in the #367 source PR), or label these files hand-maintained and track the 0.2.1→next update path as release work. Consider a CI check that the standalone update file equals the embedded $pgque_update_body$.

LOW docs/examples.md:116. The text says receive() "still hands back an empty batch token to ack". This PR's docs/reference.md:181 now says (correctly, per receive.sql) that receive() also auto-finishes empty batches, so the two docs contradict each other. The sentence predates this PR, but the PR now asserts the opposite. (Docs; confidence 8/10)

Suggestion: Drop the "unlike receive()" clause.


POTENTIAL ISSUES (13)

These are moderate-confidence findings. Review them manually; some may be false positives.

MEDIUM .github/workflows/ci.yml:96-165 (stable-smoke). No CI job exercises overflow on the frozen stable sql/pgque.sql. (confidence 7/10)

test_receive_overflow.sql runs only against devel/sql/pgque.sql and devel/sql/pgque-tle.sql. Stable smoke receives 1 event with ceiling 10. The Bugs agent confirmed by script that stable receive/receive_coop/version are byte-identical to the update body and that the embedded TLE install body equals sql/pgque.sql. The release artifact's headline behavior is still verified only by local logs and #367's CI.
Suggestion: Add an N+1 → 54000 / retry-N → success assertion for receive and receive_coop plus version() = '0.2.1' to stable-smoke.

MEDIUM tests/test_tle_upgrade_v0_2.sql:52-53. The upgrade test covers only TLE 0.2.0→0.2.1 with data. (confidence 6/10)

Several cases are untested: the fresh-registration branch of stable sql/pgque-tle.sql (install_extension(..., extension_sql)), a rerun after ALTER EXTENSION UPDATE (the existing_version = '0.2.1' branch), the unsupported-origin raise, and plain non-TLE \i sql/pgque.sql over a populated 0.2.0 install. The Bugs agent reviewed every branch statically and found the logic correct.
Suggestion: Add a fresh-registration check that create extension installs 0.2.1 and raises 54000 on overflow, a post-update rerun, and a plain 0.2.0→0.2.1 reinstall with data.

MEDIUM tests/test_receive_overflow.sql:161, tests/test_receive_overflow.sql:424. INT_MAX is exercised only on empty batches. (confidence 7/10)

receive(..., 2147483647) runs after ack (an empty batch). The partitioned INT_MAX call is a bare perform with no assertion, on an already-acked slot. receive_coop INT_MAX is untested. The non-empty path is the one the probe comment exists for, and the PR description lists INT_MAX as covered.
Suggestion: Run INT_MAX on a 2-3 event batch for all three variants and assert the full count.

MEDIUM tests/test_receive_overflow.sql:370-427. The rollback after a failed receive_partitioned (lease and slot subscription state) and after a plain receive_coop overflow is not asserted. Only plain receive and coop takeover check before/after state. (confidence 6/10)

Suggestion: Snapshot the slot subscription and lease rows and the coop member sub_batch/sub_last_tick/sub_next_tick, then assert they are unchanged.

MEDIUM docs/reference.md:122-135. The reference never says that the default max_return of 100 is below the default ticker_max_count of 500, so a bare receive(q, c) or receive_coop(q, c, w) can raise 54000 under normal load. (confidence 6/10)

The owner deliberately keeps the default, and this review does not challenge that choice. Validation of CLI-gate finding #1: CONFIRMED as fact, intentional by design. Line 135 says that setting the value to the threshold cannot guarantee success, but it does not connect this to the default.
Suggestion: Add one sentence to the receive and receive_coop reference entries stating the default/threshold relationship and that callers should pass an explicit burst-sized ceiling.

LOW docs/installation.md:441. The snippet hard-codes alter extension pgque update to '0.2.1';. CLAUDE.md disallows concrete version tags in docs/ "unless it really helps the reader", and this one will go stale. (confidence 7/10)

Suggestion: Use alter extension pgque update;. The wrapper calls set_default_version, so this goes to the registered default. Also name the supported origin ("the immediately preceding release") rather than "unsupported upgrade origins are rejected".

LOW sql/pgque-tle.sql:7258. if existing_version = '0.2.0' and '0.2.1' = '0.2.1' then contains a constant tautology that looks like template residue. (confidence 7/10)

Suggestion: Use if existing_version = '0.2.0' then, and fix it in the #367 source so the byte-for-byte match is preserved.

LOW tests/test_receive_overflow.sql:109, :334, :486, tests/test_api_receive.sql:91. Overflow catches use when others without asserting SQLSTATE. The active-batch (:109) and takeover (:334) cases accept any error, and the :486 catch only checks the SQLSTATE afterwards. (confidence 7/10)

Suggestion: Use when sqlstate '54000' or assert returned_sqlstate.

LOW tests/test_receive_overflow.sql:204. NULL max_return is tested for receive only. receive_coop(..., null) (documented as unbounded) and receive_partitioned(..., null) (rejected) are untested. N-1 is tested for plain receive only. (confidence 7/10)

LOW clients/typescript/src/consumer.ts and the Go/Python/Ruby consumers. No client test covers how the Consumer loop behaves when overflow persists, and the TS client has no overflow test at all. (confidence 6/10)

Validation of CLI-gate finding #2: REFUTED as a correctness bug. Python _poll_once runs in with conn.transaction() on autocommit, which rolls back. Ruby conn.transaction do rolls back and re-raises. Go uses pool implicit transactions. TS catches, logs and sleeps. No "current transaction is aborted" masking occurs. The test gap remains.

LOW docs/tutorial.md:232, :271. The nack blocks still use receive('orders', 'processor', 1) limit 1. This is safe now, since it fails closed on batches of 2 or more events, but it is the pattern the new tutorial note warns against copying. (confidence 5/10)

LOW docs/installation.md:369-392. The plain (non-TLE) Upgrading section does not mention that re-running sql/pgque.sql changes receive/receive_coop from truncating to raising. (confidence 5/10)

Suggestion: Add one behavior-framed sentence about sizing ceilings. The migration note belongs in the release notes.

LOW sql/pgque-tle.sql:7258-7263 (Security). If a 0.2.0→0.2.1 update path is already registered, the wrapper reuses it without checking that it matches update_sql. A different pgtle_admin member could pre-plant one. (confidence 4/10)

This needs pgtle_admin and has the same trust model as the old same-version no-op.
Suggestion: Optionally compare the stored script and raise on mismatch.


Validation of the deterministic CLI gate report (pr-366-samorev-gate.md)

Gate finding Verdict Evidence
#1 Default max_return=100 < ticker_max_count=500 → default callers stall CONFIRMED (intentional) Owner-declared design in the PR. Kept only as a potential docs clarification (above), not a defect.
#2 Python/Ruby/Go Consumer loops don't roll back after 54000 REFUTED Python with conn.transaction() (autocommit) rolls back. Ruby conn.transaction do rolls back and re-raises. Go uses pool implicit transactions. TS catches and sleeps. The consumers default to INT_MAX.
#3 Go *pgque.SQLError / SQLSTATE may not exist; integration test may not compile REFUTED clients/go/errors.go defines SQLError{Op, SQLSTATE, Err}, and wrapSQLError fills it from pgconn.PgError.Code. integration_test.go has no build tag, so CI's go test -race ./... compiles and runs it.
Dropped by gate defender: frozen-installer byte-match to v0.2.1 not CI-enforced Agree it is not a defect of this PR The merge-order gate (#367 tag first) is explicit in the PR. #367 is still OPEN (head 22e0b24), so do not merge #366 until #367 is merged and tagged.

Prior review (round 1 @ 6e87070) follow-up

Round-1 finding Status at 6544e35
Default 100 vs 500 Deliberate owner decision; not re-flagged as a bug
Overflow used generic P0001 Resolved: errcode = '54000' in all three variants, asserted in SQL, Go, Python and Ruby tests
No pg_tle 0.2.0→0.2.1 path Resolved: install_update_path + set_default_version; CI test covers populated ALTER EXTENSION UPDATE and fresh create
receive_partitioned probe may see other-slot rows Resolved / verified: get_batch_cursor(4) applies the slot predicate inside the cursor SQL (… _evs where <pred> order by 1) and fetches exactly i_max. The probe can only see matching rows. A multi-slot regression was added.
Frozen artifacts without tag Handled by the stated merge order (#367 first)

Specialized agent results

Agent Model Status Raw result
Security opus ✅ complete No CRITICAL, HIGH or MEDIUM findings. Every replaced function keeps security definer set search_path = pgque, pg_catalog. Signatures are unchanged, so create or replace preserves ACLs and no PUBLIC-executable overload is created. The dynamic fetch/close use quote_ident on an internal bigint-derived name. The CI step uses no untrusted ${{ }} and is not pull_request_target. 2 LOW at 4/10 (pre-planted update path → potential; availability shift → duplicate of the intentional design).
Bug Hunter opus ✅ complete No CRITICAL, HIGH or MEDIUM findings. Verified that partition probe placement, GET DIAGNOSTICS row_count and the empty-slice finish are ordered correctly. Overflow is raised before any finish_batch, and allocation, takeover, auto-registration and lease renewal all roll back. All 4 wrapper branches are correct. A script confirmed the embedded install body equals sql/pgque.sql (260466 bytes) and the update bodies are byte-identical. The only deltas from v0.2.0 are 3 functions plus the header. 2 LOW (stale client docs → merged into blocking #1; hand-maintained "auto-generated" headers → non-blocking).
Test Analyzer sonnet ✅ complete 4 MEDIUM, 3 LOW. All are coverage gaps, none is a failing test. They are listed above as potential issues: stable CI overflow, upgrade branches, non-empty INT_MAX, partitioned/coop rollback assertions, NULL/N-1 per variant, when others catches, consumer-loop and TS tests.
Guidelines sonnet ⚠️ complete, reduced rule set 2 MEDIUM (README migration framing → blocking #2; installation version tag → potential), 4 LOW, 1 INFO. SQL style, search_path, lowercase and left-aligned keywords, shell set -Eeuo pipefail and commit subjects all pass. devel sources match the generated devel SQL.
Docs sonnet ✅ complete 6 MEDIUM, 3 LOW. Stale SDK docs (blocking #1), examples.md contradiction (non-blocking), default-vs-threshold note, TS README, migration framing, version tag, tutorial receive(…, 1), plain-upgrade note, and receive_partitioned undocumented (pre-existing → filtered).
Sqitch Migration Checker — N/A postgres-ai/platform only
Visual Evidence — N/A No UI or visual paths changed (docs .md only, no site/ changes)

Coverage and incompleteness

  • Diff: all 1903 diff lines (78 KiB) were read in full by the orchestrator and every agent. Nothing was truncated. Where diff context was insufficient, full source was read at the head SHA: partition_keys.sql, get_batch_cursor, stable sql/ against git show v0.2.0:…, build/transform.sh, ci.yml, and the Go/Python/Ruby/TS client sources and tests.
  • ⚠️ Guidelines coverage was incomplete. The organizational postgres-ai/rules files could not be loaded (the fetch was denied in this environment). The Guidelines agent checked against PgQue CLAUDE.md only, which restates the core of the Postgres.AI SQL style guide and takes precedence on conflict.
  • ⚠️ The review was static. No agent ran databases or test suites. Runtime behavior relies on the green CI at 6544e35 (18/18) and the author's local logs.
  • ⚠️ Validation was not independent. The orchestrator validated findings against full source in a single pass instead of spawning a separate validator subagent per finding.
  • SOC2 was skipped, as PgQue's CLAUDE.md requires.

Summary

Area Findings Potential Filtered
CI/Pipeline 0 0 0
Security 0 1 1
Bugs 1 0 0
Tests 0 7 0
Guidelines 1 2 2
Docs 2 3 2
Metadata 0 0 0

Note:

  • Findings are high-confidence issues (8-10/10), blocking or non-blocking according to severity. This table counts the 4 high-confidence findings (2 blocking, 2 non-blocking) by area, and each cross-agent finding is counted once.
  • Potential are medium-confidence issues (4-7/10) to review manually.
  • Filtered are low-confidence, pre-existing or duplicate findings: the Security availability note (duplicate of the intentional default), the Guidelines multi-line -- comment style and long hint strings, receive_partitioned absent from the reference (pre-existing), and tutorial 0.2.0 version output (pre-existing, not touched by this PR).

Merge gate: fix the 2 blocking items, then re-review the delta. Merge only after #367 is merged and v0.2.1 is tagged, and confirm that the frozen sql/ files match the tag byte-for-byte.


samorev-assisted review (AI analysis by Tanya301/samorev)

Resolve review findings on SDK documentation and default ceilings.\n\nRefs #365.
@NikolayS

NikolayS commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

samorev Code Review Report — round 3 (delta review)

Reviewed head: 1df5e363b1531edb5d130d4e7aff4500e05d0aa4 (base 95d5c3c12bccab8f917167659cfe0ba4d64e38ca). Any later commit voids this review, so re-review the delta before merging.

  • PR: fix: reject truncated receive batches #366 - fix: reject truncated receive batches
  • Author: @NikolayS (commits by samo-agent)
  • AI-Assisted: Yes
  • Linked issue: fix: reject truncated receive batches #365 (open)
  • Mode: --blocking. SOC2 checks are skipped, as PgQue's CLAUDE.md requires.
  • Scope: this is a delta review of one commit, 1df5e36 ("docs: clarify complete-batch receive contract"). It sits on top of 6544e35, which got a full-source review in round 2. The delta is 13 files, +71/-70. It changes only docs, READMEs, docstrings, JSDoc and Go comments, plus one Go test comment, and no executable code. The Security, Test and Bug agents each confirmed this independently. The SQL, CI, installers and tests are byte-identical to 6544e35, so the round-2 full-source findings on them carry forward (see "Prior evidence carried forward").
Pipeline Coverage
✅ success: 18/18 checks on 1df5e36 (CI run 36880437185, headSha verified = 1df5e36) Not reported

Verdict: ✅ PASSED. There are no blocking findings at 1df5e36. Both round-2 blocking findings are fixed. All five specialized reviewers completed, and an independent validator confirmed the findings. The remaining items are LOW/INFO or potential issues.

The merge gate still applies, and this review does not lift it. Do not merge #366 until #367 is merged, the v0.2.1 tag is published, and git diff --exit-code v0.2.1 -- sql/pgque.sql sql/pgque-tle.sql sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql passes. That result must be posted with the exact tag and head. As of this review, #367 is OPEN, its head is dc0b698 (it has moved past 22e0b24), and no v0.2.1 tag exists. I hashed the three frozen files with sha256 and they are currently byte-identical between #366 1df5e36 and #367 dc0b698. This review did not cover #367's own delta 22e0b24..dc0b698.


Round-2 blocking findings: validation

Round-2 finding Status at 1df5e36 Evidence
B1 SDK API docs still describe removed truncation semantics ✅ FIXED Every cited site now documents the complete-batch / SQLSTATE 54000 / never-ack contract:
• Go: options.go:18-25, :125-132; pgque.go:153-158, :327-330; concurrency_test.go:48-49
• TypeScript: README.md:77, :124; client.ts:152-156, :375-379; types.ts:59-65
• Python: client.py:229-238, :411-415
A case-insensitive sweep of clients/ and docs/ found no leftover "up to N", "rows beyond", "skipped after ack", "drop rows", "match ticker_max_count", "older servers" or "overflow guard" text in tracked files. (pgque_py.egg-info/PKG-INFO is gitignored.) One stale pre-existing comment outside the B1 list remains; see P2 below.
B2 Migration framing in client READMEs, which PgQue CLAUDE.md forbids ✅ FIXED "On servers with the overflow guard…" and "Older servers can truncate… upgrade first" are gone from the Go (:96), Python (:73-78) and Ruby (:88-93) READMEs. The text now describes only current behavior. Ruby ack() is now ack. The migration note now lives in #367's draft GitHub release notes ("Older servers can silently truncate receive results…", plus the default-100 "Before upgrading" warning). That matches CLAUDE.md ("Migration notes belong in release notes"). No new migration framing, PR/issue numbers or version tags were added to the README or docs/.

Other round-2 items:

Round-2 item Status
Non-blocking: docs/examples.md:116 contradicted reference.md on receive() empty batches ✅ Fixed. The text now says receive() also auto-finishes, which matches receive.sql:79-82 and frozen sql/pgque.sql:5380-5382.
Non-blocking: "Auto-generated by build/transform.sh" headers on hand-maintained files ❌ Not fixed; carried below as N4.
Potential: reference never ties the default of 100 to the ticker default of 500 ✅ Fixed: reference.md:127 (receive) and :185 (receive_coop).
Potential: plain-SQL upgrade note on sizing ceilings ✅ Fixed: installation.md:383-386, worded as current behavior.
Potential: TypeScript README ✅ Fixed (lines 77 and 124).
Potential: installation.md hard-coded update to '0.2.1' ❌ Not fixed (now at :446); carried as P4.
Potential: tutorial.md:232, :271 use receive(…, 1) limit 1 ❌ Not fixed; carried.
Potential: sql/pgque-tle.sql:7258 tautology '0.2.1' = '0.2.1' ❌ Not fixed (file untouched); carried.
Potential (security): a pre-registered pg_tle update path is reused without comparison Unchanged (file untouched); carried.
7 test-coverage potentials (stable CI overflow, upgrade branches, non-empty INT_MAX, partitioned/coop rollback assertions, NULL/N-1 per variant, when others catches, consumer-loop/TS tests) Unchanged. The delta touches no test, SQL or workflow file. Carried.

Validation of the CLI gate report (pr-366-samorev-gate.md), checked against full source at 1df5e36

Gate finding Verdict Evidence
#1 No CI check that frozen sql/ byte-matches the v0.2.1 tag CONFIRMED as fact; intentional process gate, not a defect .github/workflows/ci.yml references only v0.2.0 (git show v0.2.0:sql/pgque-tle.sql). There is no v0.2.1 tag. The PR description defines an explicit release-operator gate instead. A CI step remains advisable but optional. See the merge gate above.
#2 No CHANGELOG or release notes for the new failure mode REFUTED as a missing file in this PR PgQue has no CHANGELOG.md. Per CLAUDE.md, release notes are GitHub releases. #367's draft release notes contain the default-100 / 54000 / "size ceilings before upgrading" warning, which I verified in the #367 description. #366 now also documents the current behavior in reference.md:127, :185 and installation.md:383-386. Correction to the gate's wording: the defaults did not change. SQL and TypeScript were already 100 at v0.2.0 (git show v0.2.0:sql/pgque.sql, client.ts). What changed is that overflow now raises 54000 where it used to truncate.
Note The gate report does not record its head SHA. Its timestamp follows the 1df5e36 commit. Either way, both findings were re-checked against source at 1df5e36.

NON-BLOCKING (4)

High-confidence LOW/INFO findings, which can be addressed later. All were independently validated as TRUE.

LOW clients/go/concurrency_test.go:48-49 - The new comment says "maximum producer count", but the ceiling bounds the message count (2*expected, where expected = goroutines*perGoroutine). The code is correct. (Bugs; validated 9/10; introduced by delta)

Suggestion: "This test knows its maximum message count, so use a ceiling above that count."

LOW clients/python/README.md:73, clients/ruby/README.md:88 - Both still open with "controls the per-receive limit", and the next sentence calls the same value a "complete-batch safety ceiling". "Limit" suggests the pagination model this PR removes. (Guidelines + Docs + Bugs; validated 8/10; untouched context line next to the rewritten text)

Suggestion: "sets the complete-batch safety ceiling for each receive."

LOW sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql:2, sql/pgque-tle.sql:2 - Previously flagged, still unresolved. The headers say "Auto-generated by build/transform.sh", but transform.sh never generates install_update_path, set_default_version or the update body. (Docs + Guidelines; validated 8/10; in PR, outside delta)

Suggestion: Label these files hand-maintained, or generate them in the #367 source, and keep the byte-for-byte tag match.

INFO commit 1df5e36 message body - The body contains a literal \n\n ("…default ceilings.\n\nRefs #365.") where it should have real newlines, confirmed with od -c. The subject is compliant. (Guidelines; validated 10/10; development__git-commit-standards.mdc)

Suggestion: Don't amend (both CLAUDE.md and the rules say so). Fix it in the squash-merge message if one is used.


POTENTIAL ISSUES

Moderate-confidence findings (4-7/10). Review them manually.

New in this delta, or newly surfaced:

P1 LOW clients/go/pgque.go:153-158, :327-330, options.go:22-25, :129-132, clients/go/README.md:96, :242, clients/typescript/src/client.ts:152-156, :375-378, types.ts:63-65, clients/typescript/README.md:77, :124; also the Python README Consumer section - "Roll back and retry" is accurate for the Python low-level client, which defaults to autocommit=False (client.py:22). It doesn't fit Go/TS pool calls, which are single autocommit statements with no caller transaction to roll back, or the Python Consumer (autocommit=True, consumer.py:187) and Ruby (autocommit-like). It is harmless, because retrying returns the same batch and nothing is lost, but it can confuse readers. (confidence 7/10; introduced by delta)

Suggestion: For pool-based APIs, write: "The failed call changed nothing; retry with a larger resource-safe ceiling. If you call it inside your own transaction, roll back first."

P2 LOW clients/typescript/src/consumer.ts:7-11 - The DEFAULT_MAX_MESSAGES rationale still says the default exists so ack(batch_id) "does not strand events the client never saw". With the overflow guard, the real reason is avoiding 54000. This is pre-existing (the PR doesn't touch the file). The Python, Go and Ruby counterparts are accurate. (confidence 7/10)

Suggestion: "Request the whole batch so normal bursts do not raise SQLSTATE 54000."

P3 INFO Terminology drift in the delta: "ticker thresholds", "ticker threshold of 500", "ticker event-count threshold" and ticker_max_count; also "safety ceiling" alongside "complete-batch safety ceiling". (confidence 7/10; writing__terminology-consistency.mdc)

Suggestion: Use one term per concept, for example "ticker_max_count (event-count threshold)" and "complete-batch safety ceiling" at first mention.

P4 LOW docs/installation.md:446 - Previously flagged, still unresolved. alter extension pgque update to '0.2.1'; hard-codes a version tag in docs/. The wrapper calls set_default_version, so alter extension pgque update; works. (confidence 6/10)

Carried forward unchanged from round 2 (files untouched by the delta; details in the round-2 report):

  • Tests (7):
    • no overflow assertion on the frozen stable installer in stable-smoke;
    • upgrade-wrapper branches untested (fresh registration, post-update rerun, unsupported origin, plain 0.2.0→0.2.1 with data);
    • INT_MAX exercised only on empty batches;
    • no rollback-state assertions after partitioned/coop overflow;
    • NULL/N-1 tested only for plain receive;
    • when others catches without SQLSTATE asserts;
    • no consumer-loop overflow test, and no TS overflow test.
  • Docs: tutorial.md:232, :271 use receive(…, 1) limit 1, which is safe but an anti-pattern; the pg_tle snippet should name the supported origin.
  • Guidelines: sql/pgque-tle.sql:7258 has the constant tautology '0.2.1' = '0.2.1'.
  • Security (4/10): sql/pgque-tle.sql:7258-7263 reuses a pre-registered update path without comparing it to update_sql. Exploiting this requires pgtle_admin.

Specialized agent results

Agent Model Status Raw result
Security opus ✅ complete NO_FINDINGS. Confirmed the delta is byte-identical to git diff 6544e35 1df5e36 and that every Go/Python/TS hunk is comment or docstring only. No signature, default, SQL string or logic changes. The only SQL snippet change (100→1000) uses literal arguments, and the clients still use parameterised queries. The guidance steers users away from null (unbounded) and toward resource-safe ceilings plus alerting on 54000, so the DoS-relevant guidance improved. Prior LOW (pre-planted pg_tle path) unchanged.
Bug Hunter opus ✅ complete No MEDIUM+. Verified every factual claim against source:
• SQL receive/receive_coop default 100 (receive.sql:30, cooperative_consumers.sql:1116, frozen sql/pgque.sql:5343, :6595); queue_ticker_max_count default 500 (devel/sql/pgque.sql:110).
• The ticker only gates whether to tick (:760), so "thresholds do not cap batch size" is true.
• 54000 is raised before any finish_batch in all three variants, so no partial result reaches the caller.
• receive() auto-finishes empty batches.
• All Go/Python/TS/Ruby client and consumer defaults match the docs.
3 LOW (pool "roll back" wording → P1; test comment → non-blocking; TS consumer comment → P2). B1 FIXED. Gate #1 confirmed/intentional, #2 refuted with the defaults correction.
Test Analyzer sonnet ✅ complete NO_FINDINGS. No executable code or test bodies changed; the only test-file edit is a comment, and the Receive(…, 2*expected) call is unchanged. New doc claims are covered: empty-batch auto-finish (tests/test_receive_empty_batch.sql), 54000 in SQL plus Python/Ruby/Go client tests. All 7 round-2 test potentials are unchanged. Note: it took the round-2 list from the prompt and did not open the round-2 report itself.
Guidelines sonnet ✅ complete 5 INFO (per-receive "limit" wording; installation version tag; tautology; literal \n in commit body; terminology drift). B2 FIXED; no new migration framing, PR/issue numbers or version tags in the README/docs/. Org rules loaded locally from pgai-rules/rules (round 2 couldn't load them): professional-communication, terminology-consistency, binary-units, title-capitalization, git-commit-standards, ai-coding-guidelines, core-principles. The agent skipped python-style-guide and db-sql-style-guide, so the orchestrator checked them: added Python docstring lines are ≤79 chars (max 75), and the single changed SQL snippet is lowercase and schema-qualified.
Docs sonnet ✅ complete No new defects in the delta. Carried: version tag (P4), auto-generated headers (N4), tutorial receive(…,1). New LOW: per-receive "limit" wording. Defaults verified against SQL/Go/Python/TS/Ruby sources. The installation paragraph is correctly placed in plain-SQL Upgrading; INFO: the pg_tle subsection lacks an equivalent note. Full prior-validation table: B1 fixed, B2 fixed, examples.md fixed, 4 potentials fixed. Gate #2: partial, not a blocker.
Validator (independent) opus ✅ complete All 8 findings TRUE. Confidences: F1 8, F2 7, F3 9, F4 7, F5 10, F6 7, F7 6, F8 8. None reaches MEDIUM+ or blocking.
Sqitch Migration Checker — N/A postgres-ai/platform only
Visual Evidence — N/A No UI or visual paths changed (Markdown and code comments only; no web/ or site/ changes)

Prior evidence carried forward

  • Round 2 (full source at 6544e35) reviewed the SQL fix (all three receive variants, the partition probe that applies the slot predicate inside get_batch_cursor, statement rollback of allocation/takeover/registration), the pg_tle 0.2.0→0.2.1 wrapper (all 4 branches), the frozen-artifact identity (embedded TLE body == sql/pgque.sql; update bodies byte-identical; only 3 functions plus the header differ from v0.2.0), CI wiring, and the client sources and tests. No CRITICAL/HIGH/MEDIUM defects were found in executable code. These files are byte-identical at 1df5e36, so the conclusions carry forward.
  • Runtime evidence: CI 18/18 green at 6544e35 (run 36878058582) and re-run 18/18 green at the exact reviewed head 1df5e36 (run 36880437185). The re-run covered PG 14–19beta1 regression, Go/Python/Ruby/TS client tests, the pg_tle install path (including the 0.2.0→0.2.1 upgrade test), the frozen stable install smoke, the v0.1.0→HEAD upgrade, pg_cron, pg_timetable, verify and build.
  • Not carried forward: the author's local Postgres 18.3 and pg_tle v1.5.2 logs. They don't identify an exact SHA, so they are cited only as author claims.

Coverage and incompleteness

  • Diff: all 347 lines (23 KiB) of git diff 6544e35 1df5e36 were read in full by the orchestrator and every agent. Nothing was truncated or clipped. Full source at 1df5e36 was read wherever a claim needed checking: SQL receive/coop/partitioned/ticker, frozen sql/, v0.2.0 artifacts, all four client libraries and consumers, the touched docs sections, ci.yml, transform.sh and CLAUDE.md.
  • This is a delta review. Files outside the delta were not re-reviewed line by line in this round. Their coverage comes from round 2 (identical bytes) and CI at the exact head.
  • This review was static. No agent ran databases or test suites. Runtime behavior relies on CI at 1df5e36.
  • Out of scope: fix: backport receive safety to 0.2.1 #367's own delta (22e0b24..dc0b698, including its new docs/upgrading.md) was not reviewed here. Only the byte-identity of the three frozen sql/ files and the release-notes text in its description were checked.
  • All five specialized reviewers and the validator completed. The one rule-coverage gap (two rule files skipped by the Guidelines agent) was closed by the orchestrator. SOC2 was skipped, as PgQue's CLAUDE.md requires.

Summary

Area Findings Potential Filtered
CI/Pipeline 0 0 0
Security 0 1 (carried) 0
Bugs 1 2 0
Tests 0 7 (carried) 0
Guidelines 1 2 (1 new, 1 carried) 0
Docs 2 (1 carried) 3 (carried) 0
Metadata 0 0 0

Notes:

  • Findings are high-confidence issues (8-10/10), all non-blocking here (LOW/INFO). Cross-agent findings are counted once. The Docs row covers the per-receive wording (new) and the auto-generated headers (carried).
  • Potential are moderate-confidence issues (4-7/10).
  • Filtered: none. The Docs INFO (pg_tle subsection lacks a sizing note) and the "Monitor 54000" actionability remark were judged below the reporting bar and are listed in the agent table only.

Merge gate: there is no review blocker at 1df5e36. Merge only after #367 is merged and v0.2.1 is tagged, and only after the release operator posts a successful git diff --exit-code v0.2.1 -- sql/pgque.sql sql/pgque-tle.sql sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql. Any further commit to #366 needs a delta re-review.


samorev-assisted review (AI analysis by Tanya301/samorev)

Clarify ticker thresholds and consumer comments. Refs #365.
@NikolayS

NikolayS commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

samorev Code Review Report — round 4 (delta review)

Reviewed head: e8074faf4112f768b5f0e21e161c64d751b5ad85 (#366, base main). This is the delta from the round-3 reviewed head 1df5e363b1531edb5d130d4e7aff4500e05d0aa4. Any later commit voids this review, so re-review the delta before merging.

Companion delta also reviewed: #367 head 034f3f6b353caeedd00d3f8cb1041cde58520692, the delta from its round-3 reviewed head dc0b698afce03bd4d9043d9a75b6d30584e78253. This report is posted on #366 only.

Pipeline Coverage
✅ #366: 18/18 checks pass on e8074fa (CI run 36881988881 + Website build 36881989067, headSha verified = e8074fa) Not reported
✅ #367: 28/28 checks pass on 034f3f6 (CI runs 36881837480 push + 36881843351 pull_request, headSha verified = 034f3f6) Not reported

Verdict: ✅ PASSED. There are no blocking findings at e8074fa, and none in #367's delta at 034f3f6.

  • All five specialized reviewers completed on the delta and returned NO_FINDINGS.
  • The one potential finding raised during the round (Docs) was rejected by an independent validator (FALSE, 3/10) and filtered.
  • There were no open blocking findings after round 3, and none were reopened.
  • The round-3 PASS carries forward for unchanged code.

Merge gates still apply, and this review does not lift them:

  1. fix: reject truncated receive batches #366 release-operator gate (from the fix: reject truncated receive batches #366 description): do not merge fix: reject truncated receive batches #366 until fix: backport receive safety to 0.2.1 #367 is merged, the v0.2.1 tag is published, and git diff --exit-code v0.2.1 -- sql/pgque.sql sql/pgque-tle.sql sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql passes, with the exact tag and head posted. As of this review, no v0.2.1 tag exists on the remote (git ls-remote --tags).
  2. Release notes: the complete draft v0.2.1 GitHub release notes are still in the fix: backport receive safety to 0.2.1 #367 description. They include the "Before upgrading" default-100 warning, SQLSTATE 54000, and the "older servers can silently truncate" note. Keep them when publishing.
  3. fix: backport receive safety to 0.2.1 #367 merge prerequisite carried from its round 3: the real v0.2.0 upgrade/reapply evidence (CLAUDE.md step 3) has still not been posted on fix: backport receive safety to 0.2.1 #367. Its comments are only the Codex notice and the samorev round 1–3 reports.

Prior findings: validation at the new heads

Blocking: round 3 had no open blocking findings on either PR. Round 2's B1 and B2 on both PRs were validated FIXED in round 3, and this delta does not touch those files' fixed text, so they stay fixed. The CLI gate items (no CI tag byte-match; missing release notes) were validated in round 3 as an explicit operator gate and as present in the #367 description. They are not new findings, and both gates are kept above.

Non-blocking and potential items this delta targets:

Round-3 item Status Evidence (exact head)
#366 N1 / #367 P (LOW): clients/go/concurrency_test.go:48-49 says "maximum producer count" ✅ FIXED (both PRs) It now says "maximum message count". The test sends 10×20 = 200 events (expected), and the ceiling is 2*expected = 400. The Receive call is unchanged.
#366 N2 (LOW): Python/Ruby README "controls the per-receive limit"; #367 P: Python README:73 ✅ FIXED (Python on both, Ruby on #366) It now says "sets the complete-batch safety ceiling for each receive". Python consumer.py and Ruby consumer.rb default to 2_147_483_647 and pass the value straight through to receive/receive_coop → pgque.receive(..., i_max_return), which raises 54000 at the (N+1)th event with no partial rows.
#366 P2 (LOW): clients/typescript/src/consumer.ts:7-11 "strand events" rationale ✅ FIXED (both PRs) It now reads "so ordinary bursts do not exceed a smaller receive ceiling (SQLSTATE 54000)". DEFAULT_MAX_MESSAGES = 2_147_483_647 is used as opts.maxMessages ?? DEFAULT_MAX_MESSAGES (line 46) for receive/receiveCoop. The plain client defaults to 100. The comment is accurate.
#367 N1 (LOW): docs/pgq-concepts.md:67 "(batch-size cap)" ✅ FIXED It now reads "event-count threshold for creating a tick; not a batch-size cap". The ticker in sql/pgque.sql:757-763 skips a tick only while new_events < queue_ticker_max_count and lag < queue_ticker_max_lag, so the value is a tick trigger, not a cap. This agrees with reference.md:135.
#366 P3 (INFO): terminology drift ⬆️ Improved, not eliminated The last "limit"/"cap" wording is gone. The remaining "event-count threshold" vs "tick-trigger threshold" and "creating" vs "requesting a tick" variants were judged interchangeable (Guidelines: 3/10, below the bar).

Carried forward unchanged (files untouched by this delta; details in the round-3 reports):

  • fix: reject truncated receive batches #366:
    • N3 "Auto-generated by build/transform.sh" headers on hand-maintained sql/ files.
    • N4 literal \n\n in commit 1df5e36's body. The new commit e8074fa is clean, confirmed with od -c.
    • P1 "roll back and retry" wording for pool/autocommit APIs.
    • P4 docs/installation.md:446 hard-codes update to '0.2.1'.
    • tutorial.md:232, :271 use receive(…,1) limit 1.
    • sql/pgque-tle.sql has the '0.2.1' = '0.2.1' tautology.
    • Security 4/10: a pre-registered pg_tle update path is reused without comparison.
    • 7 test-coverage potentials (listed in round 3).
  • fix: backport receive safety to 0.2.1 #367:
    • No plain-SQL v0.2.0→HEAD CI job (7/10).
    • Demo 100 snippets without a caveat.
    • when others overflow assertions.
    • upgrading.md pg_tle origin actionability.
    • Untested wrapper branches.
    • No TS overflow test.
    • Tutorial receive(…,1).
    • pgque-tle.sql trailing \echo / README link.
    • Security 4/10: transform.sh dollar-quote guard.

BLOCKING ISSUES (0)

None.

NON-BLOCKING (0 new)

None new. The round-3 non-blocking items still open are listed under "Carried forward" above.

POTENTIAL ISSUES (0 new)

None survived validation. Filtered item, shown for transparency:

  • Docs, LOW (6/10), filtered by validator (FALSE, 3/10): clients/go/concurrency_test.go:253-257 on both heads.
    • The comment "prior to WithMaxMessages the Consumer hardcoded Receive(..., 100), capping every poll at 100 rows and stranding any extras" was flagged as stale framing.
    • The validator's reasons for rejecting it:
      • The comment is explicitly historical and was accurate for the pre-PR server.
      • It is unchanged since v0.2.0 and outside the delta.
      • The suggested rewrite ("rejected with 54000") would itself be historically wrong, because the old Consumer never ran against the new contract.
      • The test it explains is still valid.

Informational, not a finding (Bug Hunter): the inherited upstream PgQ table comment queue_ticker_max_count - batch should not contain more events remains at sql/pgque.sql:87 and sql/pgque-tle.sql:152. It also appears in the generated devel/sql/pgque.sql:87 and devel/sql/pgque-tle.sql:177, and in v0.2.0. It predates this PR, is untouched by the delta, and lives in frozen or generated artifacts. If wanted, fix it at the source in a separate change.


Specialized agent results (all 5 completed on the delta)

Agent Model Status Result
Security opus ✅ complete NO_FINDINGS.
• Confirmed both ranges are exactly one commit, with an empty git diff --stat -- sql/. The only source-file edits are a Go test comment and a TS JSDoc block. DEFAULT_MAX_MESSAGES and the Receive(..., 2*expected) call are unchanged.
• No SQL, shell, credential, auth or network surface.
• The README guidance still steers away from unbounded use and from acking a failed receive. The TS consumer logs, sleeps and continues on receive error, with no ack.
• The carried pg_tle potential is unaffected.
Bug Hunter opus ✅ complete NO_FINDINGS. Verified all four factual claims against exact-head source in both worktrees:
• Go ceiling 400 > 200 messages.
• Python/Ruby max_messages passthrough, with 54000 raised at N+1 in devel and frozen SQL.
• TS default used as the receive ceiling.
• The #367 ticker threshold semantics (sql/pgque.sql:757-763); get_batch_events returns the whole tick window.
No logic changed. One informational note (the inherited PgQ table comment above).
Test Analyzer sonnet ✅ complete NO_FINDINGS.
• No test logic or executable code changed, so no new tests are required.
• The doc claims are covered by existing tests: int-max defaults asserted in TS consumer.test.ts:373 and Go consumer_internal_test.go:169-170; 54000 in tests/test_receive_overflow.sql, tests/test_tle_upgrade_v0_2.sql and the Python/Ruby/Go client tests.
• The 7 carried test potentials are unchanged, including no consumer-loop and no TS overflow test.
Guidelines sonnet ✅ complete NO_FINDINGS.
• Org rules read locally from pgai-rules/rules: professional-communication, terminology-consistency, title-capitalization, binary-units-standards, git-commit-standards, ai-coding-guidelines, core-principles. CLAUDE.md read at both heads.
• Comment style matches CLAUDE.md (TS JSDoc, Go //).
• No PR/issue numbers, version tags or migration framing in the changed README/docs lines.
• The commit message is compliant on both heads: 40-character conventional subject, body lines ≤72, no literal \n (od -c).
• The new README lines are about 92 characters, but no rule limits Markdown line length.
• Terminology drift improved; the remaining variants are 3/10.
• Not applicable to a comment/Markdown-only delta: the python, db-sql and shell style guides. No Python, SQL or shell code changed.
Docs sonnet ✅ complete 1 LOW potential (6/10), rejected by the validator (see Filtered).
• Items (a)–(d) verified FIXED.
• A sweep at both exact heads across clients/, docs/ and README.md found no remaining stale cap, pagination or truncation wording. These returned nothing: "per-receive", "strand events", "rows beyond", "skipped after ack", "drop rows", "match ticker_max_count", "older servers", "overflow guard", and "batch-size cap" used affirmatively. The remaining hits were checked and are accurate.
• P4, tutorial receive(…,1) and the auto-generated headers are unchanged.
Validator (independent) opus ✅ complete The single finding (Docs, Go test comment :253-257) was ruled FALSE, 3/10: historical, pre-existing since v0.2.0, outside the delta, and the suggested rewrite would be inaccurate.
Sqitch Migration Checker — N/A postgres-ai/platform only
Visual Evidence — N/A No UI or visual paths changed (a Go test comment, a TS JSDoc block and Markdown)

Prior evidence carried forward

  • fix: reject truncated receive batches #366 round 2, full source at 6544e35: reviewed the SQL fix in all three receive variants, the partition probe (slot predicate inside get_batch_cursor), statement rollback, the pg_tle 0.2.0→0.2.1 wrapper, frozen-artifact identity, CI wiring, and client sources and tests. Round 3 at 1df5e36 was a docs-only delta on top of that. Every executable, SQL, CI and test file except the one Go test comment is byte-identical between 1df5e36 and e8074fa, so those conclusions carry forward.
  • fix: backport receive safety to 0.2.1 #367 round 2, full source at 22e0b24, and round 3 at dc0b698: these carry forward the same way. The 034f3f6 delta is comments and docs only.
  • Runtime evidence: CI is green at the exact reviewed heads:
  • Not carried forward as verified: the author's local Postgres 18.3 and pg_tle v1.5.2 logs. They don't identify an exact SHA, so they remain author claims.

Coverage and incompleteness

  • Diff: the full combined delta (110 lines, about 5 KiB, both PRs) was read by the orchestrator and by every agent. Nothing was clipped or truncated.
  • Full source read where a claim needed checking:
    • Go concurrency_test.go
    • Python consumer.py, Ruby consumer.rb and TS consumer.ts, plus client.*
    • devel receive.sql and frozen sql/pgque.sql (the 54000 sites and the ticker)
    • docs/reference.md and docs/pgq-concepts.md
    • CLAUDE.md at both heads
    • both round-3 reports
  • This is a delta review. Unchanged files were not re-audited line by line this round. Their coverage comes from the prior full-source rounds (identical bytes) and from CI at the exact heads.
  • This review was static. No agent ran databases or test suites, so runtime behavior relies on CI at the exact heads.
  • Coverage is complete for the declared scope. All five specialized reviewers and the validator completed, and the org rules were loaded locally. fix: backport receive safety to 0.2.1 #367 is covered only for its dc0b698..034f3f6 delta, the byte-identity of its frozen files, and the presence of its release notes. The full fix: backport receive safety to 0.2.1 #367 report remains its own round-3 report.

Summary

Area Findings Potential Filtered
CI/Pipeline 0 0 0
Security 0 0 new (1 carried per PR) 0
Bugs 0 0 0
Tests 0 0 new (carried unchanged) 0
Guidelines 0 0 0
Docs 0 0 1
Metadata 0 0 0

Notes:

  • Findings are high-confidence issues (8–10/10).
  • Potential are moderate-confidence issues (4–7/10).
  • Filtered are 0–3/10 after validation.
  • Round-3 items carried unchanged are listed above and not recounted.

Result: PASSED (delta scope) at #366 e8074faf4112f768b5f0e21e161c64d751b5ad85. #367's delta 034f3f6b353caeedd00d3f8cb1041cde58520692 has no findings.

This is a review verdict, not merge approval. Merge #366 only after #367 is merged (with its v0.2.0 reapply evidence posted), v0.2.1 is tagged, and the release operator posts a successful git diff --exit-code v0.2.1 -- sql/pgque.sql sql/pgque-tle.sql sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql. Any further commit to either PR needs a delta re-review.


samorev-assisted review (AI analysis by Tanya301/samorev)

@NikolayS

NikolayS commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

Post-review real-test evidence

Exact reviewed/tested head: e8074faf4112f768b5f0e21e161c64d751b5ad85. SamoRev round 4 PASS is posted above; no source changes followed it. All commands below completed after that review with exit status 0.

Environment: isolated Postgres 18.6 containers, no published ports, localhost listening, connection logging enabled. pg_tle upgrade uses v1.5.2.

Commands and results

From the candidate checkout, the regression harness executes:

psql -X -v ON_ERROR_STOP=1 -f devel/sql/pgque.sql
psql -X -v ON_ERROR_STOP=1 -f tests/run_all.sql
psql -X -v ON_ERROR_STOP=1 -f tests/acceptance/run_acceptance.sql
psql -X -v ON_ERROR_STOP=1 -f devel/sql/pgque.sql -f tests/test_install_idempotency.sql

Outputs: ALL TESTS PASSED, ALL ACCEPTANCE TESTS PASSED, and PASS: main post-review regression, acceptance, reapply.

Additional executable harnesses (all exit 0):

  • bash doc-real-test/run.sh: six snippets extracted byte-for-byte from current main/stable reference and examples; reference receive, fan-out, exactly-once transaction; 1,001-event burst against the documented 1,000 ceiling. Asserts SQLSTATE 54000, explicit rollback preserves the cursor, then retry processes all 1,001 distinct event identities, range 1–1,001, before acknowledgment. Both surfaces PASS.
  • bash upgrade-smoke.sh: installs tagged v0.2.0, creates populated normal/cooperative queues, active batch, retry/DLQ/config state; applies the maintenance installer twice; asserts state retained, version 0.2.1, overflow and complete recovery. Output: PASS: isolated v0.2.0 to v0.2.1 upgrade smoke test.
  • bash tle-upgrade-smoke.sh: installs tagged v0.2.0 pg_tle wrapper, registers the candidate update path twice, updates populated extension, checks both versions and preserved active batch/subscription state, verifies plain/cooperative 54000 plus full retry. Also validates fresh CREATE EXTENSION through the registered default update edge. Output: test_tle_upgrade_v0_2: ALL PASSED.

The upgrade harnesses use the maintenance candidate; its three frozen release SQL files are byte-identical to this PR's frozen files. No throughput benchmark or client package publication is claimed.

Non-blocking review dispositions

  • Generated-file provenance: the maintenance build/transform.sh does generate the pg_tle wrapper and function-only update edge. Main's frozen files are copied from that maintenance build; they are not independently hand-maintained. The published-tag comparison remains mandatory before fix: reject truncated receive batches #366 merges.
  • The plain-SQL v0.2.0 CI-matrix gap is covered for this release by the populated post-review test above. Optional additional per-variant boundaries, TypeScript/consumer-loop error tests and wrapper-branch coverage remain non-blocking test-hardening work; existing SQL coverage checks all receive paths, event recovery and SQLSTATE.
  • One-event tutorial/demo ceilings are valid for their fixtures; the reference and release notes explicitly warn about default 100, ticker thresholds, and operator-selected resource limits. Real pagination is feat: design safe paged batch consumption #364.
  • Automatic statement rollback in pooled/autocommit APIs requires no explicit client rollback; the general rollback instruction applies to an explicit failed transaction. No runtime change is needed.
  • Reusing an already registered pg_tle update path requires a trusted pg_tle administrator; additional catalog-integrity checks are optional defense in depth. The generated dollar-quote update body is extracted from the already scanned installer sources.
  • Historical commit formatting is retained (no amend). Remaining wording/style and optional origin-introspection suggestions do not change the verified contract.

Evidence integrity

Logs retained in pgque-overflow-evidence/; SHA-256:

4bd42b90d868945ac2f9d9423b74cd5d85016c15afa73b13baf4208b36f75cdd  post-review-main-regression.log
b867a9c516d88b3028c33feb5185774042a66aa2e775347e272d6cccdeb68395  post-review-doc-real-test.log
8377c976e4837577be296a8972a6f7c80e70d55ea08bd38d3bd109224d84b2eb  post-review-upgrade-smoke.log
413aad50bf41aa3217f9985e39041a34e79a1723040d8be7e095bcb087db7d1b  post-review-tle-upgrade.log

Release sequencing remains: merge maintenance, verify exact merge CI, publish and verify v0.2.1, prove frozen artifacts match the tag, then merge development.

@NikolayS

NikolayS commented Oct 1, 2026

Copy link
Copy Markdown
Owner Author

Published-tag gate: PASS

  • Maintenance PR fix: backport receive safety to 0.2.1 #367 is MERGED at 32bfbe2119c8fcadf6d49efe886435fdde12500d; its tree exactly matches reviewed head 034f3f6b353caeedd00d3f8cb1041cde58520692.
  • Exact-merge CI run 36883000305: 14/14 jobs passed.
  • v0.2.1 is published, not draft/prerelease, and confirmed by GitHub's latest-release endpoint.
  • git rev-parse 'v0.2.1^{commit}' = 32bfbe2119c8fcadf6d49efe886435fdde12500d.
  • Downloaded all three SQL release assets plus SHA256SUMS; sha256sum -c SHA256SUMS reports OK for all three, and each file matches the exact merge artifact.

At development PR #366 head e8074faf4112f768b5f0e21e161c64d751b5ad85:

git diff --exit-code v0.2.1 -- sql/pgque.sql sql/pgque-tle.sql sql/pgque-tle-updates/pgque--0.2.0--0.2.1.sql

Exit 0, no diff. The frozen SQL/tag-match release-operator gate is satisfied.

A fresh detached checkout of the published tag also passed (all exit 0):

  • fresh SQL install, pgque.version() = '0.2.1', and the complete overflow regression;
  • populated v0.2.0 plain-SQL upgrade/reapply, retained state and complete recovery;
  • populated pg_tle v1.5.2 update plus fresh CREATE EXTENSION through the registered update edge.

Output markers: PASS: tagged fresh SQL install, version 0.2.1, overflow regression; PASS: isolated v0.2.0 to v0.2.1 upgrade smoke test; test_tle_upgrade_v0_2: ALL PASSED.

No client packages were published. Real pagination remains tracked separately in #364.

@NikolayS
NikolayS merged commit b8933a8 into main Oct 1, 2026
18 checks passed
@NikolayS
NikolayS deleted the fix-receive-overflow branch October 1, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants