Polling fallback: read each list by updated_at and keep a cursor per list

README.md

@@ -238,10 +238,10 @@ told where Node is instead of looking.
 | `GITHUB_WEBHOOK_SECRET_FILE` | when `GITHUB_WEBHOOK_PORT` is set | — | Path to the webhook secret, mode 600. |
 | `GITHUB_WEBHOOK_PORT` | no | unset = listener off | TCP port of the webhook listener (bind it to loopback, terminate TLS in front). |
 | `GITHUB_WEBHOOK_PATH` | no | `/github/webhook` | Path the listener answers on. |
-| `GITHUB_POLL_INTERVAL_MS` | no | `0` (off) | Polling fallback: issues and PRs, issue comments and review comments updated since the cursor, plus the checks on the heads of the open PRs seen. `0` is off; a non-zero value below `60000` is refused. Partial — it cannot see a submitted review, a review request or a push ([what polling misses](docs/deploy.md#option-3--polling-only-no-webhook-at-all)) — and it is not meant to run alongside the webhook. |
+| `GITHUB_POLL_INTERVAL_MS` | no | `0` (off) | Polling fallback: issues and PRs, issue comments and review comments updated since each list's cursor, plus the checks on the heads of the open PRs seen. `0` is off; a non-zero value below `60000` is refused. Partial — it cannot see a submitted review, a review request or a push ([what polling misses](docs/deploy.md#option-3--polling-only-no-webhook-at-all)) — and it is not meant to run alongside the webhook. |
 | `GITHUB_REPOS` | yes | — | Comma-separated `owner/name` allow-list; a write anywhere else is refused by name. Several entries must share one owner (one App installation, one token); a second owner is a second server process. With several, every tool call names its repository. |
 | `GITHUB_PR_BASE_ALLOW` | no | empty = no PR target allowed | Branches a pull request may target or be retargeted to. A plain `branch` applies to every repository, `owner/name:branch` to that one: `anima-research/rayforce:dev,main`. |
-| `GITHUB_STATE_FILE` | no | unset = in-memory only | Durable JSON: checkpoint lineage, rollback bookkeeping, the poll cursor. |
+| `GITHUB_STATE_FILE` | no | unset = in-memory only | Durable JSON: checkpoint lineage, rollback bookkeeping, and the polling state: a cursor for each repository and list (`pollCursors`), `lastPollAt` where a list with none starts, and the rows announced as new (`pollNew`). A list with a cursor of its own starts from it, not from `lastPollAt`. |
 | `GITHUB_BOT_LOGIN` | yes | — | The App's bot login, e.g. `rayforce-clone[bot]`. From-self and mention detection depend on it. |
 | `GITHUB_MAINTAINERS` | no | empty | Comma-separated logins marked `isMaintainer` in metadata and tagged `github:actor=maintainer`. |
 

changelog.d/fix-poll-paging.fixed.md

@@ -0,0 +1,23 @@
+- The polling fallback no longer loses rows in three ways it did. A page
+  shorter than 100 no longer ends a list while `Link: rel="next"` says more
+  follow. A read stopped at the three-request bound keeps that list's cursor
+  where it stopped, and a later pass reads on from there, instead of moving
+  past the rows it never read. And each list is read by `updated_at` rather
+  than by page offset, so a row updated mid-read can't push an unread one onto
+  a page already read, except among rows that share one `updated_at` second,
+  which are still paged by offset.
+- Cursors are whole seconds. GitHub's `since` part way through a second leaves
+  out every row stamped in that second, so a cursor at the pass's start made
+  the next pass miss rows changed later in the second the pass started in.
+- Each repository's three lists keep a cursor of their own in the state file
+  (`pollCursors`), starting from `lastPollAt` until they have one.
+  `lastPollAt` never passes where a list stopped.
+- A row announced as opened isn't announced as opened again when it's read
+  again inside its window, after a stopped read or a refused delivery
+  (`pollNew` in the state file). A first pass that a delivery refused keeps
+  the window it read.
+- `getCheckRuns` reads on past a short page that isn't empty while
+  `total_count` says more runs follow, so such a head is no longer left
+  undecided. A response
+  without a `total_count` reads on past a full page instead of stopping at
+  the first.

docs/contract-notes.md

@@ -33,23 +33,43 @@ camel case, so a reviewer can map a contract bullet to a type by sight.
   `issues?state=all&sort=updated&since=`), `listIssueCommentsSince`
   (`issues/comments?sort=updated&since=`) and `listReviewCommentsSince`
   (`pulls/comments?sort=updated&since=`), returning `PolledIssue` /
-  `PolledComment` rows. The poller (.3) keeps the cursor `lastPollAt`, dedups
+  `PolledComment` rows, each list read by `updated_at` rather than by page
+  offset, three requests a pass and more only to finish a second. The poller
+  (.3) keeps a cursor for each repository's list in whole seconds
+  (`pollCursors`, starting from `lastPollAt` until it has its own), which a
+  read stopped at that bound leaves where it stopped, dedups
   with the key `<kind>:<id>:<updated_at>` through `StateStore.seenPollKey`
   (ring of `POLL_DEDUP_RING` = 2000 keys, .7), names poll events
   `poll:<kind>:<id>:<updated_at>`, and asks `getCheckRuns` for the heads of
   the open PRs it saw, at most 20 per poll. These endpoints carry no
   `X-Poll-Interval`; `MIN_POLL_INTERVAL_MS` is the floor.
+  - Each `…Since` method returns `PollRead<T>`, `{ rows, resumeAt? }`:
+    `resumeAt` is set when the request bound stopped the read, and names the
+    second the next read asks from.
+  - `StateSnapshot` gains `pollCursors` (`<owner>/<name>:<list>` →
+    `{ since, resumeAt? }`) and `pollNew` (the same key → id → `created_at`),
+    read and written through `StateStore.pollCursors`/`setPollCursors` and
+    `polledNew`/`setPolledNew`. Both fields are **additive**: a state file
+    without them loads with no cursors and nothing remembered, so every list
+    starts from `lastPollAt`, and a release before them ignores them and
+    reads `lastPollAt`, up to which every list has delivered.
 - **Amendment 2a: a poll row's kind comes from `created_at` against the
   cursor.** Amendment 1 left the polling ingress unable to tell an opened
   issue from an edited one, so every open issue row arrived as `issue.edited`
   and a recipe keyed on `github:issue=opened` could never fire on it (found by
   the WP10 end-to-end run). A `since` query still carries no action, but it
   does carry `created_at`, and that is enough: `PolledIssue` gains `createdAt`
-  and `normalizePolledIssue(repo, row, receivedAt, since, previousHeadSha?)`
+  and `normalizePolledIssue(repo, row, receivedAt, since, previousHeadSha?, announced?)`
   derives the kind from it.
-  - An **issue** row created at or after `since` is `issue.opened`; otherwise
-    `issue.edited`, as before. Closed rows stay `issue.closed`.
-  - A **pull-request** row created at or after `since` is `pull.opened`. An
+  - An **issue** row created at or after `since`, and not announced as new
+    already (`announced`), is `issue.opened`; otherwise
+    `issue.edited`, as before. Closed rows stay `issue.closed`. `since` is
+    the start of the list's window, which stays put while a stopped read is
+    caught up. The state also keeps the rows announced as new while their
+    `created_at` is inside the window (`pollNew`), so a row read again there
+    is not opened twice.
+  - A **pull-request** row created at or after `since`, and not announced
+    as new already, is `pull.opened`. An
     open row seen again is `pull.synchronize` when its head sha moved away
     from the one the last pass recorded, and is **dropped** (null) when it did
     not: every comment on a pull request bumps its row, and the comment and
@@ -60,7 +80,9 @@ camel case, so a reviewer can map a contract bullet to a type by sight.
     rather than not at all, which is the bounded cost of not leaving a cold
     start blind to what is already open.
   - The comparison is of instants, not strings: GitHub stamps rows without
-    milliseconds while the cursor carries them, and `'Z'` sorts after `'.'`.
+    milliseconds while a cursor is written with them (`.000Z`, and any
+    precision in a state file an older release wrote), and `'Z'` sorts after
+    `'.'`.
   - `StateSnapshot` gains `pollHeads` (`<owner>/<name>#<number>` → head sha),
     bounded by `POLL_HEAD_MEMORY` = 200 and recorded on a dropped row as well
     as on a delivered one. The field is **additive**: a state file written

docs/deploy.md

@@ -15,7 +15,7 @@ user the agent runs as (`ubuntu` on a typical agent host):
 | `~/.config/github-mcpl/github-mcpl.env` | 600 | variable names and paths, **no secrets** |
 | `~/.config/github-mcpl/app-private-key.pem` | **600** | the App private key |
 | `~/.config/github-mcpl/webhook-secret` | **600** | the webhook secret, one line |
-| `~/.config/github-mcpl/state.json` | 600 | checkpoint lineage and the poll cursor |
+| `~/.config/github-mcpl/state.json` | 600 | checkpoint lineage and the polling state |
 
 The server refuses to start if either secret file is readable by group or
 other: `readConfig` stats both and reports the mode by name. That check is the