Skip to content

feat(sdk): add lazy pagination pagers - #3256

Merged
drew merged 4 commits into
refactor/2802-continuation-pagination/drewfrom
2802-sdk-pagination-pagers/drew
Sep 11, 2026
Merged

feat(sdk): add lazy pagination pagers#3256
drew merged 4 commits into
refactor/2802-continuation-pagination/drewfrom
2802-sdk-pagination-pagers/drew

Conversation

@drew

@drew drew commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add lazy, single-pass page iterators to the curated Rust, Python, Go, and TypeScript SDKs. List methods now return a pager without issuing an RPC, while explicit list_all/ListAll helpers preserve the convenient exhaustive-collection path.

Related Issue

Part of #2802. Stacked on #3249.

Changes

  • Add shared Page<T> and Pager<T> abstractions in each curated SDK, with one request per page and support for resuming from an initial page token.
  • Make list methods lazy and expose page boundaries plus next_page_token; add explicit helpers that exhaust all remaining pages.
  • Preserve the current continuation token after a fetch failure so callers can retry the same page.
  • Update Go client interfaces and fakes, SDK examples, E2E callers, and user/architecture documentation for the breaking list API.
  • Add tests for lazy execution, page-by-page traversal, collection, resume tokens, retry behavior, and non-nil empty Go results.

Testing

  • mise run pre-commit passes
  • OPENSHELL_SYSTEM_GATEWAY_DIR=/tmp/openshell-system-config-empty-2802-followup mise run test passes, including 1,450 server tests
  • cargo test -p openshell-sdk passes (21 unit and 33 mock-client tests)
  • uv run pytest python/openshell/sandbox_test.py passes (130 tests)
  • mise run sdk:ts:test passes (105 tests); the earlier full mise run sdk:ts:ci passes (104 tests before the added review test)
  • mise exec -- go test ./openshell/v1 ./openshell/v1/fake passes
  • Unit tests added/updated
  • E2E callers updated (full gateway E2E not run)

mise run ci passes all Rust, Python, TypeScript, lint, format, and documentation gates. Its Go lane fails only in existing gateway-discovery tests because this host has /etc/openshell/gateways/default; the changed Go SDK and fake packages pass independently.

Checklist

@drew
drew added this pull request to stack #3257 September 10, 2026 16:32
@drew
drew requested review from a team and derekwaynecarr as code owners September 10, 2026 16:32
@github-actions

Copy link
Copy Markdown

@drew
drew force-pushed the 2802-sdk-pagination-pagers/drew branch from 00659bd to 9f162b0 Compare September 10, 2026 16:59
@drew
drew force-pushed the 2802-sdk-pagination-pagers/drew branch 2 times, most recently from fc56d57 to c0b1bc0 Compare September 10, 2026 23:04

@johntmyers johntmyers left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gator-agent

PR Review Status

The lazy pager implementations are coherent across the four curated SDKs, but the Go all-workspaces migration leaves published examples that no longer compile and a Fern guide that describes the removed scope contract.

Action required: update the Go and Fern all-workspaces examples to use explicit scope arguments and AllWorkspaces: true.

Blocking findings:

  • GATOR-c0b1bc02-01: Go all-workspaces documentation still uses the removed zero-scope ListAll calls.

Carried findings:

  • None
Gator metadata
  • Validation: Maintainer-authored, issue-backed SDK work within #2802 and its two-PR implementation stack.
  • Docs: Blocking drift remains in Go API examples and Fern workspace guidance.
  • Checks: Current-head Branch Checks are still running; DCO, Trivy, Helm, docs preview, and published E2E/GPU E2E gates are green.
  • E2E: Test-label decision deferred until blocking review feedback is resolved.
  • Head SHA: c0b1bc024efc7a81af270fcfe943a874259f77e2
  • Base SHA: bd7da2eba89328f9b113dd57732ec4608ee5a3d7
  • Merge base SHA: bd7da2eba89328f9b113dd57732ec4608ee5a3d7
  • Patch ID: 83e5ec631ce0abce9bc1d4f73803265cd720cbcc
  • Gator payload: 9
  • Review mode: initial
  • Previous reviewed SHA: none
  • Review budget exhausted: no
  • Maintainer decision required: no
  • Next state: gator:in-review

Comment thread sdk/go/docs/src/api/sandboxes.md
@johntmyers johntmyers added the gator:in-review Gator is reviewing or awaiting PR review feedback label Sep 10, 2026
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@drew
drew force-pushed the 2802-sdk-pagination-pagers/drew branch from c0b1bc0 to 6c4b941 Compare September 10, 2026 23:44
@johntmyers johntmyers added the test:e2e Requires end-to-end coverage label Sep 10, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for 6c4b941. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@johntmyers johntmyers left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gator-agent

PR Review Status

Thanks @drew. I checked your current-head update against the prior all-workspaces finding: the sandbox, provider, and service examples now pass the required explicit scope arguments with AllWorkspaces: true, the Fern guide describes the same selector, and the added request-level tests cover the sandbox and provider wire requests. The carried finding is resolved, and the follow-up review found no new blocking issues.

Blocking findings:

  • No blocking findings remain

Carried findings:

  • GATOR-c0b1bc02-01: resolved by the current-head documentation and request-test updates
Gator metadata
  • Validation: Maintainer-authored, issue-backed SDK work within #2802 and its stacked implementation.
  • Docs: Go SDK examples and Fern workspace guidance are updated for the explicit all-workspaces selector.
  • Checks: Required Branch Checks are pending; DCO, Helm Lint, Trivy Changes, and docs preview are green.
  • E2E: test:e2e applied; E2E Label Help requested a rerun, and Branch E2E Checks run 34543507320 attempt 2 is queued for the current head.
  • Head SHA: 6c4b94166d6d299cfab887a936c782f2e8175ac6
  • Base SHA: fefe01e07f5f167413939b4e24589a4309f8624b
  • Merge base SHA: fefe01e07f5f167413939b4e24589a4309f8624b
  • Patch ID: 870acdfd599835eff3cf1cbf522c71f1b47132e1
  • Gator payload: 9
  • Review mode: follow_up
  • Previous reviewed SHA: c0b1bc024efc7a81af270fcfe943a874259f77e2
  • Review budget exhausted: no
  • Maintainer decision required: no
  • Next state: gator:watch-pipeline

@johntmyers johntmyers added gator:watch-pipeline Gator is monitoring PR CI/CD status and removed gator:in-review Gator is reviewing or awaiting PR review feedback labels Sep 10, 2026
@drew
drew added this pull request to the merge queue Sep 11, 2026
@johntmyers johntmyers added gator:merge-ready and removed gator:watch-pipeline Gator is monitoring PR CI/CD status labels Sep 11, 2026
Merged via the queue into main with commit 1860010 Sep 11, 2026
115 of 117 checks passed
@drew
drew deleted the 2802-sdk-pagination-pagers/drew branch September 11, 2026 00:15
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Final status: Gator completed its follow-up review with no blocking findings, the required E2E workflow was dispatched, required checks passed, and maintainer approval was present before merge.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants