Skip to content

feat(source-api): follow the paged product list - #253

Draft
alukach wants to merge 1 commit into
mainfrom
feat/paged-product-list
Draft

alukach wants to merge 1 commit into
mainfrom
feat/paged-product-list

Conversation

@alukach

@alukach alukach commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

source-cooperative/source.coop#590 (#651) pages GET /api/v1/products/{account}: it answers {"items": [...], "next_cursor": ...} where it answered {"products": [...]}, which was also silently cut off at 50. Today SourceProductList requires products, so once that deploys every S3 ListObjects at an account root (s3://{account}/) would fail to parse and answer 500 — while CI stayed green, because the stub serves the old shape.

The account-root listing now asks for ?limit=100 and follows next_cursor until there is none, collecting every product ID. items also accepts products as an alias, and the current API ignores the query and sends no cursor, so this reads one unpaged list from it exactly as before: this can merge and deploy before or after source.coop, which is the point.

Each page is its own URL, so each is cached for PRODUCT_LIST_CACHE_SECS on its own; an account with more than 100 products costs one API call per 100 on a cold listing, where today it silently lists only the first 50. ADR-007's lookup table still holds (same URL, same anonymous caller, same 60s per response), so it isn't amended.

Tests: tests/stub_api.py serves the new shape; fixtures.rs pins the paged, last-page and old unpaged shapes; the nightly test_contract.py asks for ?limit=100 and reads items, falling back to products until source.coop deploys, after which the fallback can go. cargo test, cargo fmt --check and cargo clippy --target wasm32-unknown-unknown -- -D warnings pass locally.

🤖 Generated with Claude Code

source.coop#590 pages GET /api/v1/products/{account} as {"items": [...], "next_cursor": ...}, where it answered {"products": [...]} (silently truncated at 50). The account-root ListObjects now asks for pages of 100 and follows next_cursor to the end, and still reads the old unpaged shape, which ignores the query and carries no cursor, so this can deploy before or after source.coop. The stub and the nightly contract test move to the new shape; the contract test accepts either until source.coop deploys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @alukach's task in 10s —— View job


I'll analyze this and get back to you.


💰 Estimated review cost: $0.10 · 0m09s · 4 turns

@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

🚀 Latest commit deployed to https://source-data-proxy-pr-253.source-coop.workers.dev

  • Date: 2026-10-09T06:47:58Z
  • Commit: 5fafdc6

This branch was successfully deployed

1 active deployment
preview — 59bd4aff Deployed Oct 9, 2026 by alukach via Deploy & Test / Deploy #423
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant