feat(entities): new entity APIs — cursor pages, count, aggregate, upsert

[arosenan]

Why

Fifty apps with the deepest skip reads on the entities cluster were reviewed. None of them pages because a user asked for page N. Every deep-skip loop exists to get something the SDK cannot give:

Need	Apps of 50	New API
A number: count, sum, group-by, avg, max-per-key, count-distinct	~37	aggregate(), count()
"Does this key already exist?" before insert; duplicate cleanup	~23	upsert(), aggregate({ having })
Walk a big table in order: export, sitemap, resumable job	~15	cursor on list() / filter()
Distinct values for dropdowns / autocomplete	5	distinct option on list() / filter()

Three apps built binary search with limit-1 probes as a substitute for count (skips of 10.2M, 900k and 64k). Builders persist lastSkip as a resume token because nothing else exists.

The server routes land in base44-dev/apper#24939. This PR is the SDK half, and its purpose right now is to settle the API shape. Version stays 0.8.x: everything is additive, so existing apps on ^0.8.x pick it up on their next install.

Proposed API

// Cursor pages: same list()/filter(), an options object instead of positional args.
// The token carries the query, sort and fields, so later pages need only the cursor.
let page = await base44.entities.Order.filter(
  { status: 'open' },
  { sort: '-created_date', limit: 1000 }
);                                        // -> { items, next_cursor, has_more }
while (page.has_more) {
  page = await base44.entities.Order.filter({ status: 'open' }, { cursor: page.next_cursor, limit: 1000 });
}

await base44.entities.Order.list('-created_date', 100);   // unchanged, -> Order[]

// A number, not rows.
const open = await base44.entities.Order.count({ status: 'open' });

// Distinct values of a field, for dropdowns and autocomplete: a page of values, not records.
const { items: brands } = await base44.entities.Product.filter({ active: true }, { distinct: 'brand' });
// items: ['Adidas', 'Nike', ...]  (ascending, cursor-paged like any other page)

// Group-by aggregates, computed server-side.
const { rows } = await base44.entities.Sale.aggregate({
  query: { sale_date: { $gte: '2026-09-01' } },
  groupBy: 'agent_id',
  sum: 'amount',
  sort: '-sum_amount',
});
// rows: [{ agent_id, count, sum_amount }, ...]; also dateBucket, avg/min/max, countDistinct, having, limit

// Create or update by a key of your own, up to 500 records per call.
const { created, updated, records } = await base44.entities.Contact.upsert(rows, { key: 'crm_id' });

Design notes:

• The cursor token carries the query, sort and fields of the walk, like Wix Data's cursorPaging: a later page needs only cursor and limit, and a different query or sort with a cursor is a 400 rather than silently ignored. The server re-sanitizes the query and applies scope and RLS on every page, so the token adds no new trust.
• distinct is an option on list()/filter(), like distinct() on a Wix Data or Mongo query, rather than a group-by spelled through aggregate(). It returns the same page envelope with values as items.
• The cursor lives on list()/filter() rather than a new method, matching Stripe/Firestore/Prisma. Passing an options object is what switches the return type to a page envelope; positional calls keep returning arrays, so no existing code changes. skip stays and is documented as deprecated for loops. The options form defaults to 100 rows per page (Notion-style), the positional form keeps its 5,000 default.
• Vocabulary follows the SDK, not Mongo: the aggregate filter is query like filter(query), spec keys are camelCase like the SDK's methods; server-produced fields stay snake_case (next_cursor, has_more, sum_amount) like created_date.
• updateMany is unchanged here. Making it update every match, instead of batches the caller loops over, is a server-side change tracked in the backend PR.

What

• list(options) / filter(query, options) → GET /{entity}/v2/list → EntityPage<T>; EntityListOptions<T, K> typed against the schema. With EntityDistinctOptions<T, K> ({ distinct, limit, cursor }) the same route returns EntityPage<T[K]>. Same route family as list, versioned like /conversations/v2/… in the agents module; the backend route takes this path once the API is closed.
• count(query?) → GET /{entity}/count?q= → number.
• aggregate(spec) → POST /{entity}/aggregate → EntityAggregateResult (server caps at 1000 rows); EntityAggregateSpec<T> typed against the schema. Spec keys are query, groupBy, dateBucket, count, sum, avg, min, max, countDistinct, having, sort, limit; the spec is the wire body.
• upsert(records, { key }) → POST /{entity}/upsert → EntityUpsertResult<T>.

New types are exported from the package entry and listed in types-to-expose.json for the docs pipeline. JSDoc follows the sdk-docs-writing skill: description, params, returns, examples on every method.

Tests

• tests/unit/entities-primitives.test.ts: nock tests for each route, params and body; positional list/filter still hit the list route and return arrays.
• tests/types/entities-primitives.types.ts: spec and option fields are tied to the entity type; unknown fields, units, sorts and keys fail to compile.
• npm run build, npm run test:types, npm run test:unit (308 tests) and npm run lint pass.

No version bump, following the repo's separate chore: bump version commits.

🤖 Generated with Claude Code

[netanelgilad]

would this use countDocumentes or estimatedDocumentsCount?
should we expose both options? will estimatedDocumentsCount even work for us?

[netanelgilad]

why not expose it in the mongo way of 1 or 0? calling it projection to be more mongo compatible?

[arosenan]

Superseded by #287, the same branch pushed to this repo so the preview-publish workflow can publish a testable package (OIDC trusted publishing is not available to PRs from forks).