Add opt-in cursor pagination for Maven package lists

[kdelee]

Related issue

Part of the LWLP-3265 pagination spike.

Summary

Adds opt-in cursor pagination for Maven repository catalog endpoints. Existing limit/offset pagination remains the default and unchanged.

Supported endpoints:

/repositories/maven/maven/<repository>/packages/
/repositories/maven/maven/<repository>/packages/flat/
/repositories/maven/maven/<repository>/versions/

Enable with:

?pagination=cursor

Cursor mode uses fixed server-side ordering, opaque next/previous links, no count, and rejects client-provided ordering with HTTP 400. Existing filters, repository-version selection, RBAC, and bounded page-size handling are preserved.

Repository-version cursor pagination is a Maven-only pilot. If validated, it may move to pulpcore so all plugins share one cursor contract.

Traffic analysis

Sanitized ingress analysis covered a 14-day window.

/packages/

38,667 total requests
38,653 with limit
38,647 with offset
38,603 with ordering

maximum offset: 5,700
p90 offset: 4,200
p99 offset: 5,500

/packages/flat/

30 total requests
30 with limit
27 with offset
0 with ordering

maximum offset: 27,000
p90 offset: 25,000
p99 offset: 27,000

This is a low-volume control endpoint, but its observed offsets are deeper than
those on /packages/.

/versions/

2,194 total requests
2,109 with limit
1,233 with offset
429 with ordering

maximum offset: 1,200,000
p90 offset: 76,525
p99 offset: 85,419

GABI query-plan comparison

All plans were sanitized and run with:

EXPLAIN (ANALYZE, BUFFERS, TIMING OFF)

Package rows (/packages/flat/ query shape)

Using a large completed repository-version fixture with approximately 37,810 MavenPackage rows:

COUNT(*):       6,518 ms cold; 164 ms warm
OFFSET 30,000:    129 ms warm
CURSOR seek:       47 ms warm

The cursor plan avoided COUNT(*) and OFFSET, but repository-version membership resolution remained the dominant cost. A membership-first alternative was tested at approximately 3,198 ms and was not adopted.

Repository versions (/versions/)

Using the largest completed repository-version fixture found, with approximately 24,646 complete versions:

COUNT(*):       34.2 ms
OFFSET 20,000:  20.4 ms
CURSOR seek:     2.2 ms

The offset plan scanned and sorted matching versions before skipping 20,000 rows. The cursor plan used the existing repository-id/version-number composite index.

Grouped package rows (/packages/)

A separate grouped-query GABI run returned approximately 5,707 distinct group/artifact rows from 37,810 package rows:

COUNT(DISTINCT group_id, artifact_id): 8,096 ms cold
OFFSET 30,000:                         288 ms warm; 0 rows at this fixture size
CURSOR seek:                           271 ms warm

The grouped cursor plan still grouped all membership rows and removed 4,441 groups after aggregation. Cursor avoids OFFSET, but aggregation and repository-version membership work remain dominant; this is not a constant-time grouped-page claim.

Testing

ruff check              passed
ruff format --check     passed
git diff --check        passed
OpenAPI regenerated     passed
Maven catalog suite     15 passed

The functional coverage includes default compatibility, cursor first/next/previous navigation for package lists, repository-version cursor navigation, omission of count, page-size handling, and rejection of client ordering.

Clients were generated with:

oci-env agent generate-client control-main

Compatibility

This is opt-in only. Existing limit/offset behavior and response contracts remain unchanged. No database migrations are included.

Example usage

First request:

  GET .../packages/flat/?pagination=cursor&page_size=100

Response includes:

  {
    "next": ".../packages/flat/?pagination=cursor&page_size=100&cursor=<opaque>",
    "previous": null,
    "results": [...]
  }

Follow next verbatim. It includes both:

  pagination=cursor
  cursor=<opaque value>