Skip to content

feat: serve PostQuery - #25

Open
mattdjenkinson wants to merge 1 commit into
feat/post-release-changelogfrom
feat/postquery
Open

mattdjenkinson wants to merge 1 commit into
feat/post-release-changelogfrom
feat/postquery

Conversation

@mattdjenkinson

Copy link
Copy Markdown
Contributor

Serves PostQuery: search, filter, sort and facet posts in one create call. Nothing is stored. Closes #5.

Stacked on #23 (which is on #22 and #16). Review those first; GitHub retargets this to main as they merge.

What it does

  • Filters: state, categories, orgs (case-insensitive) and deliveryStage. votedByMe and followedByMe work in the user context only; elsewhere the query is rejected as Invalid.
  • Sorts: top, newest, trending, recentlyChanged, and relevance (the default when text or similarTo is set).
  • Facets count over every match, not just the current page.
  • Paging: limit (default 20, max 100) and continue. A continue token only works with the query that produced it.
  • Hidden posts never appear, in any context.
  • text: sends a ResourceSearchQuery for Post to Milo's search service (--search-kubeconfig), applies filters in Postgres, and keeps search's ranking. If search fails the query returns 503; browsing and similarTo keep working, as in the design. Without the flag, text uses Postgres keyword matching: a new weighted tsvector column with title over summary over body, matching any of the words.
  • similarTo: embeds the text through an OpenAI-compatible gateway (--ai-gateway-url, --embedding-model, --ai-gateway-token-file) and ranks the 100 nearest posts by pgvector cosine similarity. It falls back to keyword matching, with a Warning header, when the gateway fails or nothing is embedded yet. Each user is rate limited (--similar-to-rate 30/min, --similar-to-burst 10, per replica), returning 429 with Retry-After.
  • Embeddings: a post-start hook embeds new and changed posts every --embed-interval. Staleness is keyed on model plus an md5 of the text, so any change to title, summary or body is picked up, and changing the model re-embeds everything. A Postgres advisory lock keeps it to one replica at a time.

Not in this PR

  • Embedding model: still unresolved in the design. The gateway serves no embedding model today, so with no --ai-gateway-url the server runs with similarTo on keyword matching. Choosing the model and dimension is the open question on PostQuery #5.
  • Search index: the ResourceIndexPolicy for posts is Search integration #12. Until it exists the search service reports posts as denied, which this treats as unavailable.
  • Milo IAM and discovery for postqueries go with the other resources in Identity, contexts and authorization #10.
  • Deployment flags (gateway URL, token projection, search kubeconfig) go with Deploy to staging #13.

Testing

  • Postgres integration tests (testcontainers):
    • sorts, paging and token reuse;
    • every filter and facet;
    • votedByMe and followedByMe across users and contexts;
    • keyword text, including body text and stop words only;
    • search-service ordering, with filters and hidden or unknown posts dropped, and search down returning 503;
    • similarTo with a fake embedder: ranking, fallback before anything is embedded, fallback when the gateway is down, re-embedding on a summary change and on a model change;
    • rate limiting.
  • Unit tests: search-service client (paging, max, denied index, error status), gateway client (index order, token re-read), HTML stripping and truncation, limiter.
  • Kind cluster with the real image and a fake OpenAI-compatible embeddings server:
    • the embedder embedded all 5 seeded posts;
    • user and staff queries returned the expected order and facets;
    • votedByMe worked from the user context and was refused from the platform context;
    • similarTo ranked the ALB and billing posts first;
    • the third call in a burst got 429 with Retry-After: 10s;
    • scaling the embedder to zero fell back to keywords with the warning.

PostQuery searches, filters, sorts and facets posts without storing
anything.

- Filters on state, categories, orgs and delivery stage, plus votedByMe
  and followedByMe in the user context. Sorts top, newest, trending,
  recentlyChanged and relevance. Facets count over every match. Offset
  continue tokens are tied to the query that made them.
- text runs a ResourceSearchQuery against the posts index when
  --search-kubeconfig is set, and Postgres keyword matching otherwise.
- similarTo embeds the text through an OpenAI-compatible AI gateway and
  ranks posts by pgvector cosine distance, falling back to keyword
  matching with a Warning when embeddings aren't available. It is rate
  limited per user.
- The apiserver embeds new and changed posts in the background, one
  replica at a time, keyed by model and a hash of the text.
- Migration 00004 adds a weighted tsvector column on posts and a GIN
  index on categories.

Part of datum-cloud/enhancements#943. Closes #5.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant