Skip to content

Server-side ad templates design spec#641

Open
jevansnyc wants to merge 7 commits intomainfrom
server-side-ad-templates-spec
Open

Server-side ad templates design spec#641
jevansnyc wants to merge 7 commits intomainfrom
server-side-ad-templates-spec

Conversation

@jevansnyc
Copy link
Copy Markdown
Collaborator

Summary

  • Adds design spec for server-side ad templates, replacing client-side SDK slot discovery (Prebid.js, Smart Slots) with edge-configured creative opportunities
  • TS matches URL patterns to pre-configured slot templates, fires the full auction (PBS + APS + all providers) at request receipt in parallel with origin fetch, and injects winning bids into head before the browser receives HTML
  • Estimated ~2,000ms improvement in time-to-ad-visible (from ~3,200ms to ~1,200ms)

Test plan

  • Review spec for accuracy against current auction pipeline
  • Validate creative-opportunities.toml schema covers publisher slot inventory
  • Confirm open questions with publisher (URL patterns, homepage slot count, PBS stored request setup)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@prk-Jr prk-Jr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On review against the current codebase, 8 design questions need answers before implementation planning can begin. 7 are inline below; one cross-cutting question (win notifications) is here:


❓ Win notifications (nurl/burl): fire server-side or defer?

When GAM renders the creative, TS is out of the render path. The nurl/burl fields from PBS/APS responses are known to TS at auction time but have no path to the client or to a server-side firing mechanism. SSP win counting breaks silently for server-auctioned impressions. Should TS fire nurl as a background HTTP request at bid-selection time, or is this explicitly deferred to a later phase?

2. Immediately fire the full server-side auction (all providers: PBS, APS, future wrappers) in
parallel with the origin HTML fetch — before the browser receives a single byte
3. Inject GPT slot definitions into `<head>` so the client can define slots without any SDK
4. Return pre-collected winning bids to the browser's lightweight `/auction` POST before the
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/auction POST role after this ships

This goal says "return pre-collected winning bids to the browser's lightweight /auction POST." Sections 4.4 and 4.5 say "no /auction POST needed — bids are already in the page." These are contradictory. The JS client currently always POSTs to /auction. Definitive answer needed: does __ts_bids replace the POST entirely, or does /auction remain as a fallback for URLs that don't match any slot template?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. _ts_bids replaces the POST for URLs that match templates. /auction also stays as the fallback path for URLs without matching templates (preserves backward compatability for non-templated pages and for publishers who haven't adopted creative-opportunities.toml yet)

params (placement IDs, account IDs) live in Prebid Server stored requests, keyed by slot ID — not
in this file.

Loaded at build time via `include_str!()`, parsed into `Vec<CreativeOpportunitySlot>` at startup.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

creative-opportunities.toml: compile-time include_str!() or Fastly KV at runtime?

include_str!() bakes the file into the WASM binary — every slot config change requires a full rebuild + Fastly deploy (~15 min). The phrase "ad ops can edit this file independently" (line 71) does not hold under that model. The RuntimeServices abstraction already exposes services.kv_store(). Which model is intended: compile-time (fast reads, deploy required per change) or runtime KV (live edits, no rebuild)?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fastly deploy's don't take 15 minutes first of all. That said, lets reduce dependence on KV store where we can for timing and publisher cost perspective and keep it simple in the WASM binary with include_str!()

with the origin fetch.

The orchestrator's existing behaviour is unchanged:
- All providers (PBS, APS, any configured wrappers) are dispatched simultaneously
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APS in Phase 1: included or excluded, and param source

APS bidder params currently flow from the browser's AdRequest POST — there is no client for a server-triggered auction. creative-opportunities.toml has no [slot.providers.aps] section and APS does not use PBS stored requests. Is APS in scope for Phase 1? If yes, where do per-slot APS params come from?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, APS in scope. Lets add them as [slot.provider.aps] in the toml. They should complete with all the other demand unless you see something i don't?


```json
{
"atf_sidebar_ad": { "hb_pb": "2.50", "hb_bidder": "kargo", "hb_adid": "abc123" },
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hb_pb price bucketing: granularity table and full key set

hb_pb is a Prebid price bucket string (discretized CPM bin), not a raw price. No bucketing logic exists in the codebase — this is a net-new component. Which granularity table: low / medium / high / auto / dense / custom? Is it per-publisher configurable? And what is the complete __ts_bids key set — just hb_pb, hb_bidder, hb_adid, or also hb_size, hb_deal, hb_format?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets make "dense" the default with a publisher override on the granularity setting. Keep the key set as is to match GAM standard.

var bids = window.__ts_bids || {};
googletag.cmd.push(function() {
slots.forEach(function(slot) {
var gptSlot = googletag.defineSlot(slot.id, slot.formats, slot.id)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slot.id as GPT adUnitPath

GPT's defineSlot(adUnitPath, size, optDiv) first argument must be the full GAM network path (e.g., /21765378893/homepage-banner). Using a short key like atf_sidebar_ad silently produces a non-functional slot — GAM will not serve to it. Is slot.id the full GAM path, or does creative-opportunities.toml need a separate field (e.g., gam_unit_path)?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're correct. We should add an optional gam_unit_path field per slot, plus a top-level gam_network_id config. Default behavior: gam_unit_path = "/{network_id}/{slot.id}". Publishers can override per-slot for non-standard paths.

**Origin is slow (NextJS 14, buffered)** — auction has more time; results more likely to be
complete. No change to streaming behavior.

**NextJS 16 streaming** — TS must flush `<head>` before `</head>` tag passes through. If auction
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Streaming mode </head> boundary: Phase 1 or deferred?

The current streaming pipeline (merged in #562) buffers until end-of-document when post-processors run. "Buffer only until </head>, inject bids, then resume streaming immediately" is a new mode not currently implemented — it requires new infrastructure in the HTML processor. Is this required for Phase 1 launch, or is injecting at document end acceptable as an initial release?

corresponding stored requests configured in the publisher's PBS instance before this goes live.
3. **Homepage slot count** — the example shows slots 0 and 1. Are there slots 2–5 following
the same pattern? Slot IDs and count to be confirmed with ad ops.
4. **Auction timeout for server-side trigger** — current `[integrations.prebid].timeout_ms`
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auction timeout: new config key or reuse existing? (Section 9, Q4)

The spec recommends 500ms for server-triggered auctions vs the current 1,000ms client-side budget. There are currently three overlapping timeout values: [auction].timeout_ms, [integrations.prebid].timeout_ms, [integrations.aps].timeout_ms. Does the server-triggered path get a new dedicated key (e.g., [creative_opportunities].auction_timeout_ms) or does it override an existing one?

Copy link
Copy Markdown
Collaborator Author

@jevansnyc jevansnyc Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets use an optional dedicated key with fallback that makes sense:

[creative_opportunities]
"# Optional. Defaults to [auction].timeout_ms if not set.
"# Recommended: 500ms (vs client-side 1000-1500ms) due to lower edge→PBS RTT.
auction_timeout_ms = 500"

  • Optional + fallback keeps it backward compatible. Publishers not setting it inherit [auction].timeout_ms
  • Per-provider config should stay untouched. [integrations.prebid].timeout_ms and [integrations.aps].timeout_ms continue to define provider-level ceilings ... the orchestrator's existing min(remaining_global_budget, provider_timeout) enforcement applies as today.
  • Single source of truth for 4.6 streaming. A_deadline (the buffer deadline) = T₀ + creative_opportunities.auction_timeout_ms (or fallback). The same value gates both the auction and the head-boundary buffer hold ... they're the same deadline.

@jevansnyc
Copy link
Copy Markdown
Collaborator Author

On review against the current codebase, 8 design questions need answers before implementation planning can begin. 7 are inline below; one cross-cutting question (win notifications) is here:

❓ Win notifications (nurl/burl): fire server-side or defer?

When GAM renders the creative, TS is out of the render path. The nurl/burl fields from PBS/APS responses are known to TS at auction time but have no path to the client or to a server-side firing mechanism. SSP win counting breaks silently for server-auctioned impressions. Should TS fire nurl as a background HTTP request at bid-selection time, or is this explicitly deferred to a later phase?

Yes, lets split the calls by where the truth lives:

nurl: TS orchestrator fires fire-and-forget BG HTTP req from the edge at bid-selection time. Edge→SSP RTT ~20–30ms, no auction-path latency cost. Per-provider switch (e.g., [integrations.prebid].fire_nurl_at_edge, default true) for cases where PBS deployment already handles win events and we'd otherwise double-fire. APS handled per its own notification spec.

burl: Fired client-side when the ad actually renders, using the existing GPT slotRenderEnded event. Logic lives inside the tsjs bundle (__tsAdInit registers the listener after defining slots). On render: if !event.isEmpty AND event.slot.getTargeting('hb_adid')[0] === bidData.hb_adid, fire ... this confirms both render and that our specific Prebid bid (not a direct deal or backfill) won the GAM line item match. TS embeds burl per slot in __ts_bids so the client has it without a round-trip.

Copy link
Copy Markdown
Collaborator

@aram356 aram356 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Design spec for server-side ad templates. The diagnosis (browser is the slowest place to run an auction; edge→PBS RTT is much shorter; slot inventory is URL-derivable) is sound, but the spec materially understates the architectural lift and skips several correctness concerns. Cross-referenced against auction/orchestrator.rs, auction/types.rs, auction/endpoints.rs, integrations/registry.rs, integrations/prebid.rs, html_processor.rs, streaming_processor.rs, and publisher.rs.

Blocking

🔧 wrench

  • CI is red on this file: cd docs && npx prettier --check superpowers/specs/2026-04-15-server-side-ad-templates-design.md fails (format-docs job). Fix: cd docs && npm run format:write and commit.
  • Cache contract missing for __ts_bids — see inline.
  • Consent / GDPR flow missing for the new trigger — see inline.
  • IntegrationHeadInjector is sync; spec assumes async wait — see inline.
  • "Auction in parallel with origin" incompatible with current sync handle_publisher_request — see inline.
  • Schema mismatch with AdFormat (missing media_type) — see inline.
  • Glob example is wrong against standard glob libraries — see inline.
  • Untrusted bid strings injected as inline JSON without escape contract (XSS) — see inline.
  • include_str!() contradicts "ad ops can edit independently" — see inline.
  • APS bidder isn't keyed by PBS stored requests — see inline.
  • Spec references the wrong timeout config key — see inline.

❓ question

  • Kill-switch / rollback when config is empty or absent? — see inline.
  • Who fires nurl/burl; what replaces Prebid analytics adapters? — see inline.

Non-blocking

🤔 thinking

  • Latency numbers are modeled, not measured — see inline.
  • Pattern-matching cost is O(slots × patterns) — see inline.
  • No telemetry contract for the new trigger — see inline.

♻️ refactor

  • Validate slot IDs at startup (XSS + cross-reference safety) — see inline.
  • googletag.defineSlot(slot.id, ..., slot.id) is wrong (ad-unit path ≠ div ID) — see inline.
  • __ts_ad_slots belongs in a dedicated head injector, not prebid.rs — see inline.

🌱 seedling

  • Plan for Edge Dictionary / KV-backed config swap — see inline.
  • Sketch the Phase 2 server-side-GAM approach — see inline.

📌 out of scope

  • Browser integration test coverage: the existing browser-integration tests today exercise the client-side ad pipeline. Implementation PRs need parallel coverage — a fixture page with creative-opportunities.toml matched, a stubbed PBS, and asserts that __ts_bids lands and googletag.setTargeting is invoked. Worth flagging now so it lands with the implementation, not after.

⛏ nitpick

  • Italics convention around the date — see inline.

CI Status

  • format-docs: FAIL (prettier delta on this file; 45+/32− to fix)
  • cargo fmt: PASS
  • cargo clippy: PASS
  • cargo test: PASS
  • vitest: PASS
  • browser/integration tests: PASS
  • CodeQL / Analyze (rust, javascript-typescript, actions): PASS

"atf_sidebar_ad": { "hb_pb": "2.50", "hb_bidder": "kargo", "hb_adid": "abc123" },
"below-content-ad": { "hb_pb": "1.00", "hb_bidder": "appnexus", "hb_adid": "def456" }
}
```
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench — Per-request bid injection without a cache contract is unsafe.

__ts_bids is per-request, per-user data injected into a publisher HTML body that flows through Fastly. The spec never states the cache contract. If the response is cacheable (default for HTML on Compute unless explicitly suppressed), bids leak across users.

Fix: Add a section that mandates Cache-Control: private, no-store (or equivalent surrogate-control + cache-key segmentation) on any response with __ts_bids injected, and explain what happens when the publisher origin sends conflicting cache headers. This is a P0 omission — the entire design rests on it.

- Per-provider timeout budgets are enforced from the remaining auction deadline
- Floor price filtering, bid unification, and winning bid selection are applied as today
- PBS resolves bidder params from its stored requests by slot ID — no bidder params travel
through TS or the browser
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench — Consent / GDPR flow is missing for the new auction trigger.

Today's /auction handler in auction/endpoints.rs:66-77 builds a ConsentContext (TCF, GPP, US Privacy, GPC) before running an auction. The spec triggers the auction at request receipt — before the page's consent UI (Didomi) can update preferences — and never describes how consent is propagated, suppressed, or reconciled.

Required additions:

  • Explicit consent-gating decision (when auction is suppressed entirely)
  • Handling of mid-page consent revocation when bids are already in <head>
  • EC-ID / cookie behavior at the new trigger
  • Treatment in §8 Edge Cases as a first-class "no consent" case

Without this the design is a regulatory regression.

auction runs in parallel. Bid injection into `<head>` must complete before the `</head>` tag
is forwarded. If the auction has not returned by the time `</head>` is encountered, TS waits
up to the remaining auction budget, then flushes with whatever bids have arrived (partial
results) or no targeting if timed out. Content after `</head>` is never held.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrenchIntegrationHeadInjector is synchronous; spec assumes it can wait on async results.

The current trait at integrations/registry.rs:392-397 is fn head_inserts(&self, ctx: &IntegrationHtmlContext<'_>) -> Vec<String> and runs inside a lol_html element callback (html_processor.rs:236-267) — fully sync, no .await.

Pick one approach and document it:

  1. Run the auction before building the HtmlProcessorConfig and pass pre-resolved bids in (simpler — but means buffering origin until auction completes), or
  2. Introduce a new async/late-injection mechanism plus a chunk-holding state machine in the streaming pipeline (which today has no "hold and wait" primitive — see html_processor.rs:46-128).

Also note lol_html element!("head", ...) fires on the opening tag; "before </head>" requires el.on_end_tag(). Spec must call this out and pick an approach.


1. Match an incoming page request URL against a set of pre-configured slot templates
2. Immediately fire the full server-side auction (all providers: PBS, APS, future wrappers) in
parallel with the origin HTML fetch — before the browser receives a single byte
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench — "Auction in parallel with origin fetch" is incompatible with the current publisher path.

publisher.rs:527 calls req.send(&backend_name) synchronously inside a non-async handle_publisher_request. Real concurrency requires send_async + a select (or join) between the orchestrator future and the pending origin request, and propagating async up through publisher handling.

Spec §3 lists "no orchestrator changes" and §7 lists this as a one-line "Request handler modification" — the actual lift is a meaningful restructuring of the publisher request path. List it explicitly in §7 as a multi-step migration including the async propagation.

pub struct CreativeOpportunitySlot {
pub id: String,
pub page_patterns: Vec<String>,
pub formats: Vec<AdFormat>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 wrench — Schema mismatch: formats = [{ width, height }] won't deserialize into AdFormat.

auction/types.rs:47-52 defines AdFormat { media_type: MediaType, width: u32, height: u32 }media_type is required and MediaType has no default. The TOML examples and Rust struct in §4.1 omit media_type.

Fix: Either change the schema/struct to include media_type (or default to Banner via #[serde(default)]), or define a separate CreativeOpportunityFormat plus an explicit converter to AdFormat.

var bids = window.__ts_bids || {};
googletag.cmd.push(function() {
slots.forEach(function(slot) {
var gptSlot = googletag.defineSlot(slot.id, slot.formats, slot.id)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactorgoogletag.defineSlot(slot.id, ..., slot.id) is wrong.

defineSlot(adUnitPath, sizes, divId)adUnitPath is the GAM ad-unit path (e.g., /12345/network/article-atf) and divId is the page DOM container ID. Reusing slot.id for both will break GAM trafficking.

Fix: Add separate gam_ad_unit_path and div_id fields to the slot config.

### Modified

- `crates/trusted-server-core/src/integrations/prebid.rs` head injector — emit
`window.__ts_ad_slots` from matched slots
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ refactor__ts_ad_slots belongs in a dedicated head injector, not stuffed into prebid.rs.

The §7 plan modifies integrations/prebid.rs to emit __ts_ad_slots. But the spec also says "Prebid.js is eliminated." Prebid integration shouldn't own the new GPT-facing global.

Fix: Put the head injection in a new creative_opportunities integration (or extend the existing gpt integration at integrations/gpt.rs) and decouple it from prebid.

params (placement IDs, account IDs) live in Prebid Server stored requests, keyed by slot ID — not
in this file.

Loaded at build time via `include_str!()`, parsed into `Vec<CreativeOpportunitySlot>` at startup.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌱 seedling — Plan for Edge Dictionary / KV-backed config swap.

Even if Phase 1 ships with include_str!(), sketch the migration path to dictionary/KV-backed config (read at request time, hot-reload, validation contract). Prevents painting yourselves into a corner when ad ops actually does want to make changes between deploys.

## 3. Non-Goals

- Eliminating client-side GPT / Google Ad Manager — GAM remains in the rendering pipeline
for Phase 1. The GAM call (`securepubads.g.doubleclick.net`) moves server-side in a future phase.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌱 seedling — Server-side GAM is the real win.

Phase 1 keeps GAM in the browser, capping the savings ceiling. Briefly outline the Phase 2 server-side-GAM approach (securepubads proxy, creative bundling) so reviewers can evaluate whether the Phase 1 architecture is shape-compatible with the eventual end state.

@@ -0,0 +1,363 @@
# Server-Side Ad Templates Design

*April 2026*
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick — Italics convention around the date.

Other specs in docs/superpowers/specs/ use a header like _Author · YYYY-MM-DD_ or omit the date entirely. Pick a convention so spec metadata is grep-able.

prk-Jr and others added 5 commits April 30, 2026 14:43
- Incorporate all review feedback (aram356 + jevansnyc): cache contract,
  consent/GDPR gating, async restructuring detail, CreativeOpportunityFormat
  schema, glob pattern fix, XSS escaping, win notifications, APS params,
  timeout config key, defineSlot fix, gpt.rs ownership, KV migration path,
  Phase 2 sketch
- Fix Prettier formatting (format-docs CI)
- Add implementation plan (12 tasks, TDD, ordered by dependency)
- Incorporate all review feedback (aram356 + jevansnyc): cache contract,
  consent/GDPR gating, async restructuring detail, CreativeOpportunityFormat
  schema, glob pattern fix, XSS escaping, win notifications, APS params,
  timeout config key, defineSlot fix, gpt.rs ownership, KV migration path,
  Phase 2 sketch
- Fix Prettier formatting (format-docs CI)
- Add implementation plan (12 tasks, TDD, ordered by dependency)
Replace the head-injected __ts_bids design with a server-cached bid
delivery model fetched by the client via a new /ts-bids endpoint. The
auction never blocks page rendering — </head> flushes immediately, body
parses without waiting for bids, and the client fetches bids in parallel
with content paint.

Key changes:
- §2 Goal: bid delivery decoupled from page rendering; FCP unchanged from
  no-TS baseline
- §4.3 Auction Trigger: drop buffered/streaming dichotomy; single mode
  forces chunked encoding on all origins (WordPress, NextJS, etc.)
- §4.4 Head Injection: only __ts_ad_slots and __ts_request_id injected at
  <head> open; bid results moved to /ts-bids endpoint
- §4.6 Client Residual: __tsAdInit defines slots immediately, fetches
  bids via /ts-bids, applies targeting and fires refresh() after resolve
- §4.7 (new) Caching Behavior: explicit cacheability table for HTML, JS,
  CSS, tsjs bundle, bid results; Fastly edge HTTP cache leveraged for
  origin HTML
- §5 Request-Time Sequence: full mermaid diagram covering content +
  creative + burl flow with cache-hit and cache-miss branches; separate
  text sequences for cache-hit (~80ms FCP, ~900ms ad-visible) and
  cache-miss (~250ms FCP, ~1,050ms ad-visible)
- §6 Performance Summary: cache-hit and cache-miss columns; FCP added
  as a tracked metric
- §7 Implementation Scope: add bid_cache.rs, /ts-bids endpoint, force
  chunked encoding step
- §8 Edge Cases: origin-agnostic entries; new entries for /ts-bids 404
  and client-never-fetches-/ts-bids

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create Spec for Server-Side Ad-Templates

3 participants