Explainable, verified match intelligence from synthetic football events. Built for the Microsoft × Premier League Inside the Game hackathon.
- Live demo: https://ca-chalkline.greenforest-cf5df294.northcentralus.azurecontainerapps.io (no login; Azure Container Apps, Microsoft Foundry, keyless)
- Demo video (under 2 minutes): https://youtu.be/nsK9wkz4UUw
- Every number on screen comes from a deterministic, tested tool, and every claim Chalkline makes about the game is measured against ground truth the simulator plants and never shows to the metrics or the agents.
All teams, players and matches are synthetic, generated by this repository. No real
match data, footage, club or player names, or league marks are used. (The Python
package and Azure resources keep the project's working name, pitchside.)
Chalkline replays a synthetic match and, as it unfolds, tells you what matters and why, with the proof one click away.
| Stage | What Chalkline does |
|---|---|
| Ingest | A seeded simulator produces a football-realistic event stream (passes, shots, tackles, pressure, possession) with all 22 players' positions at every event. Same seed, same bytes, on any OS. |
| Interpret | Eleven deterministic tools: expected goals and pass completion (fitted models), win probability, expected threat, control vs chaos, pressing shifts, and off-ball credit: which teammate's run made a pass possible, by undoing it and recomputing the space. |
| Explain | An agent desk on Microsoft Agent Framework (analyst → narrator → verifier) explains moments for a casual fan or an analyst, in English or Spanish. The verifier checks every number against the facts each sentence cites; a narrative that fails twice is never shown, only its evidence. |
| Render | A broadcast view: perspective pitch, score bug, win-probability bar, a telestrator trail of the current move, lower thirds aired by a producer agent that only knows what has happened so far, and "the run that made it" replayed with and without the run. A documented, versioned overlay JSON contract. |
| Personalize | Casual fan or analyst, English or Spanish, side by side; follow any player; ask a question; recap so far; and fact-check any commentator's line against what is known at that minute. |
- Open the demo. The badge next to the logo says Foundry live.
- Pick Halvesham Town v Westerlake City and press Next story: "The run that made it" (11'). The pitch replays the pass without Cyril Ulkov's run, then with it; the bars show his control of the pass destination, 52% → 92%. The desk's verified explanation appears beside it (precomputed for every aired story, so it is instant). Tap an underlined sentence to see its facts, the method, caveats and what Chalkline cannot tell.
- Press Back to live, then Next goal (27'). The goal airs as a lower third and the win-probability bar moves: Westerlake 45% → 77%.
- Press play: the telestrator draws the current move, and new stories air as lower thirds for eight seconds, chosen by the producer agent from what is known so far.
- Switch Analyst / ES, or tick Side by side.
- In Fact-check a line, press Try an example: "…85% possession" is flagged, and the data says 52%.
- Open How it's measured for the held-out numbers below.
Seeds 0-99 build and tune; seeds 200-399 are held out. Full report: docs/EVALUATION.md, docs/data/CALIBRATION.md.
| Claim | Held-out result |
|---|---|
| The simulated league is football-realistic | ~1,030 passes, 85% completion, 26 shots, 2.9 goals per match; every target at least 2.5 standard errors inside its bound |
| Expected goals are right | within 0.014 on average of the simulator's true scoring probability (5,193 shots); Brier 0.0943 vs 0.0938 for the truth itself |
| Win probability is calibrated | Brier 0.455 against final results (no information: 0.656) |
| "The run that made it" credits the right runner | first 86% of the time on planted runs (crediting the receiver: 44%; whoever moved most: 54%); AUC 0.94 |
| Pressing changes are real | precision 0.71 against planted pressing spells; 0.68 false alarms per match with nothing planted |
| The verifier agent keeps errors off screen | a single agent with tools airs an unsupported number, wrong attribution or misspelled name 35-40% of the time; with the verifier, 0% (80 held-out moments, live Foundry) |
We also report what did not help: splitting analyst and narrator added no measurable quality over one agent plus the verifier, and the in-match xG rate barely improves win probability over score and time in a league of equal teams.
simulator ──events──> metrics (11 pure tools, as_of_seq: no look-ahead)
│ truth sidecar │ registry ── MCP server
│ (never read by ├──> producer agent ──> timed overlay (deterministic, instant)
│ metrics or agents) └──> agent desk: analyst ─> narrator ─> verifier ─┐
v ^ retry once │ │
evaluation on held-out seeds └──────────────────────────────┘ v
verified insight, or evidence only
FastAPI + static UI on Azure Container Apps (managed identity)
├─ Microsoft Foundry (gpt-4.1-mini, gpt-5-mini fallback), keyless
└─ Azure Blob insight cache, keyless; daily cap on model calls
- Agents: analyst (picks tools), narrator (structured sentences citing evidence ids), verifier (deterministic attribution check), producer (causal air schedule with an attention budget). Failure recovery: fallback model, tool-error correction, default evidence, one retry with the verifier's feedback, then evidence only. ADR 0005.
- Models are logistic regressions and a Poisson model fitted in numpy and shipped as generated code: microseconds, deterministic, auditable, no extra service. ADR 0008.
- Ground truth: pressing spells, off-ball runs and every shot's and pass's true probability are written to a sidecar file; a test fails if metrics or agents read it.
- Methods for every number: docs/METHODS.md. API and the overlay contract: docs/API.md. Deploy: docs/DEPLOY.md. Path to production: docs/PRODUCTION.md.
| Technology | Used for |
|---|---|
| Microsoft Foundry | gpt-4.1-mini and gpt-5-mini deployments behind the analyst and narrator; keyless account (disableLocalAuth) |
| Microsoft Agent Framework | the desk as a workflow of executors with a retry edge; agents with tools |
| Model Context Protocol | the eleven metric tools served over MCP (pitchside.agents.mcp_server) |
| Azure Container Apps | the app (API + UI) with a user-assigned managed identity |
| Azure Container Registry, Azure Blob Storage | image pulls and the insight cache, both with the managed identity |
| Entra ID / Azure RBAC | Foundry User, AcrPull and Storage Blob Data Contributor roles; no keys anywhere |
| Bicep, GitHub Actions | infrastructure as code; tests and the cross-OS reproducibility check on every push |
Requires Python 3.12 and uv.
uv sync
uv run pytest -q # 313 tests, ~1 min
uv run python -m pitchside.api # http://127.0.0.1:8000 (evidence-only mode without a model)With a model (keyless): az login, deploy infra/foundry.bicep, then set
FOUNDRY_PROJECT_ENDPOINT, AI_MODEL_DEPLOYMENT and optionally
AI_MODEL_FALLBACK_DEPLOYMENT in your environment.
uv run python -m pitchside.simulator --seed 42 --out data/match_42.jsonl # one match + truth
uv run python scripts/calibrate.py # realism, detectors, models on held-out seeds
uv run python scripts/fit_models.py # refit xG, xPass and the prior (dev seeds only)
uv run python scripts/evaluate_agents.py --round v2 --n 40 # agent ablation (uses Foundry)
uv run python -m pitchside.agents.mcp_server --match data/match_42.jsonl # tools over MCP- The league is synthetic: models and thresholds describe it, not real football. Teams are equal by construction.
- Positions are per event, without velocities; there are no corners, cards or substitutions in the simulator.
- Fact-checking free text attributes numbers with a fixed English and Spanish word list; unusual phrasings fall to "found" or "unsupported", never to a wrong "supported".
MIT. See NOTICE.md.