feat: Complete Genie Readiness page in Streamlit app

[mick2004]

Summary

The feat/genie-readiness branch contains a new Genie Readiness page in the Streamlit app (streamlit-waf-automation/app.py). The page runs live SQL checks against system tables and shows a readiness tier badge (🟢/🟡/🔴). The core logic is done but the page needs polish work before it can be considered complete.

Branch: feat/genie-readiness

---

What's already done (in the branch)

• ?page=genie_readiness routing and session state handling
• _run_genie_readiness_checks() — single-scan audit SQL + SCIM users API + billing SKU
• _compute_readiness_tier() — 3-tier (green/yellow/red) logic
• Core Requirements section: Unity Catalog, Premium/Enterprise plan, Identity Provisioning, Federated SSO
• Genie-Ready Criteria section: SQL-only users, Group cloning/IdP sync
• Unified Login optional check (AWS only)
• SP audit-access warning banner (when all checks return 0 due to RLS)
• Genie Usage section: MAU, total messages, 7-day trend
• Re-check button with 5-min session cache
• Error expander for failed SQL checks
• 🔮 Genie Readiness link button on the main dashboard page

---

What still needs to be done

1. Genie-specific sidebar

File: streamlit-waf-automation/app.py — the with st.sidebar: block (around line 451)

Currently the sidebar always renders the WAF Guide (pillar controls) regardless of which page is active. When the user is on the Genie Readiness page, this is irrelevant.

Add: Conditional sidebar — when st.session_state.get("waf_page") == "genie_readiness", render Genie-specific guidance instead:

• Tier meanings (🟢/🟡/🔴 explained)
• Table of core requirements + why they matter
• Troubleshooting guidance for ⚠️ signs (SP GRANT instructions)
• Links to relevant Databricks docs (Genie, UC, SCIM, identity federation)

Wrap the existing WAF Guide content in else: so it only shows on non-Genie pages.

2. Next Steps section

File: streamlit-waf-automation/app.py — insert after the Genie Usage section, before the final st.caption(...) (around line 1109)

Add a st.subheader("🚀 Next Steps") block that gives actionable guidance based on tier:

• 🟢 green: "Ready to deploy — contact your Databricks account team to enable Genie, then share the link with your SQL users."
• 🟡 yellow: "Core infra is ready. To reach 🟢: (1) confirm SQL-only user entitlement in your IdP, (2) enable IdP group sync for SQL consumer groups, (3) click Re-check."
• 🔴 red: List the specific ❌ items from core flags so the user knows exactly what to fix (UC, plan upgrade, SSO config). If all show ⚠️, suggest running the SP GRANT and re-checking.

3. WAF score integration (bigger work — separate issue or task)

The Genie Readiness tier (green/yellow/red) is not reflected anywhere in the WAF dashboard score. This is a known gap. Genie readiness should eventually contribute to the overall WAF score (likely under Data & AI Governance or Performance Efficiency pillar). This requires:

• Adding a genie_readiness score row to dashboard_queries.yaml + waf_controls_with_recommendations.csv
• Running waf_reload.py to populate waf_cache
• Updating the dashboard JSON to include it in the pillar score calculation

This is scoped out of the current feature branch and should be tracked separately.

---

Key files

File	Relevance
streamlit-waf-automation/app.py	All Genie Readiness logic (lines 99–1116 on branch)
streamlit-waf-automation/dashboard_queries.yaml	Source of truth for WAF SQL / waf_cache — needed for score integration
streamlit-waf-automation/waf_controls_with_recommendations.csv	Controls list — needed for score integration
DONOTCHECKIN/CLAUDE.md	Project conventions, architecture invariants
DONOTCHECKIN/tasks/state.md	Full project state

---

Architecture invariants to keep in mind

• Genie Readiness checks run live against system tables (NOT waf_cache) — this is intentional, these are operational checks not historical metrics
• All other app queries must go through waf_cache only (never system tables directly)
• wait_timeout="50s" on all SQL statement executions (cold warehouse takes 30–60s)
• No hardcoded workspace-specific values (INSTANCE_URL, DASHBOARD_ID, WORKSPACE_ID all via env vars)