ShatterGlass: The Brutally Honest AI Pitch Coach

Submission for the #GeminiLiveAgentChallenge

🚀 Test the Live Demo

https://shatterglass-699882871925.us-central1.run.app

🛠️ Spin-Up Instructions (For Judges)

Prerequisites:

• A Google Cloud Project
• Vertex AI enabled
• Paste your Project ID and Region in .env

---

☁️ Create & Configure a Google Cloud Project

1. Create a Project

1. Go to https://console.cloud.google.com
2. Click Project Selector → New Project
3. Enter a Project Name
4. Click Create

---

2. Open Cloud Shell

1. Click the Cloud Shell icon (☁️ >_) in the top-right.
2. Wait for the terminal to initialize.

---

3. Set Your Project

Run in Cloud Shell:

# set your project ID, replace YOUR_PROJECT_ID with the ID from step 1
gcloud config set project YOUR_PROJECT_ID --quiet

---

4. Enable Required Services

Run:

gcloud services enable compute.googleapis.com \
                       artifactregistry.googleapis.com \
                       run.googleapis.com \
                       cloudbuild.googleapis.com \
                       iam.googleapis.com \
                       aiplatform.googleapis.com

💻 Run It Locally

# 1. Clone the repository
git clone https://gitlab.com/hackathons1782711/shatterglass.git
cd shatterglass

# 2. Start the Backend / WebSocket Server
cd backend
npm install

# Set your env first with a Vertex AI enabled project id and region
cp .env.example .env

# Then run the dev server
npm run dev

# 3. Start the frontend
cd ../frontend
npm install
npm run dev

Open in browser:

http://localhost:3000

⚠️ Use Chrome or Edge. Grant Microphone and Camera permissions when prompted.

---

📂 Documentation

• Architecture Details: This diagram illustrates the complex bidirectional data flows between the frontend, backend, Gemini Live API, Google Search Grounding, and the post-session report generation.
• Instructions for CI/CD setup on Gitlab: see GITLAB_GCP_DEPLOYMENT.md for a step-by-step guide to setting up secure, keyless deployments to Google Cloud using Workload Identity Federation.
• Features Deep Dive: see FEATURES.md for a detailed breakdown of the technical features.
• More Documentation in docs/ folder: see DOCS

☁️ Proof of Google Cloud Deployment

For the judges looking to verify our GCP backend:

• The Service: This application is live-deployed on Google Cloud Run.

• The Code Shortcut: You can verify where the Gemin API gets called via the GenAI SDK directly in our repository here: See backend/src/gemini-session.js

• Our GitLab CI/CD pipeline in .gitlab-ci.yml actively authenticates to Google Cloud via stateless OIDC Identity Tokens to deploy Docker containers exclusively to GCP.

  The "feature creep" stops here. We set out to build a Next-Gen Live Agent, not a traditional chatbot. ShatterGlass is a multimodal communication coach that actually interrupts you mid-sentence if your pitches fall flat, your body language is completely off, or you confidently invent fake numbers.

No "turn-taking" buttons. Just a hot microphone and a brutally honest AI utilizing native WebRTC-speed WebSockets.

✅ Minimum Criteria Met

• ✔️ Leverage a Gemini model: Powered by the Gemini Live API (Gemini 2.0 Flash Audio Native) for real-time bidirectional audio streaming.
• ✔️ Framework Used: Built with the official Google GenAI SDK for Node.js (@google/genai).
• ✔️ Google Cloud Deployment: Hosted seamlessly on Google Cloud Run using Workload Identity Federation and Artifact Registry for automated CI/CD.

🪟 The Multimodal "Wow" Factor (Live Agent)

We audited our application straight against the Live Agents and Multimodal rubrics:

1. The "Barge-In" Interruption: We specifically engineered the system prompt and eliminated client-side audio noise gates so the AI can natively cancel your audio playback and interrupt your speech if you use too many "ums" or drag on your value proposition.
2. Visual Rate-Limiting: The app streams your live webcam as base64 JPEG canvas frames at 0.5 fps (every 2000ms). The AI sees your eye contact and posture in real-time without suffocating WebSocket bandwidth.
3. Hybrid Fact-Checking: If you state a massive numerical lie (e.g., "The software market is $900 Trillion"), Gemini leverages Google Search Grounding to instantly find the correct metric and interrupts you to correct you live.

---