Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1 +1,12 @@
OPENAI_API_KEY=
# Provider: "groq" (default) or "openai"
LLM_PROVIDER=groq

# Groq — https://console.groq.com/keys (free tier available)
GROQ_API_KEY=your_groq_api_key_here

# OpenAI — only needed if LLM_PROVIDER=openai
# OPENAI_API_KEY=your_openai_api_key_here

# Audio source default: mic | system | both
# Use "system" to capture Teams/Zoom interviewer audio (requires Stereo Mix or VB-Cable)
AUDIO_SOURCE=system
49 changes: 41 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,16 @@ Hack Interview application is a tool designed to assist in job interviews using
## Features

- **Real-Time Audio Processing**: Records and transcribes audio seamlessly.
- **Voice Recognition**: Uses OpenAI's Whisper model for accurate voice recognition.
- **Intelligent Response Generation**: Leverages OpenAI's GPT models for generating concise and relevant answers.
- **Voice Recognition**: Uses Groq Whisper (or OpenAI Whisper) for accurate voice recognition.
- **Intelligent Response Generation**: Uses Groq Llama models (or OpenAI GPT) for concise and relevant answers.
- **Cross-Platform Functionality**: Designed to work on various operating systems.
- **User-Friendly Interface**: Simple, intuitive and hideous GUI for easy interaction.

## Requirements

- **Python 3.10+**: Ensure Python is installed on your system.
- **OpenAI API Key**: To use OpenAI's GPT models, you will need an API key.
- **Groq API Key** (recommended): Free tier at [console.groq.com](https://console.groq.com/keys). Used for Whisper transcription and Llama answers by default.
- **OpenAI API Key** (optional): Only if you set `LLM_PROVIDER=openai` in `.env`.
- **BlackHole for MacOS**: An essential tool for recording your computer's audio output (e.g. from Zoom calls or browser tabs). Microphone Fallback: If BlackHole isn't installed or properly configured, the application can still function by recording your microphone input.

## Installation
Expand All @@ -40,15 +41,47 @@ Hack Interview application is a tool designed to assist in job interviews using
3. **BlackHole**: If using MacOS, install [BlackHole](https://github.com/ExistentialAudio/BlackHole) and set up a [Multi Output Device](https://github.com/ExistentialAudio/BlackHole/wiki/Multi-Output-Device)

4. **Environment Setup**:
- Add your OpenAI API key to the `.env` file. If you don't have one, you can get it [here](https://platform.openai.com/api-keys).
- Copy `.env.example` to `.env`.
- Add your Groq API key (`GROQ_API_KEY`) from [console.groq.com](https://console.groq.com/keys).
- Default provider is Groq (`LLM_PROVIDER=groq`). For OpenAI instead, set `LLM_PROVIDER=openai` and add `OPENAI_API_KEY`.

## Usage

- **Starting the Application**: Run `python main.py` to launch the GUI.
- *(optional)* **Setup**: You can choose the OpenAI model to use for response generation and the position you are being interviewed for. The default settings are set in the `src/config.py` file.
- **Recording**: Press `R` or click the big red toggle button to start/stop audio recording. It will create a `recording.wav` file in the project directory.
- **Transcription and Response Generation**: Press `A` or click the 'Analyze' button to transcribe the recorded audio and generate answers.
- **Viewing Responses**: Responses are displayed in the GUI, offering both a quick and detailed answer.
- **Audio source**: In the GUI, set **Audio** to **System (Teams/Zoom)** for live calls (default). Use **Microphone** only when testing alone.
- **Recording**: Press `R` to start, speak or let the interviewer ask a question, press `R` again to stop. Saves `record.wav` in the project folder.
- **Transcription**: Press `A` to transcribe and generate short/full answers.

## Capture Microsoft Teams / Zoom audio (Windows)

Your **microphone only hears you**, not the interviewer. Teams plays their voice through **speakers/headphones** (system audio). Use one of these setups:

### Option A — Stereo Mix (free, if your PC supports it)

1. Right-click the **speaker icon** → **Sound settings** → **More sound settings**.
2. Open the **Recording** tab → right-click empty area → check **Show disabled devices**.
3. Enable **Stereo Mix** → Set as **Default Device** (or note its name).
4. In the app, set **Audio** to **System (Teams/Zoom)**.
5. In Teams, use your normal **speakers or headset** for call audio.

### Option B — VB-Audio Virtual Cable (works on most PCs)

1. Install [VB-Audio Virtual Cable](https://vb-audio.com/Cable/) (free).
2. In **Teams** → Settings → **Devices** → set **Speaker** to **CABLE Input**.
3. In **Windows Sound** → **Recording**, enable **CABLE Output** as default (or the app will detect it automatically).
4. Listen on your headset via Teams **Test call** or use **Listen to this device** on CABLE Output if you need to hear the call.

### During the interview

1. Set **Audio** → **System (Teams/Zoom)**.
2. When the interviewer asks a question, press **R** (record) → wait for the question → press **R** (stop).
3. Press **A** to transcribe and get answers.

Use **Both** if you want system audio plus your mic in one recording.

### macOS

Install [BlackHole](https://github.com/ExistentialAudio/BlackHole), create a **Multi-Output Device** that includes BlackHole + your headphones, and set Teams output to that device. Set **Audio** to **System (Teams/Zoom)**.

## Contributions

Expand Down
268 changes: 268 additions & 0 deletions audio.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,268 @@
import sys
import threading
from typing import Any, Dict, List, Optional, Tuple

import numpy as np
import PySimpleGUI as sg
import sounddevice as sd
import soundfile as sf
from loguru import logger

from src.config import OUTPUT_FILE_NAME, SAMPLE_RATE

_PREFERRED_HOSTAPIS = ("MME", "Windows DirectSound", "Windows WASAPI")

# Devices that capture speaker / call audio (not a physical mic).
_SYSTEM_DEVICE_KEYWORDS = (
"blackhole",
"stereo mix",
"stereomix",
"what u hear",
"wave out",
"loopback",
"cable output",
"vb-audio",
"vb audio",
"virtual cable",
"soundflower",
)


def _hostapi_name(device: Dict[str, Any]) -> str:
return sd.query_hostapis()[device["hostapi"]]["name"]


def _is_system_device(name: str) -> bool:
lower = name.lower()
return any(keyword in lower for keyword in _SYSTEM_DEVICE_KEYWORDS)


def _device_priority(device_id: int) -> int:
"""Lower is better. Prefer stable host APIs on Windows."""
api = _hostapi_name(sd.query_devices(device_id))
if sys.platform == "win32":
order = {name: i for i, name in enumerate(_PREFERRED_HOSTAPIS)}
return order.get(api, len(_PREFERRED_HOSTAPIS))
return 0


def find_system_device_id() -> Optional[int]:
"""
Find a device that records system/call audio (Teams, Zoom, browser).

Windows: enable Stereo Mix, or install VB-Audio Cable (see README).
macOS: install BlackHole and route call audio to it.
"""
candidates: List[Tuple[int, int]] = []
for device_id, device in enumerate(sd.query_devices()):
if device["max_input_channels"] < 1:
continue
if _is_system_device(device["name"]):
candidates.append((device_id, _device_priority(device_id)))

if not candidates:
return None

device_id = min(candidates, key=lambda item: item[1])[0]
device = sd.query_devices(device_id)
logger.debug(
f"Using system audio device: {device['name']} ({_hostapi_name(device)})"
)
return device_id


def find_microphone_device_id() -> Optional[int]:
"""Find a normal microphone, excluding virtual system-capture devices."""
system_id = find_system_device_id()
candidates: List[Tuple[int, int]] = []

for device_id, device in enumerate(sd.query_devices()):
if device["max_input_channels"] < 1:
continue
if device_id == system_id:
continue
if _is_system_device(device["name"]):
continue
candidates.append((device_id, _device_priority(device_id)))

if candidates:
device_id = min(candidates, key=lambda item: item[1])[0]
device = sd.query_devices(device_id)
logger.debug(
f"Using microphone: {device['name']} ({_hostapi_name(device)})"
)
return device_id

default_input = sd.default.device[0]
if default_input is not None and default_input >= 0:
device = sd.query_devices(default_input)
if not _is_system_device(device["name"]):
logger.debug(f"Using default input device: {device['name']}")
return default_input

return None


def resolve_device_ids(audio_source: str) -> List[int]:
"""Return device id(s) to record for the given source mode."""
source = audio_source.lower()
mic_id = find_microphone_device_id()
system_id = find_system_device_id()

if source == "mic":
return [mic_id] if mic_id is not None else []
if source == "system":
return [system_id] if system_id is not None else []
if source == "both":
ids = []
if system_id is not None:
ids.append(system_id)
if mic_id is not None and mic_id not in ids:
ids.append(mic_id)
return ids

logger.warning(f"Unknown audio source '{audio_source}', using microphone.")
return [mic_id] if mic_id is not None else []


def _device_sample_rate(device_id: int) -> int:
info = sd.query_devices(device_id, "input")
rate = int(info["default_samplerate"])
return rate if rate > 0 else SAMPLE_RATE


def _to_mono(audio_data: np.ndarray) -> np.ndarray:
if audio_data.ndim == 1:
return audio_data
return audio_data.mean(axis=1)


def _mix_tracks(tracks: List[np.ndarray]) -> np.ndarray:
if len(tracks) == 1:
return tracks[0]
min_len = min(track.shape[0] for track in tracks)
mono_tracks = [_to_mono(track[:min_len]) for track in tracks]
mixed = np.sum(mono_tracks, axis=0)
peak = np.max(np.abs(mixed))
if peak > 1.0:
mixed = mixed / peak
return mixed


def _record_device(
device_id: int,
button: sg.Element,
frames: List[np.ndarray],
lock: threading.Lock,
) -> None:
samplerate = _device_sample_rate(device_id)
device_info = sd.query_devices(device_id, "input")
channels = min(int(device_info["max_input_channels"]), 2)

def callback(
indata: np.ndarray,
frame_count: int,
time_info: Any,
status: sd.CallbackFlags,
) -> None:
if status:
logger.warning(f"Audio stream status: {status}")
if button.metadata.state:
with lock:
frames.append(indata.copy())

with sd.InputStream(
samplerate=samplerate,
device=device_id,
channels=channels,
dtype="float32",
callback=callback,
blocksize=int(samplerate * 0.1),
):
while button.metadata.state:
sd.sleep(100)


def record(button: sg.Element, audio_source: str = "system") -> None:
"""
Record audio while the record button is active.

Args:
button: The record toggle button.
audio_source: "mic", "system", or "both".
"""
logger.debug(f"Recording (source={audio_source})...")
device_ids = resolve_device_ids(audio_source)

if not device_ids:
if audio_source.lower() in ("system", "both"):
logger.error(
"No system audio device found. Enable Stereo Mix in Windows Sound "
"settings, or install VB-Audio Virtual Cable (see README)."
)
else:
logger.error("No microphone found.")
return

if audio_source.lower() in ("system", "both") and find_system_device_id() is None:
logger.warning(
"System audio device not found; only microphone will be used. "
"See README to capture Teams/Zoom audio."
)

all_frames: List[List[np.ndarray]] = [[] for _ in device_ids]
locks = [threading.Lock() for _ in device_ids]
errors: List[str] = []

def worker(index: int, device_id: int) -> None:
try:
_record_device(device_id, button, all_frames[index], locks[index])
except Exception as e:
errors.append(str(e))
logger.error(f"Recording error on device {device_id}: {e}")

threads = [
threading.Thread(target=worker, args=(i, device_id), daemon=True)
for i, device_id in enumerate(device_ids)
]
for thread in threads:
thread.start()
for thread in threads:
thread.join()

if errors and not any(all_frames):
return

tracks = []
for frames in all_frames:
if frames:
tracks.append(np.concatenate(frames, axis=0))

if not tracks:
logger.warning("No audio recorded.")
return

audio_data = _mix_tracks(tracks)
samplerate = _device_sample_rate(device_ids[0])
save_audio_file(audio_data, samplerate=samplerate)


def save_audio_file(
audio_data: np.ndarray,
output_file_name: str = OUTPUT_FILE_NAME,
samplerate: int = SAMPLE_RATE,
) -> None:
"""Save audio data to a WAV file."""
if audio_data.ndim == 1:
data = audio_data
else:
data = audio_data

sf.write(
file=output_file_name,
data=data,
samplerate=samplerate,
format="WAV",
subtype="PCM_16",
)
logger.debug(f"Audio saved to: {output_file_name}...")
Loading