Senior Data Engineer — Technical Session

What this session is

This is a live, 2-hour agentic coding session. You will pick one of two problems, then work on it with an AI agent while you share your screen.

We are evaluating: How you approach a problem, define a plan, ensure quality, and build toward a solution.

What we are NOT evaluating: Your ability to finish or have a perfect solution.

You are free to:

• Ask questions when something is ambiguous
• Make assumptions
• Use the AI agent throughout the session to help you explore and implement your plan

Schedule

When	Time	What happens
Before the day	5 min	Set up your environment (below). If the check does not print READY, tell your recruiter.
0:00	10 min	Intros. Run git pull and the setup check, set the agent key that we send you, and start the agent.
0:10	5 min	Read the two problems that your interviewer sends, and pick one.
0:15	75 min	Work on the problem, and think out loud. You can take a 5-minute break at any time.
1:30	15 min	Walk us through your plan and what you built.
1:45	15 min	Two questions from us about how you work. Then your questions for us, about us and the team.

The scenario

You have just joined the data engineering team at Waypoint. They sell a directory of every physical business location in the US: coffee shops, gas stations, dentists, and gyms. Delivery apps, mapping companies, and retail analysts pay for it. Other engineering teams at Waypoint build paid features on the data that your team ships. Waypoint does not collect this data itself. It buys data feeds from three providers:

• atlas
• beacon
• cartograph.

The problems

At the start of the session, your interviewer sends you the two problems in the chat, as a file. Save it as PROBLEMS.md in the root of this repo, so that your agent can read it too. Then pick one.

Your environment

You work on your own computer, with your own editor and terminal. Set it up before the session. It takes about 5 minutes.

Install

• git
• uv. uv installs Python 3.14 and every package for you.
• The DuckDB CLI, version 2.0 or later
• Claude Code

Set up

git clone https://github.com/crexi-dev/de-interview-exercise.git
cd de-interview-exercise
uv sync
uv run python setup/build_db.py
uv run python setup/check.py

The last command must print READY. If it prints NOT READY, follow what it lists. If you cannot fix it, tell your recruiter before the session. On macOS and Linux, make setup runs the same three commands.

What you have

• The agent. At the start, your interviewer sends you a key in the chat. In the terminal, run export ANTHROPIC_API_KEY=<key> (in PowerShell, $env:ANTHROPIC_API_KEY="<key>"). Then start Claude Code with claude in the same terminal. The key works only for this session.
• The database. To explore the data, run duckdb -readonly data/waypoint.duckdb in the terminal. In read-only mode, you, the agent, and dbt can use the database at the same time.
• dbt, SQL, or Python. We build our warehouse transformations in dbt. In the session, you can work in dbt, in plain SQL scripts, or in Python jobs.
• dbt. The project in dbt/ has one source per provider (atlas, beacon, cartograph), a waypoint source for categories, and a health_inspections source for the OCR output. The sources hold data tests, for example the accepted values of event_type for each provider. dbt writes models to data/dev.duckdb. To build, run dbt build from the dbt/ folder. If dbt is new to you, dbt/LEARN_DBT.md has a prompt that teaches you and your agent the basics, and a free practice project.
• SQL scripts. sql/README.md tells you how to write and run plain SQL scripts against the data. They write their tables to data/work.duckdb.
• Python. The repo is one uv project. Its environment, .venv, holds dbt, Jupyter, and your Python jobs. It has dbt, duckdb, pandas, polars, jupyterlab, and anthropic. Activate it in each terminal (source .venv/bin/activate, or .venv\Scripts\activate on Windows), or put uv run in front of a command. To add a package, run uv add <package>. extract_load/README.md tells you how to run a job.
• Docs. SCHEMA.md describes every table and column, as the providers document them. AGENTS.md tells the agent how to work with the data and dbt here.
• A dev container, if you prefer one. .devcontainer/ builds the same environment in Docker, for VS Code Dev Containers.

You can also use your own docs and web searches.