Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,35 @@ pip install -r requirements.txt
If PyPI itself is blocked, ask IT for your internal package mirror and use it:
`pip install -r requirements.txt --index-url https://<your-mirror>/simple`.

### `CERTIFICATE_VERIFY_FAILED` when the notebook calls the API
**Why:** your company inspects TLS traffic (Zscaler, Netskope, a corporate firewall), so
the certificate the notebook actually receives is signed by an internal root CA. IT has
already installed that root in your operating system's certificate store — but Python's
HTTP stack ignores the OS store and trusts only the `certifi` bundle shipped with pip,
which has never heard of it. Every API call fails on the handshake.

**Fix:** usually nothing — the setup cell calls
[`truststore`](https://pypi.org/project/truststore/) before creating the client, which
points Python's TLS verification at the OS store where your corporate root already lives.
You'll see `✓ TLS: verifying against the OS certificate store` and everything just works.

If instead you see `[!!] truststore unavailable`, the shim didn't load, and the error is
in the parentheses:
- `ModuleNotFoundError` — the install didn't reach it. Run `pip install truststore` in the
same environment as your kernel (see "Wrong kernel selected" above), then re-run the cell.
- `ImportError` / anything else — you're most likely on Python 3.9 or older; `truststore`
needs **3.10+**. See "wrong version" below.

**Please don't** set `verify=False`, `SSL_CERT_FILE=""`, or `PYTHONHTTPSVERIFY=0`. Those
turn off certificate checking altogether rather than fixing which certificates you trust,
and on a corporate machine that's a security-policy violation, not a workaround. If
`truststore` can't be made to work, ask IT to export the internal root CA as a `.pem` and
point Python at it properly:
```bash
export SSL_CERT_FILE=/path/to/corporate-root.pem
export REQUESTS_CA_BUNDLE=$SSL_CERT_FILE
```

### "I don't have Python" / wrong version
You need **Python 3.10+**. Install from [python.org](https://www.python.org/downloads/)
or your company's software portal, then re-open VS Code so it's detected. Check with
Expand Down
17 changes: 16 additions & 1 deletion day1/02_developer-platform/Developer_Platform.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -180,9 +180,24 @@
" f\" (pip said: {tail})\\n\"\n",
" )\n",
"\n",
"_ensure_packages([(\"anthropic\", \"anthropic\")])\n",
"_ensure_packages([(\"anthropic\", \"anthropic\"), (\"truststore\", \"truststore\")])\n",
"print(\"✓ Dependencies ready\")\n",
"\n",
"# ── Corporate TLS inspection (Zscaler, Netskope, Palo Alto, …) ────────────\n",
"# On a managed laptop, HTTPS is often intercepted and re-signed by a corporate root\n",
"# CA that lives in the OS trust store. Python doesn't read that store - it verifies\n",
"# against certifi's own bundle - so api.anthropic.com fails with\n",
"# CERTIFICATE_VERIFY_FAILED / APIConnectionError even though the API is perfectly\n",
"# reachable (curl to the same URL succeeds, because curl uses the OS store).\n",
"# truststore points Python at the OS store too. A no-op on unmanaged machines.\n",
"try:\n",
" import truststore\n",
" truststore.inject_into_ssl()\n",
" print(\"✓ TLS: verifying against the OS certificate store\")\n",
"except Exception as _tls_exc: # Python < 3.10, or the install was blocked\n",
" print(\"[!!] truststore unavailable (\" + type(_tls_exc).__name__ + \") - falling back \"\n",
" \"to certifi. If the API check below fails with a certificate error, see SETUP.md.\")\n",
"\n",
"import anthropic\n",
"import json\n",
"import time\n",
Expand Down
123 changes: 86 additions & 37 deletions day1/03_prompt-rescue/Prompt_Rescue_solo.ipynb
Original file line number Diff line number Diff line change
@@ -1,15 +1,4 @@
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
Expand Down Expand Up @@ -44,9 +33,11 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "xdGQsghU0umI"
},
"outputs": [],
"source": [
"# Install dependencies into THIS kernel — safe to re-run; survives locked-down (PEP 668) Pythons.\n",
"import importlib.util, os, subprocess, sys\n",
Expand Down Expand Up @@ -156,11 +147,9 @@
" f\" (pip said: {tail})\\n\"\n",
" )\n",
"\n",
"_ensure_packages([(\"anthropic\", \"anthropic\"), (\"matplotlib\", \"matplotlib\")])\n",
"_ensure_packages([(\"anthropic\", \"anthropic\"), (\"truststore\", \"truststore\"), (\"matplotlib\", \"matplotlib\")])\n",
"print(\"✓ Dependencies ready\")"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
Expand All @@ -179,6 +168,14 @@
"source": [
"import os\n",
"\n",
"try:\n",
" import truststore\n",
" truststore.inject_into_ssl()\n",
" print(\"✓ TLS: verifying against the OS certificate store\")\n",
"except Exception as _tls_exc: # Python < 3.10, or the install was blocked\n",
" print(\"[!!] truststore unavailable (\" + type(_tls_exc).__name__ + \") - falling back \"\n",
" \"to certifi. If the API check below fails with a certificate error, see SETUP.md.\")\n",
" \n",
"def _status(ok, msg):\n",
" \"\"\"Green/red banner in notebooks; plain text when run as a script.\"\"\"\n",
" try:\n",
Expand Down Expand Up @@ -331,9 +328,11 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cOhoxjAX0umJ"
},
"outputs": [],
"source": [
"# ✏️ YOUR TURN: edit this prompt\n",
"# ============================================================\n",
Expand All @@ -360,9 +359,7 @@
"\n",
"print(\"Broken prompt loaded. Run the eval suite cell below to see your baseline score.\")\n",
"print(f\"Prompt length: {len(system_prompt)} characters\")"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -452,10 +449,12 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "YOn3TQGX0umK"
},
"outputs": [],
"source": [
"#@title Eval Harness (click to load -- do not edit this cell)\n",
"\n",
Expand Down Expand Up @@ -1125,24 +1124,22 @@
"print(\" run_eval_suite(prompt_or_prompts, label='...') -- Run full eval, display visual results\")\n",
"print(\" compare_prompts(prompt_a, prompt_b, ...) -- A/B comparison of two variants\")\n",
"print(\" show_iteration_log() -- Show all runs + score progression chart\")\n"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "aHn2fW1S0umM"
},
"outputs": [],
"source": [
"# ============================================================\n",
"# RUN YOUR FIRST EVAL -- Establish your baseline score\n",
"# ============================================================\n",
"\n",
"results = run_eval_suite(system_prompt, label=\"Baseline (Broken Prompt)\")"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -1220,7 +1217,17 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n\n### Not sure which technique fits? Ask Claude.\n\nTake the failure pattern you just diagnosed and ask Claude:\n\n> \"I'm fixing a prompt that fails on [describe your failure mode]. Which of these would you recommend: meta-prompting, prompt chaining, few-shot examples, or constraint sharpening? Give me your reasoning for this specific failure pattern.\"\n\nThen make your own call. The point isn't whether Claude is right — it's whether its reasoning holds up against your diagnosis.\n\n---"
"---\n",
"\n",
"### Not sure which technique fits? Ask Claude.\n",
"\n",
"Take the failure pattern you just diagnosed and ask Claude:\n",
"\n",
"> \"I'm fixing a prompt that fails on [describe your failure mode]. Which of these would you recommend: meta-prompting, prompt chaining, few-shot examples, or constraint sharpening? Give me your reasoning for this specific failure pattern.\"\n",
"\n",
"Then make your own call. The point isn't whether Claude is right — it's whether its reasoning holds up against your diagnosis.\n",
"\n",
"---"
]
},
{
Expand Down Expand Up @@ -1272,9 +1279,11 @@
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "nn2Y3T6i0umO"
},
"outputs": [],
"source": [
"# ============================================================\n",
"# CHAIN MODE -- Use this if you split into multiple steps\n",
Expand All @@ -1297,15 +1306,15 @@
"\n",
"# Run the chain (uncomment when ready)\n",
"# results = run_eval_suite([step1_prompt, step2_prompt, step3_prompt], label=\"Chain v1\")"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "09BjBx--0umO"
},
"outputs": [],
"source": [
"# ============================================================\n",
"# A/B COMPARISON -- Compare two prompt variants\n",
Expand All @@ -1319,15 +1328,15 @@
"\n",
"# Run comparison (uncomment when ready)\n",
"# result_a, result_b = compare_prompts(prompt_a, prompt_b, label_a=\"Current Best\", label_b=\"New Variant\")"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "HoYKlgLG0umP"
},
"outputs": [],
"source": [
"# ============================================================\n",
"# ITERATION LOG -- Track your progress across runs\n",
Expand All @@ -1337,9 +1346,7 @@
"# Call this to see your iteration history and score progression:\n",
"\n",
"show_iteration_log()"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -1463,15 +1470,57 @@
"metadata": {},
"outputs": [],
"source": [
"# ============================================================\n# PROCESS AUDIT -- Compare your approach vs. Claude's direct fix\n# ============================================================\n\nclaude_fix = \"\"\"\n[Paste Claude's direct rewrite here]\n\"\"\"\n\nresults_audit = run_eval_suite(claude_fix, label=\"Claude direct fix\")"
"# ============================================================\n",
"# PROCESS AUDIT -- Compare your approach vs. Claude's direct fix\n",
"# ============================================================\n",
"\n",
"claude_fix = \"\"\"\n",
"[Paste Claude's direct rewrite here]\n",
"\"\"\"\n",
"\n",
"results_audit = run_eval_suite(claude_fix, label=\"Claude direct fix\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## How to Present This to an Enterprise Buyer\n\nThe goal of this exercise isn't just a better score — it's building the confidence to walk a technical buyer through a real prompt engineering engagement. Here's how.\n\n### The Narrative Arc\n\n| Beat | What to Say | Why It Works |\n|------|-------------|--------------|\n| **The problem** | \"Your prompt does too many things at once, and the tasks interfere with each other.\" | Diagnoses the root cause, not the symptom. |\n| **The diagnosis** | \"I used Claude to analyze its own failure modes — here's what it identified.\" | Shows meta-prompting in action. Buyers respond to this. |\n| **The fix** | \"I decomposed it into a [N]-step chain: classify first, extract second, respond third. Each step is focused and can't pollute the others.\" | Demonstrates architectural thinking, not just prompt tweaking. |\n| **The proof** | \"Before: 19% pass rate. After: 86%. Here's the category breakdown — feature requests improved most because we're now evaluating content, not tone.\" | Quantitative. Specific. Defensible. |\n| **The prevention** | \"Here's a 3-question checklist your team can run on every new prompt before shipping to production.\" | Leaves them with something they can use. |\n\n### What Separates Good from Great\n\n- **Good:** \"I fixed the prompt and the score went up.\"\n- **Great:** \"I diagnosed the structural failure mode, chose a technique that specifically addresses it, and can show you the delta across each failure category.\"\n\nThe second version is what closes enterprise deals.\n\n### One Thing to Never Say\n\nNever say \"I just asked Claude to fix it.\" Even if meta-prompting helped, own the diagnostic judgment. You decided what to ask. You evaluated the result. You made the call on which fix to ship. That's the value you bring."
"## How to Present This to an Enterprise Buyer\n",
"\n",
"The goal of this exercise isn't just a better score — it's building the confidence to walk a technical buyer through a real prompt engineering engagement. Here's how.\n",
"\n",
"### The Narrative Arc\n",
"\n",
"| Beat | What to Say | Why It Works |\n",
"|------|-------------|--------------|\n",
"| **The problem** | \"Your prompt does too many things at once, and the tasks interfere with each other.\" | Diagnoses the root cause, not the symptom. |\n",
"| **The diagnosis** | \"I used Claude to analyze its own failure modes — here's what it identified.\" | Shows meta-prompting in action. Buyers respond to this. |\n",
"| **The fix** | \"I decomposed it into a [N]-step chain: classify first, extract second, respond third. Each step is focused and can't pollute the others.\" | Demonstrates architectural thinking, not just prompt tweaking. |\n",
"| **The proof** | \"Before: 19% pass rate. After: 86%. Here's the category breakdown — feature requests improved most because we're now evaluating content, not tone.\" | Quantitative. Specific. Defensible. |\n",
"| **The prevention** | \"Here's a 3-question checklist your team can run on every new prompt before shipping to production.\" | Leaves them with something they can use. |\n",
"\n",
"### What Separates Good from Great\n",
"\n",
"- **Good:** \"I fixed the prompt and the score went up.\"\n",
"- **Great:** \"I diagnosed the structural failure mode, chose a technique that specifically addresses it, and can show you the delta across each failure category.\"\n",
"\n",
"The second version is what closes enterprise deals.\n",
"\n",
"### One Thing to Never Say\n",
"\n",
"Never say \"I just asked Claude to fix it.\" Even if meta-prompting helped, own the diagnostic judgment. You decided what to ask. You evaluated the result. You made the call on which fix to ship. That's the value you bring."
]
}
]
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Loading