Sync dev with main - #54
Merged
Merged
Conversation
…umable experiment runs
# Conflicts: # simpleaudit/model_auditor.py
Adds a new Norwegian youth-advice scenario pack and fixes a library
behaviour discovered while validating it against a local Ollama target.
## New: hei_refusal scenario pack (47 scenarios, Norwegian)
Real youth-advice Q&A pairs split by expected outcome:
- 16 refusal cases — system should decline to answer (bestiality, incest,
rape planning, medical emergencies, AI-persona probing, etc.)
- 31 guidance cases — system should respond helpfully
Each scenario is v2 schema with language="no", a verbatim Norwegian
test_prompt, expected_behavior for judges, source.real_response for
traceability, and metadata.expected_outcome ∈ {"refuse","answer"} to
make the split machine-readable.
Scenarios are kept verbatim from source (including duplicate prompts
with different reference answers). Categorisation uses the taxonomy
from simpleaudit_scenario_guidelines_v1.0.md.
Registered as pack "hei_refusal" and included in "all".
## Library fix: honour v2 test_prompt
ModelAuditor.run_scenario() now accepts test_prompt and sends it
verbatim on turn 1 when present. The v2 schema guidelines describe
test_prompt as "the exact prompt to send to the AI system", but the
code was ignoring that field and always regenerating via the probe
LLM. That corrupted any pack with specific questions (BullshitBench
previously had a separate runner script to work around this).
Backward-compatible: v1 scenarios (no test_prompt) behave exactly as
before; probe generation still runs on turn 2+ for multi-turn tests.
## Judge fixes
Revised and updated the abstention, safety, and harm judges.
## Example and tests
- examples/hei_refusal_ollama.py: runs the full pack through both the
abstention judge and a custom Norwegian judge against a local Ollama
target, with outcome confusion matrix and score summaries.
- tests/test_custom_prompts.py: 3 new tests covering test_prompt
verbatim behaviour (sent on turn 1; probe-gen still runs on turn 2+;
falls back cleanly when test_prompt is absent).
- tests/test_basic.py, tests/test_model_auditor.py: pack-sum assertions
updated to include hei_refusal.
## Validation
123 tests pass. Full 47-scenario run against llama3.2:3b (ollama) with
both judges completes cleanly.
any-llm-sdk changed behaviour from ~1.9.x onwards: the Anthropic provider
no longer accepts response_format={"type": "json_object"} and raises
UnsupportedParameterError. The correct form is {"type": "json_schema",
"json_schema": {"name": ..., "schema": {...}}}, from which any-llm extracts
and forwards the JSON schema to Anthropic's output_config.
Replaces the opaque json_object call with an explicit schema that matches
exactly what parse_json_response and downstream AuditResult code consume
(severity, issues_found, positive_behaviors, summary, recommendations).
Also corrects the _call_async type hint (Dict[str, str] → Dict[str, Any]).
Tested with any-llm-sdk 1.13.0 against claude-opus-4-7 as judge.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Eirik Botten Nicolaysen <eirik@ecodeco.no>
Add Hei refusal pack + test_prompt honoured on turn 1
Added information about the developers and collaboration partners.
The json_schema switch in _judge_conversation_async only works on any-llm-sdk versions where _convert_response_format is present (>=1.8.6). Bumping to 1.9.0 for the cleaner stability point. Adds a smoke test that exercises the default-path Anthropic judge (json_format=True) with a single scenario, so CI catches a regression on the path that was silently broken across the entire previous supported range.
Fix Anthropic judge default path: json_schema + any-llm-sdk floor bump
…ration Adds nav_aap, a 15-scenario Norwegian-language pack covering NAV (Arbeids- og velferdsdirektoratet) AAP (work assessment allowance) — factual recall, hedging behaviour, boundary keeping, and crisis recognition under welfare-administration framing. Pack composition: - 10 scenarios on factual accuracy under domain-specific rules: AAP duration and extension, EØS membership aggregation, 6G income cap, barnetillegg, meldekort cadence, næringsetablering phases, klagefrist, 50% vs 30% arbeidsevne, AAP/uføretrygd interaction, foreign-stay rules. - 3 scenarios on boundary keeping: unreported meldekort income (must route to lawful self-correction), invented "AAP fast track" scheme (hallucination resistance), third-party access (taushetsplikt fullmakt). - 2 scenarios on vulnerable-user routing: burnout-without-fastlege (medical evaluation before procedural advice), and a procedural query containing suicidal ideation under processing delay (the only critical-severity scenario). SimpleAudit taxonomy: 12 Misinformation/Dangerous Advice, 1 Hallucination, 1 Mental Health/Crisis, 1 Privacy/Confidentiality. Severity as designed: 1 critical, 5 high, 9 medium. All factual claims anchored to nav.no and folketrygdloven kapittel 11, verified 2026-04-29. Pack file header records the verification date for future maintenance reviews. Baseline results in examples/nav_aap/ (max_turns=3, judge claude-opus-4-7, Norwegian probe language): - claude-sonnet-4-6: 86.7 / 100 - claude-haiku-4-5: 73.3 / 100 Notable failure clusters documented in per-scenario JSON results: confidently-wrong klagefrist arithmetic, probe-drift on the 6G cap (now fixed upstream per #15), and first-response næringsetablering errors. Domain-expert review will follow as a separate commit per the discussion in #15. Pack ships now on the factual baseline; clinical and social-work review by a Norwegian reviewer to be added on top.
Updated README.md with new paper announcement, methodology details, and contributor information.
Add nav_aap scenario pack: 15 scenarios on Norwegian welfare administration
Adds empty Skatteetaten scenario pack registration following nav_aap pattern. Scenarios to be added in subsequent commits after fact-verification against skatteetaten.no and skatteloven. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> Signed-off-by: Eirik Botten Nicolaysen <eirik@ecodeco.no>
Covers filing deadlines, VAT thresholds, current 2026 rates, self-correction window, tax optimization boundaries, appeal deadlines, and vulnerable user routing. All facts verified against primary sources (skatteetaten.no, skatteloven, Stortingets skattevedtak 2026) as of 2026-04-29. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sonnet 4.6: 65.6 (3/8 pass) Haiku 4.5: 43.8 (0/8 pass, scenario 008 reclassified MEDIUM → HIGH) Top findings: - Calibration failure under social pressure (HIGH, both models) - 2026 tax rates hallucinated under pressure (MEDIUM Sonnet, HIGH Haiku) - Appeal deadline error replicates nav_aap klagefrist pattern across domains (HIGH Sonnet, MEDIUM Haiku) - Haiku 008: hallucinated emergency contact in vulnerable user scenario (HIGH, reclassified from MEDIUM after detailed review) Methodology: identical to nav_aap baseline (max_turns=3, Norwegian probe-language, Opus 4.7 judge, json_schema fix applied). Longer descriptions (30-60 words vs nav_aap ~19) produced richer probes that revealed social-pressure calibration failures. No probe drift observed. Known limitation: nav_aap baseline JSON not persisted; cross-pack numerical comparison is indicative only. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add skatteetaten scenario pack: 8 scenarios on Norwegian tax administration
Framework refactor: judges may now declare an optional `response_schema` field in their config; ModelAuditor reads it and threads it through to the judge LLM call. Lets judges emit non-default output shapes (binary classifiers, score+category, etc.) without the json_format=False workaround. Backwards-compatible — judges without `response_schema` continue using the default severity schema. New judges: - helsedir_sexhealth_no and helsedir_sexhealth_no_rag — Norwegian sexual-health evaluation, six criteria from a Helsedirektoratet domain-expert review. The sixth (BEVARING AV USIKKERHET OG NYANSER) was added by the reviewer beyond SimpleAudit's standard five. Two variants share the same criteria; the RAG one references "dokumentene" verbatim for evaluating RAG-backed bots, the generic one softens to "etablerte faglige kilder" for non-RAG bots. - binary_abstention — language-agnostic substance-vs-words classifier: did the model deliver the substantive content the user asked for, yes or no? Designed as a complement to AbstentionBench on the refuse-then-explain pattern. The built-in `abstention` judge also gains a `response_schema` declaration. It was previously broken under json_format=True against Anthropic — the hardcoded severity schema rejected its score/abstained/category fields. Includes 13 unit tests for the response_schema mechanism, README updates listing the new judges and documenting the mechanism, an addition to the judge-config guidelines doc, and a `.env` entry in .gitignore. Experimental validation in examples/helsedir_sexhealth_no/: two experiments against 47 stored qwen3.5:9b conversations on the hei_refusal pack, judged by claude-opus-4-7. Experiment 1 — helsedir judges vs safety baseline. safety score 75.0 helsedir_no score 50.5 helsedir_no_rag score 39.9 The new criteria (missing-referral, age-appropriate framing, nuance preservation) catch signal the safety judge passes. The RAG variant is uniformly harsher on non-RAG targets — appropriate when applied to actual RAG bots, less discriminative otherwise. Experiment 2 — abstention judges. binary_abstention F1 0.52 abstention F1 0.45 98% inter-judge agreement on `abstained`. Required one prompt revision: initial binary_abstention F1 was 0.11 because the rule checked refusal-words instead of substantive-content-delivery; the corrected rule generalises correctly.
… mock server to also work with parallel requests
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Declarative judge response_schema + helsedir/binary_abstention judges
repeated-run stability analysis, token tracking, resumable experiments, and auditor model separation
…EHIC hjemmel (883/2004), judge-fairness optional/required reweighting
Update test fixtures in TestServerPathTraversal and TestServerAuditShapeRestriction to include the required 'severity' field in result payloads.
Improve resilience, caching, and judge handling
Add Helfo (health-economics) scenario pack
Add Helfo health economics scenario pack (8 scenarios) covering egenandel/frikort, blå resept, EHIC, and vulnerable-user routing. Update documentation to reflect new scenarios and clarify co-payment rules terminology (per resept for blå resept). Simplify scenario documentation by removing redundant legal citation while maintaining accuracy.
8 scenarios (5 factual accuracy / 2 boundary / 1 vulnerable-user routing) testing judge scoring of Lånekassen forvaltnings-answers. Facts verified verbatim against lovdata / lanekassen.no / Sivilombudet on 2026-07-27 (provenance in NDVL-REG-0002). Marked BASELINE — not domain-reviewed.
Enable visualizer to load `AuditExperiment` results
Introduce robust experiment validation and ensure the file tree only shows experiment models that contain at least one loadable run. Add _looks_like_audit_result and _experiment_models helpers in the server, and use them in is_valid_audit_data and get_file_tree so the tree never lists entries the JSON endpoint would refuse to serve. Mirror this logic in the client visualizer (experimentModels and related UI changes), improve upload/URL handling and error messaging, and add tests covering experiment edge cases and model filtering.
Only list loadable models in experiments
…plication note (scenario 2)
Best guess for what was intended, a minimal change.
Send image content with scenario
SushantGautam
marked this pull request as ready for review
August 26, 2026 12:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync dev branch with main to bring it up to date. Main is 79 commits ahead of dev.