-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathground_truth_layers.html
More file actions
321 lines (276 loc) · 23.9 KB
/
Copy pathground_truth_layers.html
File metadata and controls
321 lines (276 loc) · 23.9 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Ground-Truth Layers: Knowledge Graphs, Pathways & Evidence in Vibe Rounds</title>
<style>
:root {
--primary: #0e7490;
--secondary: #0891B2;
--accent-promo: #27ae60;
--accent-provo: #e74c3c;
--bg-light: #f8fafc;
--card-bg: #ffffff;
--text-main: #2d3748;
--text-muted: #718096;
--border-color: #e2e8f0;
}
* { box-sizing: border-box; }
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
line-height: 1.7;
color: var(--text-main);
background-color: var(--bg-light);
margin: 0;
padding: 15px 10px;
}
.article-container {
max-width: 900px;
margin: 0 auto;
background: var(--card-bg);
padding: 24px 16px;
border-radius: 12px;
box-shadow: 0 10px 25px -5px rgba(0, 0, 0, 0.05), 0 8px 10px -6px rgba(0, 0, 0, 0.01);
}
header { border-bottom: 3px solid var(--primary); padding-bottom: 20px; margin-bottom: 30px; }
h1 { color: var(--primary); font-size: 1.8em; line-height: 1.2; margin: 0 0 12px 0; }
.subtitle { font-size: 1.05em; color: var(--secondary); font-weight: 600; margin: 0 0 20px 0; }
.cta-header-btn {
display: inline-block; background-color: var(--secondary); color: white; font-weight: bold;
padding: 10px 20px; border-radius: 6px; text-decoration: none; font-size: 0.95em;
transition: background-color 0.2s, transform 0.1s;
}
.cta-header-btn:hover { background-color: var(--primary); color: white; }
.cta-header-btn:active { transform: scale(0.98); }
/* Series navigation */
.series-nav { display:flex; flex-wrap:wrap; gap:8px; margin-top:20px; padding-top:18px; border-top:1px dashed var(--border-color); }
.series-nav a {
font-size:0.78em; font-weight:600; padding:6px 13px; border-radius:20px;
background:#e2e8f0; color:var(--text-main); text-decoration:none; white-space:nowrap;
transition: background-color .2s, color .2s;
}
.series-nav a.active { background:var(--primary); color:#fff; }
.series-nav a:hover { background:var(--secondary); color:#fff; }
h2 { color: var(--primary); font-size: 1.4em; margin-top: 35px; margin-bottom: 15px; border-bottom: 1px solid var(--border-color); padding-bottom: 10px; }
h3 { color: var(--secondary); font-size: 1.15em; margin-top: 25px; margin-bottom: 10px; }
p { margin-bottom: 20px; font-size: 1em; }
.disclaimer { background: #fffbe6; color: #8a6d3b; padding: 15px 16px; border-radius: 8px; font-size: 0.9em; margin-bottom: 30px; border-left: 5px solid #ffe79a; }
.video-container { position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%; background: #000; border-radius: 8px; margin: 25px 0; }
.video-container iframe { position: absolute; top: 0; left: 0; width: 100%; height: 100%; border: 0; }
.table-responsive { width: 100%; overflow-x: auto; -webkit-overflow-scrolling: touch; margin: 25px 0; }
table { width: 100%; border-collapse: collapse; font-size: 0.9em; min-width: 550px; }
th, td { padding: 12px 14px; text-align: left; border-bottom: 1px solid var(--border-color); }
th { background-color: var(--primary); color: white; font-weight: 600; }
tr:nth-child(even) { background-color: #f8fafc; }
.concept-card { border-left: 5px solid; padding: 16px 18px; margin: 20px 0; background: #f8fafc; border-radius: 0 8px 8px 0; }
.promo-card { border-color: var(--accent-promo); }
.promo-card h3, .promo-card h4 { color: var(--accent-promo); }
.provo-card { border-color: var(--accent-provo); }
.provo-card h3, .provo-card h4 { color: var(--accent-provo); }
.case-card { border-color: var(--primary); }
.info-card { border-color: var(--secondary); }
.info-card h3, .info-card h4 { color: var(--secondary); }
.risk-card { border-color: var(--accent-provo); background:#fdf1ef; }
.risk-card h3, .risk-card h4 { color: #b0392b; }
.module-tag { display: inline-block; background: #e2e8f0; color: #4a5568; padding: 4px 8px; border-radius: 4px; font-size: 0.8em; margin-right: 4px; margin-bottom: 6px; font-weight: 600; }
.case-walker { background: #f1f5f9; padding: 18px 14px; border-radius: 10px; border: 1px solid var(--border-color); margin: 30px 0; }
.step { display: none; animation: fadeIn 0.4s ease-in-out; }
.step.active { display: block; }
@keyframes fadeIn { from { opacity: 0; transform: translateY(8px);} to { opacity: 1; transform: translateY(0);} }
.btn { display: block; width: 100%; box-sizing: border-box; text-align: center; padding: 12px 20px; margin-top: 20px; border: none; border-radius: 6px; cursor: pointer; color: white; font-weight: bold; font-size: 0.95em; transition: opacity 0.2s, transform 0.1s; }
.btn:hover { opacity: 0.9; }
.btn:active { transform: scale(0.98); }
.btn-start { background-color: var(--primary); }
.btn-promo { background-color: var(--accent-promo); }
.btn-provo { background-color: var(--accent-provo); }
ul, ol { padding-left: 18px; }
li { margin-bottom: 8px; }
.case-link a { color: var(--secondary); font-weight: bold; text-decoration: none; word-break: break-all; }
.case-link a:hover { text-decoration: underline; }
blockquote { font-style: italic; border-left: 3px solid var(--text-main); margin: 20px 0; padding: 6px 20px; color: var(--text-muted); }
code { font-family: "SFMono-Regular", Consolas, monospace; background: var(--border-color); padding: 0.1rem 0.3rem; border-radius: 3px; font-size: 0.88em; }
.badge-row { display:flex; flex-wrap:wrap; gap:10px; margin: 18px 0 24px; }
.badge { background: var(--card-bg); border:1px solid var(--border-color); border-radius:6px; padding:8px 12px; min-width:110px; }
.badge .k { font-size:0.65rem; text-transform:uppercase; letter-spacing:0.08em; color:var(--text-muted); display:block; }
.badge .v { font-weight:700; font-size:0.92rem; color:var(--text-main); }
.badge.urgent .v { color: var(--accent-provo); }
.grid3 { display:grid; grid-template-columns:repeat(3,1fr); gap:14px; margin:18px 0 6px; }
.grid2 { display:grid; grid-template-columns:1fr 1fr; gap:14px; margin:18px 0 6px; }
@media (max-width:680px){ .grid3{grid-template-columns:1fr;} .grid2{grid-template-columns:1fr;} }
.grid-card { background: var(--card-bg); border:1px solid var(--border-color); border-radius:8px; padding:16px; }
.grid-card .tag { font-size:0.68rem; color:var(--secondary); text-transform:uppercase; letter-spacing:0.08em; display:block; margin-bottom:6px; font-weight:700; }
.grid-card h4 { margin:0 0 6px; font-size:0.95rem; color:var(--text-main); }
.grid-card p { font-size:0.85rem; color:var(--text-muted); margin:0; }
.grid-card.pro { border-top:3px solid var(--accent-promo); }
.grid-card.con { border-top:3px solid var(--accent-provo); }
.thesis { font-size: 1.15em; line-height: 1.5; color: var(--primary); border-top: 2px solid var(--secondary); border-bottom: 2px solid var(--secondary); padding: 18px 0; margin: 24px 0; text-align:center; font-style:italic; }
.chain { display:flex; flex-wrap:wrap; gap:8px; align-items:center; margin:14px 0 20px; font-size:0.85rem; }
.chain .cstep { background:#eef4f3; border:1px solid #cfe3e1; color:var(--primary); padding:6px 10px; border-radius:5px; white-space:nowrap; font-weight:600; }
.chain .arrow { color:var(--text-muted); }
.compare2 { display:grid; grid-template-columns:1fr 1fr; gap:14px; margin:18px 0; }
@media (max-width:600px){ .compare2{grid-template-columns:1fr;} }
.compare2 .col { background: var(--card-bg); border:1px solid var(--border-color); border-radius:8px; padding:16px 18px; }
.compare2 .col.was { border-top:3px solid var(--accent-provo); }
.compare2 .col.now { border-top:3px solid var(--accent-promo); }
.compare2 .col h5 { margin:0 0 8px; font-size:0.78rem; text-transform:uppercase; letter-spacing:0.06em; color:var(--text-muted); }
.checklist { background: var(--card-bg); border:1px solid var(--border-color); border-radius:8px; padding:18px 20px; margin:16px 0; }
.checklist h4 { margin:0 0 10px; font-size:0.95rem; color:var(--secondary); }
.checklist ul { margin-bottom:0; }
.checklist li { list-style:none; position:relative; padding-left:26px; }
.checklist li::before { content:"\2610"; position:absolute; left:0; color:var(--secondary); font-size:1.05em; }
.arc-box { font-size:0.85rem; color:var(--text-muted); background:#f8fafc; border:1px solid var(--border-color); border-radius:8px; padding:18px 20px; line-height:1.9; }
footer { margin-top: 40px; padding-top: 20px; border-top: 1px solid var(--border-color); font-size: 0.85em; color: var(--text-muted); display: flex; flex-direction: column; gap: 12px; align-items: flex-start; }
footer a { color: var(--secondary); font-weight: 600; text-decoration: none; }
footer a:hover { text-decoration: underline; }
footer .footer-links { display:flex; flex-wrap:wrap; gap:6px 14px; }
@media (min-width: 768px) {
body { padding: 30px 15px; }
.article-container { padding: 50px; }
.case-walker { padding: 30px; }
h1 { font-size: 2.4em; }
h2 { font-size: 1.75em; }
.btn { display: inline-block; width: auto; }
footer { flex-direction: row; justify-content: space-between; align-items: center; }
}
</style>
</head>
<body>
<div class="article-container">
<header>
<h1>Fact Graphs, Pathways & Appraised Evidence</h1>
<p class="subtitle">Raising the Floor Under Promption and Provocation</p>
<p>Author - Dr. Avinash Kumar Gupta</p>
<a href="https://avi33tbtt.github.io/#quick-tutorial" target="_blank" rel="noopener noreferrer" class="cta-header-btn">Get started →</a>
<nav class="series-nav">
<a href="article.html">Index</a>
<a href="modern_clinical_reasoning.html">Part 1 · Modern Clinical Reasoning</a>
<a href="ground_truth_layers.html" class="active">Part 2 · Ground-Truth Layers</a>
<a href="Case_Complexity.html">Part 3 · Case Complexity</a>
<a href="using_the_stack.html">Part 4 · Using the Stack</a>
<a href="journey.html">Part 5 · The Build Itself</a>
</nav>
</header>
<div class="disclaimer">
<strong>Architectural Note:</strong> this analysis, like Vibe Rounds itself, is a learning-stack companion piece for studying clinical reasoning on deidentified case material — not a clinical decision tool.
</div>
<section>
<h2>0. Recap: Two Cognitive Modes, One Shared Risk</h2>
<p>The base framework runs two modes over case narratives — <strong>Promption</strong>, which scaffolds messy narrative data into structured illness scripts, and <strong>Provocation</strong>, which stress-tests the resulting hypothesis for anchoring and premature closure. Both modes are fluent by design. Neither, on its own, is anchored to anything outside the model's own internal coherence.</p>
<div class="chain">
<span class="cstep">#17 Semantic Qualifiers</span><span class="arrow">→</span>
<span class="cstep">#12 Diff Dx Deepdive</span><span class="arrow">→</span>
<span class="cstep">#30 Anchor Extractor</span><span class="arrow">→</span>
<span class="cstep">#45 Adversarial Shadow</span><span class="arrow">→</span>
<span class="cstep">#28 Diagnostic Time-Out</span>
</div>
<p>That shared blind spot is the subject of this piece: two structured ground-truth layers — a <strong>knowledge graph</strong> and an <strong>institutional pathway</strong> — examined against real published examples, plus a third layer, <strong>appraised evidence</strong>, that answers a question neither of the first two can.</p>
</section>
<section>
<h2>1. Layer One — The Knowledge Graph</h2>
<h3>Worked example: Chronic Kidney Disease</h3>
<div class="concept-card info-card">
<h4>Reference tool</h4>
<p style="margin-bottom:6px;">Vibe Rounds' <strong>Clinical Data Explorer</strong> is a browsable concept graph: diseases, findings, and investigations linked by system, prevalence tier, and a two-level "single-concept explorer" that shows which diseases connect to a finding, and which further concepts those diseases connect to in turn.</p>
<a href="https://avi33tbtt.github.io/tools/concepts/" target="_blank" rel="noopener" style="font-size:0.85em;word-break:break-all;">avi33tbtt.github.io/tools/concepts/</a>
</div>
<p>Applied to CKD, the graph does something a free-text answer can't: it turns a hedge into a checkable criterion. Instead of the model asserting "this looks like chronic kidney disease" as an internally coherent but unverified judgment, the graph holds the actual diagnostic threshold — <strong>eGFR below 60 mL/min/1.73m² persisting for three months or more</strong> — as a structured, queryable entry, alongside its linked findings, complications, and differential set.</p>
<div class="grid3">
<div class="grid-card"><span class="tag">Before the graph</span><h4>Qualitative hedge</h4><p>"Findings are consistent with chronic kidney disease" — fluent, but not falsifiable against anything.</p></div>
<div class="grid-card"><span class="tag">After the graph</span><h4>Criterion-based claim</h4><p>eGFR < 60 for ≥ 3 months, cross-checked against the graph's own CKD entry and its linked differential field.</p></div>
<div class="grid-card"><span class="tag">New capability</span><h4>Coverage flag</h4><p>A computable overlap score between case findings and graph entries — a visible proxy for "how sparse is my ground truth here."</p></div>
</div>
<h3>What the graph fixes — and what it doesn't</h3>
<ul>
<li>Converts tacit illness-scripts into explicit, checkable structure, so modules can validate against verified criteria rather than internal coherence alone.</li>
<li>Enables a real calibration ledger: confidence tracked against documented criteria across many cases, not just one.</li>
<li>Turns the adversarial shadow module into a systematic differential engine, drawing on the graph's own differential-diagnosis field rather than free-associating alternatives.</li>
<li>Enables rule-based, auditable red-flag and escalation logic instead of prose-shaped urgency.</li>
</ul>
<div class="concept-card risk-card">
<h4>Limitation</h4>
<p style="margin:0;">A graph raises the floor by making fabrication checkable — it does not stop correct data from being misapplied to an atypical or comorbid case that doesn't actually match the entry being invoked.</p>
</div>
</section>
<section>
<h2>2. Layer Two — The Institutional Pathway</h2>
<h3>Worked example: Acute Pancreatitis</h3>
<div class="concept-card info-card">
<h4>Reference pathway</h4>
<p style="margin-bottom:6px;">Vibe Rounds' <strong>Acute Pancreatitis</strong> pathway page encodes a full time-boxed management sequence: immediate steps (0–24h), first-line management, definitive/escalation care, monitoring, complications, and red flags — each tied to a concrete time window rather than a general instruction.</p>
<a href="https://avi33tbtt.github.io/pathways/acute-pancreatitis.html" target="_blank" rel="noopener" style="font-size:0.85em;word-break:break-all;">avi33tbtt.github.io/pathways/acute-pancreatitis.html</a>
</div>
<div class="badge-row">
<div class="badge"><span class="k">ICD-10</span><span class="v">K85.9</span></div>
<div class="badge"><span class="k">Setting</span><span class="v">ER / Ward / ICU</span></div>
<div class="badge urgent"><span class="k">Urgency</span><span class="v">High</span></div>
<div class="badge"><span class="k">Escalation gate</span><span class="v">48h organ failure</span></div>
</div>
<p>Where the CKD graph gives modules a fact to check, the pancreatitis pathway gives them something the fact graph structurally cannot: a <strong>sequence</strong>. Diagnosis requires 2 of 3 criteria; fluid resuscitation is titrated hour-by-hour against urine output; the well-known 48-hour organ-failure threshold defines severe disease under the Revised Atlanta Classification and is exactly the kind of concrete pause point the framework's Diagnostic Time-Out module needs, instead of a generic "pause and reconsider" prompt.</p>
<div class="table-responsive">
<table>
<thead><tr><th>Ground-truth axis</th><th>Knowledge graph (CKD)</th><th>Institutional pathway (pancreatitis)</th></tr></thead>
<tbody>
<tr><td>What it checks</td><td>Facts & criteria</td><td>Sequence & timing</td></tr>
<tr><td>Failure mode it targets</td><td>Fabricated or ungrounded facts</td><td>Right action, wrong time window</td></tr>
<tr><td>Concrete anchor</td><td>eGFR < 60 for ≥3 months</td><td>48h organ-failure threshold</td></tr>
<tr><td>Module it sharpens</td><td>#45 Adversarial Shadow</td><td>#28 Diagnostic Time-Out</td></tr>
<tr><td>New risk introduced</td><td>Misapplied-but-correct data</td><td>False institutional authority ("per protocol")</td></tr>
</tbody>
</table>
</div>
<h3>A new risk: borrowed authority</h3>
<div class="concept-card risk-card">
<h4>Risk</h4>
<p style="margin:0;">A pathway-derived module doesn't just state a fact — it issues a timed, directly actionable instruction that carries the implicit weight of "per protocol." A wrong instruction here compounds faster and is trusted more readily than a wrong fact, which argues for stricter provenance and citation requirements on pathway-derived modules than on fact-based ones.</p>
</div>
<p>The net effect: a mature system needs to separately audit <strong>diagnostic correctness</strong>, <strong>sequencing correctness</strong>, and <strong>contextual/population correctness</strong> (special populations, contraindication branching) — rather than collapsing all three into a single aggregate "reasoning quality" score.</p>
</section>
<section>
<h2>3. Layer Three — Appraised Evidence</h2>
<p>Facts answer <em>what</em>. Pathways answer <em>when</em>. Appraised evidence — systematic reviews, meta-analyses, GRADE-rated guidelines — answers a third question neither layer touches: <strong>how strong is the justification, and under what conditions does it hold?</strong></p>
<ul>
<li>Un-flattens uniform declarative confidence into GRADE-style certainty tiers, instead of one flat register of assertion.</li>
<li>Surfaces genuine guideline disagreement — ACG vs. IAP/APA vs. NICE on pancreatitis management — as a teaching moment rather than silently picking one synthesis to present as consensus.</li>
<li>Converts evidence-based-medicine modules from generic EBM-sounding prose into citation-anchored critique of a specific study's actual limitations.</li>
<li>Restores legitimate use of precise, sourced numbers — effect sizes, confidence intervals — in place of unfalsifiable hedge words. The earlier problem was <em>ungrounded</em> precision, not precision itself.</li>
</ul>
<div class="concept-card risk-card">
<h4>Sharpest new risk</h4>
<p style="margin:0;">Evidence-appraisal-shaped language is the most convincing hallucination surface in the entire stack, because it carries the highest inherent credibility signal. It is only safe when backed by real, traceable retrieval — not GRADE-style prose generated without an actual retrieval step behind it.</p>
</div>
</section>
<section>
<h2>4. The Complexity/Bias Tradeoff</h2>
<p>Stacking all three layers isn't free. For common, densely-documented diseases, the base model is often already close to ground truth — adding a fact graph, a pathway, and an evidence layer adds seams where independently-sourced components can quietly contradict each other, for a comparatively small accuracy gain.</p>
<p>For rare or atypical presentations, the picture is sharper: unstructured reasoning over-fits the case to the nearest common script, while a structure-first approach can force a sparse or absent entry into the nearest available match — both produce confident wrongness, just via different mechanisms, and the structured failure is <strong>harder to detect</strong> because it wears the appearance of verified rigor.</p>
<div class="thesis">
"The highest-leverage fix isn't adding or withholding ground-truth layers uniformly — it's making 'how much ground truth do I actually have for this case' a first-class, visible output of the system itself."
</div>
<p>A well-designed system can fail <em>loudly</em> — an explicit "no strong match" signal — rather than silently inheriting the base model's overconfidence. Disagreement across layers (fact graph vs. pathway vs. evidence) is itself a usable audit signal, provided the seams are surfaced rather than papered over. That argues for differential deployment: heavy structure is low-value where the base model is already reliable, and highest-value exactly where unstructured reasoning fails worst — conditional on the structure honestly reporting its own coverage density.</p>
</section>
<section>
<h2>5. Proposed Next Step</h2>
<p>An empirical study: run matched common and rare cases both with and without the ground-truth layers (fact graph, pathway, evidence), and measure three outcomes across both conditions —</p>
<div class="grid3">
<div class="grid-card"><span class="tag">Metric 1</span><h4>Diagnostic accuracy</h4><p>Does the added structure actually move the answer closer to ground truth?</p></div>
<div class="grid-card"><span class="tag">Metric 2</span><h4>Detectability of failure</h4><p>When it's wrong, does it fail loudly (flagged) or quietly (confidently wrong)?</p></div>
<div class="grid-card"><span class="tag">Metric 3</span><h4>Learner calibration</h4><p>Does the learner's own confidence track the true reliability of the answer they were given?</p></div>
</div>
</section>
<p>Continue to <a href="Case_Complexity.html">Part 3 · Weighting Ground-Truth Layers by Case Complexity →</a></p>
<footer>
<span>Vibe Rounds CCOS Framework · Companion analysis for educational and clinical-reasoning practice only — not a clinical decision tool.</span>
<div class="footer-links">
<a href="article.html">Index</a>
<a href="modern_clinical_reasoning.html">Part 1</a>
<a href="Case_Complexity.html">Part 3</a>
<a href="using_the_stack.html">Part 4</a>
<a href="journey.html">Part 5</a>
<a href="https://avi33tbtt.github.io/pathways/acute-pancreatitis.html" target="_blank" rel="noopener">Pancreatitis Pathway</a>
<a href="https://avi33tbtt.github.io/tools/concepts/" target="_blank" rel="noopener">Data Explorer</a>
</div>
</footer>
</div>
</body>
</html>