Repository navigation
Commit b6d35c5
fix(gooddata-eval): record a dashboard chat fault instead of letting it escape
ChatClient.send_message can raise ChatError, and the dashboard loop was the only
one that let it through. Escaping run_agentic_dashboard_skill discarded every
K-run already completed along with this one's exit_reason, so the item surfaced
as a bare error with no per-run diagnosis -- exactly what the shared LoopExit
contract exists to prevent. metric_skill.py and conversation.py already record
it; this matches them.
A bare catch would have discarded ChatError.partial_result. What the stream
delivered before it died is still the turn's work, so its reasoning, tool calls
and step count are merged -- shifted and indexed the way a completed turn is,
since left raw a late turn's call_ts restarts near zero and timeline_detail
would report it as overlapping the first.
DashboardRunResult's comment claimed CHAT_ERROR was unreachable here because the
loop had no try/except. That had it backwards: the missing try/except was the
bug, not a reason the fault could not occur. SIMULATED_USER_FAILED really is
unreachable -- the follow-up is built by build_simulated_reply with no judge
call to fail -- and the comment now says so.
Three tests, each verified to fail without the fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>1 parent 9dd2b28 commit b6d35c5
2 files changed
Lines changed: 80 additions & 4 deletions
File tree
- packages/gooddata-eval
- src/gooddata_eval/core/agentic
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
| 31 | + | |
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| |||
667 | 667 | | |
668 | 668 | | |
669 | 669 | | |
670 | | - | |
671 | | - | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
672 | 673 | | |
673 | 674 | | |
674 | 675 | | |
| |||
879 | 880 | | |
880 | 881 | | |
881 | 882 | | |
882 | | - | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
883 | 910 | | |
884 | 911 | | |
885 | 912 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
1104 | 1105 | | |
1105 | 1106 | | |
1106 | 1107 | | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
| 1119 | + | |
| 1120 | + | |
| 1121 | + | |
| 1122 | + | |
| 1123 | + | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
| 1134 | + | |
| 1135 | + | |
| 1136 | + | |
| 1137 | + | |
| 1138 | + | |
| 1139 | + | |
| 1140 | + | |
| 1141 | + | |
| 1142 | + | |
| 1143 | + | |
| 1144 | + | |
| 1145 | + | |
| 1146 | + | |
| 1147 | + | |
| 1148 | + | |
| 1149 | + | |
| 1150 | + | |
| 1151 | + | |
| 1152 | + | |
| 1153 | + | |
| 1154 | + | |
| 1155 | + | |
1107 | 1156 | | |
1108 | 1157 | | |
1109 | 1158 | | |
| |||
0 commit comments