Repository navigation
fix(gateway): log the capture status a request ends with - #5981
Merged
ZaynJarvis merged 2 commits intoOct 15, 2026
Merged
Conversation
t0saki
marked this pull request as ready for review
October 15, 2026 04:26
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
The request log row carried the capture status read when the request was prepared. A long request that started while delivery was retrying finished after the worker's delivery_recovered row, so its stale "retrying" became the newest status for the session and /admin/overview kept counting the capture in capture_issues until another request arrived. Re-read the capture document just before the row is written.
t0saki
force-pushed
the
fix/gateway-capture-status-refresh
branch
from
October 15, 2026 10:05
a6343cc to
469fcca
Compare
ZaynJarvis
approved these changes
Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
/admin/overviewcan keep reporting a capture asretryingafter its delivery recovered.Each request log row carries the
capture_statusread when the request was prepared (kernel.prepare/CapturePipeline.confirm), but the row is written only when the request finishes (ProxyRequest.finish).overview_summarytakes the newest row per (session, protocol). Here is how it goes wrong:errorset, so a request prepared now recordsretrying.errorand logs thekind=capturerowactive / delivery_recovered.retrying. That row is newer than the recovery row.capture_issues.retrying = 1until another request on that session logsactive. If no request comes, it stays that way.This PR re-reads the capture document just before the request row is written and recomputes the status with the existing
CapturePipeline.metrics. The row now records the status at the time it is logged. A recovery during the request no longer leaves a staleretrying, and a failure during the request no longer leaves a staleactive. One small window remains: if the worker inserts its capture row between the re-read and the request row's INSERT, the request row is still the newest. That window is a few milliseconds instead of the whole request duration (25s in the ovtest case). The cost is one capture read per enhanced request.The re-read has its own
try. If it fails, the row keeps the capture fields computed at prepare time, and the request row (token usage, duration, recall metrics) is still logged.I rejected counting only
kind=capturerows in the overview.reset_capture, the deleted-key path and ahistory_changedrestart all clearerrorwithout writing a capture row, so the overview would keep showingretryingafter any of them.Found by the ovtest integration run on OpenViking main
92285e5(2026-10-13), casejourney-g13-d-sdk-chat-to-sdk-chat-request-not-reached. After an injected 503, the retriedmessages/batchreturned 200 and the capture log wrotedelivery_recovered. A turn slowed by a 25s search delay finished after that, and the overview three seconds later still showedcapture_issues{paused:0,retrying:1}.Human Involvement
Related Issue
Related: #5960 also touches
proxy.pyand movesoverview_summary/ renamesmanagement.log. It does not fix this issue. Whichever PR lands second needs a small rebase.Type of Change
Changes Made
openviking_gateway/proxy.py: infinish(), re-read the capture document and recompute the capture fields withCapturePipeline.metricsbeforemanagement.log. A failed re-read is logged and falls back to the prepare-time fields.tests/gateway/test_app.py: addtest_overview_counts_capture_recovered_during_a_request. It seeds a retrying capture, clears it and logsdelivery_recoveredwhile the next request is upstream, then asserts that the overview reports no capture issues.tests/gateway/test_app.py: addtest_capture_status_refresh_failure_still_logs_request. It makes the re-read fail and asserts that the request row is still written with the prepare-timecapture_status.Testing
Checklist