smoke: wait until grcli events has subscribed - #753
Merged
Merged
Conversation
Comment on lines
+489
to
+490
| local end=$(wc -l < $tmp/grout.log) | ||
| local level=$(grcli -j log level show | jq -re '.[] | select(.name == "grout.api") .level') |
There was a problem hiding this comment.
1. Smoke setup fails without local logs 🐞 Bug ☼ Reliability
_start_events_stream reads $tmp/grout.log even when _start_grout was not used to create that file. With the default event-following mode, tests connected to an existing daemon and tests launched under the debugger exit at this command before running their test body.
Agent Prompt
## Issue description
Event startup assumes `$tmp/grout.log` exists, but existing-daemon and debugger modes do not call `_start_grout` and therefore never create or populate it.
## Fix Focus Areas
- smoke/_init.sh[444-490]
## Recommended Fix
Make every supported startup mode expose daemon logs through `$tmp/grout.log`, or use a subscription readiness mechanism that does not depend on that local file. Preserve support for both `run_grout=false` and `GDB=true`.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+512
to
+514
| { tail -f -n +$((end + 1)) $tmp/grout.log || : ; } | | ||
| timeout 10 grep -qE 'GR_EVENT_SUBSCRIBE.*Success' || | ||
| fail "timeout after 10s waiting for grcli events" |
There was a problem hiding this comment.
2. Event startup can hang indefinitely 🐞 Bug ☼ Reliability
The readiness pipeline waits for the tail -f producer after grep -q exits, while the shell's ignored pipe signal means the producer only notices the closed reader on a later write. If the subscription message is the final log record, the function never reaches the level restoration, and the stated ten-second timeout does not bound the whole pipeline.
Agent Prompt
## Issue description
The timeout covers only `grep`, while the shell still waits for the persistent `tail -f` process after a successful match.
## Fix Focus Areas
- smoke/_init.sh[512-516]
## Recommended Fix
Run the log follower under explicit lifecycle management: capture its PID, wait for the bounded matcher, and terminate and reap the follower immediately on either success or timeout. Preserve the matcher status before cleanup so timeout still calls `fail`.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+512
to
+514
| { tail -f -n +$((end + 1)) $tmp/grout.log || : ; } | | ||
| timeout 10 grep -qE 'GR_EVENT_SUBSCRIBE.*Success' || | ||
| fail "timeout after 10s waiting for grcli events" |
There was a problem hiding this comment.
3. Another subscriber can bypass the wait 🐞 Bug ≡ Correctness
The readiness check accepts any successful GR_EVENT_SUBSCRIBE log record without verifying the logged client PID belongs to the newly launched event command. During daemon restart, the surviving routing process reconnects and issues the same request, so its subscription can satisfy the check while the event command is still racing with subsequent interface creation.
Agent Prompt
## Issue description
The readiness expression cannot distinguish the smoke event client from other processes subscribing concurrently, particularly the routing process reconnecting after restart.
## Fix Focus Areas
- smoke/_init.sh[498-514]
- main/api.c[335-345]
## Recommended Fix
Launch `grcli events` so its actual process PID is captured, then require both that PID and the successful subscription request in the daemon log. Ensure pipelines or process substitutions do not replace the captured PID with that of `tee` or `awk`.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
The event stream is started in the background and races with interface creation. If the subscription lands after an interface is created, its add event is missed and grcli prints the interface by its numeric id, which breaks wait_event pattern matching. This happens regularly on slow CI runners and never locally. Redirect the grout logs to files and tail them, so the harness can watch them. When starting the event stream, temporarily raise the grout.api log level to debug and block until the daemon logs the successful GR_EVENT_SUBSCRIBE before returning. Interfaces are then only created once the stream is guaranteed to receive their events. Signed-off-by: Robin Jarry <rjarry@redhat.com>
rjarry
force-pushed
the
arp-ndp-smoke-fix
branch
from
September 17, 2026 18:05
3b634c9 to
43fb6e6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The event stream is started in the background and races with interface creation. If the subscription lands after an interface is created, its add event is missed and grcli prints the interface by its numeric id, which breaks wait_event pattern matching. This happens regularly on slow CI runners and never locally.
Redirect the grout logs to files and tail them, so the harness can watch them. When starting the event stream, temporarily raise the grout.api log level to debug and block until the daemon logs the successful GR_EVENT_SUBSCRIBE before returning. Interfaces are then only created once the stream is guaranteed to receive their events.