Skip to content

Functest: keep_alive + wait_client_online after ACPI SHUTDOWN - #1088

Open
satishnaidu77 wants to merge 1 commit into
HCK-CI:masterfrom
satishnaidu77:feat-functest-wait-client-online
Open

Functest: keep_alive + wait_client_online after ACPI SHUTDOWN#1088
satishnaidu77 wants to merge 1 commit into
HCK-CI:masterfrom
satishnaidu77:feat-functest-wait-client-online

Conversation

@satishnaidu77

Copy link
Copy Markdown

Summary

Functest guest recovery after intentional ACPI / QMP SHUTDOWN so mid-batch NetKVM hotplug cases can continue.

Problem

Some NetKVM functest cases intentionally power off the guest (shutdown /s /t 0 /f) and wait for QMP SHUTDOWN. That exits QEMU.

Functest used to boot clients without keep_alive. After those cases, CL1 stayed dead. Post-test minidump probing hit WinRM (No route to host) and aborted the whole batch, so later cases never ran.

Testcases this fixes

Testcase Polarion / KAR Why it needs this
NetKVM/netkvm_hotplug_shutdown VIRT-95886 / KAR pci_hotplug.with_shutdown Hot-plug NIC, then clean guest shutdown + QMP SHUTDOWN
NetKVM/netkvm_hotplug_unplug_shutdown VIRT-95888 / KAR nic hotplug + shutdown after unplug Hot-plug, unplug, then clean guest shutdown + QMP SHUTDOWN
NetKVM/netkvm_hotplug_unplug_reboot (downstream in same run) VIRT-95889 / reboot after unplug Does not use SHUTDOWN itself, but fails to start if a prior shutdown case left QEMU dead

What we changed (FW) and why

Change Why
Boot functest clients with keep_alive: true (same idea as HCKTest) After ACPI/SHUTDOWN, restart QEMU so the guest can boot again mid-batch
New step wait_client_online: true Explicitly wait until WinRM is up after QEMU restart before next steps/minidump — dumps stay strict; if guest never returns, the step fails loudly
Docs + Tools stub for wait_for_client_online FunctestTools implements the wait; base Tools gets a clear stub

Companion case JSON updates (add wait_client_online after QMP SHUTDOWN) live in the functional-tests NetKVM cases.

Test plan

  • Re-run with --pcie-spare-root-ports 1:
    NetKVM/netkvm_hotplug_shutdown,NetKVM/netkvm_hotplug_unplug_shutdown,NetKVM/netkvm_hotplug_unplug_reboot
  • Confirm log shows Waiting for client CL1 to come online after both SHUTDOWN cases
  • Confirm 3/3 PASSED

Pass log

I, [2026-08-04T14:18:55.834075 #3193375]  INFO -- : Waiting for client CL1 to come online
I, [2026-08-04T14:19:34.838514 #3193375]  INFO -- : PASSED: netkvm_hotplug_shutdown
I, [2026-08-04T14:20:04.469208 #3193375]  INFO -- : Waiting for client CL1 to come online
I, [2026-08-04T14:20:43.474416 #3193375]  INFO -- : PASSED: netkvm_hotplug_unplug_shutdown
I, [2026-08-04T14:21:34.123596 #3193375]  INFO -- : PASSED: netkvm_hotplug_unplug_reboot
I, [2026-08-04T14:21:35.649054 #3193375]  INFO -- : TEST SUMMARY
I, [2026-08-04T14:21:35.649106 #3193375]  INFO -- : Total:  3
I, [2026-08-04T14:21:35.649127 #3193375]  INFO -- : Passed: 3
I, [2026-08-04T14:21:35.649149 #3193375]  INFO -- : Failed: 0

Command used:

./bin/auto_hck --id 58 functest -p Win2025x64 -d NetKVM \
  --driver-path <NetKVM/2k25/amd64> \
  --pcie-spare-root-ports 1 \
  --testcase NetKVM/netkvm_hotplug_shutdown,NetKVM/netkvm_hotplug_unplug_shutdown,NetKVM/netkvm_hotplug_unplug_reboot

Made with Cursor

Boot functest clients with keep_alive so QMP SHUTDOWN restarts QEMU,
and add an explicit wait_client_online step so WinRM is back before
the next mid-batch case (NetKVM hotplug shutdown paths).

Signed-off-by: Satish Tagirisapu <stagiris@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@kostyanf14

Copy link
Copy Markdown
Contributor

@satishnaidu77
I have another opinion about this issue. I got the same problem for QGA testing. Do not update the PR for now. I hope I can provide more extended support in a day or two.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves Functest resilience for cases that intentionally trigger guest shutdown (ACPI / QMP SHUTDOWN) by ensuring the client VM is restarted and the framework can explicitly wait for WinRM to come back before continuing the batch.

Changes:

  • Boot Functest clients with keep_alive: true so QEMU is restarted after guest shutdown mid-batch.
  • Add a new Functest step type wait_client_online and wire it through step-type validation and execution.
  • Document the new step type and add a base-tools stub API for wait_for_client_online.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
lib/models/command_info.rb Adds wait_client_online to the unified step/command model.
lib/engines/hcktest/tools.rb Adds a stub for wait_for_client_online in the tools interface used across engines.
lib/engines/functest/step_handler.rb Recognizes wait_client_online as a step type for “exactly one step-type field” validation.
lib/engines/functest/functest.rb Boots functest clients with keep_alive: true to survive QEMU exits after shutdown.
lib/auxiliary/command_execution_manager.rb Executes wait_client_online by calling tools’ wait_for_client_online.
docs/Functest-Engine.md Documents wait_client_online and updates step-type/client-targeting docs accordingly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/Functest-Engine.md
}
```

If the guest never returns, the step fails (and the run stops) — dump collection stays strict and is not skipped.
Comment on lines +299 to +305
sig { params(command_info: Models::CommandInfo).void }
def execute_wait_client_online(command_info)
target_machines(command_info).each do |machine_name|
@logger.info("Waiting for client #{machine_name} to come online")
@tools.wait_for_client_online(machine_name)
end
end
@satishnaidu77

Copy link
Copy Markdown
Author

@satishnaidu77 I have another opinion about this issue. I got the same problem for QGA testing. Do not update the PR for now. I hope I can provide more extended support in a day or two.

@kostyanf14
Okay..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants