qt-auto-test

A CLI tool for AI agents to control and test Qt6 GUI applications on Linux.

Two components work together:

• libqt-auto-test.so — shared library injected into the target Qt6 process via LD_PRELOAD or ptrace. Runs a Unix-socket agent inside the process.
• qt-auto-test — CLI client that connects to the agent socket and issues commands, printing JSON to stdout.

Dependencies

Dependency	Version	Purpose
Qt6 (Core, Gui, Widgets, Test)	≥ 6.0	Agent library
CMake	≥ 3.16	Build system
GCC or Clang	C++17	Compiler
libX11	any	Window-title lookup in CLI
nlohmann/json	≥ 3.11	JSON (fetched automatically if absent)

Install on Debian/Ubuntu:

sudo apt install qt6-base-dev qt6-base-dev-tools cmake build-essential libx11-dev
# optional — avoids downloading nlohmann/json at configure time:
sudo apt install nlohmann-json3-dev

Build

cmake -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build --parallel $(nproc)

Build outputs:

Path	Description
build/cli/qt-auto-test	CLI binary
build/lib/libqt-auto-test.so	LD_PRELOAD agent library
build/tests/test_app/test_app	Minimal Qt6 app for testing

Install

cmake --install build --prefix /usr/local

Installs to $prefix/bin/qt-auto-test, $prefix/lib/libqt-auto-test.so, and $prefix/share/man/man1/qt-auto-test.1.

Usage

Launching a new application

qt-auto-test launch -- /path/to/myapp --any-app-flags

Sets LD_PRELOAD and execs the app. The -- separator is required.

Attaching to a running process

qt-auto-test inject --pid 4321
# or by process name:
qt-auto-test inject --name myapp

Uses ptrace to call dlopen inside the target. Requires either CAP_SYS_PTRACE or ptrace_scope ≤ 1. x86-64 only.

Inspecting widgets

# All widgets with screen positions (stable IDs for the session)
qt-auto-test widgets --name myapp

# Text content — same IDs as above, joinable on "id"
qt-auto-test text --name myapp

Screenshots

# Full primary screen
qt-auto-test screenshot --name myapp --output /tmp/screen.png

# Single widget
qt-auto-test screenshot --name myapp --widget w0004 --output /tmp/btn.png

# Pixel region (screen coordinates)
qt-auto-test screenshot --name myapp --region 100,200,400,300 --output /tmp/crop.png

Full-screen capture may fail on Wayland without a screen-sharing portal. Widget-level capture (--widget) works on both X11 and Wayland.

Keyboard input

qt-auto-test send-keys --name myapp --widget w0007 --keys '<Ctrl+A>new text<Return>'

Key specs: <Return>, <Tab>, <Escape>, <Backspace>, <Delete>, <Up>, <Down>, <Left>, <Right>, <Home>, <End>, <PageUp>, <PageDown>, <F1>–<F12>, <Ctrl+X>, <Shift+X>, <Alt+X>, <Meta+X> and combinations like <Ctrl+Shift+Z>. Text outside angle brackets is typed literally.

Mouse

# Click a widget
qt-auto-test click --name myapp --widget w0004

# Click at screen coordinates
qt-auto-test click --name myapp --x 350 --y 210

# Double-click, right-click
qt-auto-test click --name myapp --widget w0004 --double
qt-auto-test click --name myapp --widget w0004 --button right

# Drag (screen coordinates)
qt-auto-test drag --name myapp --from-x 150 --from-y 310 --to-x 400 --to-y 310

# Scroll (positive = up, negative = down)
qt-auto-test scroll --name myapp --widget w0006 --delta -3

Context menus

# Right-click a widget to open its context menu
qt-auto-test click --name myapp --widget w0012 --button right

# Find the QMenu that appeared
qt-auto-test widgets --name myapp | jq '.widgets[] | select(.class=="QMenu")'

# List its items (text, enabled, shortcut, separator, has_submenu)
qt-auto-test actions --name myapp --widget w0045

# Trigger an item by name ("Copy" matches "&Copy" automatically)
qt-auto-test trigger-action --name myapp --widget w0045 --action-text Copy

# Or by zero-based index
qt-auto-test trigger-action --name myapp --widget w0045 --action-index 0

trigger-action calls QAction::trigger() directly — no synthetic mouse event, so the menu does not need to stay focused. Works with both QMenu::exec() (modal) and QMenu::popup() (non-modal) menus.

Detach

qt-auto-test detach --name myapp

Stops the agent and removes the socket. The target process continues running.

Target selectors

Every subcommand except launch accepts one of:

Flag	Resolved by
--pid <n>	Direct process ID
--name <n>	/proc/*/comm scan
--window <title>	X11 _NET_WM_NAME + _NET_WM_PID (requires EWMH-compliant WM)

JSON output

All commands that contact the agent print a JSON response. status is "ok" or "error". Exit code is 0 on success, 1 on error.

{ "status": "error", "error": "widget not found: w9999" }

Full documentation

man -l docs/qt-auto-test.1     # from the source tree
man qt-auto-test               # after install

Limitations

• Qt6 only. The agent library is compiled against Qt6; injecting into a Qt5 process fails at load time.
• x86-64 only for inject. All other subcommands are architecture-independent.
• One client at a time. The agent processes connections sequentially.
• Widget IDs are session-scoped. IDs reset after detach / re-inject.