Repository navigation
Conversation
This was referenced Oct 2, 2026
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…lls any more 30257df (Android device queries match the whole text and id) took out the last caller of this helper in the uiautomator2 driver. The unused linter has flagged it since, so Lint has failed on every push to main, and Build, which needs Lint, has been skipped each time. The same helper in the appium and devicelab drivers is untouched.
Maestro runs a runScript file as plain JavaScript. It reads the file and
evaluates ${...} only in the step's env, when: condition and label; the
script text goes to the engine as it is (YamlFluentCommand.kt:409-424,
Commands.kt:1029-1035, Orchestra.kt:723-737).
The runner expanded ${...} and $VAR across the whole file before running
it. A template literal that used the script's own variables was replaced
ahead of the script, against variables that did not exist yet:
`/v1/x?email=${encodeURIComponent(who)}&state=${state}` came out as
`/v1/x?email=undefined&state=`.
A script file now runs as written. Inline script text, which Maestro has no
equivalent of, keeps the expansion.
Maestro gives a sub-flow its own env scope: enterEnvScope saves the env and
leaveEnvScope puts that copy back, so a key the sub-flow added is gone when
it returns (GraalJsEngine.kt:223-238, around runSubFlow at
Orchestra.kt:1159-1197, which repeat, retry and runFlow all go through).
The runner's withEnvVars restored each key to the value it had before, and
a key that had none was set to "" rather than removed. After a runFlow,
retry or sub-flow with `env: {KEY: ...}`, KEY stayed defined: typeof KEY was
"string", `$KEY` expanded to nothing, and runShell saw KEY="" in its
environment.
withEnvVars now uses applyScopedEnv, which runScript's env already used:
a key is restored when it existed and removed when it did not.
Maestro builds its script http client with 5-minute read, write and call timeouts (GraalJsEngine.kt:28-34), and the CLI passes no client of its own (Orchestra.kt:138 and 161), so every call gets them. The runner's http.* gave a call 30 s unless the call set `timeout`, so a script that calls a slow endpoint (seeding test data, waiting on a server-side job) failed with "HTTP request failed" where Maestro waits. A local server that answered after 32 s failed the call at 30 s. A call without a timeout option now gets 5 minutes. The option still wins.
On a physical iPhone reached through usbmux and an SSH tunnel, WDA began dropping connections 90 minutes into a run: about one request in fourteen came back as EOF within about 10 ms, nearly all of them element reads sent in parallel (displayed, text, rect, name). Most were harmless, but a dropped text read made a copyTextFrom come back empty and failed the flow. A fresh WDA dropped none, in seven other runs. net/http does not help here. It sends a request again by itself only when it is a GET on a connection that had been used before (shouldRetryRequest), so a fresh connection that is hung up on, and any POST, come back as errors. A GET, or a POST that only finds elements, whose connection dies before any response (EOF, reset, broken pipe) is now sent once more, with a warning in the log. An action (a tap, typing, a swipe, launching an app) is never repeated: it may have reached WDA before the connection went. One more try, not a loop. The changelog entry for keeping idle connections said those requests are sent again. That is only true with this change, so the entry is reworded.
Maestro evaluates each ${...} of the value and then reads the text as false
only when it is blank, "false" in any case, "undefined", "null" or a number
equal to zero, so "abc" or "YES" is true (Orchestra.kt:1030-1052, after
Condition.evaluateScripts at Condition.kt:15-21). An undeclared name
evaluates to undefined (GraalJsEngine.kt:198-204).
The runner took a string as true only when it was exactly "true", so
`assertTrue: ${name}` failed for any other value. A when: or while: value
fared worse: ExpandCondition replaced the ${...} with its value, and
EvalCondition then ran that value as JavaScript again. "abc" became a
ReferenceError and "YES" an undefined name, both false, and an unset
variable expanded to "", which CheckCondition took as no condition at all,
so `when: true: ${UNSET}` ran the branch Maestro skips.
A string result now goes through Maestro's rule. A condition that is one
${...} is no longer expanded ahead of the check: EvalCondition evaluates the
expression once and reads its value. Every undeclared name in a condition
is undefined, not only the ALL_CAPS ones, so `${maybe || 'x'}` keeps working
now that it is evaluated there. A condition with text around its ${...}
still expands and runs as JavaScript, as before.
Maestro runs onFlowComplete in the flow's finally, and a failing hook fails a flow that had passed, while a flow that had failed keeps its own error (Orchestra.kt:236-272). The hook runs through executeCommands, so it stops at its first failing step that is not optional. When onFlowStart fails, the body is skipped and onFlowComplete still runs. The runner ran onFlowComplete in a defer that ignored every failure, after the flow had been reported, so a flow whose cleanup failed passed. onFlowComplete now runs before the flow's result is final, after the body and after a failing onFlowStart alike. A failing step that is not optional ends the hook and fails a flow that had passed. A panic still runs the hook, as the defer did. The step counts now include the hook's steps, as they do onFlowStart's. The onFlowStart failure path also marks the flow failed for the recording, which --video on-failure had discarded as a passing flow's.
7 of 13 tasks
bulatgaleev
force-pushed
the
upstream-pr/b2-onflowcomplete-fails-flow
branch
from
October 2, 2026 04:07
65a59c6 to
99d82bf
Compare
Contributor
|
Thanks @bulatgaleev. This closes a real gap: failing cleanup used to pass silently. It now matches Maestro's |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Maestro runs
onFlowCompletein the flow'sfinally(runFlowin Orchestra.kt, at cli-2.10.0). The hook runs throughexecuteCommands, so it stops at its first failing step that is not optional. A failing hook fails a flow that had passed, and a flow that had failed keeps its own error. WhenonFlowStartfails, the body is skipped andonFlowCompletestill runs. maestro-runner ranonFlowCompletein a deferred call that ignored every failure, after the flow had already been reported, so a flow whose cleanup failed passed. The hook now runs before the flow's result is final, after the body and after a failingonFlowStartalike, and a failing step that is not optional ends the hook and fails a flow that had passed.Type of Change
Changes Made
FlowRunner.Runcalls the newrunFlowCompleteHooksbefore the result is final. It stops at the first failing step that is not optional and returnsonFlowComplete failed: <error>, which fails a flow that had passed. A flow that had failed keeps its own error.onFlowStart's are.onFlowStartis now marked failed for--video on-failure. Its status stayedpassedthere, so the recording was thrown away.pkg/executor/flow_hooks_test.go: seven tests. A failing hook fails a passing flow, and is reported after the hook ran. An optional failing step may fail. The hook stops at its first failing step. A failed body keeps its error. A failingonFlowStartstill runs the hook. A panic still runs it. The failed flow's recording is kept. Five of the seven fail without the change.Related Issues
No existing issue found.
Testing
go test ./pkg/executor/go test -raceforpkg/executor,pkg/driver/wda,pkg/jsengine,pkg/flowandpkg/driver/uiautomator2passes at the top of the stack;go vet ./...,gofmt -land golangci-lint v2.13.2 (the version CI pins) report nothing on every branch of the stackmake testas a whole: not run, because its device tests drive whatever device is attached to the machinemake lint: the Makefile has nolinttarget, and the other lintersmake checkruns (staticcheck, revive, errcheck, nilaway, gosec) are not installed hereChecklist
Additional Notes
A flow whose
onFlowCompletehas a failing step used to pass and now fails, as it does under Maestro. Cleanup that is allowed to fail is markedoptional: true.Stack 7 of 7: #200 → #194 → #195 → #196 → #197 → #198 → #199. Merge in that order. This branch is built on #200, #194, #195, #196, #197 and #198, so it also contains their commits. This PR's own change is the top commit,
99d82bf. Once the PRs below it merge, the rest of the diff disappears, and nothing conflicts. All seven sit onmainatd3f738f(the merge of #185).