OT-2 (legacy): fix the position read, and bound every request instead of blocking forever - #1240
Draft
miikee wants to merge 2 commits into
Draft
OT-2 (legacy): fix the position read, and bound every request instead of blocking forever#1240miikee wants to merge 2 commits into
miikee wants to merge 2 commits into
Conversation
_current_channel_position called ot_api.lh.save_position, which ot_api does not define, so every caller raised: move_channel_x, move_channel_y and move_channel_z could not work at all. It now enqueues savePosition itself and polls the command the way ot_api's own command wrapper does. The poll awaits rather than sleeping, so a robot that never answers does not hold the event loop for the whole 30s budget. With the read working, move_channel_to moves a channel to an absolute position and holds whichever axes are left out. Chaining the per-axis calls was the only way to reach an arbitrary point before, and each of those descends separately, so a three-axis move could clip labware between the steps. This lifts to the traversal height and travels once. get_channel_position exposes the read. Not verified on an OT-2. The savePosition command and its response shape are the same ones a Flex uses, where they are hardware verified.
…wn the command queue ot_api reaches the robot through urlopen and passes it no timeout, so a request the robot never answers blocked the process for good. All 24 ot_api calls are issued from an async def and ran inline, so that block also froze every other device's coordination behind it. Every call now goes through one _request helper. A daemon thread keeps the loop free, asyncio.wait_for ends the wait at a real deadline, and a lock keeps commands one at a time the way the blocking calls used to for free. The thread is not the loop's shared executor, because asyncio.run() joins that executor on the way out: an unanswered request would move the hang from mid-command to shutdown and burn a pool slot every other backend shares. Three budgets on the constructor, named to match the Flex backend: request_timeout for one request/response, command_timeout for a command that waits on motion, status_poll_interval between status reads. The budget wraps the lock as well as the request, so a read documented at 7.5s cannot silently block for as long as whatever is ahead of it. Non-positive values are refused. Robot commands are enqueued and polled here rather than through ot_api's @command decorator, whose 30s ceiling no caller can raise and whose poll loop has no sleep in it. command_timeout is the real ceiling now, so a mix declared at ten minutes is not killed at thirty seconds inside ot_api. A command that times out is contained rather than abandoned: the run is stopped so what is still queued cannot execute, and further commands are refused until setup() starts a fresh run. Retrying an aspirate used to make the pipette aspirate twice from one well. stop() halts the run through the endpoint the robot-server actually has; the two it used before do not exist, so it spent two full request budgets on 404s. The give-up type is builtins.TimeoutError on every supported version. asyncio.wait_for raises asyncio.exceptions.TimeoutError below 3.11, which is not an OSError, so a downstream "except TimeoutError" missed every OT-2 timeout on 3.9 and 3.10 and caught it on 3.11+. _save_position and _current_channel_position are async now, and the simulator's override follows. The chatterbox's canned savePosition data returned None, so a dry run could not read a channel back at all; it now answers with the position the recorded moves sent the arm to. The tests fake the robot at its command queue, which is where the backend reaches it, so they pin the params a real robot would receive.
This was referenced Sep 3, 2026
Closed
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits on the legacy OT-2 backend. The second builds on the first, so they
are one PR.
1.
_current_channel_positionis broken today. It callsot_api.lh.save_position, whichot_apidoes not define, so every callerraises.
move_channel_x,move_channel_yandmove_channel_zon this backendcannot work at all right now: they always fail with
RuntimeError: Failed to query current pipette position. It now enqueuessavePositionitself and pollsthe command the way
ot_api's own wrapper does, awaiting rather than sleeping.With the read working,
move_channel_tomoves a channel to an absolute positionand holds whichever axes are left out. Chaining the per-axis calls was the only
way to reach an arbitrary point before, and each of those descends separately,
so a three-axis move could clip labware between the steps.
2. Every request is bounded and off the event loop.
ot_apireaches therobot through
urlopenand passes it no timeout, so a request the robot neveranswers blocks the process for good. All 24
ot_apicalls are issued from anasync defand ran inline, so that block also froze every other devicecoordinating behind it.
Every call now goes through one
_requesthelper: a daemon thread keeps theloop free,
asyncio.wait_forends the wait at a real deadline, and a lock keepscommands one at a time the way the blocking calls used to for free. The thread
is deliberately not the loop's shared executor, because
asyncio.run()joinsthat executor on the way out.
Three budgets on the constructor, named to match the Flex backend:
request_timeout,command_timeout,status_poll_interval.Robot commands are enqueued and polled here rather than through
ot_api's@commanddecorator, whose 30s ceiling no caller can raise and whose poll loophas no sleep in it at all. A command that times out is contained rather than
abandoned: the run is stopped so what is still queued cannot execute. Retrying
an aspirate used to make the pipette aspirate twice from one well.
Not verified on an OT-2 -- I do not have one. The
savePositioncommand andits response shape are the same ones a Flex uses, where they are hardware
verified. I diffed all seven command payloads against the installed
ot_apiand they are byte-identical, and the tests fake the robot at its command queue
so they pin the params a real robot would receive.
Worth your call: this moves knowledge of the Opentrons HTTP command payloads
into PyLabRobot rather than leaving it in
ot_api. I think that is right, sincethe payloads are protocol facts and PLR was already pinned to
ot_api's bugs,but it is a coupling decision rather than a detail, so say if you would rather
it stayed out.
413 tests passing.
🤖 Generated with Claude Code