Skip to content

Fall back to wall-clock time in rate limiter without video metadata - #3307

Draft
hansent wants to merge 3 commits into
mainfrom
fix/rate-limiter-wall-clock-fallback
Draft

hansent wants to merge 3 commits into
mainfrom
fix/rate-limiter-wall-clock-fallback

Conversation

@hansent

@hansent hansent commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

What changed and why

roboflow_core/rate_limiter@v1 is meant to fall back to datetime.now() when video_reference_image is not a video frame. It never did: WorkflowImageData.video_metadata doesn't raise for an image without metadata. Instead it returns a placeholder with frame_number=0, fps=30, so the try/except never fell back. The limiter computed time 0 on every run and, after the first run, suppressed next_steps forever. This affects still images (HTTP requests) and images from steps that build a new WorkflowImageData without copying video metadata (stitch_images@v1, grid_visualization@v1, stability_ai_*). It is the follow-up split out of #3296, where Camera Calibration dropping metadata exposed it.

  • Add WorkflowImageData.has_video_metadata(), a public check for real (producer-supplied) video metadata, so blocks don't read _video_metadata.
  • The limiter now uses:
    • wall-clock time when there is no reference image or it has no video metadata;
    • video time (frame_number / fps) only for comes_from_video_file=True with a non-zero FPS;
    • frame_timestamp otherwise (streams, unknown source, FPS missing or 0). This is the same rule Time in Zone and Velocity use.
  • Keep the last-execution time as float seconds, so naive wall-clock and timezone-aware frame timestamps never get subtracted from each other.
  • Update the block's docs and field descriptions.

Behavior change for streams: frames with comes_from_video_file set to False/None that report an FPS used to be timed by frame_number / fps. They now use frame_timestamp. Video files are unchanged.

Execution engine changelog: not needed. has_video_metadata() is an additive accessor and changes no compile or run behavior (see .cursor/rules/execution-engine-version-changelog.mdc). The fix is recorded under ## Unreleased → ### Fixed in workflows/CHANGELOG.md.

Tests

  • New multi-frame unit tests in workflows/tests/unit_tests/core_steps/control_flow/test_rate_limiter.py, with wall-clock time simulated by patching the module's datetime:
    • an image without metadata, and no reference image (wall-clock path);
    • a video file at 1 FPS, frames 1, 3, 5 (video time; the wall clock is held constant);
    • stream, unknown source, FPS 0 and FPS missing (frame_timestamp path).
  • has_video_metadata() test in workflows/tests/unit_tests/execution_engine/entities/test_base.py.
  • The new tests fail against the old block. Its datetime.fromtimestamp call hits the patched datetime, so they error out rather than show the old output. To show the bug itself, a script with real time ran still images 0.15 s apart with a 0.1 s cooldown: the old block let only the first through ([True, False, False, False]), the fixed one let all four through.

Commands run:

cd workflows && uv sync --locked --extra test && .venv/bin/python -m pytest tests/unit_tests/core_steps/control_flow tests/unit_tests/execution_engine/entities/test_base.py
# 133 passed, 1 skipped
uvx black==26.10.1 <changed files> && uvx isort==5.13.2 <changed files>

Not run: the server integration test tests/workflows/integration_tests/execution/test_workflow_with_rate_limiter.py, which needs a model. It uses no video_reference_image, so its path is the unchanged wall-clock one.

Notes

  • Not addressed: the limiter keeps a single last-execution time shared by every video, and doesn't reset when video time goes backwards (a new file or a restart).

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely.

🚧 Right now this is a draft, so automated Claude review is paused — nothing is being spent yet. Mark it Ready for review to trigger it.

Warning

💸 The Claude reviewer bills in credits, not vibes

Automated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared.

Draft when unsure, Ready when you mean it:

  • 🌱 Not sure the PR is in good shape yet? Keep it (or set it back) as a draft — drafts pause review, so you can push and iterate without burning credits on a moving target.
  • 💪 Feel strong about the contents? Mark it Ready for review and the reviewer will take a look.

However you get there, arrive prepared:

  • 🧱 Bring a SOLID, thorough PR. Point your local agent at our skills/ to tune it to our guidelines first — or, if you are one of those fabled carbon-based contributors, read them yourself. A half-baked diff costs exactly the same to review as a finished one.
  • ✅ Resolve every comment before you re-request review. Re-requesting with threads still open means paying twice for the same conversation.
  • 🔁 Do not use CI review as an inner loop for a local agent. The reviewer is not a step-by-step debugger — do the unfolding locally and arrive with the answer, not the search.
  • 🙋 If something looks off, ask a human. One question to a maintainer is cheaper and faster than three rounds of agent re-review chasing a misread.

Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.

  • Prefer to skip automated review entirely? Add the skip-claude-review label.

…-clock-fallback

# Conflicts:
#	workflows/roboflow_workflows/core_steps/flow_control/rate_limiter/v1.py
…-clock-fallback

# Conflicts:
#	workflows/CHANGELOG.md

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant