Repository navigation
Support configurable mask grids in RF-DETR Triton postprocessing - #3154
Merged
Merged
Conversation
…factor mask_decode_mode and tradeoff_factor are accepted end to end and then ignored: they pass untouched through InferenceModelsInstanceSegmentationAdapter.map_inference_kwargs and are swallowed by **kwargs in each model's post_process. Masks are always produced at full input resolution. The plan proposes a mask_resolution_factor in [0,1], resolved in the adapter from the existing enum, carried as explicit metadata rather than inferred from array shape. Default 1.0 is byte-identical to today. Measured on 4K with 300 instances: polygon path 889 ms -> 92 ms, RLE path 3107 ms -> 271 ms, peak RSS 3256 MB -> 595 MB at t=0.25. On Jetson Orin NX the stage currently moves ~40 GB per frame. Documents the blocking prerequisite: there is no mask-to-image coordinate scaling in the inference_models response path, so reduced-resolution masks would silently corrupt every polygon and RLE payload.
The existing coverage of align_instance_segmentation_results compares chunked output against monolithic output. Both sides move together when the resize target changes, so neither test can detect a change in output geometry. crop_masks_to_boxes had no coverage at all. Add characterization tests that pin the current contract: masks sized to size_after_pre_processing, bool dtype, axis order on a non-square image (a transpose is invisible on a square one), a golden per-instance pixel count from a fixed seed, and that the static-crop paste relocates onto a larger canvas without changing the set-pixel count. Add first coverage for crop_masks_to_boxes, including one test documenting that `scaling` is a single scalar applied to both axes. Add scale_polygons_to_image, which lifts contour coordinates from mask space into image space, scaling each axis independently since the two need not share an aspect ratio. The legacy path does this via post_process_polygons / scale_polygons (inference/core/utils/postprocess.py); the inference_models rewrite dropped it, which is harmless only while masks happen to be image-sized. No caller is changed here. Tests: python -m pytest tests/unit_tests/models/common/roboflow/test_post_processing.py 45 passed (31 pre-existing, 14 added).
Supersedes the earlier plan, whose headline figures combined two code paths that never execute together and whose recommended v1 is measurably slower than today on the default RLE path. Records the team decisions from review: new block versions are the delivery mechanism, mask_size as carried metadata is accepted in principle, and the RLE wire-format decision is deferred to the end of implementation. Adds the TDD sequence, including why the existing chunking tests cannot detect a geometry change. Applies black and isort to the files touched by the previous commit.
Committed in error. This branch carries Python changes only; the files remain on disk, untracked.
Interpolate the mask resize target between the post-unpad mask grid and size_after_pre_processing, instead of always resizing to the latter. A factor of 1.0 is the default and short-circuits to the image dimensions, so round() never runs on the default path and output is bit-identical. The empty-detection branch and the static-crop canvas both follow the factor. The canvas keeps the mask grid's ratio to the image rather than planting a reduced mask on a full-size canvas, which would have been correctly placed but saved no memory; crop offsets move into the same space. Consumers that derive a canvas from mask.shape therefore get a consistent answer with and without detections. No caller passes the parameter yet. Tests: python3 -m pytest tests/unit_tests/models/common/roboflow/test_post_processing.py 58 passed (45 pre-existing, 13 added).
mask.shape[1:] and InstancesRLEMasks.image_size are currently the de facto definition of the mask canvas at roughly fifteen call sites. Once the resize target becomes adjustable they stop agreeing with the image, so the grid has to travel with the prediction rather than be inferred. Add mask_size to InstancesRLEMasks and InstanceDetections, each defaulting to the value it effectively had before: image_size for the former, the carrier's own grid for the latter. Every existing construction stays valid and every reader that ignores the field gets today's behaviour. to_coco_rle_masks still writes image_size as the COCO size field. The wire format is a separate, deferred decision and is untouched here. Tests: python3 -m pytest tests/unit_tests/models/base tests/unit_tests/models/common/roboflow/test_post_processing.py 77 passed.
Follows .cursor/rules as AGENTS.md requires: keyword-only secondary arguments via `*`, a blank line after guard clauses, and named return values instead of expressions on the return line. No behaviour change; every call site already passed these arguments by keyword.
Both parameters were accepted end to end and then ignored: they passed untouched through map_inference_kwargs and were swallowed by **kwargs in each model's post_process. Collapse the published enum onto the single resolution factor that post-processing consumes. accurate maps to 1.0, fast to 0.0, tradeoff reads tradeoff_factor; an absent mode stays at 1.0, so default behaviour is unchanged. Both source keys are consumed, since the deeper pre/post stages do not accept arbitrary kwargs. An unknown mode now raises InvalidMaskDecodeArgument, matching the legacy path, which previously accepted anything here. Scale polygon coordinates from mask space into image space where every mask branch converges, using the mask grid now carried on the detection. Equal sizes short-circuit to a no-op, so today's output is untouched; without this a reduced-resolution mask would report coordinates shrunk toward the origin. The legacy path does this via post_process_polygons; the inference_models rewrite dropped it. Tests: python -m pytest tests/inference/unit_tests/core/models/test_inference_models_adapters.py 37 passed (27 pre-existing, 10 added).
The dense path already took the factor but nothing passed it, so the parameter was reachable from a request and still inert. Thread it through all six instance-segmentation families - yolov5, yolov7, yolov8, yolo26, yolact and rfdetr - on both the dense and RLE routes, across the ONNX, TorchScript and TensorRT backends. Their post_process methods all accept **kwargs, so an un-threaded parameter is silently swallowed, which is how the original regression happened. Teach align_instance_segmentation_results_to_rle_masks the same interpolation the dense path uses, and scale its static-crop canvas and offsets to match, so mask_format no longer changes the resolution a caller receives. to_coco_rle_masks now declares the grid the counts were encoded on rather than the image size. Measured: pycocotools decodes an honest reduced size cleanly, decodes a mismatched one silently wrong, and sv.Detections.from_inference auto-resizes an honest one to the image. The previous behaviour - declaring image_size while encoding at size_after_pre_processing - is the mismatched case as soon as the two differ, so declaring the encoded grid is the correction, not the risk. RF-DETR's fused Triton post-process interpolates to the image inside the kernel and cannot honour a reduced target. It now reports mask_resolution_factor_unsupported and defers to the eager path instead of silently returning full-resolution masks. Passing the reduced output size into the kernel is the better answer and is left as a follow-up; it already parameterises its resize tables by output size. Default remains 1.0 and short-circuits before any arithmetic, so output is unchanged. Tests: python3 -m pytest tests/unit_tests -q 1597 passed. The 12 failures are pre-existing and environmental: five compare realpath-resolved against unresolved temp paths and cannot pass on macOS, seven need the optional sam3 package.
The first draft's headline figures combined two code paths that never execute together, and its recommended v1 was measurably slower than the status quo on the default RLE path. Both are corrected. Restructures to the repository implementation-plan template: one recommended solution, Before/After flows on the same 4K example, and Question/Investigation/Recommendation/Input for each open item. Records the team's review decisions - new block versions as the delivery mechanism, mask_size as carried metadata, RLE wire format settled at the end of implementation - and folds in the measured evidence that settled the wire question. Adds a sequencing section covering what is built, what remains and why the Triton gate belongs with the RLE work rather than before it. The TDD sequence is now an appendix rather than a separate document, so the plan and the test strategy stay in one file.
mask_decode_mode and tradeoff_factor are declared on every version of the block, and were forwarded to the server, which discarded them. Now that the server honours them, those versions would silently change the masks an existing workflow produces - a v3 step set to "fast" would start returning model-resolution masks on upgrade. Versions up to v4 therefore pin the outbound request to accurate/1.0, preserving the behaviour they have always had. Their manifests keep both fields, so existing workflow JSON still validates, but the fields are now deliberately inert rather than accidentally so. @v5 is the version where setting them does what their description says. v5.py and v5_tensor.py differ from their v4 counterparts in four places: the manifest version, the type literal, the class name, and the two outbound calls. The versioning mechanism is file copy rather than inheritance, so the two siblings share a class name and are selected by ENABLE_TENSOR_DATA_REPRESENTATION at import time. Tests cover the split - legacy versions pin, v5 forwards - and assert manifest parity between each version and its tensor-native sibling, which is held by convention today and nothing else. Also fixes three rfdetr backends where masks_resolution_factor was forwarded from post_process but declared on from_pretrained. flake8 F821 caught it; the TensorRT paths are not exercised by CPU tests. Tests: python3 -m pytest tests/unit_tests -q (inference_models) 1942 passed python3 -m pytest tests/workflows/unit_tests -q 1265 passed uvx flake8 --select=E9,F63,F7,F82 clean
The plan called reducing max_detections a cheaper, lossless first move. Measured, it is neither: halving it saves 55% where the resolution factor saves 90%, and capping at 150 in a scene with 300 objects deletes 150 real detections rather than degrading them. Replaces the claim with the data.
The plan called reducing max_detections a cheaper, lossless first move. Measured, it is neither: halving it saves 55% where the resolution factor saves 90%, and capping at 150 in a scene with 300 objects deletes 150 real detections rather than degrading them. Adds the fidelity curve against t, by object size, and states plainly what it is not. Aggregate mask IoU reads 0.908 at t=0.25 while boundary IoU is 0.654 and small objects are at 0.754, so evaluating this change on aggregate IoU alone reaches the wrong conclusion. Small objects pay roughly five times what large ones do, which makes a single recommended default workload-dependent. True AP_mask against ground truth cannot be produced in this workspace - there is no labelled set and no model weights - so the feature ships with a measured fidelity curve and no recommended value.
Phase 6 is split into what landed and what did not. Steps 1, 2 and 4 are done; the Triton kernel is gated rather than taught, and remote version skew is unhandled. Documents both deferrals with their impact. The Triton one is not free: the fused CUDA path is the fast path, so gating means t<1 falls back to the slower eager generator, and whether reduced-resolution-eager beats full-resolution-fused is unmeasured. Enabling the knob could make RF-DETR-seg slower on GPU. That needs one CUDA measurement before the knob is documented for GPU users. Narrows the version-skew item. run_locally uses the in-process model manager, so the block and the model layer are the same installation and skew cannot occur locally; only run_remotely can reach an older backend. The execution-engine compatibility guard does not cover this, since it gates engine schema semantics rather than post-processing behaviour. Drops the Author field, which git already records.
Four blockers and three majors from review, each reproduced before the fix and verified after. RLE-backed polygons were decoded on image_size while the counts describe the encoded grid. The adapter requests RLE whenever the model supports it, so this was the default polygon route and returned unusable contours. Both decoders now use the encoded grid. Static crop with a reduced factor could crash: the canvas size and the crop offset were rounded independently, leaving the canvas a pixel short of the pasted extent. The canvas now contains that extent. Only crops reaching the image edge under letterbox scaling expose it, which is why a sweep with zero padding found nothing. A static crop anchored at (0, 0) yields a mask smaller than the image whose coordinates are already image-space. Inferring "reduced grid" from mask_size != image_size scaled those a second time at the default factor. Scaling is now relative to size_after_pre_processing. The tensor-native local call sites were unpinned in v1-v4. All four are pinned. v5_tensor is pinned too: nothing under workflows/ reads mask_size and the sites that rebuild InstancesRLEMasks drop it, so reduced resolution is available in non-tensor mode only. The pin comment claimed these fields never took effect. That is false on the legacy ORT backend, which honours all three modes and is the default on Windows. Corrected in all twelve places and recorded in the changelog as a deliberate behaviour change there. The Triton dispatcher dropped the factor, so its gate was unreachable. to_supervision returned a reduced mask beside image-space boxes, and __iter__ declared the image size for counts on the encoded grid. Zero-detection results reported a shape the populated path would never produce. Both changelogs added. Plan document removed from the published docs tree. Formatter-only churn in three unrelated files reverted. Tests rewritten around the review's central point, that the first set could not fail: 24 golden cases captured from origin/main with non-zero padding throughout, dense-versus-RLE content comparison, the static-crop crash combination, RLE-backed polygons at reduced resolution, and a dispatcher spy rather than a direct call to the gate. Tests: inference_models 2016 passed, workflows 1274 passed, inference core/models 121 passed, flake8 clean.
Review raised that "fast" and "accurate" mislead when the image is smaller than the model's mask grid. resolve_mask_target_size interpolates between the mask grid at 0.0 and size_after_pre_processing at 1.0, so when the image is the smaller of the two a lower factor produces MORE mask pixels, not fewer: with a 160x160 grid and a 100x100 image, 0.0 yields 25,600 pixels against 10,000 at 1.0. States the endpoints explicitly in the function docstring and in the tradeoff_factor field description on both @v5 files, including that a lower factor does not guarantee smaller masks or lower latency, and that callers sensitive to this should compare size_after_pre_processing against the mask grid rather than assume a direction. Documentation only; no behaviour change.
Resolves three conflicts, all additive on both sides: an import block in the adapter tests, and Unreleased entries in both changelogs. Review raised that pinning v1-v4 unconditionally removes behaviour that currently works. The legacy backend honours mask_decode_mode and tradeoff_factor; only inference_models discarded them. Pinning everywhere traded one group's breakage for another's. The pin is now conditional on USE_INFERENCE_MODELS, so those versions keep passing the fields through on the legacy backend and behave exactly as they do today. No existing workflow changes behaviour on either backend. workflows/ cannot import inference.* - the isolation probe forbids it - so the flag travels as ModelsConfiguration.use_inference_models, bound in roboflow_workflows.environment and populated by the host from inference.core.env, matching how every other such value reaches the package. The tensor-native siblings stay pinned unconditionally. Their reason is downstream rather than the backend: nothing under roboflow_workflows/ reads mask_size and the sites rebuilding InstancesRLEMasks drop it, so a reduced grid is reinterpreted as image-sized whichever backend produced it. Tests: inference_models 2242 passed, workflows 1274 passed, inference core/models 129 passed. Twelve inference_models failures are pre-existing and environmental (triton is Linux-only; five path tests compare realpath-resolved against unresolved and cannot pass on macOS).
Completes the conditional pinning across the non-tensor block versions: v1-v4 now send accurate/1.0 only when USE_INFERENCE_MODELS is set, and pass the caller's values through otherwise. The legacy backend honours these fields, so those versions behave on it exactly as they do today. The tensor-native siblings stay pinned unconditionally. Their reason is downstream rather than the backend: nothing under roboflow_workflows/ reads mask_size and the sites rebuilding InstancesRLEMasks drop it, so a reduced grid is reinterpreted as image-sized whichever backend produced it. The comments now state which reason applies where. Documents ModelsConfiguration.use_inference_models in the changelog. It is a public field and a host serving through the legacy backend must set it to False, or v1-v4 stop forwarding fields that backend honours - reintroducing the regression this change exists to avoid. Applies black and isort. Eight files needed reformatting, all of them introduced by this branch; the equivalents on origin/main were already clean. Tests: inference_models 2242 passed, workflows 1274 passed, inference core/models 129 passed. black, isort and flake8 clean across all 47 changed Python files.
…m/MTaraszewski/inference into feat/instance-seg-tradeoff-factor
The release cut inference_models 0.39.1 and roboflow-workflows 0.2.4. Merging it dropped this branch's inference_models entries and moved its workflows entry into 0.2.4, attributing it to a release that does not contain it. Both merged without conflict, so neither was flagged.
…ce_segmentation/v3_tensor.py Co-authored-by: Grzegorz Klimaszewski <166530809+grzegorz-roboflow@users.noreply.github.com>
Co-authored-by: Grzegorz Klimaszewski <166530809+grzegorz-roboflow@users.noreply.github.com>
Place origin crops on the original-image canvas for dense and RLE outputs. Preserve tensors when encoding dense adapter results as RLE. Add real-model integration, HTTP server, and platform workflow regression coverage. Validated in the existing Python 3.12 .venv: 406 regression tests passed, 3 skipped; 6 ONNX and 3 Torch integration cases passed; 12 HTTP server cases passed; 6 local workflow checks passed using a public model. Black, isort, and git diff --check passed. TensorRT execution and authenticated staging E2E remain pending.
…ask-tradeoff # Conflicts: # docs/instance_segmentation_mask_resolution.md # inference_models/docs/changelog.md # inference_models/tests/unit_tests/models/rfdetr/test_triton_postprocess.py # workflows/CHANGELOG.md # workflows/roboflow_workflows/core_steps/common/inference_response_entities.py
dkosowski87
marked this pull request as ready for review
October 9, 2026 18:57
dkosowski87
requested review from
PawelPeczek-Roboflow,
grzegorz-roboflow,
hansent,
probicheaux,
rafel-roboflow and
yeldarby
as code owners
October 9, 2026 18:57
Contributor
|
👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely. 🚦 This PR is marked Ready for review, so automated Claude review will run — and every pass spends real credits. Warning 💸 The Claude reviewer bills in credits, not vibesAutomated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared. Draft when unsure, Ready when you mean it:
However you get there, arrive prepared:
Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.
|
PawelPeczek-Roboflow
approved these changes
Oct 9, 2026
Contributor
|
🤖 Claude review started at commit New commits are not auto-reviewed. Add the |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| "cuda_device_required" if factor == 0 else "mask_downsampling_unsupported" | ||
| ) | ||
|
|
||
|
|
Contributor
There was a problem hiding this comment.
High: none of the new CUDA parity tests run in CI.
These tests are the only coverage for the rewritten kernels: _box_coordinate_on_mask_grid, _selected_query_class_metadata_kernel, the per-query retry flag and the reduced-grid RLE path. They are gated only by skipif(not torch.cuda.is_available() ...) and carry no gpu_only / trt_extras marker. Nothing in CI collects them:
unit_tests_inference_experimental.ymlruns-m "not gpu_only"on CPU runners, so every one of them is skipped there.unit_tests_inference_experimental_gpu.ymlruns-m "gpu_only and trt_extras", so it deselects them.
This applies to test_triton_mask_grid_matches_reference_on_cuda, test_triton_native_and_large_source_grids, test_triton_global_topk_keeps_more_than_four_classes_per_query, test_ignored_classes_on_inactive_query_consume_global_topk_slots, test_triton_2xlarge_strided_queries_match_reference and test_triton_fractional_boxes_match_final_grid_reference. A later regression in box arithmetic, global top-k or mask-grid parity would merge with CI green. The Orin runs in the PR description are manual and can't be repeated by CI.
Fix: add @pytest.mark.gpu_only and @pytest.mark.trt_extras to each CUDA-gated test, as test_triton_object_detection_postprocess.py and test_triton_universal_preprocess_runtime.py already do. A module-level pytestmark won't work here because the file also has CPU tests. Please also mark this file's existing CUDA-gated tests while you are in it.
(Skill: review-topic-test-hygiene, Standard 4)
Contributor
|
@dkosowski87 — please answer the question below. @PawelPeczek-Roboflow, you approved this PR, so your input is needed too. ⏳ This PR is on hold pending your answers. The review will not advance to sign-off until the IMPORTANT question below is answered. IMPORTANT questions
New commits are not auto-reviewed. When you have answered and addressed the inline finding, add the Reviewed at HEAD: a52a201 |
Contributor
|
Claude review summary: not merge-ready. There is 1 blocking finding (inline) and 1 open IMPORTANT question (in the comment above). Skills: review-inference-models-pkg, review-topic-prediction-integrity, review-topic-test-hygiene No dedicated surface skill covers Blocking
Checked against the reference (
Tests to add: none beyond routing the existing CUDA tests into the GPU job (see the inline finding). Commands that informed this review: Reviewed at HEAD: a52a201 |
Contributor
|
Release coordination
Reviewed at HEAD: a52a201 |
grzegorz-roboflow
approved these changes
Oct 9, 2026
Collaborator
Author
The cost of around 10s latency on the first run is the Triton kernel compilation. The The memory regression is reasonable for the performance improvement and unblocking this path. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RF-DETR's Triton RLE postprocessor rejects mask-resolution factors other than
1.0, forcing reduced masks through the per-instance PyTorch/RLE loop. This change supports native and upsampled grids in the sparse Triton path, retaining reference fallback for antialiased downsampling and unsupported transforms.Aligned with merged #3156: boxes and masks share the final
mask_sizegrid insideinference_models. Both selection kernels map normalized boxes through network dimensions directly to that grid, preserving reference arithmetic and dtype boundaries. The shared finalizer retains float32 fractional boxes when the actual mask and image grids differ; equal grids use final integer rounding. Eager, multiclass, deferred, and empty results follow the same convention. Original frame metadata, HTTP opt-in behavior, andto_supervision()restoration retain the merged contract.Global top-k precedes class remapping, with one sparse mask per selected query and no four-class cutoff. Per-query retry flags avoid the shared-header race. Packed interpolation tables preserve reference coefficients, and kernels read mask strides without copying every query. Scope is the Triton postprocessor, its tests, and documentation; the existing enablement flag controls it. Profiling uses
implementation-selection-mode=base, with no execution-plan or model-forward change.Exact snapshots at factor
0.25match all 417 detections, including box dtypes/values and every mask pixel. At that factor, warm complete-inference latency improves, but cold first use and first-process RSS regress. This remains a draft and does not satisfy unconditional no-regression acceptance.In the results below, factor means
masks_resolution_factor, the model parameter selected from HTTPtradeoff_factorwhen tradeoff mode and reduced-mask opt-in are enabled. All results use RLE masks. The existing measurements cover:0.250.5and0.01.01.0The complete-inference campaign below uses factor
0.25throughout. Its baseline falls back to the non-Triton path at this factor. Base is 640×480/batch 1/confidence0.4; large is 1920×1080/batch 4/confidence0.01. Large timings are per four-image batch.0.25artifacts/base/latency.json0.25artifacts/base/latency.json0.25artifacts/base/memory.json0.25artifacts/base/memory.json0.25artifacts/large/latency.json0.25artifacts/large/latency.json0.25artifacts/large/memory.json0.25artifacts/large/memory.jsonAt factor
0.25, first-use measurements are complete first inference calls in fresh processes, not isolated compiler timers. Later processes reuse Triton's disk cache. Warmup moves this cost before serving; it does not remove it.Memory, full-resolution control, and trace details
Independent process RSS peaks and first-use times below are all at factor
0.25, with the same reference/fallback baseline as the complete-inference table. Peaks are retained individually:0.250.25The paired raw-output benchmark measures postprocessing only, on identical raw model outputs. Both workloads here use confidence
0.4and OpenCV-resized inputs: base is 640×480/batch 1, large is 1920×1080/batch 4. Its large workload differs from the complete-inference campaign's confidence0.01; do not combine their stage and end-to-end timings. There are 20 warmups and five alternating rounds of 50 calls per implementation.For factors
0.0,0.25, and0.5, the timed control directly calls the non-Triton reference implementation, the same path used when Triton postprocessing is disabled. The mode column describes current Triton; the reference control uses its ordinary synchronous implementation in both comparisons. All entries below are warm measurements; negative changes indicate lower latency.0.50.50.50.50.250.250.250.250.00.00.00.0Evidence:
rfdetr-mask-coords-benchmark-afcae76a-20261009/output/postprocess-benchmark.json. Exact parity passes in all 16 combinations, including factor1.0with the different control described next. Retained allocator growth is zero throughout. Allocation figures are incremental CUDA tensor peaks, not total process GPU memory; NVML process accounting is unavailable on this Jetson, and shared host/GPU RAM channels must not be added.At factor
1.0, the first full-grid postprocessing-only comparison showed median increases of 0.4–1.6%, within repetition spread. A longer repeat (10 alternating rounds × 100 calls per implementation, after 20 warmups) follows. Control is the previous Triton implementation, at5cfa38595219a736be92cd0d6948f32bd1e9c123, with Triton enabled. These results do not measure improvement over Triton disabled at1.0; no such timed baseline was collected. Both implementations have the same incremental CUDA allocation peaks: 393 KiB eager / 401 KiB deferred. Both sizes use confidence.4and OpenCV-resized inputs here; this differs from the large campaign's.01workload.1.01.01.01.0Nsight CUDA/NVTX evidence at factor
0.25, reference/fallback → current Triton, per inference (or batch):0.250.25The sparse RLE kernels execute in both workloads, replacing the reference per-mask resize/copy loop. Trace timing includes instrumentation and is not used as standalone latency. CPU sampling is unsupported; CUDA/NVTX capture remains valid. All five complete-inference latency repetitions at factor
0.25are retained. Host telemetry covers every selected repetition with no diagnostics or reported throttle events. CPU clocks stay at 2201 MHz; GPU clock metrics are unavailable. GPU temperature peaks are 52.03→52.59°C/base and 55.25→55.44°C/large. Median CPU/CV power is 7.34→7.32 W/base and 6.96→7.31 W/large; GPU/SoC power is 5.80→5.80 W/base and 13.09→14.24 W/large. These are named Jetson rails, not isolated GPU power.Validation (all 80 current-head PR checks pass):
884260f93, then 72 coordinate cases atafcae76a1. The first run's 72 failures were an immutable-metadata fixture assignment, fixed with_replace; no runtime code, expected values, or tolerances changed. Tests cover fp32/fp16/bfloat16, network resize and nonsquare dimensions, equal/reduced grids, empty/multiclass/deferred outputs,to_supervision(), all four factors, strided 300-query/192×192 masks, fallback and JIT handling.E9,F63,F7,F82), andgit diff --checkpass. Advisory flake8 retains the pre-existingboundary_activeF841.PYTHONPATH=inference_models python -m pytest -q \ inference_models/tests/unit_tests/models/rfdetr/test_triton_postprocess.py \ inference_models/tests/unit_tests/models/rfdetr/test_triton_jit_fallback.py \ inference_models/tests/unit_tests/models/common/roboflow/test_mask_image_dimensions.py \ inference_models/tests/unit_tests/models/base/test_instance_segmentation.py # Corrected coordinate fixture cases, run separately on afcae76a1: PYTHONPATH=inference_models python -m pytest -q \ inference_models/tests/unit_tests/models/rfdetr/test_triton_postprocess.py \ -k fractional_boxes_match_final_grid_referenceSource pins and reproducible evidence
orin-agx-jp62, CUDA:0), MAXN; CUDA 12.6, Torch 2.6, Triton 3.2.rfdetr-seg-nano, package8b8da2fe824240522a39f3cde41aafae, Torch FP32 eager.ccbaa275b342b76261c84e78a73f5a479877250f; task-local remote-runner configuration based on6bc3898d.2baa328bd1da42b28e69d43d33c8cb61e7a47cb5(merged Align instance segmentation boxes with the mask grid #3156); measured candidateafcae76a1a816ca66c8ef05b31ecc6a05200795c. Current head772da058dmerges newer main; RF-DETR runtime blob is identical (92d418b65372cb147ddaa89569f949bfb81e8064). No claim that profiling ran at the later merge SHA.roboflow/roboflow-inference-server-jetson-6.2.0:1.7.3, digestsha256:3c4b5193738433fb45b52af4e2a0441aef9d8473f4863bf8903f94ca194b770a.implementation-selection-mode=base,mask_format=rle,masks_resolution_factor=.25; basecamera_640x480_batch_1_base, confidence.4, max detections 300, grid178×218; largecamera_1920x1080_batch_4_high, confidence.01, max detections3000, grid328×538.Artifacts on the selected host:
/home/roboflow/remote-runner/runs/<run-id>/output/:rfdetr-mask-quarter-coords-{before,after}-v2-20261009-optimization-handoff-1; each containsoptimization-profile-set.json,artifacts/{base,large}/{latency,memory}.json, andartifacts/{base,large}/nsight/inference.sqlite.{base,large}-snapshot-1runs; input SHA-256, arrays and masks all match exactly.rfdetr-mask-coords-benchmark-afcae76a-20261009; longer full-grid repeat:rfdetr-mask-full-grid-repeat-afcae76a-20261009(postprocess-benchmark.json).rfdetr-mask-coords-tests-884260f9-20261009andrfdetr-mask-coords-tests-afcae76a-20261009;run.logis besideoutput/.Earlier pre-#3156 measurements are superseded by the results above.
0.250.25, criterion ≤ baseline median and p951.0, current vs previous Triton, strict zero increase0,0.25,0.5,1; respective controls above0.25, criterion ≤ baselineThe full-grid repeat does not establish an exact zero-regression bound; median increases are below 1% and round differences change sign. All four repeat cases match the reference exactly and retain the same CUDA allocation peaks with zero retained growth.
Overall recommendation: human decision required on the cold-start cost measured at factor
0.25. Coordinate alignment is implemented and verified, and reduced-grid warm performance improves. Keep the PR draft rather than presenting it as meeting unconditional latency/memory no-regression acceptance.