Add support for Object Detector models#15
Merged
Conversation
Add ObjectDetectorView, a camera view backed by a Create ML / Core ML Object Detector model. It runs a VNCoreMLRequest on each frame and publishes the labels of detected objects through a `detectedObjects: Binding<[String]>`. Mirrors the existing model-backed views: graceful degradation on a bad/missing model (camera feed with no detections, logged, reported via the optional onError closure) and an optional camera: CameraOptions?. Also updates docs and tooling: README, DocC topics, SKILL.md, CHANGELOG, the Examples gallery, a new /prototypekit:object-detector command, plugin README + marketplace description, and tests (view init, onError, nil-model graceful-failure). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NURZKomSBVteJi6uoeTCqV
Object detection's value is localization, so expose it. Add a public DetectedObject value type (label, confidence, normalized boundingBox) and a second ObjectDetectorView initializer that publishes detectedObjects: Binding<[DetectedObject]>. The receiver now maps VNRecognizedObjectObservation into DetectedObject; the existing Binding<[String]> initializer is preserved as a convenience that maps to the top labels. Both initializers share one private designated init. Updates tests (DetectedObject value + the new initializer paths) and docs: README bounding-box example, SKILL.md both signatures + DetectedObject type, DocC topic, CHANGELOG, Examples gallery box overlay, and the object-detector scaffolding command. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NURZKomSBVteJi6uoeTCqV
JamesDale
marked this pull request as ready for review
July 2, 2026 09:06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for Create ML / Core ML Object Detector models via a new SwiftUI view,
ObjectDetectorView.It runs a
VNCoreMLRequeston each live camera frame and publishes the labels of the detected objects (the top label of eachVNRecognizedObjectObservation) through adetectedObjects: Binding<[String]>. The design deliberately mirrors the existing model-backed views (ImageClassifierView,HandPoseClassifierView):PKLog, and it's reported through the optionalonError: ((PrototypeKitError) -> Void)?closure (reusing the existing.modelLoadFailedcase).camera: CameraOptions?..scaleFillcrop/scale so objects near the frame edges aren't discarded (object detectors localize across the whole frame).Public API
Changes
Sources/PrototypeKit/Public/ObjectDetectorView.swift— the view +ObjectDetectorReceiver.Tests/PrototypeKitTests/OD/ObjectDetectorTests.swift— view init,onError, and nil-model graceful-failure coverage.GracefulFailureTestswith the equivalent object-detector cases (bad-URL init,onError, nil-model receiver ignores frames).SKILL.mdAPI reference + camera-view / model-view lists + corrected gotcha,CHANGELOG(Unreleased), Examples gallery entry, a new/prototypekit:object-detectorscaffolding command, pluginREADMEtable row, and marketplace description.Testing
GracefulFailureTests.🤖 Generated with Claude Code
Generated by Claude Code