Add support for Action Classification from body movement#16
Merged
JamesDale merged 1 commit intoJul 2, 2026
Merged
Conversation
Introduce ActionClassifierView, a camera-based SwiftUI view that classifies a person's action from their body movement using a Create ML / Core ML Action Classifier model. It detects body-pose keypoints per frame with Vision's VNDetectHumanBodyPoseRequest, accumulates a sliding window of frames, and feeds a [window, 3, 18] multi-array into the model, publishing the predicted label through a binding. - ActionClassifierConfiguration exposes the poses/label feature names, prediction window size, and prediction interval for models that differ from the Create ML template. The window size is taken from the model's own input constraint when advertised. - Degrades gracefully on a bad/missing model (camera feed with no predictions, logged, reported via optional onError), matching the other model-backed views, and accepts a camera: CameraOptions?. - Frames without a detected body are zero-padded so a person briefly leaving the frame doesn't corrupt the temporal window. Adds graceful-failure tests and documents the view across the README, DocC catalog, CHANGELOG, Examples gallery, and the plugin skill/command set. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018LXbPcM8eVnaCQwtAR8y6G
JamesDale
marked this pull request as ready for review
July 2, 2026 09:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Action Classification to PrototypeKit — classifying a person's action from their body movement using a Create ML / Core ML Action Classifier model.
The new
ActionClassifierViewis a camera-based SwiftUI view that:VNDetectHumanBodyPoseRequest,[window, 3, 18]multi-array into the model, publishing the predicted label through aBinding<String>.It mirrors the API and behaviour of the existing model-backed views (
ImageClassifierView,ObjectDetectorView,HandPoseClassifierView) and works on both iOS and macOS.Details
ActionClassifierConfigurationexposesposesFeatureName("poses"),labelFeatureName("label"),predictionWindowSize(60), andpredictionInterval(15) for models that differ from the Create ML template. The window size is taken from the model's own input constraint when it advertises one.onErrorclosure. Accepts acamera: CameraOptions?.predictionIntervalframes once the window is full.Docs & tests
onError, nil-model ignores frames).Live Action Classificationsection), DocC catalog,CHANGELOG.md, theExamples/gallery, and the plugin (skill guidance, README command table, new/prototypekit:action-classifiercommand).Notes
No Swift toolchain is available in this environment, so the package was not compiled here; the code follows the established patterns of the sibling views.
🤖 Generated with Claude Code
Generated by Claude Code