From 14c9c33f7ab43df9dd07e98c0b7e4c45f5e94ce3 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 2 Jul 2026 09:18:12 +0000 Subject: [PATCH] Add support for Action Classification from body movement Introduce ActionClassifierView, a camera-based SwiftUI view that classifies a person's action from their body movement using a Create ML / Core ML Action Classifier model. It detects body-pose keypoints per frame with Vision's VNDetectHumanBodyPoseRequest, accumulates a sliding window of frames, and feeds a [window, 3, 18] multi-array into the model, publishing the predicted label through a binding. - ActionClassifierConfiguration exposes the poses/label feature names, prediction window size, and prediction interval for models that differ from the Create ML template. The window size is taken from the model's own input constraint when advertised. - Degrades gracefully on a bad/missing model (camera feed with no predictions, logged, reported via optional onError), matching the other model-backed views, and accepts a camera: CameraOptions?. - Frames without a detected body are zero-padded so a person briefly leaving the frame doesn't corrupt the temporal window. Adds graceful-failure tests and documents the view across the README, DocC catalog, CHANGELOG, Examples gallery, and the plugin skill/command set. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_018LXbPcM8eVnaCQwtAR8y6G --- .claude-plugin/marketplace.json | 2 +- CHANGELOG.md | 8 + Examples/PrototypeKitExamples.swift | 23 ++ README.md | 55 ++++ .../PrototypeKit.docc/PrototypeKit.md | 2 + .../Public/ActionClassifierView.swift | 253 ++++++++++++++++++ .../ErrorHandling/GracefulFailureTests.swift | 19 ++ plugin/README.md | 1 + plugin/commands/action-classifier.md | 51 ++++ plugin/skills/prototypekit/SKILL.md | 49 +++- 10 files changed, 458 insertions(+), 5 deletions(-) create mode 100644 Sources/PrototypeKit/Public/ActionClassifierView.swift create mode 100644 plugin/commands/action-classifier.md diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 1f65a31..7149261 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -12,7 +12,7 @@ { "name": "prototypekit", "source": "./plugin", - "description": "Scaffold and wire up PrototypeKit SwiftUI views for rapid on-device ML prototyping (camera, image classification, object detection, text/barcode/hand-pose/sound recognition, natural-language analysis).", + "description": "Scaffold and wire up PrototypeKit SwiftUI views for rapid on-device ML prototyping (camera, image classification, object detection, text/barcode/hand-pose/sound recognition, action classification, natural-language analysis).", "version": "0.1.0", "author": { "name": "Friday Technologies", diff --git a/CHANGELOG.md b/CHANGELOG.md index 675b815..aaf9197 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,14 @@ once tagged releases begin. ## [Unreleased] ### Added +- **Action classification.** A new `ActionClassifierView` classifies a person's action from their + body movement on the live camera feed using a Create ML / Core ML **Action Classifier** model. It + detects body-pose keypoints with Vision's `VNDetectHumanBodyPoseRequest` and feeds a sliding window + of frames (two seconds by default) into the model, publishing the predicted label through a binding. + An `ActionClassifierConfiguration` exposes the prediction window, prediction interval, and feature + names for models that differ from the Create ML template. Like the other model-backed views it + degrades gracefully on a bad/missing model (camera feed with no predictions, logged, reported via the + optional `onError` closure) and accepts a `camera: CameraOptions?`. - **Object detection.** A new `ObjectDetectorView` runs a Create ML / Core ML **Object Detector** model on the live camera feed and publishes the objects found in each frame. Two initializers are offered: one takes a `detectedObjects: Binding<[String]>` for the labels only, and one takes a diff --git a/Examples/PrototypeKitExamples.swift b/Examples/PrototypeKitExamples.swift index 86790c3..4311d21 100644 --- a/Examples/PrototypeKitExamples.swift +++ b/Examples/PrototypeKitExamples.swift @@ -25,6 +25,7 @@ struct ExampleGallery: View { NavigationLink("Image classification", destination: ImageClassifierExample()) NavigationLink("Object detection", destination: ObjectDetectorExample()) NavigationLink("Object detection (boxes)", destination: ObjectDetectorBoxesExample()) + NavigationLink("Action classification", destination: ActionClassifierExample()) } Section("Sound") { NavigationLink("Sound recognition", destination: SoundExample()) @@ -213,6 +214,28 @@ struct SoundExample: View { } } +/// Classifies a person's action from their body movement with a Create ML action classifier. +/// +/// Replace `modelURL` with your own model, e.g. `ActionClassifier.urlOfModelInThisBundle`. +struct ActionClassifierExample: View { + @State private var latestPrediction = "" + @State private var errorMessage: String? + + private let modelURL = URL(fileURLWithPath: "/path/to/YourActionClassifier.mlmodelc") + + var body: some View { + VStack(spacing: 16) { + ActionClassifierView(modelURL: modelURL, latestPrediction: $latestPrediction) { error in + errorMessage = error.localizedDescription + } + Text(latestPrediction.isEmpty ? "Detecting…" : latestPrediction).font(.title) + if let errorMessage = errorMessage { + Text(errorMessage).foregroundStyle(.red).font(.footnote) + } + } + } +} + // MARK: - Motion /// Classifies device activity with a Create ML activity classifier. diff --git a/README.md b/README.md index 809500e..cbd8650 100644 --- a/README.md +++ b/README.md @@ -282,6 +282,61 @@ struct HandPoseClassifierViewSample: View { +### Live Action Classification + +Classify a person's action from their body movement in real-time using a Create ML / Core ML +**Action Classifier** model. + +1. **Required Step:** Drag in your Create ML / Core ML action classifier model into Xcode. +2. Change `ActionClassifier` below to the name of your Model. +3. You can use `latestPrediction` as you would any other state variable. + +An action unfolds over time, so `ActionClassifierView` detects body-pose keypoints with Vision and +feeds a sliding window of frames (two seconds by default) into your model, updating `latestPrediction` +as the window advances. + +Utilise `ActionClassifierView` + +```swift +ActionClassifierView(modelURL: ActionClassifier.urlOfModelInThisBundle, + latestPrediction: $latestPrediction) +``` + +If your model uses different feature names or a different prediction window, supply an +`ActionClassifierConfiguration`: + +```swift +ActionClassifierView( + modelURL: ActionClassifier.urlOfModelInThisBundle, + configuration: ActionClassifierConfiguration(predictionWindowSize: 90), + latestPrediction: $latestPrediction +) +``` + +
+Full Example +
+ +```swift +import SwiftUI +import PrototypeKit + +struct ActionClassifierViewSample: View { + + @State var latestPrediction: String = "" + + var body: some View { + VStack { + ActionClassifierView(modelURL: ActionClassifier.urlOfModelInThisBundle, + latestPrediction: $latestPrediction) + Text(latestPrediction) + } + } +} +``` +
+ + ### Live Text Recognition Utilise `LiveTextRecognizerView` diff --git a/Sources/PrototypeKit/PrototypeKit.docc/PrototypeKit.md b/Sources/PrototypeKit/PrototypeKit.docc/PrototypeKit.md index beba268..de33b68 100644 --- a/Sources/PrototypeKit/PrototypeKit.docc/PrototypeKit.md +++ b/Sources/PrototypeKit/PrototypeKit.docc/PrototypeKit.md @@ -62,6 +62,8 @@ directory for a runnable gallery covering every feature. - ``ObjectDetectorView`` - ``DetectedObject`` - ``HandPoseClassifierView`` +- ``ActionClassifierView`` +- ``ActionClassifierConfiguration`` - ``LiveTextRecognizerView`` - ``LiveBarcodeRecognizerView`` - ``LiveAnimalRecognizerView`` diff --git a/Sources/PrototypeKit/Public/ActionClassifierView.swift b/Sources/PrototypeKit/Public/ActionClassifierView.swift new file mode 100644 index 0000000..44121dd --- /dev/null +++ b/Sources/PrototypeKit/Public/ActionClassifierView.swift @@ -0,0 +1,253 @@ +// +// ActionClassifierView.swift +// +// +// Created by James Dale on 2/7/2026. +// + +import SwiftUI +import CoreML +import Vision + +/// Configuration options for live action classification via ``ActionClassifierView``. +/// +/// The defaults match the input and output feature names produced by a standard Create ML +/// **Action Classifier**, which is trained on sequences of human body-pose keypoints. If your model +/// uses different feature names or window size, override them here so they line up with your model. +public struct ActionClassifierConfiguration { + + /// The model input feature name for the window of body-pose keypoints. + /// + /// Create ML's Action Classifier names this input `"poses"` and expects a multi-array shaped + /// `[window, 3, 18]` — one `[1, 3, 18]` slice per frame, as produced by + /// `VNHumanBodyPoseObservation.keypointsMultiArray()`. + var posesFeatureName: String + + /// The model output feature name for the predicted action label. + var labelFeatureName: String + + /// The number of frames of body-pose data that inform a single prediction. + /// + /// This should match the model's prediction window. When the model advertises a fixed input + /// window, ``ActionClassifierView`` uses the model's value; otherwise it falls back to this one. + /// Create ML's Action Classifier template defaults to a two-second window. + var predictionWindowSize: Int + + /// How many new frames to collect between predictions once a full window is available. + /// + /// Running the model on every frame is wasteful, so predictions are throttled: after the window + /// fills, a new prediction is made every `predictionInterval` frames while the window slides + /// forward. A smaller value is more responsive; a larger value is cheaper. + var predictionInterval: Int + + /// Creates an action classifier configuration. + /// + /// - Parameters: + /// - posesFeatureName: The body-pose window input feature name. Defaults to `"poses"`. + /// - labelFeatureName: The predicted label output feature name. Defaults to `"label"`. + /// - predictionWindowSize: The number of frames per prediction. Used only when the model does + /// not advertise a fixed input window. Defaults to `60` (two seconds at 30 fps). + /// - predictionInterval: The number of new frames between predictions once the window is full. + /// Defaults to `15`. + public init(posesFeatureName: String = "poses", + labelFeatureName: String = "label", + predictionWindowSize: Int = 60, + predictionInterval: Int = 15) { + self.posesFeatureName = posesFeatureName + self.labelFeatureName = labelFeatureName + self.predictionWindowSize = predictionWindowSize + self.predictionInterval = predictionInterval + } +} + +final class ActionClassifierReceiver: PKCameraViewReceiver, ObservableObject { + + private let mlModel: MLModel? + private let configuration: ActionClassifierConfiguration + + @Published var latestPrediction: String? + + /// The number of frames that make up one prediction window. + private let windowSize: Int + + /// A rolling buffer of per-frame body-pose keypoint arrays, each shaped `[1, 3, 18]`. + private var poseWindow: [MLMultiArray] = [] + + /// How many frames have been collected since the last prediction was made. + private var framesSinceLastPrediction = 0 + + private lazy var bodyPoseRequest: VNDetectHumanBodyPoseRequest? = { + guard self.mlModel != nil else { return nil } + return VNDetectHumanBodyPoseRequest { [weak self] request, error in + guard let self = self else { return } + if let error = error { + PKLog.vision.error("Body pose request failed: \(error.localizedDescription)") + return + } + + // Take the most prominent person in the frame. When nobody is detected we still advance + // the window with a zero-filled slice, matching how Create ML pads missing frames — so a + // person leaving and re-entering the frame doesn't corrupt the temporal window. + let observation = (request.results as? [VNHumanBodyPoseObservation])?.first + let pose = (try? observation?.keypointsMultiArray()) ?? self.emptyPose() + guard let pose = pose else { return } + + self.append(pose) + } + }() + + /// Creates a receiver for the given action model. + /// + /// - Parameters: + /// - mlModel: The Core ML action-classification model, or `nil` when the model failed to load. + /// When `nil`, frames are ignored and no predictions are published. + /// - configuration: The window and feature-name configuration. + init(mlModel: MLModel?, configuration: ActionClassifierConfiguration) { + self.mlModel = mlModel + self.configuration = configuration + + // Prefer the model's own window size when it advertises one, so the buffer always matches. + let inputs = mlModel?.modelDescription.inputDescriptionsByName ?? [:] + let modelWindow = inputs[configuration.posesFeatureName]? + .multiArrayConstraint?.shape.first?.intValue + if let modelWindow = modelWindow, modelWindow > 0 { + self.windowSize = modelWindow + } else { + self.windowSize = configuration.predictionWindowSize + } + } + + func processImage(_ cgImage: CGImage) { + guard let bodyPoseRequest = bodyPoseRequest else { return } + + let handler = VNImageRequestHandler(cgImage: cgImage, + orientation: .up, + options: [:]) + + do { + try handler.perform([bodyPoseRequest]) + } catch { + PKLog.vision.error("Failed to perform body pose detection: \(error.localizedDescription)") + } + } + + /// A zero-filled `[1, 3, 18]` keypoint slice, used to pad frames where no body is detected. + private func emptyPose() -> MLMultiArray? { + guard let pose = try? MLMultiArray(shape: [1, 3, 18], dataType: .float32) else { return nil } + for i in 0.. windowSize { + poseWindow.removeFirst(poseWindow.count - windowSize) + } + + framesSinceLastPrediction += 1 + if poseWindow.count == windowSize && framesSinceLastPrediction >= configuration.predictionInterval { + framesSinceLastPrediction = 0 + predict() + } + } + + /// Runs the model over the current window and publishes the predicted label. + private func predict() { + guard let mlModel = mlModel else { return } + guard let modelInput = try? MLMultiArray(concatenating: poseWindow, axis: 0, dataType: .float32) else { + return + } + + guard + let featureProvider = try? MLDictionaryFeatureProvider(dictionary: [ + configuration.posesFeatureName: modelInput + ]), + let prediction = try? mlModel.prediction(from: featureProvider) + else { return } + + if let label = prediction.featureValue(for: configuration.labelFeatureName)?.stringValue { + DispatchQueue.main.async { + self.latestPrediction = label + } + } + } +} + +/// A SwiftUI view that shows a live camera feed and classifies a person's action from their body +/// movement using a Core ML model. +/// +/// The view detects human body-pose keypoints with Vision's `VNDetectHumanBodyPoseRequest` and feeds a +/// sliding window of them into the Create ML / Core ML **Action Classifier** you provide, publishing the +/// predicted label through a binding. Because an action unfolds over time, the view collects a window of +/// frames (two seconds by default) before its first prediction and updates as the window slides forward. +/// +/// It drives a ``PKCameraView`` internally, so your app target must declare the `NSCameraUsageDescription` +/// (Privacy - Camera Usage Description) key in its Info properties. +/// +/// ```swift +/// @State var latestPrediction = "" +/// +/// ActionClassifierView(modelURL: JumpingJacksClassifier.urlOfModelInThisBundle, +/// latestPrediction: $latestPrediction) +/// ``` +/// +/// - Note: The most prominent person in the frame is classified; frames without a detected body are +/// padded so a person briefly leaving the frame doesn't corrupt the window. +public struct ActionClassifierView: View { + + @State var receiver: ActionClassifierReceiver + + @Binding var latestPrediction: String + + private let cameraOptions: CameraOptions? + + private let onError: ((PrototypeKitError) -> Void)? + + private let loadError: PrototypeKitError? + + /// Creates an action classifier view backed by a Core ML model. + /// + /// - Parameters: + /// - modelURL: The location of the compiled Core ML / Create ML action-classification model to + /// load, typically `YourModel.urlOfModelInThisBundle`. + /// - configuration: An ``ActionClassifierConfiguration`` describing the prediction window and the + /// model's feature names. Defaults match a standard Create ML Action Classifier. + /// - latestPrediction: A binding updated with the most recent predicted action label. Defaults to a + /// constant empty string when you only need the on-screen camera feed. + /// - camera: Optional ``CameraOptions`` selecting the camera position and device type. Pass `nil` + /// to use the default back wide-angle camera. (Ignored on macOS.) + /// - onError: An optional closure called (on the main thread) if the model fails to load. The view + /// still shows the camera feed without classification; use this to surface the failure in your UI. + public init(modelURL: URL, + configuration: ActionClassifierConfiguration = .init(), + latestPrediction: Binding = .constant(""), + camera: CameraOptions? = nil, + onError: ((PrototypeKitError) -> Void)? = nil) { + self._latestPrediction = latestPrediction + self.cameraOptions = camera + self.onError = onError + do { + let mlModel = try MLModel(contentsOf: modelURL) + self.receiver = ActionClassifierReceiver(mlModel: mlModel, configuration: configuration) + self.loadError = nil + } catch { + // Degrade gracefully: show the camera feed without classification rather than + // crashing the host app when the model can't be loaded. + PKLog.model.error("Failed to load action model at \(modelURL.path): \(error.localizedDescription)") + self.receiver = ActionClassifierReceiver(mlModel: nil, configuration: configuration) + self.loadError = .modelLoadFailed(url: modelURL, underlying: error) + } + } + + public var body: some View { + PKCameraView(receiver: receiver, options: cameraOptions) + .onReceive(receiver.$latestPrediction, perform: { newPrediction in + guard let newPrediction = newPrediction else { return } + self.latestPrediction = newPrediction + }) + .onAppear { + if let loadError = loadError { onError?(loadError) } + } + } +} diff --git a/Tests/PrototypeKitTests/ErrorHandling/GracefulFailureTests.swift b/Tests/PrototypeKitTests/ErrorHandling/GracefulFailureTests.swift index f08cef8..49ed552 100644 --- a/Tests/PrototypeKitTests/ErrorHandling/GracefulFailureTests.swift +++ b/Tests/PrototypeKitTests/ErrorHandling/GracefulFailureTests.swift @@ -39,6 +39,11 @@ final class GracefulFailureTests: XCTestCase { XCTAssertNoThrow(view.body) } + func testActionClassifierViewInitWithBadURLDoesNotCrash() throws { + let view = ActionClassifierView(modelURL: bogusModelURL) + XCTAssertNoThrow(view.body) + } + // MARK: - The optional onError hook is accepted and does not crash construction func testImageClassifierViewAcceptsOnError() throws { @@ -56,6 +61,11 @@ final class GracefulFailureTests: XCTestCase { XCTAssertNoThrow(view.body) } + func testActionClassifierViewAcceptsOnError() throws { + let view = ActionClassifierView(modelURL: bogusModelURL, onError: { _ in }) + XCTAssertNoThrow(view.body) + } + // MARK: - PrototypeKitError provides human-readable descriptions func testPrototypeKitErrorDescriptions() { @@ -96,6 +106,15 @@ final class GracefulFailureTests: XCTestCase { XCTAssertTrue(receiver.detectedObjects.isEmpty) } + func testActionClassifierReceiverWithNilModelIgnoresFrames() throws { + let receiver = ActionClassifierReceiver(mlModel: nil, configuration: .init()) + let frame = try makeSolidColorCGImage(width: 64, height: 64) + + receiver.processImage(frame) + + XCTAssertNil(receiver.latestPrediction) + } + // MARK: - Activity classification (iOS only) must stay inert without a model #if os(iOS) diff --git a/plugin/README.md b/plugin/README.md index 937caba..48bdd5d 100644 --- a/plugin/README.md +++ b/plugin/README.md @@ -39,6 +39,7 @@ claude --plugin-dir ./plugin | `/prototypekit:body-pose-detector` | Scaffold live body pose detection (`LiveBodyPoseDetectorView`) | | `/prototypekit:rectangle-detector` | Scaffold live rectangle detection (`LiveRectangleDetectorView`) | | `/prototypekit:hand-pose` | Scaffold hand-pose classification (`HandPoseClassifierView`, Core ML) | +| `/prototypekit:action-classifier` | Scaffold action classification from body movement (`ActionClassifierView`, Core ML) | | `/prototypekit:sound-recognizer` | Scaffold sound recognition (`.recognizeSounds`, iOS 15+) | | `/prototypekit:natural-language` | Scaffold on-device text analysis — sentiment, language ID, entities (no camera/model) | diff --git a/plugin/commands/action-classifier.md b/plugin/commands/action-classifier.md new file mode 100644 index 0000000..f1d784d --- /dev/null +++ b/plugin/commands/action-classifier.md @@ -0,0 +1,51 @@ +--- +description: Scaffold a PrototypeKit action classifier view (ActionClassifierView) that classifies a person's action from their body movement, backed by a Core ML model. +argument-hint: "[optional: Core ML model class name, e.g. ActionClassifier]" +--- + +Add action classification using PrototypeKit's `ActionClassifierView`. It detects a +person's body-pose keypoints with Vision and classifies their action from a sliding +window of frames using your Core ML model. Follow the exact API in the `prototypekit` +skill. + +Signature to use: +`ActionClassifierView(modelURL:, configuration: ActionClassifierConfiguration = .init(), latestPrediction: Binding = .constant(""), camera: CameraOptions? = nil, onError: ((PrototypeKitError) -> Void)? = nil)`. + +An action unfolds over time, so the view collects a window of frames (two seconds by +default) before its first prediction and updates as the window advances. If the model +uses different feature names or a different window, pass an `ActionClassifierConfiguration` +(`posesFeatureName`, `labelFeatureName`, `predictionWindowSize`, `predictionInterval`). + +1. Generate a SwiftUI `View` that: + - declares `@State var latestPrediction: String = ""`, + - embeds `ActionClassifierView(modelURL: .urlOfModelInThisBundle, latestPrediction: $latestPrediction)`, + - displays `Text(latestPrediction)`. + Use the model class name from `$ARGUMENTS` if provided; otherwise use a + placeholder like `ActionClassifier` and tell the user to replace it. +2. Ensure `import SwiftUI` and `import PrototypeKit`. +3. Remind the user to: + - train/drag in an action-classifier `.mlmodel` (Xcode generates the class used above), and + - add `NSCameraUsageDescription` to Info.plist. + A bad/missing model URL degrades gracefully (the camera feed shows but no predictions + are produced); use the optional `onError` closure to surface the failure in the UI. + +Reference example: + +```swift +import SwiftUI +import PrototypeKit + +struct ActionClassifierSample: View { + @State var latestPrediction: String = "" + + var body: some View { + VStack { + ActionClassifierView(modelURL: ActionClassifier.urlOfModelInThisBundle, + latestPrediction: $latestPrediction) + Text(latestPrediction) + } + } +} +``` + +Prefer editing the user's actual files over just printing code when a target is clear. diff --git a/plugin/skills/prototypekit/SKILL.md b/plugin/skills/prototypekit/SKILL.md index 88400d8..5848e30 100644 --- a/plugin/skills/prototypekit/SKILL.md +++ b/plugin/skills/prototypekit/SKILL.md @@ -49,8 +49,8 @@ Then `import PrototypeKit` in any file that uses it. Any camera-based view (`PKCameraView`, `ImageClassifierView`, `ObjectDetectorView`, `LiveTextRecognizerView`, `LiveBarcodeRecognizerView`, `LiveAnimalRecognizerView`, `LiveFaceDetectorView`, `LiveBodyPoseDetectorView`, `LiveRectangleDetectorView`, -`HandPoseClassifierView`) requires a camera-usage description, or the app crashes on -launch of that view: +`HandPoseClassifierView`, `ActionClassifierView`) requires a camera-usage description, or the app +crashes on launch of that view: - **Camera:** key `NSCameraUsageDescription` — shown in Xcode's Info tab as `Privacy - Camera Usage Description`. Value: a human-readable reason. @@ -66,7 +66,8 @@ To add: select the project ▸ your target ▸ **Info** tab ▸ right-click the ## Core ML models Views that classify or detect (`ImageClassifierView`, `ObjectDetectorView`, -`HandPoseClassifierView`) take a `modelURL: URL` pointing at a compiled Core ML model. +`HandPoseClassifierView`, `ActionClassifierView`) take a `modelURL: URL` pointing at a compiled +Core ML model. 1. Drag your `.mlmodel` (from Create ML / Core ML) into the Xcode project. 2. Xcode generates a Swift class named after the model. @@ -432,6 +433,46 @@ struct HandPoseSample: View { } ``` +### `ActionClassifierView` — action classification from body movement (Core ML) + +Detects a person's body-pose keypoints with Vision and classifies their action from a sliding +window of frames using a Create ML / Core ML **Action Classifier** model. An action unfolds over +time, so the view collects a window (two seconds by default) before its first prediction and +updates as the window advances. + +Signature: + +```swift +public init(modelURL: URL, + configuration: ActionClassifierConfiguration = .init(), + latestPrediction: Binding = .constant(""), + camera: CameraOptions? = nil, + onError: ((PrototypeKitError) -> Void)? = nil) +``` + +`ActionClassifierConfiguration` exposes `posesFeatureName` (default `"poses"`), `labelFeatureName` +(default `"label"`), `predictionWindowSize` (default `60`), and `predictionInterval` (default `15`) +for models that differ from the Create ML template. + +Full example: + +```swift +import SwiftUI +import PrototypeKit + +struct ActionClassifierSample: View { + @State var latestPrediction: String = "" + + var body: some View { + VStack { + ActionClassifierView(modelURL: ActionClassifier.urlOfModelInThisBundle, + latestPrediction: $latestPrediction) + Text(latestPrediction) + } + } +} +``` + ### `.recognizeSounds` — live sound recognition (iOS 15+ only) A `View` modifier. Uses the built-in system sound classifier by default, or a @@ -529,7 +570,7 @@ struct SentimentView: View { - **Missing privacy key = crash.** Add `NSCameraUsageDescription` (camera views) and `NSMicrophoneUsageDescription` (`recognizeSounds`) before running. - **Bad model URL.** `ImageClassifierView` / `ObjectDetectorView` / - `HandPoseClassifierView` degrade gracefully if the model fails to load — the camera + `HandPoseClassifierView` / `ActionClassifierView` degrade gracefully if the model fails to load — the camera feed still shows but no predictions/detections are produced, and the failure is logged (and reported to `onError` where available). Verify the model is in the target's bundle and the generated class name matches.