Connect visual scans to AI interpretation

.github/workflows/native-ios.yml

@@ -5,9 +5,25 @@ on:
   pull_request:
     paths:
       - "ios/**"
+      - "backend/**"
       - ".github/workflows/native-ios.yml"
 
 jobs:
+  backend-test:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: backend
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 24
+          cache: npm
+          cache-dependency-path: backend/package-lock.json
+      - run: npm ci
+      - run: npm test
+
   build:
     runs-on: macos-latest
     steps:

.gitignore

@@ -3,6 +3,7 @@
 # dependencies
 /node_modules
 /web/node_modules
+/backend/node_modules
 /.pnp
 .pnp.*
 .yarn/*

README.md

@@ -26,9 +26,10 @@ The private web prototype is available at
 3. Choose your personal development team under Signing & Capabilities.
 4. Build and run. Approve only the sensors you want to test.
 
-The first native slice includes a real ARKit/LiDAR camera surface, live
-microphone level, motion, heading, location, and barometric context. Its AI
-synthesis remains a clearly labeled guided demo until the backend is connected.
+The native slice includes a real ARKit/LiDAR camera surface, live microphone
+level, motion, heading, location, and barometric context. Visual scans can now
+send one deliberately captured frame to the optional server-owned AI service;
+without a configured service, the app falls back to its clearly labeled demo.
 
 ## Cloud-first workflow
 

backend/README.md

@@ -1,6 +1,27 @@
 # FIELD backend boundary
 
-This directory will hold the server-owned multimodal interpretation service.
-No API key belongs in either client. The first endpoint will accept the shared
-scan envelope, return structured evidence and inference separately, and retain
-nothing unless the user explicitly saves an observation.
+This directory contains the server-owned multimodal interpretation service. It
+runs as a small Cloudflare Worker; no OpenAI API key belongs in either client.
+
+The first endpoint, `POST /v1/interpret/visual`, accepts a JPEG from an explicit
+visual scan plus direct depth/light context. It calls the OpenAI Responses API
+with image input and a strict JSON schema, returns concise evidence and inference
+separately, and sets `store: false`. FIELD does not retain the image or response.
+
+## Local setup
+
+```bash
+npm install
+npx wrangler secret put OPENAI_API_KEY
+npm test
+npm run dev
+```
+
+Use `OPENAI_MODEL` to override the default `gpt-5.6-luna`. For an iPhone build,
+set the Xcode build setting `FIELD_API_BASE_URL` to the deployed Worker origin,
+without a trailing `/`.
+
+This first deployment is suitable for private device testing. Before a public
+release, put Cloudflare rate limiting and Apple App Attest verification in front
+of the endpoint; a static secret embedded in the app would not provide a real
+security boundary.