API.md

Tiiny device API reference

Everything the Tiiny Pocket answers on the network, in one place. Unofficial. First collected 2026-10-25 against TiinyOS 0.1.35, rechecked 2026-10-27 after the update to device services 0.1.36 (gateway spec unchanged, 37 endpoints; vault 54 → 59). Sources, marked per endpoint:

mark	meaning
tested	called and checked against a real device
spec	in an OpenAPI spec the device publishes (exact paths and schemas)
docs	in Tiiny's developer docs (dev.tiiny.ai), with example requests
app	only seen in the TiinyOS desktop app's code; behaviour not checked

After a TiinyOS update, run python tools/api_snapshot.py --check: it refetches the device's specs, lists every added/removed/changed endpoint, regenerates endpoints/gateway.md and endpoints/kb.md, and probes the read-only routes below.

Services and ports

port	service	reachable over	notes
80	gateway: routes to everything below	Wi-Fi/LAN and USB	the only port you normally need
80 /api/v1/sys, /api/v1/account, /api/v1/apps, /api/v1/webapp, /api/v1/chat/history	system & account service (FastAPI)	"	spec: /openapi.json, /docs → endpoints/gateway.md
80 /api/v1/models, /api/v1/npu, /v1/*, /api/tags, /api/chat	model service (also on 8800 on the device)	"	no published spec; documented below
80 /kb/*	knowledge base / vault ("SimpleMem")	"	spec: /kb/openapi.json, /kb/docs → endpoints/kb.md
80 /v1/config, /v1/<connector>/…, /v1/mcp/…	on-device assistant: connectors and MCP	"	docs
80 /api/services	app services (Tiiny App Farm)	"	docs/app
dynamic	each loaded model's own container	Wi-Fi/LAN only	port from GET /api/v1/models/running; no auth
8800	model service directly	on the device only	the docs use :8800; not reachable from the network
5555	upgrade/OTA service	on the device only	/api/upgrade/*
39218	discovery metadata GET /device.json	Wi-Fi/LAN	no auth; names, addresses, USB link details
39217/udp	discovery broadcast (GADGET_DISCOVER_V1)	LAN	_gadget._tcp mDNS/DNS-SD too
9109	community NPU dashboard + /metrics (optional, not part of TiinyOS)	Wi-Fi/LAN	only if installed on the device

Interactive docs: only two services publish them: /docs and /redoc (system & account) and /kb/docs, /kb/redoc (vault). Models/inference, the assistant's connectors and /api/services have no docs or spec (/v1/docs 404); model containers vary (Foundation-1 has /openapi.json, the llama.cpp chat container doesn't). The pages need the API key for "Try it out" calls.

Auth: Authorization: Bearer <API key> on port 80 (the key TiinyOS shows; the device accepts more than one). The Anthropic-style route also accepts x-api-key (tested); no key → 401. Discovery (39218) and model containers need no key. After the Tiiny is powered off and on, every API call answers 502 Bad Gateway, over USB and Wi-Fi alike, until it's unlocked. Two ways unlock it within seconds: open the TiinyOS desktop app, or run tiiny login (the Tiiny CLI; on a PC that's already signed in it just says "Already logged in as …", and that's enough). After that the API keeps working even if the app is closed, and closing the app while the Tiiny runs doesn't lock it (tested, 0.1.38, 2026-10-30, three power cycles). TiinyOS "Auto Start" avoids it. Reported to Tiiny.

Host names (TiinyOS maps them to the device): openai.api.tiiny, anthropic.api.tiiny, ollama.api.tiiny, api.tiiny. All of them are the same port 80 — the routes below work on the plain IP too (tested).

System and power

method	path	what	mark
POST	/api/v1/sys/shutdown/	power off (empty body). Replies {"status":"ok","message":"System shutdown will be initiated in seconds via host API"}; off within ~10 s. 0.1.37 replies …initiated in seconds via D-Bus (earlier: via host API). Broken on 0.1.36 (same ok reply, device stays on); fixed in 0.1.37 (tested over USB and Wi-Fi, 2026-10-29), still working on 0.1.38 (2026-10-30)	tested
POST	/api/v1/sys/reboot/	restart	spec
GET / POST	/api/v1/sys/ac_power/	power on automatically when plugged in: GET → {"status","ac_power": bool}; POST {"switch": 0|1}	tested (GET), app (POST)
GET	/api/v1/sys/status	live cpu, memory, gpu, npus, disk	tested
GET	/api/v1/sys/performance	request timings per endpoint: summary, endpoints (limit, sort_by, include_stats_endpoint)	tested
GET	/api/v1/sys/device_info	cpu, gpu, dnpu, ram, storage, network, versions, device name, settings	tested
POST	/api/v1/sys/setting	change a device setting	spec
POST	/api/v1/sys/base_urls	runtime base URLs	spec
GET	/api/v1/sys/storage, /storage/llm, /storage/agent	storage use	spec
DELETE	/api/v1/sys/storage/agent/{agent_name}	delete an agent's data	spec
GET/POST	/api/v1/sys/wifi/wifi_list, wifi_connect, wifi_disconnect, connection_status, connection_switch, wifi_change, wifi_auto_connect	Wi-Fi setup	docs/app — 404 on port 80 over Wi-Fi (2026-10-25); probably USB/first-setup only
GET	/api/v1/npu/status	NPU memory budget and utilisation (util only while a model is loaded)	tested
GET	:39218/device.json	discovery: device name, IPv4/IPv6, USB network (usb.network, device_ip, host_ip), metadata URLs	tested
POST	:5555/api/upgrade/check, download_and_ota_prepare?target_version=, upgrade_to?target_version=prepared, pause_prepare, resume_prepare, reboot; GET status, status/v2	firmware updates	docs/app (on-device only)

Models

All through port 80 (the model service). {id} is the URL-encoded model id, e.g. Qwen%2FQwen-Image-2.1.

method	path	what	mark
GET	/api/v1/models/?including_downloading=false	installed models: id, status (running/stopped…), type, input, supported (e.g. Image Editing, Tool Use), npu_usage (% of NPU), load_time, desc	tested
GET	/api/v1/models/running	running instances, each container's port	tested
GET	/api/v1/models/{id}/unload_candidate	does it fit? unload_required, model_id[] to unload, npu_required/available	tested
POST	/api/v1/models/{id}/launch {"model_id": id}	load (async; poll running)	tested
POST	/api/v1/models/{id}/launch/stream	load with progress stream	app
POST	/api/v1/models/{id}/stop	unload	tested
POST	/api/v1/models/unload_all	unload everything	tested
POST	/api/v1/models/{id}/download, /pause_download, /interrupt; /api/v1/models/interrupt_all	downloads	docs/app
DELETE	/api/v1/models/{id}	delete a downloaded model	docs
POST	/api/v1/models/import, /import/inspect; GET /import/toolkits	import a custom model (checks toolkit support)	docs/app
GET	/api/v1/models/types, /storage, /online_models, /api/v1/model-stats	model types, disk use, store catalogue, stats	probed
GET (ws)	/api/v1/models/running/ws, /api/v1/models/npu/status/ws	live updates	app
GET	/api/v1/statistics/week_usage?start_date=…, /app_week_usage	usage stats	app (needs start_date)

Inference: OpenAI-compatible (/v1)

The device handles one request at a time; parallel requests to different models slow each other down badly.

method	path	what	mark
GET	/v1/models	loaded models; each has aliases (default = the loaded chat model), supports_vision, supports_chat, thinking, context_window	tested
POST	/v1/chat/completions	OpenAI chat. Streaming (reasoning_content deltas when thinking), tools/tool calls, images as image_url data URIs. Thinking off: chat_template_kwargs: {"enable_thinking": false} (gpt-oss rejects it; use reasoning_effort). Model default = whatever is loaded. temperature, top_p and seed are passed through to the model (0.1.38: temp 0 gives the same answer every time; temp 1 varies, gateway and direct port alike; no seed = random, a fixed seed gives the same answer every time even at temp 1). "Tell me a joke" returns the same joke most of the time even at temp 1: a predictable prompt, not a dropped setting	tested
GET	/v1/capabilities	capabilities of the running model (503 when none)	probed
POST	/v1/embeddings	embeddings (load an embedding model, e.g. Qwen3-Embedding-0.6B)	tested
POST	/v1/rerank	{model, query, documents[]} (not in the OpenAI spec)	docs
POST	/v1/images/generations	create: model, prompt, size "WxH", negative_prompt, seed, steps, guidance_scale, response_format; edit: add image (one, base64/data URI) or images (array, up to 10) with an edit-capable model. n > 1 fails: one image per request. /images/edits and /variations don't exist	tested
POST	/v1/image/generate/stream	image generation with progress	app
POST	/v1/audio/speech	model (the full id), input, voice, language, instruct, ref_audio (base64 WAV) + ref_text, response_format wav|pcm|mp3. What each model needs differs (see "Text to speech: which voice to send" below): voice must be one of the model's own speakers; OpenAI's default alloy gives a 500 on most models	tested
POST	/v1/audio/transcriptions	multipart model, file, language; json/text only. Resample 48 kHz stereo to 16 kHz mono first	tested
POST	/v1/ocr?model=<id>	{"input": data URI, "response_format": "json"} → data[0].text + lines. PP-OCRv6 Small/Medium work (100% on the test image, 0.1.36). GLM-OCR doesn't work through the API on 0.1.36: here it answers 404 Endpoint not found, and /v1/chat/completions answers 400 does not support chat. It works on its own port (see Model containers)	tested
GET/POST	/v1/music/info, /v1/music/sessions, /v1/music/generate[/wav|/preview|/resume], /v1/music/sessions/{id}/download?format=mp3	music (Foundation-1 loops via /generate/wav with {model, prompt, config}; SongGeneration via /generate/preview + /resume)	tested
GET/POST	/v1/3d/capabilities, /v1/3d/tasks, /v1/3d/tasks/{id}, /v1/3d/tasks/{id}/cancel	Hunyuan3D stages: multipart stage (shape, simplify, texture, export), params JSON, files image/mesh/material	tested
GET	/v1/video/info?model=<id>	a loaded video model's options (0.1.38): canvases (the only sizes it takes; MiniMax-H3-Turbo: 1344x768, 768x1344, 832x480, 480x832), step_options (4-8), default_steps (7), fps (24), max_keyframes (2), audio (32 kHz stereo)	tested
POST/GET/DELETE	/v1/videos, /v1/videos/{id}, /v1/videos/{id}/content	OpenAI-style video jobs (0.1.38). POST {model, prompt, size "WxH", seconds "2", steps, seed, image} (image: a start frame as a data URI) answers {id, status: queued}. Poll GET /v1/videos/{id} (status queued → in_progress → completed/failed, progress 0-100); GET …/content is the MP4 (H.264 + AAC). Max 7 s: 8 s and longer fail at once with "no exported bucket fits a request needing [audio=…, target_video=…]" (the submit itself accepts any length; a size not in canvases fails a few seconds in). Times (MiniMax-H3-Turbo): 2 s at 832x480, 4 steps: ~7 min; 7 s at 832x480, 4 steps: ~12 min; 5 s at 1344x768, 4 steps: ~32 min. Renders are cut off after 1 hour (see Known quirks)	tested

Inference: Anthropic- and Ollama-compatible

method	path	what	mark
POST	/v1/messages	Anthropic Messages API (on anthropic.api.tiiny or the IP). With a thinking model it thinks by default (a thinking block before the text, which can use up max_tokens); send "thinking": {"type": "disabled"} for a plain answer	tested
GET	/v1/models	model list in Anthropic format	probed
POST	/api/chat	Ollama chat ("stream": false, "think": false work)	tested
GET	/api/tags	Ollama model list (all installed models)	tested

Model containers (direct, Wi-Fi/LAN only, no auth)

Port from GET /api/v1/models/running; changes on every load. Blocked over USB.

model	call
Qwen3-TTS (Base)	POST :port/v1/synthesize {"text", "config": {"ref_audio", "ref_text", "language", "instruct"}} → audio_data (24 kHz WAV)
Foundation-1	POST :port/v1/generate/wav {"prompt", "config": {seconds_total, steps, cfg_scale, seed}} → WAV; has /openapi.json
SongGeneration-v2	POST :port/v1/generate {"config": {caption, lyrics, generation_type: mixed|bgm|vocal|separate, seconds_total, seed, steps, temperature, top_k, top_p}} → audio_data (48 kHz); only config at the top level
Hunyuan3D-2.1	POST :port/v1/tasks (multipart, as /v1/3d/tasks); GET :port/v1/tasks/{id}
GLM-OCR	POST :port/v1/chat/completions with the image and the text Text Recognition: (100% on the test image, 0.1.36; the only working route for GLM-OCR)

What runs each kind of model

Read from each model's own port with plain GETs (/, /openapi.json, /health, /v1/models, /props) and the Server header; tested 2026-11-01, TiinyOS 0.1.38, one model per kind. Every model runs in its own container, started on load; the gateway on port 80 routes to it.

models	what answers on the model's port
chat (Qwen3.5-9B)	llama.cpp server (Server: llama.cpp, /props, owned_by: llamacpp); a GGUF file (model.q8_0.gguf for Qwen3.5-9B), 1 slot, 128K context
embeddings, rerankers, ASR (Qwen3-ASR, Whisper; both list a qwen3_forced_aligner too), PP-OCRv6	owned_by: "mirage" in /v1/models
video (MiniMax-H3-Turbo)	"Mirage video server (minimax_h3)"; /v1/models lists canvases, step_options, buckets, dynamic_geometry: false, progressive_resolution: true
image (Z-Image-Turbo, FLUX.2-klein-4B)	an unnamed server; /v1/models gives type: image_generation, context_length: 512
TTS (Qwen3-TTS, VibeVoice, Supertonic)	one shared TTS server: /v1/synthesize, /v1/synthesize/stream, /v1/audio/speech, /v1/capabilities; Qwen3-TTS reports itself as …-Awq (AWQ-quantized)
music (Foundation-1, ACE-Step)	Python FastAPI (Server: uvicorn): "Text-to-Music Server" 1.0.0 (ACE-Step: "using a configurable TTM model")
3D (Hunyuan3D-2.1, TRELLIS.2)	FastAPI "Image-to-3D Server" 2.0.0: stage tasks with /v1/tasks/{id}/pause, /resume, /cancel, /artifacts, /v1/version
GLM-OCR	FastAPI "OpenAI Compatible Server" 1.0.0 (owned_by: system)

Hardware (GET /api/v1/sys/device_info): CPU ARMv9.2, GPU G720, dnpu "160 BF16 TOPs", peak_int8 "190 TOPS (30 TOPS on SoC, 160 TOPS on dNPU)", ram "80 GB (32 GB on SoC, 48 GB on dNPU)". So there are two AI processors and two memory pools: the main chip's NPU with 32 GB, and a separate NPU (the "dNPU") with its own 48 GB.

Inferred, not confirmed by Tiiny:

• "Mirage" looks like Tiiny's own engine for everything that isn't llama.cpp (it names itself only in the video server and in owned_by).
• The main chip matches the CIX P1 (12-core Armv9.2, Mali-G720, 30 TOPS NPU), as third-party write-ups suggest. The dNPU vendor isn't public; write-ups disagree.
• "No exported bucket fits" plus dynamic_geometry: false and a fixed list of canvases suggest the video model is compiled ahead of time for a fixed set of sizes and lengths, as NPU toolchains generally require. If so, other sizes or lengths over 7 s would need Tiiny to export new versions of the model; nothing on the device can add them. Image models' "verified sizes" may be the same idea.
• Tiiny credits PowerInfer (its open-source CPU/GPU inference engine, github.com/Tiiny-AI/PowerInfer) as core tech; nothing on the API names it.

Knowledge base / vault (/kb)

Full list (54 endpoints, from the device's own spec): endpoints/kb.md. The vault service calls itself "SimpleMem Knowledge Base Service" 2.0.0.

Before you start: files are only indexed while an embedding model is loaded (e.g. Qwen/Qwen3-Embedding-0.6B, ~1% NPU), and a normal search (and the nightly summaries) want a chat model loaded too; without one TiinyOS shows "model not loaded" alerts and searches stall. fast_mode: true skips the chat model. The docs send Host: kb.tiiny.local; it isn't needed on the IP. Same Authorization: Bearer <API key> as everything else.

Files: upload → index → search → rename → download → delete (tested 2026-10-25)

step	call	body	real response (trimmed)
upload	POST /kb/upload	multipart file (optional finalize_now, source_id, timestamp)	{"success": true, "source_id": "file-8cb17251b7f4", "filename": "…", "chunks": 1, "vectorized": false, "finalized": false} (0.1 s)
list	GET /kb/sources?limit=500	—	{"count", "total", "limit", "offset", "items": [{"source_id", "filename", "source_type", "chunk_count", "finalized", "vectorize_status", "dream_status", …}]}
index	POST /kb/finalize	{"source_id": "file-…"}	{"success": true, "generated_entries": 1} (0.9 s)
search	POST /kb/retrieve	{"question": "…", "fast_mode": false, "include_memory_context": false}	{"success", "query", "count", "results": [{"entry_id", "lossless_restatement": "Source file: …\n\n<text>", "keywords", …}]} (26 s with no chat model loaded)
rename	PATCH /kb/sources/{source_id}	{"filename": "new name.txt"}	{"success": true, "item": {…}}
download	GET /kb/files/{source_id}	—	the original file, byte for byte
delete	POST /kb/delete-memory	{"source_id": "file-…"}	{"success": true, "deleted_entries": 1, "removed_chunks": 1, "deleted_source": true}; gone from search too

include_memory_context defaults to true: the search then also pulls in your personal memory (profile, facts). Set it to false to search only files. GET /kb/retrieve?question=… works too, and POST /kb/query takes the same body.

KEY=your-api-key; TIINY=http://192.168.1.50
curl -H "Authorization: Bearer $KEY" -F "file=@notes.txt" $TIINY/kb/upload
curl -H "Authorization: Bearer $KEY" "$TIINY/kb/sources?limit=500"
curl -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d '{"source_id":"file-…"}' $TIINY/kb/finalize
curl -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d '{"question":"…","include_memory_context":false}' $TIINY/kb/retrieve

Conversations, daily summaries and memory (spec; read-only parts checked)

TiinyOS chats are stored in the vault as conversations and summarised ("dreamed") into memory entries by a background worker, nightly 23:00–05:00 by default (/kb/memory/schedule). All of it can be read and managed:

method	path	what
POST	/kb/upload_conversation	add conversation text: text, time (/stats for counts)
POST	/kb/add	add a single memory: speaker, content, timestamp, source_id, auto_finalize
POST	/kb/finalize_conversation	summarise pending conversation text now
GET	/kb/sources/conversations, /kb/sources/files	conversation sources / file sources only
GET / DELETE	/kb/conversation_days, /kb/conversation_days/{day}	stored conversations per day
GET / PUT	/kb/memory/schedule	the nightly window (nightly_start_hour* …); GET also shows the summary worker's state (tested)
GET / PUT	/kb/memory/model	which model writes the summaries (model_id); GET → {"model_id": "", "automatic_summary_enabled": true, "running_models": []} (tested)
POST	/kb/memory/process	run memory processing now
GET / POST / PATCH / DELETE	/kb/memory/facts, /kb/memory/facts/{fact_id}	extracted facts (fact_type, subject, predicate, object, …), filterable (status, fact_type, subject, q)
GET / POST	/kb/memory/proposals, /{id}/approve, /{id}/reject	facts waiting for approval
POST	/kb/context	build the memory context the assistant uses: query*, include_user_context, include_facts, include_vector_memories, max_tokens
GET / PATCH	/kb/user-context, /kb/user-context/profile/{field}, /kb/user-context/versions?limit=1	your profile (occupation, preferences, more_about_you)

How the memory pipeline behaves (read from the spec and the live data shapes, 2026-10-25; not stress-tested):

• Writers: anything with the API key — files (/kb/upload), single memories with a free-form speaker (/kb/add), conversation text (/kb/upload_conversation, which TiinyOS chats feed as "conversation days"). There's no per-agent permission model in the API.
• "Dreaming": every source has dream_status, dream_attempt_count, dream_deferred_until; conversation days have has_summary, summary_memory_worthy, summary_consumed, summary_refinement_status (queued/success/skipped). A conversation_summary worker runs in the nightly window (default 23:00–05:00, /kb/memory/schedule) with the model from /kb/memory/model (empty model_id = automatic summaries off) and extracts facts (subject/predicate/object, confidence, evidence, last_seen_at, created_by, e.g. curation_auto_approve). Riskier ones become proposals (risk_level) waiting for approve/reject.
• Eviction: none found — no TTL, size cap or retention setting. Facts are replaced by newer facts (supersedes_fact_id, status other than active) rather than aged out; removal is explicit (/kb/delete-memory, DELETE /kb/conversation_days/{day}, DELETE /kb/memory/facts/{id}). The day delete's response reports deleted_facts and deleted_proposals, so it is meant to remove what was summarised from that day too (not yet tested).
• Memory scope (added in the 2026-10 update, vault 59 endpoints): GET/PUT /kb/memory/scope, body {"scope": "all" | "tiinyos"}. The spec describes it as the summary scope: tiinyos "summarizes TiinyOS client chats only", all "summarizes every LLM output". The GET also reports supported_scopes and origin_tagging (true here). Tested 2026-10-27 with all: on an idle device the day's /kb/conversation_days/{day} entry didn't change for 9 minutes; one chat sent straight to the gateway's /v1/chat/completions (not TiinyOS) changed its content_updated_at 14 s later. So any client's chats through the gateway (other apps, scripts) are captured into that day's conversation for the daily summary. The captured text isn't readable through the API (journal_text stays empty while queued; /kb/files/{source_id} 404s for the day file) and TiinyOS doesn't show it. Without a summary model nothing becomes memory, but the days stay queued. Imports are batched (whole minutes). Switching to tiinyos did not stop the capture: a gateway chat sent afterwards updated the day the same way (2026-10-27). See PRIVACY.md for what you can delete.

On-device assistant: connectors and MCP

From Tiiny's docs (docs), not tested here.

method	path	what
GET	/v1/{gmail|outlook|calendar|outlook/calendar}/status, /auth-url	OAuth email/calendar connectors
DELETE	/v1/{connector}/connection	disconnect
POST	/v1/telegram/check-with-token, /v1/discord/check-with-token, /v2/x/check-with-token	verify bot/API tokens
PATCH	/v1/config/{telegram|discord|email|x|whatsapp}, /v1/config/channels/{…}	save connector settings
POST	/v1/email/imap/verify	check IMAP settings
POST / GET	/v1/whatsapp/qr, /v1/whatsapp/bridge/status, /bridge/logout	WhatsApp bridge
GET / POST / DELETE	/v1/mcp/servers, /v1/mcp/servers/import-json, /v1/mcp/servers/{id}, /{id}/check	custom MCP servers
GET / POST	/v1/mcp/tools, /v1/mcp/tools/call	MCP tools (app)
GET	/api/v1/chat/history/, /sessions; DELETE /{session_id}	TiinyOS chat history (spec)

Account and pairing

/api/v1/account/* (auth, mail codes, passwords, recovery keys, device name, feedback), /api/v1/connect (pair a client), /api/v1/auth/init, /api/v1/auth/login: see endpoints/gateway.md (spec) and the TiinyOS docs.

Apps

GET /api/v1/apps, /api/v1/apps/icons; GET /api/v1/webapp/, /running, POST /start, /stop (spec); /api/services, /api/services/{id}, /download/{id}/status, /updates/check, /resource-usage/ws (docs/app): the app farm.

Known quirks (checked on 0.1.35 and 0.1.36)

• One request at a time; images are one per request (n > 1 fails).
• A model that just reported running may answer its first request with xh2a backend fail (seen with Qwen-Image right after a load): wait ~10 s and retry.
• Since 0.1.36, /v1/models reports thinking as an object (supported, toggleable, enabled_by_default, levels, default_level) instead of a boolean.
• After a restart every call answers 502 until tiiny login runs or the TiinyOS app is opened (see Auth above; 0.1.38).
• Over USB only port 80 is open (still true on 0.1.38): no model ports, so no GLM-OCR and no SongGeneration direct route (separate tracks, exact-length songs). The USB link's address can land in Docker's 172.17.x.x (move Docker's bip if a container can't reach it).
• Hunyuan3D background stage fails (BriefNet missing); SongGeneration style reference fails.
• Plain /v1/music/generate fails for SongGeneration ("Session not found"); /resume ignores seconds_total.
• 0.1.36 to 0.1.38: GLM-OCR isn't reachable through the API at all (/v1/ocr 404, chat 400); use its own port over Wi-Fi/LAN, or PP-OCRv6 through /v1/ocr.
• 0.1.36: power off (/api/v1/sys/shutdown/) answers ok but the device stays on. Fixed in 0.1.37.
• With the NPU nearly full, unload_candidate can say a model fits (e.g. 55% needed, 56% free) while the load never starts. Unload other models first (/api/v1/models/unload_all), then launch.
• Streaming chat: the last chunk before [DONE] can carry usage with an empty choices list; don't index choices[0] blindly.
• Thinking counts toward max_tokens. With a thinking model (e.g. Qwen3.6) and a small limit like 256, the reply can come back with empty content, finish_reason: "length" and the budget spent in reasoning_content: it looks like the model refused. Turn thinking off for the request ("chat_template_kwargs": {"enable_thinking": false}; Anthropic style: "thinking": {"type": "disabled"}) or raise max_tokens to 1-2K.
• Video renders over 1 hour fail but keep running (0.1.38, 2026-11-01, MiniMax-H3-Turbo, 1344x768, 7 s, 8 steps, which would need about 2 hours): after exactly 60 minutes GET /v1/videos/{id} reports status: failed, error.code: GENERATION_TIMEOUT ("video generation exceeded the gateway time limit"), yet the NPU stays at 100%: the render goes on in the background. DELETE /v1/videos/{id} answers deleted: true but doesn't stop it. Stopping the video model (POST /api/v1/models/{id}/stop) does, slowly (no answer within 60 s), and afterwards every model was unloaded and /api/v1/npu/status showed no NPU readings for a few minutes. The next launch of the video model (hours later) answered 409 cleanup_failed / physical_stop_failed ("runtime cleanup failed and must be retried"); launching again worked (loaded in 5 s). Keep renders well under an hour.

Test runs: USB vs Wi-Fi (2026-10-30, TiinyOS 0.1.38, device services 0.1.30)

Same device, same tools/api_test.py run twice: once over the USB cable, once over Wi-Fi. Everything behaves the same on both links except the models' own ports, which the USB link still blocks.

check	USB cable	Wi-Fi
chat (Qwen3.6-35B-A3B): plain, streaming, tool calls, vision, Anthropic /v1/messages, Ollama /api/chat	PASS	PASS
embeddings (Qwen3-Embedding-0.6B)	PASS	PASS
OCR: PP-OCRv6 Small / Medium via /v1/ocr (100% on the test image)	PASS	PASS
OCR: GLM-OCR via /v1/ocr	FAIL (404)	FAIL (404)
OCR: GLM-OCR via /v1/chat/completions	FAIL (400 "does not support chat")	FAIL (400)
OCR: GLM-OCR on its own port	not reachable	PASS (100%)
image (Z-Image-Turbo, 512x512)	PASS	PASS
voice clone (Qwen3-TTS Base)	PASS	PASS
transcription (Qwen3-ASR, 98% match)	PASS	PASS
music loop (Foundation-1)	PASS	PASS
3D shape (Hunyuan3D-2.1, ~7 min)	PASS	PASS
power off (/api/v1/sys/shutdown/)	PASS (tested by hand, see below)	PASS (tested by hand, see below)

Re-run 2026-11-02 after the TiinyOS desktop app updated to 1.0.3: the device itself was unchanged (TiinyOS 0.1.38, services 0.1.30), and so was everything here: the same results over Wi-Fi, both API specs unchanged (gateway 37 endpoints, kb 59), and video still capped at 7 s with the same four sizes.

Re-run 2026-11-04 (Tiiny CLI updated to v0.0.5 the same day, device unchanged): USB and Wi-Fi identical again, same results as above.

Power off on 0.1.38 was tested by hand later the same day: the device turned off, and on power-up it came back on 0.1.38 (no rollback). 0.1.37 (2026-10-29) gave the same results, with power off working on both links (off in ~9 s). 0.1.36 (2026-10-28) was the same except power off, which answered ok but left the device on. Compare your own runs: python tools/api_test.py --via usb, then --via wifi, then python tools/api_compare.py snapshots/api-test-*/report.json.

Chat speed (measured 2026-11-01, TiinyOS 0.1.38)

One model loaded at a time, one request at a time, over Wi-Fi, thinking off, prompt caching off (cache_prompt: false), two runs each (they agreed within 1-2%). "Writing" = 400-token answers; "reading" = 2K and 6K-token prompts. Numbers from llama.cpp's own timings in the reply (predicted_per_second, prompt_per_second).

model	kind	writing (tok/s)	reading a prompt (tok/s)	a 6K-token prompt takes
Qwen3.6-35B-A3B	MoE, ~3B active, 17.4 GB	27.4	~790	~8 s
gpt-oss-20b	MoE, ~3.6B active, 10.3 GB	24.1	~1,300	~5 s
Qwen3.5-9B	dense, 6.2 GB	18.4	1,200-1,420	~4.5 s
Qwen3-4B-Instruct-2507	dense, 2.3 GB	18.1	1,650-1,950	~4 s
Qwen3.8-27B	dense, 15.4 GB	6.2	~500	~13 s

What it means:

• The MoE models are the fast ones. Qwen3.6-35B-A3B writes 4x faster than the dense 27B and is the best speed/quality pick for chat. Long prompts cost a few seconds before the first word.
• Small dense models stop at ~18 tok/s. The 4B is no faster than the 9B, so something other than model size sets that limit (about 55 ms per token). Inferred: a fixed per-token cost, for example handing work between the CPU and the NPU.
• Effective bandwidth: the dense 27B reads all 15.4 GB of weights for every token, so 6.24 tok/s means the device moves about 96 GB/s while running it. That's the device as a whole: the Tiiny has a CPU with 32 GB and a separate NPU with 48 GB (device_info), and Tiiny says its runtime splits work between them, so this is the combined figure, not one memory's speed. Treat it as the real-world number to plan with; the hardware's peaks are higher.
• Rough rule for a big dense model: writing speed ≈ 96 GB/s ÷ its size in GB (a 30 GB dense model would write ~3 tok/s), but small dense models don't go above ~18 tok/s. MoE models come in at about half of what the same rule gives for their active part (35B-A3B: ~1.5 GB active → rule says ~64, measured 27). A rough estimate, not a law.

Tiiny CLI vs the direct API (CLI v0.0.5, 2026-11-04, TiinyOS 0.1.38)

The same tests run through the tiiny CLI and straight against the API, over USB, same models, same inputs, scored the same way. Run it yourself: python tools/cli_compare.py (needs the CLI installed and signed in).

feature	CLI	API
chat (tiiny run -m … -p …)	PASS	PASS
chat with an image (tiiny run -m <chat model> -f image.png)	FAIL: the image isn't sent	PASS
embeddings (tiiny run embed)	PASS	PASS
OCR, PP-OCRv6 Small/Medium (tiiny run ocr)	PASS (100%)	PASS (100%)
OCR, GLM-OCR	FAIL (HTTP 40x), same as the API	FAIL (404/400)
image (tiiny run image)	PASS	PASS
transcription (tiiny run asr)	PASS (100%)	PASS (98%)
writing speed, Qwen3.6-35B-A3B (tiiny bench)	30.6 tok/s	30.7 tok/s

What to know about the CLI (v0.0.5):

• -f with a chat model doesn't attach the image. Asked to read the test image, the model answers that no image was provided, and it makes up the invoice number every time. Send images through the API (image_url) instead.
• tiiny load doesn't return when run from a script (no terminal attached), even with -y: the model is up in seconds, but the command keeps waiting. In scripts, load through the API (POST /api/v1/models/{id}/launch) or wrap the CLI in a timeout. Not checked in an interactive terminal.
• Output formats: file results print as File: <path> Content: <the API's JSON> (OCR); transcription prints File "<path>" …: <text>. --json doesn't change the OCR output.
• --device-address takes a bare IP only (no port); the CLI always uses port 80. Without it, the CLI uses the address it discovered, which can be an IPv6 address on Wi-Fi.
• Speeds match the API: the CLI adds little more than its own start-up time (~0.3 s per call).

Downloading models over the USB cable

Models can't be downloaded with the Tiiny's Wi-Fi off (tested 2026-11-04, CLI v0.0.5, TiinyOS 0.1.38, over USB from a PC with internet): tiiny download <model> refuses at once with "Tiiny needs a network connection. Use 'tiiny wifi connect' to scan available networks", and nothing is queued. The Tiiny downloads models from the internet itself, and the USB link doesn't give it a way out. Keep Wi-Fi on (any network with internet) for downloads; the USB cable is fine for everything else. Check it yourself with python tools/usb_download_test.py (turns Wi-Fi off, tries a small download, turns Wi-Fi back on).

Text to speech: which voice to send (2026-11-05, TiinyOS 0.1.38)

POST /v1/audio/speech answers 500 "The upstream TTS model failed." when the request doesn't suit the model, most often because voice isn't one of the model's own speakers. OpenAI clients (Hermes, Open WebUI, SDK examples) send voice: "alloy" by default, which most of these models reject. Use the full model id as model (tts-1 or a short name gives 404 "not loaded").

model	send	voice: "alloy"
MOSS-TTS-Nano-100M	voice: "default" (its only speaker)	500
Qwen3-TTS CustomVoice	voice: aiden, dylan, eric, ono_anna, ryan, serena, sohee, uncle_fu, vivian	works (falls back)
Qwen3-TTS VoiceDesign	instruct: a description, e.g. "A calm, warm female voice."	500
Qwen3-TTS Base, VibeVoice-1.5B	clone only: ref_audio + ref_text	500
VibeVoice-Realtime-0.5B	voice: one of 61 speakers, by language: en-Emma_woman, en-Carter_man, … (de-…, fr-… for other languages)	500
Supertonic-3	broken through the API: every request fails, because the API passes the full id Supertone/supertonic-3 and the model's server only accepts Supertonic-3. On its own port (Wi-Fi/LAN) it works with model: "Supertonic-3" and voice F1-F5 / M1-M5	500

• VibeVoice-Realtime can fail the first request after it loads, and fails short text with a speaker of another language: retry once, and match the speaker to the text's language.
• response_format: "mp3" works wherever wav works.
• Kokoro-82M isn't covered yet (not downloaded on the test device).
• List the voices on your own Tiiny and check each model: python tools/tts_voices.py (needs a Wi-Fi/LAN address: the voice lists come from each model's own port). The harness checks one preset voice: api_test.py audio-tts-preset.