Skip to content

api-reference.mdx

Latest commit

 

History

History
1006 lines (757 loc) · 38.1 KB

api-reference.mdx

File metadata and controls

1006 lines (757 loc) · 38.1 KB
Raw
title API Reference
description Error handling, thread safety, security, deployment, and complete API for the C++ agent framework
icon book-open
**Source Code:** [`cpp/`](https://github.com/amd/gaia/tree/main/cpp) in the GAIA repository. **See also:** [Overview](/cpp/overview) for architecture, execution flow, and getting started.

Error Handling & Recovery

The framework handles failures at every layer — LLM connection, JSON parsing, and tool execution — so your agent doesn't crash on transient errors.

LLM Connection Failures

If the LLM server is unreachable or returns an error, the agent retries once automatically, then exits gracefully:

Call LLM → fails → retry once → fails again → return error result

The return value on LLM failure:

{
  "result": "Unable to complete task due to LLM error: Connection refused",
  "steps_taken": 1,
  "steps_limit": 20
}

Your application should check the result field — there is no exception to catch. HTTP timeouts: 30s connection, 120s read.

Tool Calling Protocols

The agent drives tools two ways. Which one it uses is decided per model.

Native OpenAI tool calling

For models known to support it, the request carries an OpenAI tools array built from the tool registry plus a tool_choice, the response-format template is left out of the system prompt entirely, and the model replies with choices[0].message.tool_calls. Results go back as spec-correct role: tool messages carrying tool_call_id, so the model sees a well-formed tool exchange. Parallel calls — several tool_calls in one response — all execute, each with its own reply. Streaming works too: tool_calls deltas are reassembled across SSE chunks.

Prompt-JSON fallback

For every other model the agent appends a response-format template asking for a JSON envelope ({"thought": ..., "tool": ..., "tool_args": {...}}) and recovers the call from the reply text. Tool results become [Result from <tool>]: user turns. This is the path every C++ agent used before native tool calling landed, and it is unchanged.

Choosing a protocol

gaia::AgentConfig cfg;
cfg.modelId = "Gemma-4-E4B-it-GGUF";

cfg.nativeToolCalls = gaia::NativeToolCalls::Auto;    // default — decide per model
cfg.nativeToolCalls = gaia::NativeToolCalls::Always;  // force native
cfg.nativeToolCalls = gaia::NativeToolCalls::Never;   // force prompt-JSON

cfg.toolChoice = "auto";  // or "required"

Auto consults gaia::isToolCallingModel(modelId), a mirror of the MODELS table in src/gaia/llm/lemonade_client.py. An unrecognised model id resolves to false — the C++ framework targets any OpenAI-compatible server, where an unknown id says nothing about tool-calling support, so it keeps the fallback that works everywhere. Running a tool-calling model this build does not know? Set NativeToolCalls::Always.

Response modes

responseMode picks the template used on the prompt-JSON path. It is ignored under native tool calling, which sends no template at all.

Mode Behavior
ResponseMode::Planning (default) JSON-only replies with thought / goal / plan / tool structure
ResponseMode::Conversational Plain text for conversation, a bare {"tool": ..., "tool_args": {...}} object only when calling a tool

Malformed JSON Recovery

This applies to the prompt-JSON path only — native tool calls arrive as structured JSON and are parsed strictly (a malformed tool_calls entry raises rather than falling back to prose parsing).

Local LLMs often return imperfect JSON. The parser applies six extraction strategies in sequence:

  1. Direct JSON parse
  2. Extract from markdown code blocks (```json ... ```)
  3. Bracket-matching — find first complete {...} in mixed text
  4. Fix common syntax errors (trailing commas, single quotes, missing brackets)
  5. Regex extraction of individual fields ("thought", "tool", "answer")
  6. Treat entire response as a plain-text conversational answer

This means the agent recovers from most LLM formatting errors without any intervention.

Tool Execution Errors

When a tool callback throws an exception or returns {"status": "error", ...}, the agent enters error recovery mode:

  1. The error is captured (exceptions are caught, not propagated)
  2. The error context is sent back to the LLM: "Tool execution failed. Please try an alternative approach."
  3. The LLM reasons about the error and may try a different tool or strategy
  4. If the LLM cannot recover within maxSteps, the agent returns the last error as the result

Tool errors never crash the agent. The error flow:

try {
    result = tool->callback(args);
} catch (const std::exception& e) {
    result = {{"status", "error"}, {"error", "Tool execution failed: " + e.what()}};
}
// → error context sent to LLM → LLM adapts → loop continues

MCP Auto-Reconnect

If an MCP server disconnects mid-session (process crash, timeout), the agent reconnects automatically:

MCP tool call → fails → reconnect to server → retry tool call → success or return error

The subprocess is re-launched and re-initialized. If reconnection fails, the tool call returns an error and the LLM is notified.

Loop Detection

The agent detects infinite tool call loops — when the LLM calls the same tool with the same arguments 4+ times in a row. When detected, the agent stops and returns:

"Task stopped due to repeated tool call loop."

Thread Safety

Blocking Semantics

processQuery() is fully blocking. It runs the complete agent loop (LLM calls, tool executions, history management) on the calling thread and returns only when a final answer is produced or the step limit is reached.

This means:

  • Do not call processQuery() from a UI thread — it will freeze the UI for the duration of the agent run
  • Use a background thread or async wrapper for GUI integration

Concurrent Agent Instances

Different Agent instances are fully independent and can run in parallel on separate threads. Each agent owns its own conversation history, tool registry, MCP connections, and output handler.

// SAFE — separate instances on separate threads
Agent agent1(config1);
Agent agent2(config2);

std::thread t1([&] { agent1.processQuery("query 1"); });
std::thread t2([&] { agent2.processQuery("query 2"); });
t1.join();
t2.join();

Single-Agent Rules

Do NOT call processQuery() concurrently on the same agent instance. The Agent enforces this with an atomic in-flight guard: the second concurrent call throws std::runtime_error("Agent::processQuery is not re-entrant"). Callers must serialize access to a single Agent (or create one per thread/session).

// NOT SAFE — same instance, two threads
Agent agent(config);
std::thread t1([&] { agent.processQuery("query 1"); });  // race condition
std::thread t2([&] { agent.processQuery("query 2"); });  // race condition

Similarly, do not call connectMcpServer() or disconnectMcpServer() while processQuery() is running.

The same rule covers the skill-set API: loadSkillSet() and setSkillLoader() mutate agent state and must be serialized by the caller. They are setup-time calls in practice. A lock inside loadSkillSet() would not make them safe — the SkillLoader may call back into the agent to register tools or rebuild the prompt, and the agent's config mutex is not recursive. Reading activeSkillSet() or skillSetLoaded() from another thread is safe; both return by value under the lock.

Security Model

Tool Registration Is Explicit

Only tools registered via registerTool() or discovered from a connected MCP server are available. There is no reflection, auto-discovery, or dynamic code execution. The LLM can only call tools that your code has explicitly registered.

Tool Callback Responsibility

The framework does not validate tool arguments before passing them to your callback. Each tool is responsible for:

  • Validating its input parameters (types, ranges, formats)
  • Sanitizing paths and shell arguments
  • Rejecting unexpected or dangerous inputs

Example — a safe file-reading tool:

toolRegistry().registerTool("read_file", "Read a text file",
    [](const gaia::json& args) -> gaia::json {
        std::string path = args.value("path", "");

        // Validate: reject path traversal
        if (path.find("..") != std::string::npos) {
            return {{"status", "error"}, {"error", "Path traversal not allowed"}};
        }

        // Validate: restrict to allowed directory
        if (path.find("/allowed/dir/") != 0) {
            return {{"status", "error"}, {"error", "Access denied"}};
        }

        // Safe to read
        std::ifstream f(path);
        std::string content((std::istreambuf_iterator<char>(f)),
                             std::istreambuf_iterator<char>());
        return {{"content", content}};
    },
    {{"path", gaia::ToolParamType::STRING, true, "File path to read"}}
);

MCP Server Trust

MCP servers are trusted implicitly — all tools they expose are registered without review. Only connect to MCP servers you control. In production, audit the tool list returned by each server before deployment.

Prompt Injection

The LLM decides which tool to call based on user input and conversation history. A malicious user could craft input that causes the LLM to misuse a tool. Mitigations:

  • Validate in the tool callback — don't trust the LLM's argument choices blindly
  • Use restrictive tool descriptions — describe exactly what the tool does and what arguments it accepts
  • Limit tool scope — register only the tools needed for your use case
  • Consider confirmation flows — for destructive operations, require user confirmation before executing

Conversation History

Conversation history persists between processQuery() calls on the same agent. Previous queries and tool results are visible to subsequent LLM calls. For multi-user scenarios, create a new Agent instance per user session to prevent data leakage.

Production Deployment

Binary Sizes

Measured with MSVC 2022 Release build (x64):

Artifact Size Notes
gaia_core.lib (static) ~18 MB Includes statically linked nlohmann_json and cpp-httplib
Example executable ~400-440 KB Linked against static library
Shared library (DLL) Smaller Build with -DBUILD_SHARED_LIBS=ON — ships only framework code

The static library is large because it bundles all dependencies. When building as a shared library (DLL), the binary is significantly smaller since dependencies are linked dynamically.

DLL / Shared Library

The framework supports both static and shared library builds. DLL export macros (GAIA_API) are already applied to all public classes:

# Build as shared library (DLL on Windows, .so on Linux)
cmake -B build -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

When consuming the DLL, the GAIA_API macro automatically switches from __declspec(dllexport) to __declspec(dllimport).

Install Targets

The CMake install target produces a complete SDK package:

cmake --install build --prefix /path/to/install

This creates:

/path/to/install/
  include/gaia/          # All public headers
  lib/gaia_core.lib      # Library (static or import lib)
  lib/cmake/gaia_core/   # CMake config for find_package()
  bin/gaia_core.dll      # DLL (shared builds only)

Consumers use find_package(gaia_core) to link against the installed SDK.

Runtime Configuration

The LLM endpoint can be configured at runtime via environment variable — no recompilation needed:

# Override the default LLM server URL
set LEMONADE_BASE_URL=http://my-server:8080/api/v1
my_agent.exe

All other AgentConfig fields are set at construction time. For dynamic configuration, read from a config file or registry in your makeConfig() function.

HTTPS Support

HTTPS is enabled automatically when CMake finds OpenSSL on the system (find_package(OpenSSL QUIET) in cpp/CMakeLists.txt). If OpenSSL is not present, GAIA builds with HTTP-only transport and skips the SSL-specific code paths. There is no GAIA_ENABLE_SSL option — to force-disable OpenSSL, pass the CMake built-in -DCMAKE_DISABLE_FIND_PACKAGE_OpenSSL=ON:

cmake -B build -DCMAKE_DISABLE_FIND_PACKAGE_OpenSSL=ON

API Quick Reference

Agent

class Agent {
public:
    explicit Agent(const AgentConfig& config = {});
    virtual ~Agent();

    // Main execution — blocking, returns {"result": "...", "steps_taken": N}
    json processQuery(const std::string& userInput, int maxSteps = 0);

    // Vision-language (VLM) overloads — send images alongside text.
    // See the Vision Language Models (VLM) section below for usage.
    json processQuery(const std::string& userInput,
                      const std::vector<Image>& images,
                      int maxSteps = 0);
    json processQuery(const std::vector<Message>& messages, int maxSteps = 0);

    // MCP server management
    // Discovered tools are registered with ToolPolicy::CONFIRM unless the
    // server proves them read-only — see /cpp/security.
    bool connectMcpServer(const std::string& name, const json& config);
    bool connectMcpServerById(const std::string& id);  // resolves via MCPRegistry
    void disconnectMcpServer(const std::string& name);
    void disconnectAllMcp();

    // Output handler (for custom UI integration)
    OutputHandler& console();
    void setOutputHandler(std::unique_ptr<OutputHandler> handler);

    // Tool registry access
    const ToolRegistry& tools() const;
    ToolRegistry& toolRegistry();

    // System prompt
    std::string systemPrompt() const;
    void rebuildSystemPrompt();  // call after adding tools dynamically

    // Skill sets — see the Skill sets section below
    const SkillSets& skillSets() const;
    SkillSetResolution resolveSkillSet(const std::optional<std::string>& requested = {}) const;
    std::vector<std::string> loadSkillSet(const std::optional<std::string>& requested = {});
    std::optional<std::string> activeSkillSet() const;
    std::vector<std::string> skillSetLoaded() const;
    void setSkillLoader(SkillLoader* loader);

protected:
    virtual void registerTools() {}           // override to register domain tools
    virtual std::string getSystemPrompt() const;  // override for agent-specific instructions
    virtual std::optional<std::string> selectSkillSet() const;  // override to pick a set at runtime
    void init();  // call at end of subclass constructor
};

Skill sets (gaia::SkillSets)

An agent can carry more than one interchangeable capability set and activate exactly one per launch. Declare them in the agent's gaia-agent.yaml:

skills:                      # always-on, whichever set is active
  - mailbox-hygiene
  - name: incident-review
    version: ">=0.1.0"
    required: false
skill_sets:                  # exactly ONE active per launch
  personal: [inbox-triage, newsletter-digest]
  work:     [inbox-triage, meeting-scheduling]
default_skill_set: personal  # required when skill_sets is non-empty

The active set is chosen in this order: AgentConfig::skillSet (the --skill-set flag's home) → the agent's selectSkillSet() hook → the manifest's default_skill_set. A name the manifest does not declare always throws SkillSetError naming the valid sets — it is never quietly downgraded to the default, because launching with the wrong capability bundle is worse than not launching.

gaia::AgentConfig cfg;
cfg.skillManifest = "/opt/mailroom/gaia-agent.yaml";  // empty = look beside the executable
cfg.skillSet      = "work";                            // empty = hook, then default

MyAgent agent(cfg);
agent.setSkillLoader(&loader);       // supplies load/unload; see below
agent.loadSkillSet();                // throws on an undeclared name
agent.activeSkillSet();              // "work"

Switching sets mid-session is loadSkillSet("personal"). The new set is loaded before the old one is retired, and only the skills the previous set brought in are unloaded — an always-on skill, and anything loaded outside a set, both survive. A failure part-way through rolls back completely, so the agent is never left reporting one set while carrying another's.

`version:` pins are parsed and reported but **not enforced** — GAIA cannot check one until versioned skill installs land. A declared pin logs a warning naming the pin and the on-disk version rather than being silently accepted. Skill *discovery* and registration arrive with the `SkillManager` work. Until an implementation of `SkillLoader` is installed via `setSkillLoader()`, `loadSkillSet()` resolves the set and warns that nothing was registered. Treat `activeSkillSet()` as "which set was chosen", not as proof the skills are loaded.

ToolRegistry

class ToolRegistry {
public:
    void registerTool(const std::string& name, const std::string& description,
                      ToolCallback callback, std::vector<ToolParameter> params = {},
                      bool atomic = false,
                      std::optional<ToolPolicy> policy = std::nullopt);

    json executeTool(const std::string& name, const json& args);

    const ToolInfo* findTool(const std::string& name) const;
    bool hasTool(const std::string& name) const;
    bool removeTool(const std::string& name);
    size_t size() const;
    void clear();
};

OutputHandler

Subclass to integrate agent output with your own UI. All methods are virtual:

class OutputHandler {
public:
    virtual void printProcessingStart(const std::string& query, int maxSteps,
                                      const std::string& modelId = "") = 0;
    virtual void printStepHeader(int stepNum, int stepLimit) = 0;
    virtual void printThought(const std::string& thought) = 0;
    virtual void printGoal(const std::string& goal) = 0;
    virtual void printToolUsage(const std::string& toolName) = 0;
    virtual void printToolComplete() = 0;
    virtual void prettyPrintJson(const json& data, const std::string& title = "") = 0;
    virtual void printError(const std::string& message) = 0;
    virtual void printWarning(const std::string& message) = 0;
    virtual void printInfo(const std::string& message) = 0;
    virtual void printFinalAnswer(const std::string& answer,
                                  const UsageStats& usage = {}) = 0;
    virtual void printCompletion(int stepsTaken, int stepsLimit) = 0;
    // ... plus printStateInfo, printPlan, startProgress/stopProgress, and debug methods
};

See the Custom Agent guide for full OutputHandler examples including headless/embedded usage.

MCPClient

class MCPClient {
public:
    static MCPClient fromConfig(const std::string& name, const json& config,
                                int timeout = 30, bool debug = false);

    bool connect();
    void disconnect();
    bool isConnected() const;

    std::vector<MCPToolSchema> listTools(bool refresh = false);
    json callTool(const std::string& toolName, const json& arguments);
};

MCPRegistry

Turns a configured server id into a launchable config, so an agent can connect to github without hardcoding how github is started.

class MCPRegistry {
public:
    MCPRegistry();                                          // default search paths
    explicit MCPRegistry(std::vector<std::string> paths);   // explicit files

    static std::string configDir();                         // $GAIA_CONFIG_DIR, else ~/.gaia
    static std::vector<std::string> defaultSearchPaths();

    std::optional<json> resolve(const std::string& id) const;  // nullopt if absent
    json require(const std::string& id) const;                 // throws if unresolvable
    bool isDisabled(const std::string& id) const;
    std::vector<std::string> listServers() const;
    std::string configPath() const;
    bool configExists() const;
    void reload();
};

It reads the same file and the same shape as the Python runtime, so a server configured once is reachable from both. Ids are whatever the file uses; gaia connectors keys entries by their catalog id (mcp-github, mcp-tavily, mcp-memory, mcp-git):

{
  "mcpServers": {
    "mcp-memory": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-memory"],
      "env": {},
      "disabled": false
    }
  }
}
gaia::Agent agent;
agent.connectMcpServerById("mcp-memory");   // throws MCPRegistryError if unconfigured

Search paths, lowest precedence first, under $GAIA_CONFIG_DIR (default ~/.gaia): mcp.json, then mcp_servers.json — the file gaia connectors maintains and the Python runtime reads, which wins on conflicting ids. Unknown top-level keys are ignored, and servers is accepted as an alias for mcpServers.

Everything that cannot produce a launchable config throws MCPRegistryError naming the id, the paths searched, and the ids that are available: a missing file, malformed JSON, an unknown id, an entry marked "disabled": true, or a non-stdio type. An agent silently losing its MCP tools is worse than one that refuses to start.

**Servers with secrets are Python-only today.** `gaia connectors configure ` stores secret env values in the OS keyring and writes a `$keyring` reference into the config. The C++ runtime has no keychain support, so those entries (`mcp-github`, `mcp-tavily`) throw rather than launching the server with the reference string as the credential. Servers whose env values are literal — or empty — work from both runtimes.

Two behaviours to know about, both of which split the runtimes if you rely on them:

  • The Python MCP path always reads ~/.gaia/mcp_servers.json and does not honor GAIA_CONFIG_DIR. Pointing it elsewhere makes the C++ registry read a file Python won't.
  • Python does not read mcp.json from the config directory, so an id that lives only there is invisible to it. Put anything both runtimes need in mcp_servers.json.

Unlike the Python runtime, the C++ registry does not read an mcp_servers.json from the current working directory: this config names commands to spawn, and a native binary that trusts whichever directory it was started from is an attack surface.

HttpClient

Blocking HTTP/HTTPS client for tools that need to call a web service. The transport (cpp-httplib) is a private dependency compiled into gaia_core, so including gaia/http_client.h does not pull a 10k-line header into your build.

#include <gaia/http_client.h>

struct HttpResponse {
    int status; std::string body; HttpHeaders headers;
    bool ok() const;                                  // 2xx
    std::string header(const std::string& name,       // case-insensitive
                       const std::string& fallback = "") const;
};

class HttpClient {
public:
    explicit HttpClient(const HttpClientConfig& config = {});  // baseUrl, timeouts,
    explicit HttpClient(const std::string& baseUrl);           // defaultHeaders, TLS

    HttpResponse get(const std::string& path, const HttpHeaders& headers = {},
                     int timeoutSec = 0, int connectTimeoutSec = 0);
    HttpResponse post(const std::string& path, const std::string& body,
                      const HttpHeaders& headers = {}, int timeoutSec = 0,
                      int connectTimeoutSec = 0);
    HttpResponse postStreaming(const std::string& path, const std::string& body,
                               HttpChunkCallback onChunk,
                               const HttpHeaders& headers = {}, int timeoutSec = 0,
                               int connectTimeoutSec = 0);
};

Every failure — connection refused, timeout, TLS unavailable, or a non-2xx status — throws HttpError (a std::runtime_error) naming the URL and the failure mode; the client never returns an empty response instead. HttpError also exposes status() and body() for programmatic handling.

gaia::HttpClient http({"https://api.example.com"});
try {
    auto res = http.get("/v1/models", {{"Authorization", "Bearer " + token}});
    process(res.body);
} catch (const gaia::HttpError& e) {
    std::cerr << e.what() << std::endl;   // "GET https://… returned HTTP 401: …"
}

For streaming responses (SSE), postStreaming hands each chunk to your callback; returning false ends the stream normally — that is how LemonadeClient stops on the [DONE] sentinel. Request path may also be an absolute URL, in which case the configured base URL is ignored. HTTPS requires an OpenSSL-enabled build (see HTTPS Support); an https:// URL on an HTTP-only build raises rather than downgrading.

VectorIndex

Flat (brute-force) vector search over float32 embeddings, with save/load persistence. Exhaustive scan on every query — no approximate index — so it returns exactly what the Python SDK's faiss.IndexFlatL2 / IndexFlatIP return, with no extra dependency to build.

#include <gaia/vector_index.h>

enum class Metric { L2, InnerProduct };

struct VectorIndexOptions {
    Metric metric = Metric::L2;
    bool normalizeOnAdd = false;   // + InnerProduct = cosine similarity
    std::string embeddingModel;    // persisted; load() rejects a mismatch
    size_t dimension = 0;          // 0 = infer from the first add()/load()
};

class VectorIndex {
public:
    using Id = std::string;

    explicit VectorIndex(VectorIndexOptions options = {});
    VectorIndex(size_t dimension, Metric metric);

    void add(const Id& id, const std::vector<float>& vector);
    void upsert(const Id& id, const std::vector<float>& vector);
    std::vector<std::pair<Id, float>> search(const std::vector<float>& query, size_t k) const;
    bool remove(const Id& id);
    bool contains(const Id& id) const;
    std::vector<float> get(const Id& id) const;

    size_t size() const;
    size_t dimension() const;
    void clear();

    void save(const std::string& path) const;
    void load(const std::string& path);
};

Scores are the Python convention, higher-is-better, sorted best-first:

Metric Score
Metric::L2 1 / (1 + d²), where d² is the squared Euclidean distance (what FAISS reports)
Metric::InnerProduct the raw dot product — cosine similarity when normalizeOnAdd is set

Ties keep insertion order, so rankings are reproducible across runs and platforms.

Mismatches raise instead of returning misleading results: a wrong-sized vector on add()/search() throws std::invalid_argument naming both dimensions, and loading a file built with a different embedding model throws std::runtime_error naming both models.

VectorIndexOptions opts;
opts.dimension = 768;
opts.embeddingModel = "nomic-embed-text-v1-GGUF";

gaia::VectorIndex index(opts);
index.add("chunk-0", embedding);
auto hits = index.search(queryEmbedding, 5);   // [(id, score), ...] best first
index.save("cache/index.vec");

The .vec file is a documented little-endian binary format (magic, version, metric, dimension, count, then float32 payload) — see cpp/include/gaia/vector_index.h. It is not interchangeable with Python's index.faiss; the two runtimes use separate cache directories and share only metadata.json.

Structured persistence (gaia::Database)

<gaia/database.h> is a RAII wrapper over SQLite for anything an agent needs to persist as structured data rather than loose JSON files. SQLite is vendored into gaia_core (cpp/third_party/sqlite/) and compiled with SQLITE_ENABLE_FTS5, so full-text search is always available and every platform build behaves identically. You do not need sqlite3.h on your include path or a SQLite package on your system.

Connection

#include <gaia/database.h>

gaia::Database db("agent.db");          // WAL + 5 s busy_timeout + foreign keys
auto mem = gaia::Database::inMemory();  // private in-memory database

Database::Options controls what is applied at open:

Option Default Meaning
walMode true WAL journaling — readers don't block the writer. Ignored for in-memory.
busyTimeoutMs 5000 How long a writer waits on a locked database before failing.
foreignKeys true Enforce FOREIGN KEY constraints.
readOnly false Open read-only; the file must exist.
createIfMissing true Create the file and any missing parent directories.

Connections are opened in SQLite's serialized mode, so one Database can be shared across threads without external locking.

Statements

Bind indices are 1-based; column indices are 0-based.

auto stmt = db.prepare("SELECT id, body FROM notes WHERE score > ?");
stmt.bindDouble(1, 0.5);
while (stmt.step()) {
    std::int64_t id  = stmt.columnInt64(0);
    std::string body = stmt.columnText(1);
}

// Or bind everything positionally and run in one call:
db.run("INSERT INTO notes (body, embedding) VALUES (?, ?)", "hello", blobBytes);

Typed accessors cover every storage class — columnInt64, columnInt, columnBool, columnDouble, columnText, columnBlob — plus isNull() and columnType() to tell a stored empty string apart from NULL.

Transactions

Transaction is a scope guard: it rolls back unless you commit.

{
    gaia::Transaction txn(db);
    db.run("INSERT INTO notes (body) VALUES (?)", "a");
    db.run("INSERT INTO notes (body) VALUES (?)", "b");
    txn.commit();          // omit this (or throw) and both inserts are discarded
}

Constructing a Transaction while one is already open produces a savepoint instead of a second BEGIN, so nesting works.

Schema migrations

Migrations mirror the Python MemoryStore approach — ordered steps, each advancing a stored version, chaining a database at any older version forward. The version lives in PRAGMA user_version, so no table is imposed on your schema; a fresh database reports 0.

std::vector<gaia::Migration> steps;

steps.push_back(gaia::Migration::fromSql(
    1, "initial schema",
    "CREATE TABLE knowledge (id TEXT PRIMARY KEY, content TEXT NOT NULL);"));

gaia::Migration v2;
v2.version     = 2;
v2.description = "add embedding column";
v2.apply       = [](gaia::Database& d) {
    d.addColumnIfMissing("knowledge", "embedding BLOB");
};
steps.push_back(std::move(v2));

db.migrate(steps);   // runs only what's newer than the stored version

Each step runs inside a transaction that also stamps the new version, so a step that throws rolls back completely and the stored version does not advance — re-running migrate() retries that same step. addColumnIfMissing() is the idempotent ALTER TABLE … ADD COLUMN, which is what makes a step that died half-way safely re-runnable.

migrate() refuses to run against a database newer than the last step it knows about rather than operating on a schema it doesn't understand.

Full-text search (FTS5)

db.execute("CREATE VIRTUAL TABLE docs USING fts5(title, body)");
db.run("INSERT INTO docs VALUES (?, ?)", "Ryzen AI", "NPU-accelerated inference");

auto hits = db.prepare("SELECT title, bm25(docs) FROM docs WHERE docs MATCH ? ORDER BY rank");
hits.bindText(1, "npu AND inference");
while (hits.step()) { /* … */ }

Database::hasFts5() reports availability so you can assert it up front instead of discovering it from a failed query.

Errors

Every failure raises gaia::DatabaseError — nothing is swallowed and no operation degrades to a placeholder value. The message carries the SQLite text plus the context needed to act on it:

cannot prepare statement: no such table: notes [code=1 SQL logic error]
  database: /home/u/.gaia/agent.db
  statement: SELECT * FROM notes

code(), dbPath(), and sql() expose the same fields programmatically.

Double-quoted string literals are rejected (`SQLITE_DQS=0`): `WHERE name = "alice"` is an error, not a silently-matching string constant. Use single quotes for literals, or a bind parameter.

Vision Language Models (VLM)

The C++ SDK supports vision-language models (VLMs) via the OpenAI-compatible /chat/completions endpoint. Images are sent inline as base64 data URIs.

gaia::Image

#include <gaia/types.h>

// Load from disk (MIME auto-detected from magic bytes).
gaia::Image img = gaia::Image::fromFile("photo.png");

// Or from bytes (explicit MIME or auto-detect).
std::vector<std::uint8_t> bytes = readSomewhere();
gaia::Image img2 = gaia::Image::fromBytes(bytes);                 // auto-detect
gaia::Image img3 = gaia::Image::fromBytes(bytes, "image/jpeg");   // explicit

Supported formats: image/png, image/jpeg, image/gif, image/webp, image/bmp. Unsupported MIME types and empty buffers throw std::invalid_argument.

Size cap. Image::fromFile rejects files larger than GAIA_MAX_IMAGE_BYTES (default 20 MiB, compile-time override). It also rejects non-regular files (directories, symlinks, FIFOs, devices) for safety.

Sending images with processQuery

Two new overloads accept images:

gaia::AgentConfig cfg;
cfg.modelId     = "Qwen3-VL-4B-Instruct-GGUF";
cfg.contextSize = 32768;   // recommended minimum for VLM
gaia::Agent agent(cfg);

// 1) Convenience overload: text + images
gaia::Image img = gaia::Image::fromFile("photo.png");
gaia::json result = agent.processQuery("Describe this image.", {img});

// 2) Caller-composed messages overload (advanced)
std::vector<gaia::Message> msgs = {
    gaia::Message::fromUser("What is in this image?", {img}),
};
gaia::json result2 = agent.processQuery(msgs);

Context size. VLM models require a large context window — 32768 is the recommended minimum. Smaller values (e.g. 2048) will surface a raw server error as std::runtime_error from processQuery.

History semantics. Both overloads are stateful and symmetric with the string overload: they read conversationHistory_ as request context, and append the input user messages (with image parts stripped) plus the assistant's final answer. Image base64 is never retained in history.

Thread safety. Agent is not re-entrant — concurrent processQuery calls on the same Agent throw std::runtime_error. See Thread Safety above.

End-to-end example

See cpp/examples/vlm_agent.cpp:

./build/vlm_agent path/to/image.png "What is in this image?"

Skills (SKILL.md)

<gaia/skill.h> reads the same SKILL.md files the Python runtime does — same schema, same constants, same refusal messages — so a skill written once loads in either runtime. The C++ side is read-only: it parses, validates, and renders skills; publishing, signing, and installing stay in the gaia skill CLI.

#include <gaia/skill.h>

// Parse a skill directory (or its SKILL.md directly).
gaia::Skill skill = gaia::parseSkillFile("~/.gaia/skills/web-search");

// Level 1 of progressive disclosure: frontmatter only, body dropped.
gaia::Skill listing = gaia::parseSkillMetadata("~/.gaia/skills/web-search");

// Render back to SKILL.md text. Round-trip is identity.
std::string text = gaia::toMarkdown(skill);

Types

struct Skill {
    std::string name;                      // ^[a-z0-9]+(-[a-z0-9]+)*$, <= 64 chars
    std::string description;               // <= 1024 chars — the trigger signal
    std::string body;                      // the Markdown instructions
    std::optional<std::string> license;
    std::optional<std::string> version;    // SemVer 2.0.0; 0.0.0 = unversioned
    GaiaMetadata gaia;                     // the metadata.gaia namespace
    nlohmann::json otherMetadata;          // metadata.<vendor> — preserved verbatim
    nlohmann::json extraFields;            // unknown top-level keys — preserved

    std::string path, root;                // provenance — not part of equality
    bool readOnly;
};

struct GaiaMetadata {
    std::string securityTier;              // verified | community | experimental
    std::vector<std::string> permissions;  // <domain>:<level>[:scope]
    SkillRequirements requirements;        // advisory
    std::vector<SkillTool> tools;          // tools the skill provides
    std::vector<std::string> toolsRequired;// registry tools it consumes
    nlohmann::json extra;                  // unknown metadata.gaia keys — preserved
};

Guarantees

  • Round-trip is identity. parseSkill(toMarkdown(parseSkill(t))) == parseSkill(t), including foreign metadata.<vendor> namespaces, keys GAIA does not model, and the author's key order. Nothing is lost by passing a third-party skill through GAIA.
  • Scalars resolve exactly as PyYAML does, because the Python runtime reads the same files with yaml.safe_load. flag: yes is a bool and mode: 0755 is 493 in both runtimes. A value PyYAML types as something JSON cannot hold — a timestamp, an infinity, the =/<< control tags — is kept as its literal text and written back unquoted, so Python still reads the value it read before.
  • compatibility, allowed-tools, and disallowed-tools are parsed, preserved, and ignored. They overlap metadata.gaia and are never a permission mechanism — permissions come only from metadata.gaia.permissions.
  • Failures are loud. Every violation throws gaia::SkillValidationError naming the field, the rule, and a doc link, and nothing partial is returned. Name must equal the directory name; a mismatch is refused.
  • BOM and CRLF tolerant, so a skill authored on Windows parses unchanged.

Next Steps

Architecture, execution flow, and getting started Custom prompts, typed tools, MCP servers, and output capture Consume gaia_core in your own CMake project Prerequisites, build steps, and running your first demo

License

Copyright(C) 2025-2026 Advanced Micro Devices, Inc. All rights reserved.

SPDX-License-Identifier: MIT