diff --git a/AGENTS.md b/AGENTS.md index c4433d90..df72f693 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,10 +13,11 @@ The project optimizes for reliable tests, lower API cost, and fast debugging. ## Current Architecture -- CLI entry point: `cmd/server/main.go` only exits through `run`; command wiring lives in `cmd/server/root.go` -- CLI command files: `cmd/server/root.go` wires `serve.go`, `migrate.go`, `db.go`, `config.go`, `doctor.go`, `provider.go`, `models.go`, `tools.go`, `audit.go`, `auth.go`, `analyze.go`, `version.go`, `schema.go`, `completion.go` (`provider_startup_probe.go` holds provider-probe helpers) -- Second binary: the CLI `cmd/trajecta` (build artifact `trajecta`) performs local, server-less operations. Today that is the one-off rename migration (`.env` prefix mapping, legacy SQLite merge into Postgres, cassette magic rewrite and structural validation, SQLite archiving) and cassette vault reorganisation (`layout plan` reads the preludes and reports the move list, `layout apply` renames the files and repoints `logs.path` and `upstream_exchanges.cassette_path` in one transaction per move). Its logic lives in `internal/legacymigrate`; every command is a dry run unless `--apply` is passed, and no legacy `LLM_TRACELAB_*` fallback exists in the binaries -- Management HTTP/MCP wiring shared by serve and tests: `cmd/server/management.go` +- CLI split: `server` is the runtime binary and `trajecta` is the operations CLI. A command belongs to `server` only if it needs the proxy process; that is `serve` (also the bare invocation) plus the per-binary `version` and `completion` +- `cmd/server`: `main.go` only exits through `run`; `root.go` wires `serve`, `version`, `completion` and the `--config`/`--format` flags and nothing else. `management.go` holds the management HTTP/MCP wiring shared by serve and tests, `provider_startup_probe.go` the startup provider probe +- `cmd/trajecta`: the operations CLI (build artifact `trajecta`). Its own local, server-less work is the one-off rename migration (`.env` prefix mapping, legacy SQLite merge into Postgres, cassette magic rewrite and structural validation, SQLite archiving) and cassette vault reorganisation (`layout plan` reads the preludes and reports the move list, `layout apply` renames the files and repoints `logs.path` and `upstream_exchanges.cassette_path` in one transaction per move); that logic lives in `internal/legacymigrate`, every command is a dry run unless `--apply` is passed, and no legacy `LLM_TRACELAB_*` fallback exists in the binaries. It also exposes every server-management command group through `internal/cli`: `analyze`, `audit`, `auth`, `config`, `db`, `doctor`, `migrate`, `models`, `provider`, `schema`, `tools`. `root.go` binds `--config`/`--format` to a viper instance and hands it to `internal/cli.Register` +- `internal/cli`: the CLI contract plus those command groups. The contract is the exit-code table, the JSON envelope, the viper-backed `Runtime` (`--config`/`--format`), the `Execute` top-level error path and the ERROR-recording log handler; both binaries share it, `cmd/server` imports the contract only and registers none of the commands +- `internal/appdb`: the config-driven application-database entry points shared by serve startup and the trajecta `db`/`auth`/`audit`/`analyze` commands - driver normalization, migration mode, `MigrateUp`/`OptimizeIndexes`, `Open`/`Initialize` and the auth store opener - Reverse proxy: `internal/proxy` - Recording pipeline: `internal/recorder` - Application store and metadata index: `internal/store` diff --git a/CHANGELOG.md b/CHANGELOG.md index 03231007..3efc12c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) ## [Unreleased] ### Changed +- The `server` binary no longer carries the commands that never needed a proxy process. Every management group - `analyze`, `audit`, `auth`, `config`, `db`, `doctor`, `migrate`, `models`, `provider`, `schema`, `tools` - moved to the `trajecta` CLI, and `server` keeps only what is about running or describing the running binary: `serve` (also the bare `server -c config.yaml` invocation), `version` and `completion`. The command bodies and the CLI contract they were written against now live together in `internal/cli`: the exit-code table, the JSON envelope (`ok`/`command`/`result`/`warnings`/`error`), the viper-backed `Runtime` that owns `-c/--config` and `--format`, and the `Execute` top-level path that classifies an error, records it and writes the envelope to stderr. `cmd/server` imports that contract for its own three commands and registers none of the migrated ones; `cmd/trajecta` binds the same two flags and calls `cli.Register`, so a command that used to be spelled `server db migrate status` is now `trajecta db migrate status` with the same subcommands, flags, exit codes and JSON shape. The application-database entry points the two binaries now share - driver normalization, the migration-mode decision, `MigrateUp`/`OptimizeIndexes`, `Open`/`Initialize` and the auth store opener - moved to `internal/appdb`, which is also where serve startup reads them from, so the startup path and the `db`/`auth` commands can no longer drift apart; `NormalizeMCPPath`/`ValidateServe` moved to `internal/config` and the provider-probe target helpers to `internal/providerprobe` for the same reason. `Taskfile.yml` runs `migrate`, `migrate:db:up`, `auth:init-user` and `auth:create-token` through `go run ./cmd/trajecta` now, and `task test:e2e` covers `./cmd/trajecta` alongside `./cmd/server`. +- `trajecta` reports failures through the shared contract instead of its own copy of it. An error envelope is now written to stderr rather than stdout, which is what lets `trajecta db migrate status --format json > state.json` keep the result parseable, and an internal failure exits `8` instead of `1`, matching `server`. Commands that emit an `ok: false` envelope and then return an error still write exactly one JSON object: `cli.Execute` records that a result was already written and does not add a second one. - The 协议 tab draws each half of the parsed exchange as a collapsed tree instead of an expanded one. Every node used to be opened at once with its own raw JSON block, so a request of forty messages pushed the response half of the page past the end of the scroll; a row is now one value, opened on demand, labelled with the parser's name for it (`message`, `tool_call`, `tool_result`, with the role beside it), showing the node's own text when it has one, and falling back to the shape of the value (`future_item`, `object`, `array`) rather than printing `unknown`. Clicking a row opens that value in a dialog named by its JSON path, where drilling further stays in the dialog, and a payload the parser could not read as JSON - an error body, a stream fragment - is shown as the text it is. - The trace detail tab strip is pinned under the page header. The 会话 tab lands on the newest turn of a long conversation, which is the bottom of the page, and the tab row that scrolled away left the reader no way to change panels except going back to the top. `AppShell` now measures every element marked `data-sticky-chrome`, hands each the offset it pins at through `--sticky-offset`, and keeps `--sticky-chrome-h` as the total for `scroll-padding-top`; the conversation rail's height is capped against that total so it still fits below the taller chrome. - The 工具调用 tab's definition dialog draws the declaration as the form it describes. A JSON Schema read as JSON puts every parameter two levels down and answers "what is the declaration" rather than "what can I pass", so parameters are now one row each with their type, requiredness, description, enumeration values, default, format, range and `additionalProperties`, nested objects and array elements indented under the parameter that declares them, and the raw schema kept behind a fold for the keywords a form cannot draw. The dialog's summary tag came from `buildToolSchemaSummary`, which spoke English inside a localized dialog, and is gone with its unit tests. diff --git a/README.md b/README.md index 00730ff5..df948fc7 100644 --- a/README.md +++ b/README.md @@ -210,7 +210,8 @@ func TestChat(t *testing.T) { | 目录 | 职责 | | --- | --- | -| `cmd/server` | CLI 入口与全部子命令（serve、db、auth、provider、models、audit、analyze、doctor…） | +| `cmd/server` | 运行时二进制入口：`serve`（以及 `version`、`completion`） | +| `cmd/trajecta`、`internal/cli` | 运维 CLI 与两个二进制共享的命令契约：upgrade、layout、db、auth、provider、models、audit、analyze、doctor… | | `internal/proxy` | 反向代理、协议入口归一化、流式响应拦截 | | `internal/recorder` | cassette 录制与落盘 | | `pkg/recordfile` | 录制格式解析与写入（V3 写入，V2 / 旧 magic 兼容读取） | diff --git a/README_EN.md b/README_EN.md index 9a0f5518..e0817fa1 100644 --- a/README_EN.md +++ b/README_EN.md @@ -210,7 +210,8 @@ The complete preset list (25 registry keys, 20 distinct providers after alias de | Directory | Responsibility | | --- | --- | -| `cmd/server` | CLI entry point and all subcommands (serve, db, auth, provider, models, audit, analyze, doctor…) | +| `cmd/server` | Runtime binary entry point: `serve` (plus `version` and `completion`) | +| `cmd/trajecta`, `internal/cli` | Operations CLI and the command contract both binaries share: upgrade, layout, db, auth, provider, models, audit, analyze, doctor… | | `internal/proxy` | Reverse proxy, protocol entry normalization, streaming interception | | `internal/recorder` | Cassette recording and persistence | | `pkg/recordfile` | Record format parsing and writing (writes V3, reads V2 and the legacy magic) | diff --git a/Taskfile.yml b/Taskfile.yml index 4fb355ae..32686e06 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -129,7 +129,7 @@ tasks: test:e2e: desc: Run local end-to-end tests without external network dependencies cmds: - - go test ./internal/proxy ./internal/monitor ./cmd/server ./unittest + - go test ./internal/proxy ./internal/monitor ./cmd/server ./cmd/trajecta ./unittest atif:validate: desc: Validate ATIF-v1.8 JSONL with pinned Harbor models (install scripts/atif-requirements.txt first) @@ -210,12 +210,12 @@ tasks: migrate: desc: Rewrite legacy V2 cassettes to V3 and rebuild SQLite index cmds: - - go run ./cmd/server migrate -c {{.CONFIG}} + - go run ./cmd/trajecta migrate -c {{.CONFIG}} migrate:db:up: desc: Apply application database migrations cmds: - - go run ./cmd/server db migrate up -c {{.CONFIG}} + - go run ./cmd/trajecta db migrate up -c {{.CONFIG}} migrate:legacy: desc: Inspect a pre-rename deployment; pass ARGS="--apply" to write changes @@ -225,12 +225,12 @@ tasks: auth:init-user: desc: Initialize an auth user, pass USER and PASSWORD cmds: - - go run ./cmd/server auth init-user -c {{.CONFIG}} --username "{{.USER}}" --password "{{.PASSWORD}}" + - go run ./cmd/trajecta auth init-user -c {{.CONFIG}} --username "{{.USER}}" --password "{{.PASSWORD}}" auth:create-token: desc: Create an API token for a user, pass USER and NAME cmds: - - go run ./cmd/server auth create-token -c {{.CONFIG}} --username "{{.USER}}" --name "{{.NAME}}" + - go run ./cmd/trajecta auth create-token -c {{.CONFIG}} --username "{{.USER}}" --name "{{.NAME}}" run: desc: Start proxy and monitor with local config diff --git a/cmd/server/completion.go b/cmd/server/completion.go index 9ed40cb4..1e49f0bf 100644 --- a/cmd/server/completion.go +++ b/cmd/server/completion.go @@ -1,6 +1,9 @@ package main -import "github.com/spf13/cobra" +import ( + "github.com/kingfs/Trajecta/internal/cli" + "github.com/spf13/cobra" +) func newCompletionCommand(root *cobra.Command) *cobra.Command { cmd := &cobra.Command{ @@ -8,7 +11,7 @@ func newCompletionCommand(root *cobra.Command) *cobra.Command { Short: "Generate shell completion scripts", RunE: func(cmd *cobra.Command, args []string) error { if len(args) != 1 { - return cliUsageError("completion requires one shell: bash, zsh, fish, or powershell", "shell") + return cli.UsageError("completion requires one shell: bash, zsh, fish, or powershell", "shell") } switch args[0] { case "bash": @@ -20,7 +23,7 @@ func newCompletionCommand(root *cobra.Command) *cobra.Command { case "powershell": return root.GenPowerShellCompletion(cmd.OutOrStdout()) default: - return cliUsageError("unsupported shell "+args[0], "shell") + return cli.UsageError("unsupported shell "+args[0], "shell") } }, } diff --git a/cmd/server/management.go b/cmd/server/management.go index 217712e0..fef9b47c 100644 --- a/cmd/server/management.go +++ b/cmd/server/management.go @@ -2,11 +2,8 @@ package main import ( "crypto/sha256" - "errors" - "log/slog" "net/http" - "strings" "time" "github.com/kingfs/Trajecta/internal/auth" @@ -60,7 +57,7 @@ func newManagementMuxWithFunctionExecutorManager( // endpoint for older clients. Stateless: true, }) - mux.Handle(normalizeMCPPathMust(cfg.MCP.Path), auth.Middleware(mcpHandler, "trajecta-mcp", verifier)) + mux.Handle(config.EffectiveMCPPath(cfg), auth.Middleware(mcpHandler, "trajecta-mcp", verifier)) } functionExecutorConfig := cfg.ResponsesFunctionExecutorsConfig() if functionExecutorManager != nil { @@ -102,33 +99,7 @@ func effectiveMCPPath(cfg *config.Config) string { if !cfg.MCP.Enabled { return "" } - return normalizeMCPPathMust(cfg.MCP.Path) -} - -func normalizeMCPPathMust(path string) string { - normalized, err := normalizeMCPPath(path) - if err != nil { - panic(err) - } - return normalized -} - -func normalizeMCPPath(path string) (string, error) { - path = strings.TrimSpace(path) - if path == "" { - return "/mcp", nil - } - if path == "/" { - return "", errors.New("mcp.path must not be /") - } - if !strings.HasPrefix(path, "/") { - path = "/" + path - } - path = strings.TrimRight(path, "/") - if path == "" { - return "", errors.New("mcp.path must not be empty") - } - return path, nil + return config.EffectiveMCPPath(cfg) } func logResolvedTargets(rtr *router.Router) { diff --git a/cmd/server/provider_startup_probe.go b/cmd/server/provider_startup_probe.go index f12bb4ab..62eb7813 100644 --- a/cmd/server/provider_startup_probe.go +++ b/cmd/server/provider_startup_probe.go @@ -41,7 +41,7 @@ func applyStartupProviderProbeToTarget(ctx context.Context, target *config.Upstr } probeTarget := *target probeTarget.Upstream = *upstreamCfg - report, err := providerprobe.Probe(ctx, providerProbeTargetFromConfig(probeTarget), client) + report, err := providerprobe.Probe(ctx, providerprobe.TargetFromConfig(probeTarget), client) if err != nil { slog.Warn("Startup provider probe failed; continuing with explicit configuration", "provider_id", strings.TrimSpace(probeTarget.ID), diff --git a/cmd/server/root.go b/cmd/server/root.go index 1139fae1..239925cd 100644 --- a/cmd/server/root.go +++ b/cmd/server/root.go @@ -1,534 +1,59 @@ package main import ( - "context" - "encoding/json" - "errors" - "fmt" - "io" - "log/slog" - "os" - "reflect" - "strings" - "sync" - + "github.com/kingfs/Trajecta/internal/cli" "github.com/spf13/cobra" - "github.com/spf13/pflag" - "github.com/spf13/viper" ) -// cliName is the server artifact name: it drives the usage text, the version -// payload and the CLI schema name. The product name (for example the Codex -// model_provider) is separate and stays "trajecta". +// cliName is the server artifact name: it drives the usage text and the +// version payload. The product name (for example the Codex model_provider) is +// separate and stays "trajecta". const cliName = "server" func run(args []string) int { - logger := slog.New(slog.NewTextHandler(os.Stderr, nil)) - slog.SetDefault(logger) - - runtime := newCLIRuntime() + runtime := cli.NewRuntime() cmd := newRootCommandWithRuntime(runtime) - cmd.SetArgs(normalizeLegacyFlagArgs(args)) - cmd.SetOut(os.Stdout) - cmd.SetErr(os.Stderr) - recorder := &cliErrorRecorder{} - slog.SetDefault(slog.New(newCLILogHandler(os.Stderr, recorder))) - - if err := cmd.Execute(); err != nil { - exit := cliExitErrorFromErr(err) - // Commands that only log their failure and return a bare code (the - // trace store, config loading) would otherwise collapse into - // COMMAND_FAILED/"command failed"; surface the logged cause instead. - if recorder.has() && fallback(exit.message, "command failed") == "command failed" { - exit.message = recorder.message() - } - // cobra fails before the persistent --format flag is bound when the - // command itself is unknown, so read it from argv as a fallback. - format := runtime.outputFormat() - if format != "json" { - if fromArgs := outputFormatFromArgs(args); fromArgs != "" { - format = fromArgs - } - } - writeCLIError(cmd.ErrOrStderr(), format, exit) - return exit.code - } - return 0 -} - -// outputFormatFromArgs finds an explicit --format value in the raw argv, for the -// failure paths where cobra never bound the flag. -func outputFormatFromArgs(args []string) string { - for i, arg := range args { - if arg == "--format" && i+1 < len(args) { - return args[i+1] - } - if value, ok := strings.CutPrefix(arg, "--format="); ok { - return value - } - } - return "" -} - -// cliExitErrorFromErr maps a command error to the documented exit contract: -// usage/validation errors exit 3, internal failures exit 8. -func cliExitErrorFromErr(err error) cliExitError { - var exit cliExitError - if errors.As(err, &exit) { - return exit - } - message := err.Error() - if isUsageError(err) { - return cliUsageError(message, "") - } - return cliExitError{ - code: exitCodeInternal, - category: errorCategoryInternal, - errCode: "INTERNAL_ERROR", - message: message, - } -} - -// isUsageError reports whether a bare cobra error is really a usage error. -func isUsageError(err error) bool { - if err == nil { - return false - } - message := err.Error() - for _, prefix := range []string{ - "unknown command ", - "unknown flag: ", - "unknown shorthand flag: ", - "required flag(s) ", - "invalid argument ", - "accepts at most ", - "accepts between ", - "accepts ", - } { - if strings.HasPrefix(message, prefix) { - return true - } - } - return false -} - -// cliErrorRecorder keeps the most recent ERROR log line so a command that only -// logs its failure can still report the real cause in the JSON envelope. -type cliErrorRecorder struct { - mu sync.Mutex - lastText string -} - -func (r *cliErrorRecorder) record(message string) { - r.mu.Lock() - defer r.mu.Unlock() - r.lastText = message -} - -func (r *cliErrorRecorder) has() bool { - r.mu.Lock() - defer r.mu.Unlock() - return strings.TrimSpace(r.lastText) != "" -} - -func (r *cliErrorRecorder) message() string { - r.mu.Lock() - defer r.mu.Unlock() - return r.lastText -} - -// cliLogHandler writes the usual text log and mirrors ERROR records into the -// recorder. -type cliLogHandler struct { - primary slog.Handler - recorder *cliErrorRecorder -} - -func newCLILogHandler(w io.Writer, recorder *cliErrorRecorder) slog.Handler { - return &cliLogHandler{ - primary: slog.NewTextHandler(w, nil), - recorder: recorder, - } -} - -func (h *cliLogHandler) Enabled(ctx context.Context, level slog.Level) bool { - return h.primary.Enabled(ctx, level) -} - -func (h *cliLogHandler) Handle(ctx context.Context, record slog.Record) error { - if record.Level >= slog.LevelError { - // The `error` attribute carries the real cause; the remaining attributes - // are context that the human-readable log line already shows. - var cause string - var context []string - record.Attrs(func(attr slog.Attr) bool { - if attr.Key == "error" { - if cause == "" { - cause = attr.Value.String() - } - return true - } - context = append(context, fmt.Sprintf("%s=%v", attr.Key, attr.Value.Any())) - return true - }) - message := record.Message - if cause != "" { - message = message + ": " + cause - } else if len(context) > 0 { - message = message + ": " + strings.Join(context, " ") - } - h.recorder.record(message) - } - return h.primary.Handle(ctx, record) -} - -func (h *cliLogHandler) WithAttrs(attrs []slog.Attr) slog.Handler { - return &cliLogHandler{primary: h.primary.WithAttrs(attrs), recorder: h.recorder} -} - -func (h *cliLogHandler) WithGroup(name string) slog.Handler { - return &cliLogHandler{primary: h.primary.WithGroup(name), recorder: h.recorder} + return cli.Execute(args, cmd, runtime) } +// newRootCommand builds the server command tree. It exists for the tests; the +// process entry point builds the tree with its own runtime. func newRootCommand() *cobra.Command { - return newRootCommandWithRuntime(newCLIRuntime()) + return newRootCommandWithRuntime(cli.NewRuntime()) } -func newRootCommandWithRuntime(runtime *cliRuntime) *cobra.Command { +// newRootCommandWithRuntime wires the server binary: the bare invocation and +// `serve` start the proxy, and completion and version describe this artifact. +// +// Every server-management operation (db, auth, analyze, audit, doctor, models, +// provider, tools, config, migrate, schema) lives in the trajecta CLI instead; +// see internal/cli. +func newRootCommandWithRuntime(runtime *cli.Runtime) *cobra.Command { cmd := &cobra.Command{ Use: cliName, Short: "Postgres-first LLM gateway with local Responses runtime and record/replay", SilenceUsage: true, SilenceErrors: true, PersistentPreRunE: func(cmd *cobra.Command, args []string) error { - return runtime.validateOutputFormat() + return runtime.ValidateOutputFormat() }, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { - return runServeWithConfig(runtime.configPath()) + return cli.RunCode(func() int { + return runServeWithConfig(runtime.ConfigPath()) }) }, } cmd.SetFlagErrorFunc(func(cmd *cobra.Command, err error) error { - return cliUsageError(err.Error(), "flag") + return cli.UsageError(err.Error(), "flag") }) cmd.PersistentFlags().StringP("config", "c", "config.yaml", "Path to configuration file") cmd.PersistentFlags().String("format", "text", "Output format: text or json") - mustBindPFlag(runtime.settings, "config", cmd.PersistentFlags().Lookup("config")) - mustBindPFlag(runtime.settings, "format", cmd.PersistentFlags().Lookup("format")) + runtime.BindPFlag("config", cmd.PersistentFlags().Lookup("config")) + runtime.BindPFlag("format", cmd.PersistentFlags().Lookup("format")) cmd.AddCommand( newServeCommand(runtime), - newMigrateCommand(runtime), - newDBCommand(runtime), - newConfigCommand(runtime), - newDoctorCommand(runtime), - newProviderCommand(runtime), - newModelsCommand(runtime), - newToolsCommand(runtime), - newAuditCommand(runtime), - newAuthCommand(runtime), - newAnalyzeCommand(runtime), newVersionCommand(runtime), - newSchemaCommand(runtime, cmd), newCompletionCommand(cmd), ) return cmd } - -type cliRuntime struct { - settings *viper.Viper -} - -func newCLIRuntime() *cliRuntime { - settings := viper.New() - settings.SetEnvPrefix("TRAJECTA") - settings.SetEnvKeyReplacer(strings.NewReplacer("-", "_")) - settings.AutomaticEnv() - return &cliRuntime{settings: settings} -} - -func (rt *cliRuntime) configPath() string { - if rt == nil || rt.settings == nil { - return "config.yaml" - } - if path := rt.settings.GetString("config"); path != "" { - return path - } - return "config.yaml" -} - -func (rt *cliRuntime) outputFormat() string { - if rt == nil || rt.settings == nil { - return "text" - } - if format := rt.settings.GetString("format"); format != "" { - return format - } - return "text" -} - -func (rt *cliRuntime) validateOutputFormat() error { - switch rt.outputFormat() { - case "text", "json": - return nil - default: - return cliExitError{ - code: exitCodeUsage, - category: errorCategoryUsage, - errCode: "INVALID_OUTPUT_FORMAT", - message: "--format must be one of: text, json", - field: "format", - } - } -} - -func runCode(run func() int) error { - if code := run(); code != 0 { - return cliExitErrorFromCode(code) - } - return nil -} - -func cliUsageError(message string, field string) cliExitError { - return cliExitError{ - code: exitCodeUsage, - category: errorCategoryUsage, - errCode: "USAGE_ERROR", - message: message, - field: field, - } -} - -func requireSubcommand(cmd *cobra.Command) error { - if cmd.Flag("format") != nil && cmd.Flag("format").Value.String() == "json" { - return cliUsageError(fmt.Sprintf("%s requires a subcommand", cmd.CommandPath()), "command") - } - if err := cmd.Help(); err != nil { - return cliExitError{ - code: exitCodeInternal, - category: errorCategoryInternal, - errCode: "HELP_RENDER_FAILED", - message: err.Error(), - } - } - return cliExitErrorFromCode(exitCodeUsage) -} - -func normalizeLegacyFlagArgs(args []string) []string { - normalized := make([]string, len(args)) - for i, arg := range args { - if strings.HasPrefix(arg, "-") && !strings.HasPrefix(arg, "--") && len(arg) > 2 { - normalized[i] = "-" + arg - continue - } - normalized[i] = arg - } - return normalized -} - -func mustBindPFlag(settings *viper.Viper, key string, flag *pflag.Flag) { - if err := settings.BindPFlag(key, flag); err != nil { - panic(err) - } -} - -type cliExitError struct { - code int - category string - errCode string - message string - field string -} - -func (e cliExitError) Error() string { - if e.message != "" { - return e.message - } - return fmt.Sprintf("exit %d", e.code) -} - -const ( - exitCodeAPI = 1 - exitCodeUsage = 3 - exitCodeInternal = 8 - errorCategoryAPI = "api" - errorCategoryUsage = "usage" - errorCategoryInternal = "internal" -) - -func cliExitErrorFromCode(code int) cliExitError { - switch code { - case 2, exitCodeUsage: - return cliExitError{ - code: exitCodeUsage, - category: errorCategoryUsage, - errCode: "USAGE_ERROR", - message: "command usage is invalid", - } - case exitCodeInternal: - return cliExitError{ - code: exitCodeInternal, - category: errorCategoryInternal, - errCode: "INTERNAL_ERROR", - message: "command failed with an internal error", - } - default: - return cliExitError{ - code: code, - category: errorCategoryAPI, - errCode: "COMMAND_FAILED", - message: "command failed", - } - } -} - -type cliEnvelope struct { - OK bool `json:"ok"` - Command string `json:"command,omitempty"` - Result any `json:"result,omitempty"` - Warnings []string `json:"warnings"` - Error *cliError `json:"error,omitempty"` -} - -type cliError struct { - Code string `json:"code"` - Category string `json:"category"` - Message string `json:"message"` - Field string `json:"field,omitempty"` - Retryable bool `json:"retryable"` - SafeToRetry bool `json:"safe_to_retry"` -} - -func writeCLIResult(w io.Writer, format string, command string, result any, text func(io.Writer) error) error { - return writeCLIResultWithWarnings(w, format, command, result, resultWarnings(result), text) -} - -// writeCLIResultWithWarnings writes the text form or a JSON envelope. The -// envelope-level warnings mirror the ones the text form prints, so a caller that -// only reads envelope.warnings does not miss them. -func writeCLIResultWithWarnings(w io.Writer, format string, command string, result any, warnings []string, text func(io.Writer) error) error { - if format != "json" { - if text == nil { - return nil - } - return text(w) - } - if warnings == nil { - warnings = []string{} - } - return writeJSON(w, cliEnvelope{ - OK: true, - Command: command, - Result: result, - Warnings: warnings, - }) -} - -// resultWarnings collects the warnings a command attached to its own result, so -// the envelope carries them whether they are a []string field or a map entry. -func resultWarnings(result any) []string { - switch value := result.(type) { - case map[string]any: - return stringSlice(value["warnings"]) - case []map[string]any: - var out []string - for _, item := range value { - out = append(out, stringSlice(item["warnings"])...) - } - return out - } - return warningsFromStruct(result) -} - -func stringSlice(value any) []string { - switch typed := value.(type) { - case []string: - return typed - case []any: - out := make([]string, 0, len(typed)) - for _, item := range typed { - if text, ok := item.(string); ok && text != "" { - out = append(out, text) - } - } - return out - case string: - if typed == "" { - return nil - } - return []string{typed} - } - return nil -} - -// warningsFromStruct reads a top-level `warnings` []string field, plus a -// `reports[].warnings` list, through reflection so the helper stays dependency -// free and works for the report structs the provider commands return. -func warningsFromStruct(result any) []string { - value := reflect.ValueOf(result) - for value.Kind() == reflect.Pointer || value.Kind() == reflect.Interface { - if value.IsNil() { - return nil - } - value = value.Elem() - } - if value.Kind() != reflect.Struct { - return nil - } - var out []string - if field := value.FieldByName("Warnings"); field.IsValid() && field.Kind() == reflect.Slice { - out = append(out, stringSlice(field.Interface())...) - } - if field := value.FieldByName("Reports"); field.IsValid() && field.Kind() == reflect.Slice { - for i := 0; i < field.Len(); i++ { - item := field.Index(i) - for item.Kind() == reflect.Pointer { - if item.IsNil() { - break - } - item = item.Elem() - } - if item.Kind() != reflect.Struct { - continue - } - if warnings := item.FieldByName("Warnings"); warnings.IsValid() && warnings.Kind() == reflect.Slice { - out = append(out, stringSlice(warnings.Interface())...) - } - } - } - return out -} - -func writeCLIError(w io.Writer, format string, exit cliExitError) { - if format != "json" { - if exit.message != "" { - fmt.Fprintln(w, exit.message) - } - return - } - _ = writeJSON(w, cliEnvelope{ - OK: false, - Warnings: []string{}, - Error: &cliError{ - Code: fallback(exit.errCode, "COMMAND_FAILED"), - Category: fallback(exit.category, errorCategoryAPI), - Message: fallback(exit.message, "command failed"), - Field: exit.field, - Retryable: exit.category != errorCategoryUsage, - SafeToRetry: exit.category != errorCategoryUsage, - }, - }) -} - -func writeJSON(w io.Writer, value any) error { - encoder := json.NewEncoder(w) - encoder.SetIndent("", " ") - return encoder.Encode(value) -} - -func fallback(value string, fallbackValue string) string { - if value != "" { - return value - } - return fallbackValue -} diff --git a/cmd/server/serve.go b/cmd/server/serve.go index 452aa870..f720ef8c 100644 --- a/cmd/server/serve.go +++ b/cmd/server/serve.go @@ -3,18 +3,17 @@ package main import ( "context" "errors" - "fmt" "log/slog" "net/http" "os" "os/signal" - "strings" "sync" "syscall" "time" - "github.com/kingfs/Trajecta/internal/auth" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/channel" + "github.com/kingfs/Trajecta/internal/cli" "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/observeworker" "github.com/kingfs/Trajecta/internal/proxy" @@ -27,19 +26,14 @@ import ( "github.com/spf13/pflag" ) -var ( - authMigrateDatabaseUp = auth.MigrateDatabaseUp - authOpenDatabase = auth.OpenDatabase -) - -func newServeCommand(runtime *cliRuntime) *cobra.Command { +func newServeCommand(runtime *cli.Runtime) *cobra.Command { return &cobra.Command{ Use: "serve", Short: "Start the proxy, recorder, monitor, and MCP management endpoints", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { - return runServeWithConfig(runtime.configPath()) + return cli.RunCode(func() int { + return runServeWithConfig(runtime.ConfigPath()) }) }, } @@ -57,7 +51,7 @@ func parseConfigPath(name string, args []string) (string, int) { fs := pflag.NewFlagSet(name, pflag.ContinueOnError) configPath := fs.StringP("config", "c", "config.yaml", "Path to configuration file") fs.SetOutput(os.Stderr) - if err := fs.Parse(normalizeLegacyFlagArgs(args)); err != nil { + if err := fs.Parse(cli.NormalizeLegacyFlagArgs(args)); err != nil { return "", 2 } return *configPath, 0 @@ -73,7 +67,7 @@ func runServeWithConfig(configPath string) int { slog.Error("Startup provider probe failed", "error", err) return 1 } - if err := validateServeConfig(cfg); err != nil { + if err := config.ValidateServe(cfg); err != nil { slog.Error("Invalid serve config", "error", err) return 1 } @@ -85,7 +79,7 @@ func runServeWithConfig(configPath string) int { // the outside - otherwise presents as a process that started cleanly, // printed nothing, and sat at 0% CPU while the database burned a core. slog.Info("Applying application database migrations...", "driver", cfg.DatabaseDriver()) - if err := migrateApplicationDatabaseUp(cfg, 0); err != nil { + if err := appdb.MigrateUp(cfg, 0); err != nil { slog.Error("Failed to migrate application database", "error", err) return 1 } @@ -93,7 +87,7 @@ func runServeWithConfig(configPath string) int { slog.Info("Starting LLM Proxy...", "version", Version, "go_version", "1.25+") - authStore, err := openAuthStore(cfg) + authStore, err := appdb.OpenAuthStore(cfg) if err != nil { slog.Error("Failed to initialize auth store", "error", err) return 1 @@ -388,68 +382,6 @@ func startTraceStoreBackgroundSync(ctx context.Context, traceStore *store.Store, }() } -func openAuthStore(cfg *config.Config) (*auth.Store, error) { - return openAuthStoreWithAutoSchema(cfg) -} - -func openAuthStoreWithAutoSchema(cfg *config.Config) (*auth.Store, error) { - driver := normalizeAuthStoreDriver(cfg.DatabaseDriver()) - switch driver { - case "sqlite": - if cfg.DatabaseAutoMigrate() { - if err := authMigrateDatabaseUp(driver, cfg.DatabaseDSN(), 0); err != nil { - return nil, fmt.Errorf("migrate database: %w", err) - } - } - case "postgres": - // Postgres auth tables are owned by the application migration set, which - // serve applies before opening the auth store. - default: - return nil, fmt.Errorf("auth store driver %q is not supported yet", driver) - } - - st, err := authOpenDatabase( - driver, - cfg.DatabaseDSN(), - cfg.DatabaseMaxOpenConns(), - cfg.DatabaseMaxIdleConns(), - ) - if err != nil { - return nil, err - } - return st, nil -} - -func normalizeAuthStoreDriver(driver string) string { - driver = strings.ToLower(strings.TrimSpace(driver)) - switch driver { - case "": - // See store.normalizeDatabaseDriver: an unset driver means Postgres. - return "postgres" - case "postgresql": - return "postgres" - default: - return driver - } -} - -func validateServeConfig(cfg *config.Config) error { - switch driver := cfg.DatabaseDriver(); driver { - case "sqlite", "postgres", "postgresql": - default: - return fmt.Errorf("database.driver %q is not supported yet; use sqlite or postgres", driver) - } - if cfg.MCP.Enabled && cfg.Monitor.Port == "" { - return errors.New("monitor.port is required when mcp.enabled=true") - } - if cfg.MCP.Enabled { - if _, err := normalizeMCPPath(cfg.MCP.Path); err != nil { - return err - } - } - return nil -} - // validateServeRouterConfig reports router-level configuration problems that // make the local Responses server unusable. It is a preflight diagnostic, not a // startup gate: callers log the returned error as a warning so the process still diff --git a/cmd/server/serve_commands_test.go b/cmd/server/serve_commands_test.go new file mode 100644 index 00000000..b67c3905 --- /dev/null +++ b/cmd/server/serve_commands_test.go @@ -0,0 +1,717 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "log/slog" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/kingfs/Trajecta/internal/auth" + "github.com/kingfs/Trajecta/internal/channel" + "github.com/kingfs/Trajecta/internal/config" + "github.com/kingfs/Trajecta/internal/router" + "github.com/kingfs/Trajecta/internal/store" + "github.com/kingfs/Trajecta/internal/upstream" + "github.com/modelcontextprotocol/go-sdk/mcp" + _ "modernc.org/sqlite" +) + +func TestLogResolvedUpstreamConfigIncludesRoutingDiagnostics(t *testing.T) { + var buf bytes.Buffer + logger := slog.New(slog.NewTextHandler(&buf, nil)) + prev := slog.Default() + slog.SetDefault(logger) + t.Cleanup(func() { + slog.SetDefault(prev) + }) + + logResolvedUpstreamConfig( + upstream.ResolvedUpstream{ + BaseURL: "https://generativelanguage.googleapis.com", + ProviderPreset: "google_genai", + ProtocolFamily: upstream.ProtocolFamilyGoogleGenAI, + RoutingProfile: upstream.RoutingProfileGoogleAIStudio, + }, + upstream.StartupDiagnostics{ + ConnectivityEndpoint: "/v1beta/models", + ConnectivityURL: "https://generativelanguage.googleapis.com/v1beta/models", + ModelRoutingHint: "model is selected in the request path", + }, + ) + + output := buf.String() + for _, want := range []string{ + "Resolved upstream config", + "provider_preset=google_genai", + "protocol_family=google_genai", + "routing_profile=google_ai_studio", + "connectivity_endpoint=/v1beta/models", + "connectivity_url=https://generativelanguage.googleapis.com/v1beta/models", + "model_routing_hint=\"model is selected in the request path\"", + } { + if !strings.Contains(output, want) { + t.Fatalf("log output = %q, want contain %q", output, want) + } + } +} + +func TestRouterConfigFromChannelsUsesDatabaseTargets(t *testing.T) { + st, err := store.New(t.TempDir()) + if err != nil { + t.Fatalf("store.New() error = %v", err) + } + defer st.Close() + + if _, err := st.UpsertChannelConfig(store.ChannelConfigRecord{ + ID: "db-channel", + Name: "DB Channel", + BaseURL: "https://db.example.com/v1", + ProviderPreset: "openai", + HeadersJSON: "{}", + Enabled: true, + }); err != nil { + t.Fatalf("UpsertChannelConfig() error = %v", err) + } + if err := st.ReplaceChannelModels("db-channel", []store.ChannelModelRecord{ + {Model: "gpt-5", Source: "manual", Enabled: true}, + }); err != nil { + t.Fatalf("ReplaceChannelModels() error = %v", err) + } + + cfg := &config.Config{} + cfg.Upstream.BaseURL = "https://yaml.example.com/v1" + cfg.Upstream.ProviderPreset = "openai" + + routerCfg, source, err := routerConfigFromChannels(cfg, channel.NewService(st)) + if err != nil { + t.Fatalf("routerConfigFromChannels() error = %v", err) + } + if source != "database" { + t.Fatalf("source = %q, want database", source) + } + if len(routerCfg.Upstreams) != 1 || routerCfg.Upstreams[0].ID != "db-channel" { + t.Fatalf("routerCfg.Upstreams = %#v", routerCfg.Upstreams) + } + if routerCfg.Upstream.BaseURL != "" { + t.Fatalf("routerCfg.Upstream.BaseURL = %q, want empty", routerCfg.Upstream.BaseURL) + } +} + +func TestRouterConfigFromChannelsFallsBackToYAML(t *testing.T) { + st, err := store.New(t.TempDir()) + if err != nil { + t.Fatalf("store.New() error = %v", err) + } + defer st.Close() + + cfg := &config.Config{} + cfg.Upstream.BaseURL = "https://yaml.example.com/v1" + cfg.Upstream.ProviderPreset = "openai" + + routerCfg, source, err := routerConfigFromChannels(cfg, channel.NewService(st)) + if err != nil { + t.Fatalf("routerConfigFromChannels() error = %v", err) + } + if source != "yaml" { + t.Fatalf("source = %q, want yaml", source) + } + if routerCfg != cfg { + t.Fatalf("routerConfigFromChannels should return original cfg on YAML fallback") + } +} + +func TestRouterConfigFromChannelsKeepsYAMLWhenExplicitCredentialsExist(t *testing.T) { + st, err := store.New(t.TempDir()) + if err != nil { + t.Fatalf("store.New() error = %v", err) + } + defer st.Close() + + if _, err := st.UpsertChannelConfig(store.ChannelConfigRecord{ + ID: "db-channel", + Name: "DB Channel", + BaseURL: "https://db.example.com/v1", + ProviderPreset: "openai", + HeadersJSON: "{}", + Enabled: true, + }); err != nil { + t.Fatalf("UpsertChannelConfig() error = %v", err) + } + + cfg := &config.Config{} + cfg.Upstreams = []config.UpstreamTargetConfig{ + { + ID: "yaml-channel", + Upstream: config.UpstreamConfig{ + BaseURL: "https://yaml.example.com/v1", + ApiKey: "$env:OPENAI_TEST_KEY", + ProviderPreset: "openai", + }, + Credentials: []config.CredentialConfig{ + { + ID: "primary", + ApiKey: "$env:OPENAI_TEST_KEY", + }, + }, + }, + } + + routerCfg, source, err := routerConfigFromChannels(cfg, channel.NewService(st)) + if err != nil { + t.Fatalf("routerConfigFromChannels() error = %v", err) + } + if source != "yaml" { + t.Fatalf("source = %q, want yaml", source) + } + if routerCfg != cfg { + t.Fatalf("routerConfigFromChannels should return original cfg when YAML credentials are explicit") + } + if got := routerCfg.Upstreams[0].EffectiveCredentials()[0].ID; got != "primary" { + t.Fatalf("credential id = %q, want primary", got) + } +} + +func TestRunServeLogsActionableInvalidUpstreamConfig(t *testing.T) { + var buf bytes.Buffer + logger := slog.New(slog.NewTextHandler(&buf, nil)) + prev := slog.Default() + slog.SetDefault(logger) + t.Cleanup(func() { + slog.SetDefault(prev) + }) + + dir := t.TempDir() + configPath := filepath.Join(dir, "config.yaml") + configBody := []byte(strings.TrimSpace(` +database: + driver: "sqlite" +server: + port: "8080" +monitor: + port: "" +upstream: + base_url: "https://api.anthropic.com" + provider_preset: "anthropic" + protocol_family: "google_genai" +debug: + output_dir: "` + dir + `" + mask_key: false +`)) + if err := os.WriteFile(configPath, configBody, 0o644); err != nil { + t.Fatalf("WriteFile() error = %v", err) + } + + code := runServe([]string{"-c", configPath}) + if code != 1 { + t.Fatalf("runServe() = %d, want 1", code) + } + + output := buf.String() + for _, want := range []string{ + "Invalid upstream config", + `upstream.provider_preset=`, + `anthropic_messages`, + `google_genai`, + } { + if !strings.Contains(output, want) { + t.Fatalf("log output = %q, want contain %q", output, want) + } + } +} + +func TestValidateServeRouterConfigRequiresLocalResponsesServerBackend(t *testing.T) { + cfg := &config.Config{ + ResponsesServer: config.ResponsesServerConfig{}, + Upstreams: []config.UpstreamTargetConfig{ + { + ID: "anthropic-messages", + ModelDiscovery: router.ModelDiscoveryStaticOnly, + StaticModels: []string{"claude-sonnet-4-5"}, + Upstream: config.UpstreamConfig{ + BaseURL: "https://api.anthropic.com/v1", + ProviderPreset: "anthropic", + }, + }, + }, + } + + err := validateServeRouterConfig(cfg, cfg) + if err == nil { + t.Fatal("validateServeRouterConfig() error = nil, want local Responses server backend validation error") + } + if !strings.Contains(err.Error(), router.LocalResponsesServerBackendRequiredError) { + t.Fatalf("validateServeRouterConfig() error = %q, want contain %q", err.Error(), router.LocalResponsesServerBackendRequiredError) + } +} + +func TestStartupProviderProbeDefaultDisabledDoesNotProbe(t *testing.T) { + var calls int + upstreamServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + calls++ + http.NotFound(w, r) + })) + defer upstreamServer.Close() + + cfg := &config.Config{} + cfg.Upstreams = []config.UpstreamTargetConfig{ + { + ID: "local", + Upstream: config.UpstreamConfig{ + BaseURL: upstreamServer.URL + "/api/v1", + }, + }, + } + + if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { + t.Fatalf("applyStartupProviderProbeSuggestions() error = %v", err) + } + if calls != 0 { + t.Fatalf("startup probe made %d HTTP calls with default config, want 0", calls) + } + if cfg.Upstreams[0].Upstream.APIType != "" || cfg.Upstreams[0].Upstream.ProtocolFamily != "" { + t.Fatalf("upstream config was modified while disabled: %+v", cfg.Upstreams[0].Upstream) + } +} + +func TestStartupProviderProbeFillsMissingAPISurface(t *testing.T) { + upstreamServer := newOpenAIProbeTestServer(t) + defer upstreamServer.Close() + + cfg := &config.Config{} + cfg.ProviderProbe.StartupFill = true + cfg.Upstreams = []config.UpstreamTargetConfig{ + { + ID: "local", + Upstream: config.UpstreamConfig{ + BaseURL: upstreamServer.URL + "/api/v1", + }, + }, + } + + if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { + t.Fatalf("applyStartupProviderProbeSuggestions() error = %v", err) + } + got := cfg.Upstreams[0].Upstream + if got.ProtocolFamily != upstream.ProtocolFamilyOpenAICompatible { + t.Fatalf("ProtocolFamily = %q, want %q", got.ProtocolFamily, upstream.ProtocolFamilyOpenAICompatible) + } + if got.APIType != upstream.APITypeResponses { + t.Fatalf("APIType = %q, want %q", got.APIType, upstream.APITypeResponses) + } + if got.Capabilities.Responses == nil || !*got.Capabilities.Responses { + t.Fatalf("capabilities.responses = %v, want true", got.Capabilities.Responses) + } + if got.Capabilities.ChatCompletions == nil || !*got.Capabilities.ChatCompletions { + t.Fatalf("capabilities.chat_completions = %v, want true", got.Capabilities.ChatCompletions) + } + if got.Capabilities.Models == nil || !*got.Capabilities.Models { + t.Fatalf("capabilities.models = %v, want true", got.Capabilities.Models) + } +} + +func TestStartupProviderProbeFailureDoesNotBlockStartup(t *testing.T) { + upstreamServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + http.Error(w, "unavailable", http.StatusServiceUnavailable) + })) + defer upstreamServer.Close() + + cfg := &config.Config{} + cfg.ProviderProbe.StartupFill = true + cfg.Upstreams = []config.UpstreamTargetConfig{ + { + ID: "local", + Upstream: config.UpstreamConfig{ + BaseURL: upstreamServer.URL + "/api/v1", + }, + }, + } + + if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { + t.Fatalf("applyStartupProviderProbeSuggestions() error = %v, want nil best-effort startup", err) + } + got := cfg.Upstreams[0].Upstream + if got.APIType != "" || got.ProtocolFamily != "" || got.Capabilities.Responses != nil || got.Capabilities.ChatCompletions != nil { + t.Fatalf("failed startup probe modified config: %+v", got) + } +} + +func TestStartupProviderProbeDoesNotOverrideExplicitAPISurface(t *testing.T) { + upstreamServer := newOpenAIProbeTestServer(t) + defer upstreamServer.Close() + + responsesDisabled := false + cfg := &config.Config{} + cfg.ProviderProbe.StartupFill = true + cfg.Upstreams = []config.UpstreamTargetConfig{ + { + ID: "local", + Upstream: config.UpstreamConfig{ + BaseURL: upstreamServer.URL + "/api/v1", + APIType: upstream.APITypeChatCompletions, + ProtocolFamily: upstream.ProtocolFamilyAnthropicMessages, + Capabilities: config.UpstreamCapabilitiesConfig{ + Responses: &responsesDisabled, + }, + }, + }, + } + + if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { + t.Fatalf("applyStartupProviderProbeSuggestions() error = %v", err) + } + got := cfg.Upstreams[0].Upstream + if got.ProtocolFamily != upstream.ProtocolFamilyAnthropicMessages { + t.Fatalf("ProtocolFamily = %q, want explicit %q", got.ProtocolFamily, upstream.ProtocolFamilyAnthropicMessages) + } + if got.APIType != upstream.APITypeChatCompletions { + t.Fatalf("APIType = %q, want explicit %q", got.APIType, upstream.APITypeChatCompletions) + } + if got.Capabilities.Responses == nil || *got.Capabilities.Responses { + t.Fatalf("capabilities.responses = %v, want explicit false", got.Capabilities.Responses) + } + if got.Capabilities.Models == nil || !*got.Capabilities.Models { + t.Fatalf("capabilities.models = %v, want missing field filled true", got.Capabilities.Models) + } +} + +func TestRunServeRejectsMCPWithoutMonitorPort(t *testing.T) { + var buf bytes.Buffer + logger := slog.New(slog.NewTextHandler(&buf, nil)) + prev := slog.Default() + slog.SetDefault(logger) + t.Cleanup(func() { + slog.SetDefault(prev) + }) + + dir := t.TempDir() + configPath := filepath.Join(dir, "config.yaml") + configBody := []byte(strings.TrimSpace(` +server: + port: "8080" +monitor: + port: "" +mcp: + enabled: true +upstream: + base_url: "https://api.openai.com/v1" +debug: + output_dir: "` + dir + `" + mask_key: false +`)) + if err := os.WriteFile(configPath, configBody, 0o644); err != nil { + t.Fatalf("WriteFile() error = %v", err) + } + + code := runServe([]string{"-c", configPath}) + if code != 1 { + t.Fatalf("runServe() = %d, want 1", code) + } + if output := buf.String(); !strings.Contains(output, "monitor.port is required when mcp.enabled=true") { + t.Fatalf("log output = %q, want monitor.port requirement", output) + } +} + +func TestRunServeRejectsUnknownDatabaseDriver(t *testing.T) { + var buf bytes.Buffer + logger := slog.New(slog.NewTextHandler(&buf, nil)) + prev := slog.Default() + slog.SetDefault(logger) + t.Cleanup(func() { + slog.SetDefault(prev) + }) + + dir := t.TempDir() + configPath := filepath.Join(dir, "config.yaml") + configBody := []byte(strings.TrimSpace(` +server: + port: "8080" +monitor: + port: "8081" +mcp: + enabled: true +database: + driver: "mysql" + dsn: "mysql://trajecta:secret@mysql:3306/trajecta" +upstream: + base_url: "https://api.openai.com/v1" +debug: + output_dir: "` + dir + `" + mask_key: false +`)) + if err := os.WriteFile(configPath, configBody, 0o644); err != nil { + t.Fatalf("WriteFile() error = %v", err) + } + + code := runServe([]string{"-c", configPath}) + if code != 1 { + t.Fatalf("runServe() = %d, want 1", code) + } + if output := buf.String(); !strings.Contains(output, "database.driver") || + !strings.Contains(output, "mysql") || + !strings.Contains(output, "is not supported yet; use sqlite or postgres") { + t.Fatalf("log output = %q, want unsupported database driver", output) + } +} + +func TestNewManagementMuxServesStreamableMCP(t *testing.T) { + t.Parallel() + + outputDir := t.TempDir() + st, err := store.New(outputDir) + if err != nil { + t.Fatalf("store.New() error = %v", err) + } + defer st.Close() + + cfg := &config.Config{} + cfg.Monitor.Port = "8081" + cfg.MCP.Enabled = true + cfg.MCP.Path = "/mcp" + + httpServer := httptest.NewServer(newManagementMux(st, nil, cfg)) + defer httpServer.Close() + + client := mcp.NewClient(&mcp.Implementation{Name: "test-client", Version: "v0.0.1"}, nil) + session, err := client.Connect(context.Background(), &mcp.StreamableClientTransport{ + Endpoint: httpServer.URL + "/mcp", + }, nil) + if err != nil { + t.Fatalf("client.Connect() error = %v", err) + } + defer session.Close() + if got := session.InitializeResult().ProtocolVersion; got != "2026-07-28" { + t.Fatalf("negotiated MCP protocol version = %q, want 2026-07-28", got) + } + + tools, err := session.ListTools(context.Background(), nil) + if err != nil { + t.Fatalf("session.ListTools() error = %v", err) + } + if len(tools.Tools) != 22 { + t.Fatalf("len(tools.Tools) = %d, want 22", len(tools.Tools)) + } +} + +func TestNewManagementMuxRejectsUnauthorizedMCP(t *testing.T) { + t.Parallel() + + outputDir := t.TempDir() + st, err := store.New(outputDir) + if err != nil { + t.Fatalf("store.New() error = %v", err) + } + defer st.Close() + + cfg := &config.Config{} + cfg.Monitor.Port = "8081" + cfg.MCP.Enabled = true + cfg.MCP.Path = "/mcp" + authStore := newTestAuthStore(t) + defer authStore.Close() + + httpServer := httptest.NewServer(newManagementMux(st, nil, cfg, authStore)) + defer httpServer.Close() + + client := mcp.NewClient(&mcp.Implementation{Name: "test-client", Version: "v0.0.1"}, nil) + _, err = client.Connect(context.Background(), &mcp.StreamableClientTransport{ + Endpoint: httpServer.URL + "/mcp", + }, nil) + if err == nil { + t.Fatalf("client.Connect() error = nil, want unauthorized error") + } + if !strings.Contains(err.Error(), "401") && !strings.Contains(strings.ToLower(err.Error()), "unauthorized") { + t.Fatalf("client.Connect() error = %v, want unauthorized", err) + } +} + +func TestNewManagementMuxServesAuthorizedStreamableMCP(t *testing.T) { + t.Parallel() + + outputDir := t.TempDir() + st, err := store.New(outputDir) + if err != nil { + t.Fatalf("store.New() error = %v", err) + } + defer st.Close() + + cfg := &config.Config{} + cfg.Monitor.Port = "8081" + cfg.MCP.Enabled = true + cfg.MCP.Path = "/mcp" + authStore := newTestAuthStore(t) + defer authStore.Close() + token, err := authStore.CreateToken(context.Background(), "admin", "mcp", auth.DefaultTokenScope, time.Hour) + if err != nil { + t.Fatalf("CreateToken() error = %v", err) + } + + httpServer := httptest.NewServer(newManagementMux(st, nil, cfg, authStore)) + defer httpServer.Close() + + client := mcp.NewClient(&mcp.Implementation{Name: "test-client", Version: "v0.0.1"}, nil) + session, err := client.Connect(context.Background(), &mcp.StreamableClientTransport{ + Endpoint: httpServer.URL + "/mcp", + HTTPClient: &http.Client{Transport: authTransport{Token: token.Token}}, + }, nil) + if err != nil { + t.Fatalf("client.Connect() error = %v", err) + } + defer session.Close() + + tools, err := session.ListTools(context.Background(), nil) + if err != nil { + t.Fatalf("session.ListTools() error = %v", err) + } + if len(tools.Tools) != 22 { + t.Fatalf("len(tools.Tools) = %d, want 22", len(tools.Tools)) + } +} + +func newTestAuthStore(t *testing.T) *auth.Store { + t.Helper() + dbPath := filepath.Join(t.TempDir(), "auth.sqlite3") + if err := auth.MigrateUp(dbPath, 0); err != nil { + t.Fatalf("auth.MigrateUp() error = %v", err) + } + authStore, err := auth.Open(dbPath) + if err != nil { + t.Fatalf("auth.Open() error = %v", err) + } + if _, err := authStore.CreateUser(context.Background(), "admin", "change-me-123"); err != nil { + _ = authStore.Close() + t.Fatalf("CreateUser() error = %v", err) + } + return authStore +} + +type authTransport struct { + Token string +} + +func TestRouterConfigurationKeepsDisabledAndDeletedChannelsAcrossBootstrap(t *testing.T) { + for _, deleted := range []bool{false, true} { + t.Run(map[bool]string{false: "disabled", true: "deleted"}[deleted], func(t *testing.T) { + st, err := store.New(t.TempDir()) + if err != nil { + t.Fatal(err) + } + defer st.Close() + cfg := &config.Config{Upstream: config.UpstreamConfig{BaseURL: "https://yaml.example.invalid/v1", ProviderPreset: "openai"}} + svc := channel.NewService(st) + if _, err := svc.BootstrapFromConfig(cfg); err != nil { + t.Fatal(err) + } + channels, err := st.ListChannelConfigs() + if err != nil { + t.Fatal(err) + } + if len(channels) != 1 { + t.Fatalf("channels=%v", channels) + } + if deleted { + err = st.DeleteChannelConfig(channels[0].ID) + } else { + record := channels[0] + record.Enabled = false + _, err = st.UpsertChannelConfig(record) + } + if err != nil { + t.Fatal(err) + } + if imported, err := svc.BootstrapFromConfig(cfg); err != nil || imported != 0 { + t.Fatalf("bootstrap=%d %v", imported, err) + } + runtimeCfg, source, err := routerConfigFromChannels(cfg, svc) + if err != nil { + t.Fatal(err) + } + if source != "database" || len(runtimeCfg.EffectiveUpstreams()) != 0 { + t.Fatalf("source=%s targets=%v", source, runtimeCfg.EffectiveUpstreams()) + } + }) + } +} + +func (t authTransport) RoundTrip(req *http.Request) (*http.Response, error) { + clone := req.Clone(req.Context()) + clone.Header = req.Header.Clone() + clone.Header.Set("Authorization", "Bearer "+t.Token) + return http.DefaultTransport.RoundTrip(clone) +} + +func TestRootCommandHelpWorksWithConfigShortcut(t *testing.T) { + t.Parallel() + + cmd := newRootCommand() + var out bytes.Buffer + cmd.SetOut(&out) + cmd.SetErr(&out) + cmd.SetArgs([]string{"-c", "config.yaml", "--help"}) + if err := cmd.Execute(); err != nil { + t.Fatalf("Execute() error = %v", err) + } + output := out.String() + for _, want := range []string{ + "Postgres-first LLM gateway with local Responses runtime and record/replay", + "Available Commands:", + "-c, --config string", + } { + if !strings.Contains(output, want) { + t.Fatalf("help output = %q, want contain %q", output, want) + } + } +} + +func TestVersionCommandSupportsJSONEnvelope(t *testing.T) { + t.Parallel() + + cmd := newRootCommand() + var out bytes.Buffer + cmd.SetOut(&out) + cmd.SetErr(&out) + cmd.SetArgs([]string{"version", "--format", "json"}) + if err := cmd.Execute(); err != nil { + t.Fatalf("Execute() error = %v", err) + } + + var envelope struct { + OK bool `json:"ok"` + Command string `json:"command"` + Result struct { + Name string `json:"name"` + Version string `json:"version"` + } `json:"result"` + } + if err := json.Unmarshal(out.Bytes(), &envelope); err != nil { + t.Fatalf("json.Unmarshal() error = %v, output=%q", err, out.String()) + } + if !envelope.OK || envelope.Command != "version" || envelope.Result.Name != cliName || envelope.Result.Version == "" { + t.Fatalf("version envelope = %+v", envelope) + } +} + +func newOpenAIProbeTestServer(t *testing.T) *httptest.Server { + t.Helper() + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch { + case r.Method == http.MethodGet && r.URL.Path == "/api/v1/models": + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{"data":[{"id":"gpt-test"}]}`)) + case r.Method == http.MethodPost && r.URL.Path == "/api/v1/chat/completions": + w.WriteHeader(http.StatusBadRequest) + _, _ = w.Write([]byte(`{"error":{"message":"model is required"}}`)) + case r.Method == http.MethodPost && r.URL.Path == "/api/v1/responses": + w.WriteHeader(http.StatusBadRequest) + _, _ = w.Write([]byte(`{"error":{"message":"input is required"}}`)) + default: + http.NotFound(w, r) + } + })) +} diff --git a/cmd/server/version.go b/cmd/server/version.go index 9aa96ad3..128bcfac 100644 --- a/cmd/server/version.go +++ b/cmd/server/version.go @@ -4,6 +4,7 @@ import ( "fmt" "io" + "github.com/kingfs/Trajecta/internal/cli" "github.com/spf13/cobra" ) @@ -22,7 +23,7 @@ type versionInfo struct { Branch string `json:"branch"` } -func newVersionCommand(runtime *cliRuntime) *cobra.Command { +func newVersionCommand(runtime *cli.Runtime) *cobra.Command { return &cobra.Command{ Use: "version", Short: "Print build version information", @@ -35,7 +36,7 @@ func newVersionCommand(runtime *cliRuntime) *cobra.Command { Date: Date, Branch: Branch, } - return writeCLIResult(cmd.OutOrStdout(), runtime.outputFormat(), "version", info, func(w io.Writer) error { + return cli.WriteResult(cmd.OutOrStdout(), runtime.OutputFormat(), "version", info, func(w io.Writer) error { _, err := fmt.Fprintf(w, "%s version=%s commit=%s date=%s branch=%s\n", cliName, Version, Commit, Date, Branch) return err }) diff --git a/cmd/trajecta/main.go b/cmd/trajecta/main.go index 955ea8c5..3946bf34 100644 --- a/cmd/trajecta/main.go +++ b/cmd/trajecta/main.go @@ -2,6 +2,8 @@ package main import ( "os" + + "github.com/kingfs/Trajecta/internal/cli" ) // Build metadata is injected with -ldflags by the Taskfile and the Dockerfile. @@ -18,15 +20,5 @@ func main() { func run() int { root := newRootCommand() - root.SetOut(os.Stdout) - root.SetErr(os.Stderr) - if err := root.Execute(); err != nil { - command := "" - if activeRuntime != nil { - command = activeRuntime.command - } - activeRuntime.writeError(os.Stdout, command, err) - return exitCodeFor(err) - } - return exitOK + return cli.Execute(os.Args[1:], root, activeOpsRuntime) } diff --git a/cmd/trajecta/root.go b/cmd/trajecta/root.go index e9acf090..690da153 100644 --- a/cmd/trajecta/root.go +++ b/cmd/trajecta/root.go @@ -1,13 +1,16 @@ -// Command trajecta is the Trajecta command-line tool: local, server-less -// Trajecta deployment needs after the project was renamed from llm-tracelab: -// it loads a pre-rename .env file, merges legacy SQLite application databases -// into Postgres, and normalizes or validates existing .http cassettes. +// Command trajecta is the Trajecta command-line tool. It is the operations CLI: +// it runs the deployment work that never needs the proxy process. // -// Every command is a dry run unless --apply is passed. +// That is the one-off rename migration (.env prefix mapping, legacy SQLite +// merge into Postgres, cassette magic rewrite and structural validation, +// SQLite archiving), the cassette vault reorganisation, and every +// server-management command group the server binary used to carry: analyze, +// audit, auth, config, db, doctor, migrate, models, provider, schema and tools. +// +// Commands that write are dry runs unless --apply is passed. package main import ( - "encoding/json" "fmt" "io" "os" @@ -15,18 +18,13 @@ import ( "strings" "github.com/spf13/cobra" + "github.com/spf13/viper" + "github.com/kingfs/Trajecta/internal/cli" "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/legacymigrate" ) -// Exit codes follow the trajecta CLI convention. -const ( - exitOK = 0 - exitFailure = 1 - exitUsage = 3 -) - // options holds every flag value; it is shared by all subcommands. type options struct { envFile string @@ -45,93 +43,36 @@ type options struct { type cliRuntime struct { opts *options - // resultWritten records that a command already emitted its result, so the - // top level error path does not print a second envelope. - resultWritten bool - - // command is the command path being executed, used to label error output. - command string -} - -// activeRuntime is the runtime of the command tree being executed. It lets the -// top level error path render failures in the selected output format. -var activeRuntime *cliRuntime - -// usageError marks an invalid invocation. -type usageError struct { - field string - message string -} - -func (e *usageError) Error() string { - if e.field == "" { - return e.message - } - return fmt.Sprintf("%s: %s", e.field, e.message) + // ops is the shared runtime the server-management command groups resolve + // --config and --format from. It is bound to the same two flags. + ops *cli.Runtime } -func usageErr(field, format string, args ...any) error { - return &usageError{field: field, message: fmt.Sprintf(format, args...)} -} - -// exitCodeFor maps an error to a process exit code. -func exitCodeFor(err error) int { - if err == nil { - return exitOK - } - var usage *usageError - if ok := asUsageError(err, &usage); ok { - return exitUsage - } - return exitFailure -} - -func asUsageError(err error, target **usageError) bool { - for err != nil { - if typed, ok := err.(*usageError); ok { - *target = typed - return true - } - unwrapper, ok := err.(interface{ Unwrap() error }) - if !ok { - return false - } - err = unwrapper.Unwrap() - } - return false -} - -// failureError marks a command that ran correctly but reported a failed -// outcome, such as a table that could not be merged. -type failureError struct{ message string } - -func (e *failureError) Error() string { return e.message } - -// fail builds a failureError. -func fail(format string, args ...any) error { - return &failureError{message: fmt.Sprintf(format, args...)} -} +// activeOpsRuntime is the shared runtime of the command tree being executed, so +// the process entry point can hand it to cli.Execute. +var activeOpsRuntime *cli.Runtime func newRootCommand() *cobra.Command { opts := &options{} runtime := &cliRuntime{opts: opts} - activeRuntime = runtime root := &cobra.Command{ Use: "trajecta", Short: "Trajecta command-line tool", Long: `trajecta is the Trajecta command-line tool. -It runs operations that do not need the server process: reading a deployment -.env file, migrating a pre-rename llm-tracelab deployment onto the current -layout, and maintaining the recorded .http cassettes. Commands that talk to a -running server are added here as well. +It runs every operation that does not need the proxy process. That is the +management command groups the server binary used to carry - analyze, audit, +auth, config, db, doctor, migrate, models, provider, schema and tools - plus the +deployment work specific to this CLI: reading a .env file, migrating a +pre-rename llm-tracelab deployment onto the current layout, and reorganising the +recorded .http cassette vault. -Nothing is written unless --apply is passed.`, +Only the rename-migration and vault-reorganisation commands are dry runs unless +--apply is passed; the management commands act when they are invoked.`, SilenceUsage: true, SilenceErrors: true, PersistentPreRunE: func(cmd *cobra.Command, args []string) error { - runtime.command = cmd.CommandPath() return runtime.validate() }, RunE: func(cmd *cobra.Command, args []string) error { @@ -145,13 +86,29 @@ Nothing is written unless --apply is passed.`, flags.StringVarP(&opts.configPath, "config", "c", "config.yaml", "path to the configuration file") flags.StringVar(&opts.format, "format", "text", "output format: text or json") flags.IntVar(&opts.jobs, "jobs", 0, "concurrent workers (default: number of CPUs)") - flags.BoolVar(&opts.apply, "apply", false, "write changes; without this flag every command only reports what it would do") + flags.BoolVar(&opts.apply, "apply", false, "write changes for the rename-migration and vault-reorganisation commands; those commands only report what they would do without it") flags.StringVar(&opts.sqliteOpen, "sqlite-open", legacymigrate.OpenModeAuto, "how to open a legacy SQLite database: auto, ro, immutable or rw") flags.StringArrayVar(&opts.traceDirs, "trace-dir", nil, "directory holding legacy SQLite databases and cassettes (repeatable)") flags.StringArrayVar(&opts.sqlitePaths, "sqlite", nil, "explicit legacy SQLite database path (repeatable)") flags.IntVar(&opts.batchSize, "batch-size", 500, "rows per Postgres insert batch") + // The server-management groups read --config and --format through the shared + // runtime, which resolves the same two persistent flags. + settings := viper.New() + settings.SetEnvPrefix("TRAJECTA") + settings.SetEnvKeyReplacer(strings.NewReplacer("-", "_")) + settings.AutomaticEnv() + if err := settings.BindPFlag("config", flags.Lookup("config")); err != nil { + panic(err) + } + if err := settings.BindPFlag("format", flags.Lookup("format")); err != nil { + panic(err) + } + runtime.ops = cli.RuntimeFromViper(settings) + activeOpsRuntime = runtime.ops + cli.Register(root, runtime.ops, cli.Options{Name: "trajecta", Version: Version}) + root.AddCommand(newUpgradeCommand(runtime)) root.AddCommand(newLayoutCommand(runtime)) root.AddCommand(newVersionCommand(runtime)) @@ -180,10 +137,27 @@ func (r *cliRuntime) validate() error { return nil } +// usageErr marks an invalid invocation with the shared exit-3 contract. +func usageErr(field, format string, args ...any) error { + return cli.UsageError(fmt.Sprintf(format, args...), field) +} + +// fail marks a command that ran correctly but reported a failed outcome, such +// as a table that could not be merged. It exits 1 with the shared error +// envelope. +func fail(format string, args ...any) error { + return cli.APIError(fmt.Sprintf(format, args...)) +} + func (r *cliRuntime) jsonOutput() bool { return strings.EqualFold(r.opts.format, "json") } +// format is the normalized --format value handed to the shared writer. +func (r *cliRuntime) format() string { + return strings.ToLower(strings.TrimSpace(r.opts.format)) +} + func (r *cliRuntime) workers() int { if r.opts.jobs > 0 { return r.opts.jobs @@ -345,47 +319,12 @@ func (r *cliRuntime) progressFunc() legacymigrate.ProgressFunc { } } -type resultEnvelope struct { - OK bool `json:"ok"` - Command string `json:"command"` - Result any `json:"result,omitempty"` - Warnings []string `json:"warnings,omitempty"` -} - // writeResult renders a command result as JSON or as human-readable text. ok // reflects the command's own verdict (a check that found errors reports // ok=false) and is what callers turn into the exit code. func (r *cliRuntime) writeResult(w io.Writer, command string, ok bool, result any, text func(io.Writer) error, warnings []string) error { - r.resultWritten = true - if r.jsonOutput() { - encoder := json.NewEncoder(w) - encoder.SetIndent("", " ") - return encoder.Encode(resultEnvelope{OK: ok, Command: command, Result: result, Warnings: warnings}) - } - if text == nil { - return nil - } - return text(w) -} - -// writeError renders a failed command. When a command already wrote a result -// (its envelope carries ok=false) nothing else is emitted, so a JSON consumer -// always sees exactly one object on stdout. -func (r *cliRuntime) writeError(w io.Writer, command string, err error) { - if r == nil || r.resultWritten || !r.jsonOutput() { - fmt.Fprintln(os.Stderr, "error: "+err.Error()) - return - } - if command == "" { - command = "trajecta" - } - encoder := json.NewEncoder(w) - encoder.SetIndent("", " ") - _ = encoder.Encode(struct { - OK bool `json:"ok"` - Command string `json:"command"` - Error string `json:"error"` - }{OK: false, Command: command, Error: err.Error()}) + r.ops.MarkResultWritten() + return cli.WriteResultWithState(w, r.format(), command, ok, result, warnings, text) } // humanBytes renders a byte count for reports. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index dff999dc..020ad564 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -23,7 +23,10 @@ Trajecta 是本地优先（local-first）的 LLM API record/replay 代理，覆 顶层目录与包职责： -- `cmd/server`：CLI 入口与 `serve` 装配。`main.go` 只通过 `run` 退出，命令接线在 `root.go`；命令文件包括 `serve.go`、`migrate.go`、`db.go`、`config.go`、`doctor.go`、`provider.go`、`models.go`、`tools.go`、`audit.go`、`auth.go`、`analyze.go`、`version.go`、`schema.go`、`completion.go`。`management.go` 提供 serve 与测试共用的管理 HTTP/MCP 装配，`provider_startup_probe.go` 提供 provider probe 辅助。 +- `cmd/server`：服务端二进制入口与 `serve` 装配。`main.go` 只通过 `run` 退出，`root.go` 只接线 `serve`、`version`、`completion` 与 `--config`/`--format`——除 `serve` 外不再承载运维子命令。`management.go` 提供 serve 与测试共用的管理 HTTP/MCP 装配，`provider_startup_probe.go` 提供启动期 provider probe 辅助。 +- `cmd/trajecta`：运维 CLI 入口。除既有的 `upgrade`、`layout`、`version` 外，还通过 `internal/cli` 暴露全部不需要代理进程的运维命令组：`analyze`、`audit`、`auth`、`config`、`db`、`doctor`、`migrate`、`models`、`provider`、`schema`、`tools`。`root.go` 用 viper 绑定 `--config`/`--format` 后交给 `internal/cli.Register`。 +- `internal/cli`：两个二进制共用的 CLI 契约与运维命令实现。契约是退出码表、JSON envelope、`Runtime`（`--config`/`--format`）、`Execute` 顶层错误路径与 ERROR 日志 recorder；命令文件是 `analyze.go`、`audit.go`、`auth.go`、`config.go`、`db.go`、`doctor.go`、`migrate.go`、`models.go`、`provider.go`、`schema.go`、`tools.go`。`cmd/server` 只引用契约，不注册其中任何命令。 +- `internal/appdb`：按配置打开 application database 的共用入口——驱动规范化、迁移模式判定、`MigrateUp`/`OptimizeIndexes`、`Open`/`Initialize` 与 auth store 打开；serve 启动与 trajecta 的 `db`/`auth`/`audit`/`analyze` 都走这里。 - `internal/proxy`：反向代理与 `/v1/responses` 入口，负责鉴权、转发、响应截获与协议感知透传调整。 - `internal/recorder`：cassette 写入与 metadata finalization。 - `internal/store`：application DB（Postgres/SQLite）schema 初始化、索引、查询与派生状态，并拥有 `ConfigurationTransaction`；`output_dir.go` 的 `Store.OutputDir()` 暴露配置的 `trace.output_dir`，供读路径缓存（会话轨迹缓存）与 cassette 放在同一目录而不新增配置项。 diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md index 0062b672..abfe6650 100644 --- a/docs/DEVELOPMENT.md +++ b/docs/DEVELOPMENT.md @@ -1,6 +1,6 @@ # 开发与测试 -本文是 Trajecta 的开发、验证、构建与集成测试入口。所有命令以仓库根目录的 `Taskfile.yml`、`.github/workflows/ci.yml` 和 `cmd/server/**` 为准；本文只记录当前实际存在、可执行的命令。 +本文是 Trajecta 的开发、验证、构建与集成测试入口。所有命令以仓库根目录的 `Taskfile.yml`、`.github/workflows/ci.yml`、`cmd/server/**`、`cmd/trajecta/**` 与 `internal/cli/**` 为准；本文只记录当前实际存在、可执行的命令。 相关阅读：[文档入口](./README.md)、[架构说明](./ARCHITECTURE.md)、[协议参考](./protocol-reference/README.md)、[Monitor 使用指南](./MONITOR_GUIDE.md)、[MCP 使用指南](./MCP_GUIDE.md)、[代理使用示例](./PROXY_USAGE_EXAMPLES.md)。 @@ -18,7 +18,7 @@ | `task test` | `go test ./...` | | `task test:race` | `go test -race ./...` | | `task test:cover` | `go test -coverprofile=coverage.out ./...` | -| `task test:e2e` | `go test ./internal/proxy ./internal/monitor ./cmd/server ./unittest` | +| `task test:e2e` | `go test ./internal/proxy ./internal/monitor ./cmd/server ./cmd/trajecta ./unittest` | | `task atif:validate` | `python3 scripts/validate_atif.py {{.CLI_ARGS}}`（需先安装 `scripts/atif-requirements.txt`） | | `task test:atif` | `go test ./internal/trajectory ./internal/monitor` + `python3 -m unittest discover -s scripts/atif_tests` | | `task test:codex-fixtures` | `env -u GOROOT go test ./internal/responses/httpapi ./internal/responses/runtime -count=1` | @@ -33,11 +33,11 @@ | `task ui:test` | 在 `web/monitor-ui` 下 `bun install --frozen-lockfile` + `bun run test:ui` | | `task ui:test:real` | 在 `web/monitor-ui` 下 `bun install --frozen-lockfile` + `bun run test:ui:real` | | `task ui:screenshots` | 在 `web/monitor-ui` 下 `bunx playwright test --config playwright.screenshots.config.js`，重新生成 README 截图 | -| `task migrate` | `go run ./cmd/server migrate -c {{.CONFIG}}`（V2 cassette 重写为 V3 并重建索引） | -| `task migrate:db:up` | `go run ./cmd/server db migrate up -c {{.CONFIG}}` | +| `task migrate` | `go run ./cmd/trajecta migrate -c {{.CONFIG}}`（V2 cassette 重写为 V3 并重建索引） | +| `task migrate:db:up` | `go run ./cmd/trajecta db migrate up -c {{.CONFIG}}` | | `task migrate:legacy` | `go run ./cmd/trajecta upgrade -c {{.CONFIG}} {{.ARGS}}`（默认 dry-run，`ARGS="--apply"` 才写盘） | -| `task auth:init-user` | `go run ./cmd/server auth init-user -c {{.CONFIG}} --username "$USER" --password "$PASSWORD"` | -| `task auth:create-token` | `go run ./cmd/server auth create-token -c {{.CONFIG}} --username "$USER" --name "$NAME"` | +| `task auth:init-user` | `go run ./cmd/trajecta auth init-user -c {{.CONFIG}} --username "$USER" --password "$PASSWORD"` | +| `task auth:create-token` | `go run ./cmd/trajecta auth create-token -c {{.CONFIG}} --username "$USER" --name "$NAME"` | | `task generate:ent` | `go generate ./ent/...` | | `task generate:secretmask` | `go generate ./pkg/secretmask/...`，即 `ragel-go -G2 -o rules_ragel.go rules.rl` + `gofmt`；需要 `ragel-go` 二进制 | | `task migrate:ent:sqlite NAME=x` | `ent/migrate/main.go --dialect sqlite --dir ent/migrations` + `update_hash.go`；Postgres 版为 `migrate:ent:postgres NAME=x DEV_URL=...`，写 `ent/postgres-migrations`；两个方向的完整生成流程与提交要求见 [Ent Migration Workflow](../ent/migrate/README.md) | @@ -67,7 +67,7 @@ task run - `task check:quick` = `fmt:check` + `lint` + `test:short`：小范围代码改动的默认验证；`task check` 与它完全相同。`test:short` 的 `-short` 只影响 Postgres 门控集成用例，其余单测与全量一致。 - `task check:full` = `check:quick` + `test` + `test:e2e` + `test:race` + `build:all`：发布前或大范围改动。 - `task test:race`：全套单元测试加 Go race detector；代理、路由、录制、存储或 streaming 改动必须补跑。 -- `task test:e2e`：本地端到端测试，覆盖 `internal/proxy`、`internal/monitor`、`cmd/server`、`unittest`，不依赖外部网络。 +- `task test:e2e`：本地端到端测试，覆盖 `internal/proxy`、`internal/monitor`、`cmd/server`、`cmd/trajecta`、`unittest`，不依赖外部网络。 - `task test:cover`：写 `coverage.out`。 - `task bench`：全部 benchmark；`task bench:core` 只跑热路径包（`internal/proxy`、`internal/router`、`internal/store`、`pkg/llm`、`pkg/recordfile`、`pkg/replay`）。 - `task lint:vet`：单独跑 `go vet ./...`，与 golangci-lint 互补。 @@ -116,7 +116,7 @@ task build:go ## 构建产物（Go 二进制、UI dist 与 go:embed 的关系） -- Go 二进制：`task build:go` 在仓库根目录生成服务端 `server`（`./cmd/server`）与 CLI `trajecta`（`./cmd/trajecta`），都使用 `-trimpath`，并通过 ldflags 注入 `main.Version`、`main.Commit`、`main.Date`、`main.Branch`（分别来自 `git describe`、`git rev-parse --short=12 HEAD`、UTC 构建时间、当前分支）。两个 `package main` 各自声明这些变量，`task clean` 同时删除两个产物。 +- Go 二进制：`task build:go` 在仓库根目录生成服务端 `server`（`./cmd/server`）与 CLI `trajecta`（`./cmd/trajecta`），都使用 `-trimpath`，并通过 ldflags 注入 `main.Version`、`main.Commit`、`main.Date`、`main.Branch`（分别来自 `git describe`、`git rev-parse --short=12 HEAD`、UTC 构建时间、当前分支）。两个 `package main` 各自声明这些变量，`task clean` 同时删除两个产物。命令分工：`server` 只保留 `serve`（以及 `version`、`completion`），其余不需要代理进程的运维命令组（`analyze`、`audit`、`auth`、`config`、`db`、`doctor`、`migrate`、`models`、`provider`、`schema`、`tools`）都挂在 `trajecta` 下，实现在 `internal/cli`；application database 的打开与迁移入口在 `internal/appdb`。 - UI 产物：`internal/monitor/ui/dist/`（`index.html` 与 `assets/`），由 `vite build` 生成，并已提交到仓库。 - 二者关系：`internal/monitor/server.go` 用 `//go:embed ui/dist/*` 把该目录编译进二进制，因此 `go build ./cmd/server` 读取的是编译时刻 `internal/monitor/ui/dist` 的内容。前端改动后必须先 `task ui:build`（或直接用 `task build` / `task build:all`，它们会先跑 `ui:build`）再编译，否则二进制里仍是旧 UI。 - 容器镜像：`task docker:build` 构建本地镜像 `trajecta:local`；`task docker:up` 用 compose 起本地栈。镜像内同时包含 `/app/bin/server`（entrypoint）与 `/app/bin/trajecta`（CLI），容器只有一个 entrypoint，用 `--entrypoint /app/bin/trajecta` 调用 CLI。 @@ -151,10 +151,10 @@ task auth:create-token USER=admin NAME=local 渠道密钥的本地加密 key 通过 `db secret` 子命令管理： ```bash -server -c config/config.yaml db secret status -server -c config/config.yaml db secret export --out trace_index.secret.backup -server -c config/config.yaml db secret rotate --yes -server -c config/config.yaml --format json db secret status +trajecta -c config/config.yaml db secret status +trajecta -c config/config.yaml db secret export --out trace_index.secret.backup +trajecta -c config/config.yaml db secret rotate --yes +trajecta -c config/config.yaml --format json db secret status ``` - `status` 只输出路径、可读性与 fingerprint，不打印密钥；`export --out` 以 `0600` 权限写备份文件，不带 `--out` 时把 key 写到 stdout。 @@ -171,10 +171,10 @@ server -c config/config.yaml --format json db secret status **预检命令**（按顺序）： ```bash -server -c config/config.yaml --format json config inspect -server -c config/config.yaml --format json doctor -server -c config/config.yaml --format json tools status -server -c config/config.yaml --format json models codex-config +trajecta -c config/config.yaml --format json config inspect +trajecta -c config/config.yaml --format json doctor +trajecta -c config/config.yaml --format json tools status +trajecta -c config/config.yaml --format json models codex-config ``` 判断标准： @@ -195,21 +195,21 @@ codex -p dgx "搜索一下今日新闻" **失败时收集**：优先用 `client_request_id`、`response_id`、`request_audit_id` 或最新请求查询。 ```bash -server -c config/config.yaml --format json audit query --list --status failed --limit 20 +trajecta -c config/config.yaml --format json audit query --list --status failed --limit 20 -server -c config/config.yaml --format json audit query \ +trajecta -c config/config.yaml --format json audit query \ --client-request-id \ --include-events --include-exchanges --include-tools --limit 200 -server -c config/config.yaml --format json audit query \ +trajecta -c config/config.yaml --format json audit query \ --response-id \ --include-events --include-exchanges --include-tools --limit 200 -server -c config/config.yaml --format json audit query \ +trajecta -c config/config.yaml --format json audit query \ --request-audit-id \ --include-events --include-exchanges --include-tools --limit 200 -server -c config/config.yaml --format json audit tool-calls --latest-by-call --limit 100 +trajecta -c config/config.yaml --format json audit tool-calls --latest-by-call --limit 100 ``` **日志定位字段**：服务日志会输出用于关联数据库与请求链路的结构化字段——`request_audit_id`、`response_id`、`conversation_id`、`client_request_id`、`trace_id`、`cassette_path`、`upstream_id`、`model`、`endpoint`、`status`、`status_code`、`duration_ms`、`ttft_ms`、`tool_type`、`tool_name`、`executor`、`call_id`、`stream`、`server_id`、`server_label`。 diff --git a/docs/IMPLEMENTATION_STATUS.md b/docs/IMPLEMENTATION_STATUS.md index ea6b6393..0e3a6e63 100644 --- a/docs/IMPLEMENTATION_STATUS.md +++ b/docs/IMPLEMENTATION_STATUS.md @@ -58,7 +58,7 @@ provider detection 属于部分实现：手动 `provider probe`、只读 `provid - eval 与实验：`datasets`、`dataset_examples`、`eval_runs`、`scores`、`experiment_runs`。 - 其他：`app_settings`、`session_summaries`、`users`、`api_tokens`。 -`semantic_nodes` 与 `overview_metric_buckets` / `overview_metric_bucket_members` 已不再被写入或读取：SQLite 启动 schema 不再建这三张表，必需表检查（`internal/appdbmigrate`、`internal/store`）也不再要求它们；语义节点树改由 Protocol/详情视图按需从 cassette 重解析（`Store.SaveObservation` 只写 `trace_observations` 的紧凑摘要），Overview 直接聚合 `logs`，路由事实则落到 `logs` 新增的 `route_target_id` / `channel_id` / `credential_id` / `sticky_status` / `sticky_previous_upstream_id` 五列上。注意 Postgres 的历史迁移（`20260622130545_add_runtime_store_tables`、`20260703110000_add_overview_metric_buckets`）仍然创建这三张表且没有 drop 迁移，所以新建 Postgres 库经 `db migrate up` 仍会得到三张空表，已有库保留旧数据；代码不会删除它们，回收空间是运维在稳定运行一段时间后手工 `DROP TABLE` 的决定（详见 [存储与部署](./STORAGE_AND_DEPLOYMENT.md)）。派生汇总命令当前只有 `server db summary rebuild sessions`，`server db summary rebuild overview` 已随派生表一起移除。 +`semantic_nodes` 与 `overview_metric_buckets` / `overview_metric_bucket_members` 已不再被写入或读取：SQLite 启动 schema 不再建这三张表，必需表检查（`internal/appdbmigrate`、`internal/store`）也不再要求它们；语义节点树改由 Protocol/详情视图按需从 cassette 重解析（`Store.SaveObservation` 只写 `trace_observations` 的紧凑摘要），Overview 直接聚合 `logs`，路由事实则落到 `logs` 新增的 `route_target_id` / `channel_id` / `credential_id` / `sticky_status` / `sticky_previous_upstream_id` 五列上。注意 Postgres 的历史迁移（`20260622130545_add_runtime_store_tables`、`20260703110000_add_overview_metric_buckets`）仍然创建这三张表且没有 drop 迁移，所以新建 Postgres 库经 `db migrate up` 仍会得到三张空表，已有库保留旧数据；代码不会删除它们，回收空间是运维在稳定运行一段时间后手工 `DROP TABLE` 的决定（详见 [存储与部署](./STORAGE_AND_DEPLOYMENT.md)）。派生汇总命令当前只有 `trajecta db summary rebuild sessions`，`trajecta db summary rebuild overview` 已随派生表一起移除。 `config inspect`、`db migrate status` 和 `doctor` 会输出 `production_storage_driver=postgres`、`production_ready`、`storage_role`、`storage_contract`。`db migrate status` 与 `db migrate up/down --dry-run` 报告迁移来源：Postgres 为 checked-in SQL，SQLite 为 `internal/store` 启动 DDL fallback；`db migrate status --check-db` 对 SQLite 只读解释 `app_schema_status` marker 与 required table 状态，不创建缺失文件、不做 destructive repair。Postgres 真实检查与代表 runtime SQL 路径由 `TRAJECTA_TEST_POSTGRES_DSN` 门控，默认测试离线。该 DSN 必须指向专用的测试库（例如 `trajecta_scratch`）而不是生产库：`TestMergeSQLiteIntoPostgresIntegration` 这类用例会向 `logs` 写入真实行，指向生产库时会在生产库留下指向不存在 cassette 的索引行。 diff --git a/docs/LEGACY_MIGRATION.md b/docs/LEGACY_MIGRATION.md index 69ca176c..a65ead4c 100644 --- a/docs/LEGACY_MIGRATION.md +++ b/docs/LEGACY_MIGRATION.md @@ -24,7 +24,7 @@ ## 前置条件 -- 目标 Postgres 可连接，并已应用应用库 schema：`server db migrate up -c config.yaml`（`trajecta upgrade db` 只写数据，不建表）。 +- 目标 Postgres 可连接，并已应用应用库 schema：`trajecta db migrate up -c config.yaml`（`trajecta upgrade db` 只写数据，不建表）。 - 一个可用的 `config.yaml` 或等价环境变量；`.env` 会被自动读取。 - 二进制：`task build:go` 会同时产出服务端 `server` 与 CLI `trajecta`，也可以单独构建： diff --git a/docs/MCP_GUIDE.md b/docs/MCP_GUIDE.md index f4f8f7dd..16cf5454 100644 --- a/docs/MCP_GUIDE.md +++ b/docs/MCP_GUIDE.md @@ -49,8 +49,8 @@ MCP endpoint 始终包在个人 token 认证中间件里（`auth.Middleware(mcpH 流程： -1. 用 `server auth init-user` 初始化首个用户。 -2. 登录 Monitor，在 `Tokens` 页面创建个人 token；或用 `server auth create-token` 创建。 +1. 用 `trajecta auth init-user` 初始化首个用户。 +2. 登录 Monitor，在 `Tokens` 页面创建个人 token；或用 `trajecta auth create-token` 创建。 3. MCP client 每个请求发送 `Authorization: Bearer `。 要点： diff --git a/docs/MONITOR_GUIDE.md b/docs/MONITOR_GUIDE.md index 2370879a..ee19ec25 100644 --- a/docs/MONITOR_GUIDE.md +++ b/docs/MONITOR_GUIDE.md @@ -14,7 +14,7 @@ Monitor 是 Trajecta 的本地 Web 工作台，由 `internal/monitor` 提供 API ```bash export TRAJECTA_DATABASE_DSN='postgres://user:pass@host:5432/trajecta?sslmode=disable' -go run ./cmd/server auth init-user -c config/config.yaml --username admin --password 'change-me-123' +go run ./cmd/trajecta auth init-user -c config/config.yaml --username admin --password 'change-me-123' ``` Monitor 使用用户名密码登录（`POST /api/auth/login`），成功后签发仅用于 Monitor 的 JWT：issuer 为 `trajecta-monitor`，audience 为 `trajecta-monitor-ui`，TTL 默认 24 小时，可用 `auth.session_ttl` 调整。前端把 JWT 存在浏览器 localStorage，并以 `Authorization: Bearer` 访问 Monitor API；该 JWT 不用于 SDK、proxy 或 MCP。 diff --git a/docs/OBSERVATION_AND_AUDIT.md b/docs/OBSERVATION_AND_AUDIT.md index e7934d01..74cc4e3f 100644 --- a/docs/OBSERVATION_AND_AUDIT.md +++ b/docs/OBSERVATION_AND_AUDIT.md @@ -286,7 +286,7 @@ type Detector interface { - `ReparseTrace` 重建 Observation IR 并保存；`RescanTrace` 只读已存观测重跑检测器；`RepairTraceUsage` 用 `llm.ResponsePipeline` 从响应体重抽 usage 并更新索引（仅 `RewriteCassette` 时重写 V3 prelude，不改变 raw payload）。 - 结果携带 `RequestNodes`、`ResponseNodes`、`StreamEvents`、`FindingCount`、`CriticalFindings`、`HighFindings`，并写入 `analysis_jobs`。 -CLI 入口（`cmd/server/analyze.go`）：`analyze reparse`、`analyze scan`、`analyze repair-usage`、`analyze reanalyze`、`analyze batch` 是 `Hidden: true` 的隐藏内部子命令（`analyze refresh`、`analyze session`、`analyze backfill-exchanges` 与 `audit query`、`audit tool-calls` 不隐藏）。完整命令清单由 [STORAGE_AND_DEPLOYMENT.md](./STORAGE_AND_DEPLOYMENT.md) 维护，这里只保留重分析相关入口。 +CLI 入口（`internal/cli/analyze.go`）：`analyze reparse`、`analyze scan`、`analyze repair-usage`、`analyze reanalyze`、`analyze batch` 是 `Hidden: true` 的隐藏内部子命令（`analyze refresh`、`analyze session`、`analyze backfill-exchanges` 与 `audit query`、`audit tool-calls` 不隐藏）。完整命令清单由 [STORAGE_AND_DEPLOYMENT.md](./STORAGE_AND_DEPLOYMENT.md) 维护，这里只保留重分析相关入口。 `analyze backfill-exchanges` 调用 `store.BackfillExchangeMetadata`，只更新 DB 索引、不重写 cassette，返回 `scanned`、`updated_model`、`updated_entry`、`legacy_or_unknown`、`missing_cassette`、`conflicts`、`dry_run`，并支持 `--dry-run`。 diff --git a/docs/POSTGRES_OPERATIONS.md b/docs/POSTGRES_OPERATIONS.md index 35dc8d59..b29b9b19 100644 --- a/docs/POSTGRES_OPERATIONS.md +++ b/docs/POSTGRES_OPERATIONS.md @@ -28,8 +28,8 @@ psql "$TRAJECTA_DATABASE_DSN" -v ON_ERROR_STOP=1 改动前先确认迁移状态（Postgres 下两个命令都读取共享的 application `schema_migrations` namespace）： ```bash -server -c config/config.yaml db migrate status --check-db -server -c config/config.yaml auth migrate status --check-db +trajecta -c config/config.yaml db migrate status --check-db +trajecta -c config/config.yaml auth migrate status --check-db ``` 两者应报告 schema 健康且 non-dirty。若处于 dirty 状态，先处理迁移一致性，不进入优化流程。 @@ -777,7 +777,7 @@ Monitor 的 Overview 页面每 60 秒轮询一次（`overview.refresh` = `refres | `acceptance.sh` | 不变量验收：派生表无孤儿、`logs.path` 不越出已知根、无活动 SQLite、迁移不 dirty、无 invalid/not-ready 索引、`parse_jobs` 探测走索引、`parse_jobs` 去重语句仍计划为反连接（写成 `NOT IN (子查询)` 时 PostgreSQL 会保留 `SubPlan`，分组 id 超出 hash 预算后逐行重扫）、仅当 `semantic_nodes` 仍存在时其反连接仍为 `Merge Anti Join`（表已 `DROP` 的库打印一行 skipped，不计失败）、reconcile 干跑无 superseded 与待清理、Monitor API 可用。**任一项失败退出码为 1** | 迁移或修复后的门禁 | | `reconcile-apply.sh` | 跑 `upgrade db --reconcile-derived-trace-ids`（默认干跑，`--apply` 才写；可加 `--prune-superseded-index-rows`），完整日志落到 `backups/` | 派生 trace id 需要修复时 | | `vacuum-after-repair.sh` | 修复后的统计刷新与死元组回收：中小编制表 `VACUUM (ANALYZE)`，大表默认只 `ANALYZE` 并报出 dead tuples | 修复结束后紧接着 | -| `optimize-indexes.sh` | 建热查询部分索引（`server db migrate optimize-indexes`，`CONCURRENTLY` 且幂等）并用 `EXPLAIN (ANALYZE)` 复测失败列表与慢请求列表 | 首次建索引、索引变更后 | +| `optimize-indexes.sh` | 建热查询部分索引（`trajecta db migrate optimize-indexes`，`CONCURRENTLY` 且幂等）并用 `EXPLAIN (ANALYZE)` 复测失败列表与慢请求列表 | 首次建索引、索引变更后 | | `verify-after-rewrite.sh` | cassette 普查 → 结构校验（`--fail-on-legacy`，旧 magic 记为错误）→ 起服务 → 无活动 SQLite → API 探针 | magic 重写之后 | | `wait-for-rewrite.sh` | 等一个长时间重写容器退出，并校验其末行计数自洽：`scanned == rewritten + current + other` | 重写跑在一次性容器里时 | | `rebuild-image.sh` | 从工作树重建 `server`/`trajecta` 并烤进镜像；替换前把旧镜像打成 `-prev-` 以便回滚 | 本地迭代后端改动 | diff --git a/docs/PROXY_USAGE_EXAMPLES.md b/docs/PROXY_USAGE_EXAMPLES.md index ff2c6636..84b0815f 100644 --- a/docs/PROXY_USAGE_EXAMPLES.md +++ b/docs/PROXY_USAGE_EXAMPLES.md @@ -20,8 +20,8 @@ CONFIG=config/examples/local-sqlite.yaml task run 首次使用需要先有用户，再签发 token（`auth create-token` 只会为已存在的启用用户签发）： ```bash -go run ./cmd/server -c config/examples/local-sqlite.yaml auth init-user --username admin --password 'change-me' -go run ./cmd/server -c config/examples/local-sqlite.yaml auth create-token --username admin --name local-dev +go run ./cmd/trajecta -c config/examples/local-sqlite.yaml auth init-user --username admin --password 'change-me' +go run ./cmd/trajecta -c config/examples/local-sqlite.yaml auth create-token --username admin --name local-dev ``` 后续示例统一使用这两个 shell 变量；如果 `server.port` 不是 `8080`，请同步修改它们以及 SDK 的 `base_url`： diff --git a/docs/RESPONSES_RUNTIME.md b/docs/RESPONSES_RUNTIME.md index 7ec71569..e9357bf4 100644 --- a/docs/RESPONSES_RUNTIME.md +++ b/docs/RESPONSES_RUNTIME.md @@ -87,7 +87,7 @@ continuation 与 input items： - Ordinary hosted `web_search` descriptor：`{"type":"web_search"}` / `{"type":"web_search_preview"}` 可被解析和审计；provider 未启用时普通 descriptor 不阻断 text create/stream，也不会暴露给上游；provider 就绪时映射为内部 function tool，由模型显式 tool call 后执行。 - Unsupported hosted tools：强制 `tool_choice` 为 `file_search` / `code_interpreter` / `computer_use_preview`（以及未就绪的 `mcp` / `web_search`）时返回稳定 OpenAI-style error envelope，`code` 为 `unsupported_tool`，message 含 `unsupported hosted tool ""`，并写不含 raw descriptor/payload 的 `tool_call_audits` rejected 记录。 -Codex TOML 生成命令 `server models codex-config `： +Codex TOML 生成命令 `trajecta models codex-config `： - 离线运行，不探测上游、不运行真实 Codex、不读取真实 API key；支持全局 `--format text|json` 与可选 `--codex-config `。 - JSON envelope 的 `command` 为 `models.codex_config`；`result.profile` 含 `model_provider`、`model`、`model_context_window`、`model_auto_compact_token_limit`。 diff --git a/docs/ROUTING_AND_CREDENTIALS.md b/docs/ROUTING_AND_CREDENTIALS.md index 06b54e92..023d06f0 100644 --- a/docs/ROUTING_AND_CREDENTIALS.md +++ b/docs/ROUTING_AND_CREDENTIALS.md @@ -13,7 +13,7 @@ 此后数据库拥有路由配置，即使所有渠道都被禁用或删除。 - `GET /api/settings/channels` 返回 `{"initialized": }`；`DELETE /api/settings/channels` 在 `ConfigurationTransaction` 内清除该标记。 仅当数据库仍然没有任何渠道时，下一次启动才会重新执行 YAML bootstrap。 -- 带显式 `credentials` 列表的 YAML 渠道保持 YAML 管理：`cmd/server` 检测到显式 credentials 时以 `WithReadOnly(true)` 构造 channel service，所有 Monitor 渠道/模型/别名写路径返回 `409` 与 `upstreams are managed by YAML credentials; edit YAML and restart instead`。 +- 带显式 `credentials` 列表的 YAML 渠道保持 YAML 管理：`cmd/server/serve.go` 检测到显式 credentials 时以 `WithReadOnly(true)` 构造 channel service，所有 Monitor 渠道/模型/别名写路径返回 `409` 与 `upstreams are managed by YAML credentials; edit YAML and restart instead`。 `BootstrapFromConfig` 也会跳过含显式 credentials 的配置，不把其导入数据库。 - 环境变量只用于启动与本地部署覆盖，不是长期的渠道/模型管理面。 @@ -133,12 +133,12 @@ Provider 与探测： ## 凭据与隐式 default credential -- 渠道负责 base URL、provider preset、routing profile、priority/weight/capacity 与模型覆盖；credential 负责 secret、`api_key` 覆盖与 route target 身份。`credentials[].headers` 目前只被 CLI `server provider probe` 读取，router/proxy 不合并它。 +- 渠道负责 base URL、provider preset、routing profile、priority/weight/capacity 与模型覆盖；credential 负责 secret、`api_key` 覆盖与 route target 身份。`credentials[].headers` 目前只被 CLI `trajecta provider probe` 读取，router/proxy 不合并它。 - 配置了显式 `credentials` 时，每个 credential 展开成一个独立 route target（`:`）；credential 的 `api_key` 覆盖 target 的 resolved key，`credential_id` 缺省由 `name` 生成 slug，再缺省为 `credential-N`。 - 没有显式 `credentials` 且 `upstream.api_key` 非空时，该 inline key 被视为隐式 credential：`credential_id = default`、`route_target_id = :default`。 - 没有显式 credentials 也没有 inline key 时，渠道仍会产生一个 route target，其 credential id 为 `default`。 - `api_key` 应引用环境变量（例如 `$env:OPENAI_PRIMARY_API_KEY`），`id` 要稳定，因为它进入 route target identity 并可能出现在 cassette metadata 中。 -- `credentials[].enabled: false` 表示该 credential 不参与路由：router 在展开 route target 时跳过它（与 CLI `server provider probe` 选探测凭据时的规则一致）。若某个渠道声明的 credentials 全部被禁用，该渠道不产生任何 route target，`New()` 会以 `no enabled upstream targets configured` 失败，而**不会**回退到渠道级 `upstream.api_key`——否则被关掉的 key 会继续发流量。跳过的 credential 不会改变其余 credential 的索引，因此由索引生成的 `credential-N` 与 route target ID 保持稳定。 +- `credentials[].enabled: false` 表示该 credential 不参与路由：router 在展开 route target 时跳过它（与 CLI `trajecta provider probe` 选探测凭据时的规则一致）。若某个渠道声明的 credentials 全部被禁用，该渠道不产生任何 route target，`New()` 会以 `no enabled upstream targets configured` 失败，而**不会**回退到渠道级 `upstream.api_key`——否则被关掉的 key 会继续发流量。跳过的 credential 不会改变其余 credential 的索引，因此由索引生成的 `credential-N` 与 route target ID 保持稳定。 - `credentials[].concurrency_limit` 目前只是配置字段，没有运行时消费者；实际并发限制由下方 limit scope 统一处理。 ## Limit 与 scope @@ -166,7 +166,7 @@ V3 cassette routing event 实际会写入的安全字段包括：`route_target_i - 渠道 API 返回值中的 `headers` 会对敏感 header 名（如 `Authorization`、`api-key`、`token` 类）掩码为 `***`；写路径接受 `{ "keep": true }` 表示保留原值，避免 UI 回传掩码覆盖真实密钥。 - 渠道 API 始终不返回明文 API key，只返回 `api_key_hint` 与 `secret_storage_mode`；`base_url` 按配置原样返回，路由事件里的候选 base URL 才会做 URL 脱敏。 - 本地 secret：API key 与敏感 header 以 `tlsec:v1:` envelope 保存，主密钥文件是 `{{output_dir}}/trace_index.secret`（权限 `0600`，本地 AES-GCM）；历史明文保持可读，下次写入时转为加密 envelope。 -- 运维入口：`server db secret status`（检查 key 文件、可读性与 fingerprint）、`db secret export --out backup.key`、`db secret rotate --yes`（备份旧 key、生成新 key、全量重加密）；Monitor 侧对应 `/api/secrets/local-key` 的状态、导出与轮换。 +- 运维入口：`trajecta db secret status`（检查 key 文件、可读性与 fingerprint）、`db secret export --out backup.key`、`db secret rotate --yes`（备份旧 key、生成新 key、全量重加密）；Monitor 侧对应 `/api/secrets/local-key` 的状态、导出与轮换。 - 没有加密能力时 `secret_storage_mode` 为 `plaintext-local`，UI 据此提示本地风险。 ## Monitor 与 MCP 的管理入口 @@ -181,7 +181,7 @@ MCP 侧提供查询工具与受控重分析动作工具，但没有渠道/模型 - `query_failures`、`summarize_failure_clusters`：失败 trace 与失败聚类。 - `reanalyze_trace`、`reanalyze_session`：对单条 trace / 单个 session 运行或入队受控重分析（动作工具，不是只读查询）。 -CLI 侧的管理入口包括 `server provider probe` / `probe-report` / `probe-apply`、`models codex-config`、`db secret status/export/rotate`。 +CLI 侧的管理入口包括 `trajecta provider probe` / `probe-report` / `probe-apply`、`models codex-config`、`db secret status/export/rotate`。 ## Replay 兼容性 diff --git a/docs/STORAGE_AND_DEPLOYMENT.md b/docs/STORAGE_AND_DEPLOYMENT.md index e5e9e862..7cc243c0 100644 --- a/docs/STORAGE_AND_DEPLOYMENT.md +++ b/docs/STORAGE_AND_DEPLOYMENT.md @@ -114,7 +114,7 @@ docker compose up -d - `TRAJECTA_DATABASE_DSN` 在 Postgres 下必填：签入的 `config/config.yaml` 设置了 `database.driver: postgres` 但 `database.dsn: ""`，本地默认值由 compose 注入。 - Postgres 服务本身读取 `POSTGRES_DB` / `POSTGRES_USER` / `POSTGRES_PASSWORD`；宿主端口由 `TRAJECTA_HOST_SERVER_PORT`、`TRAJECTA_HOST_MONITOR_PORT`、`TRAJECTA_POSTGRES_PORT` 控制。 - 首个 provider 可选：设置 `TRAJECTA_BOOTSTRAP_UPSTREAM_BASE_URL` 与 `TRAJECTA_BOOTSTRAP_UPSTREAM_API_KEY` 可导入一个 OpenAI 兼容 provider；两者留空也合法，登录后在 Monitor Web 配置 provider、凭据与模型。legacy `TRAJECTA_UPSTREAM_*` 仍支持单 upstream 迁移，新部署应使用 Web 管理的 provider 数据库。 -- 配置文件路径由 `TRAJECTA_CONFIG` 决定（CLI 的 viper 实例使用 `TRAJECTA` 前缀并开启 `AutomaticEnv`，`cmd/server/root.go`）；镜像 `Dockerfile` 与 compose 都把它设为 `/app/config/config.yaml`，未设置且未传 `-c` 时回退到 `config.yaml`。 +- 配置文件路径由 `TRAJECTA_CONFIG` 决定（CLI 的 viper 实例使用 `TRAJECTA` 前缀并开启 `AutomaticEnv`，`internal/cli/root.go`）；镜像 `Dockerfile` 与 compose 都把它设为 `/app/config/config.yaml`，未设置且未传 `-c` 时回退到 `config.yaml`。 - 输出目录：`TRAJECTA_OUTPUT_DIR` 同时覆盖 `debug.output_dir` 与 `trace.output_dir`，`TRAJECTA_TRACE_OUTPUT_DIR` 只覆盖 `trace.output_dir` 且在两者都设置时后者生效（`internal/config/config.go`）；镜像把两者都设为 `/app/data/traces`，compose 同样注入。解析优先级是 `trace.output_dir` → `debug.output_dir`（`Config.TraceOutputDir()`），**录制器、store、默认 SQLite 路径与启动日志共用这一个解析结果**；两者都为空不报错，但 `Load` 会打一条 warning，此时 cassette 与默认 SQLite 文件写在进程工作目录下。 - 响应写超时默认关闭：`http.Server.WriteTimeout` 覆盖整个响应写入而不是两次写入之间的间隔，而代理转发的 completion、本地 Responses 的 SSE、以及 Monitor 的 `GET /api/events/stream` 都可能是长连接，固定值会把它们在中途截断（客户端只看到连接关闭，没有可解析的协议错误）。因此 `server.write_timeout` 默认 `0`（不设写截止时间）；请求的上界改由调用方决定——转发到上游的请求携带入站请求的 context，SDK 取消即取消上游调用，再叠加 transport 的拨号/TLS 超时。同一段 transport 也没有等待上游响应头的上界（`ResponseHeaderTimeout`），因为代理无法区分"上游卡住"和"上游很慢"：非流式的推理调用可能在首字节前合法地花掉几分钟，而拨号、TLS 握手与空闲连接三个超时都看不见这段等待。代价是：上游接受连接后不再作答时，客户端的请求、它的并发槽位与连接会一直被占着，直到调用方自己放弃。愿意用快速失败换资源占用的部署可以设 `server.upstream_response_header_timeout`（或 `TRAJECTA_SERVER_UPSTREAM_RESPONSE_HEADER_TIMEOUT`），流式调用只需覆盖到首字节的时间而不是到最后一个 token 的时间。需要硬上限的部署（例如公网暴露、防慢读客户端）可设 `server.write_timeout`（如 `30m`）或 `TRAJECTA_SERVER_WRITE_TIMEOUT`；`server.read_timeout` 默认 `5m`，可用 `TRAJECTA_SERVER_READ_TIMEOUT` 覆盖（`cmd/server/serve.go`）。 - 排障开关默认关闭，只在排查问题时临时打开：`debug.pprof_enabled`（`TRAJECTA_DEBUG_PPROF_ENABLED`，默认 `false`）在 Monitor mux 上挂载 `net/http/pprof`（关闭时 `/debug/` 前缀返回 404，不会落到 SPA）；`debug.slow_query_threshold`（`TRAJECTA_DEBUG_SLOW_QUERY_THRESHOLD`，默认 `0`，即不记录）让 store 记录超过该阈值的语句供 System 页读取。两者都是排查手段，不是常态配置；页面细节见 [Monitor 指南](./MONITOR_GUIDE.md)。 @@ -180,22 +180,22 @@ analysis_job -> detectors -> trace_findings（可选 LLM analysis） | 类别 | 内容 | 说明 | | --- | --- | --- | | 可由 cassette 重算 | `logs` 索引、`trace_observations`、`trace_findings`、`analysis_runs`、`parser_versions`、`session_summaries`、`parse_jobs` / `analysis_jobs` 队列状态 | 派生数据，可清空重建；reparse 结果幂等（节点树不落库，详情视图按需从 cassette 重解析） | -| 由写入路径增量维护 | `session_summaries` | 随写入按 session 重建，但重建先记进进程内队列（上限 256 条）。进程内由谁消费取决于是否启动了后台 flusher：`serve` 调 `Store.StartDerivedRefresh()`，此后由后台 goroutine 唯一消费——写入会直接唤醒它，所以通常毫秒级落库，2 秒的 ticker 只作为漏唤醒与失败重试的兜底；此时读取端不再自己 flush，页面渲染不会为整会话重建买单，代价是读取端最多看到亚秒级的滞后（表现为某次请求暂时未计入会话合计，不会是错误或自相矛盾的行）。没有启动 flusher 的调用方（测试、无服务器的 CLI 命令）保持同步语义：读取端自己先 flush，因此读与它之前的写立即一致。`Store.Close()` 会先停 flusher 再 settle 队列（其余只 `Close` 的命令行入口由此覆盖），刷新失败时 work 会留在队列里等下一次 flush 重试而不是被丢弃。与 logs 的漂移用 `server db summary rebuild sessions` 修复 | +| 由写入路径增量维护 | `session_summaries` | 随写入按 session 重建，但重建先记进进程内队列（上限 256 条）。进程内由谁消费取决于是否启动了后台 flusher：`serve` 调 `Store.StartDerivedRefresh()`，此后由后台 goroutine 唯一消费——写入会直接唤醒它，所以通常毫秒级落库，2 秒的 ticker 只作为漏唤醒与失败重试的兜底；此时读取端不再自己 flush，页面渲染不会为整会话重建买单，代价是读取端最多看到亚秒级的滞后（表现为某次请求暂时未计入会话合计，不会是错误或自相矛盾的行）。没有启动 flusher 的调用方（测试、无服务器的 CLI 命令）保持同步语义：读取端自己先 flush，因此读与它之前的写立即一致。`Store.Close()` 会先停 flusher 再 settle 队列（其余只 `Close` 的命令行入口由此覆盖），刷新失败时 work 会留在队列里等下一次 flush 重试而不是被丢弃。与 logs 的漂移用 `trajecta db summary rebuild sessions` 修复 | | 持久化状态（非 cassette 可推导） | channel/upstream 配置与模型目录、`app_settings`（如 `channels.initialized`、`routing.settings`）、`users` / `api_tokens`、`responses` / `response_items`、`request_audits` / `execution_events` / `tool_call_audits`、`datasets` / `eval_runs` / `scores` / `experiment_runs`、本地 channel secret 加密密钥 | 需要独立备份 | | 可由 cassette 回填的索引字段 | `upstream_exchanges` 的 exchange metadata | `analyze backfill-exchanges` 只回填 DB 索引，不重写 cassette；`logs` 只作为推断输入被读取，不会被写入 | 重算命令（均为当前可用子命令；`analyze reparse` / `scan` / `repair-usage` / `reanalyze` / `batch` 在 cobra 里是 `Hidden`，`--help` 只列出 `refresh` 与 `session`）： ```bash -server analyze reparse --trace-id -server analyze scan --trace-id -server analyze reanalyze --trace-id # 或 --session-id -server analyze repair-usage --trace-id [--rewrite-cassette] -server analyze backfill-exchanges [--dry-run] -server analyze session --session-id -server analyze batch --all --limit 1000 # 或 --trace-id/--request-id/--session-id/过滤器 -server analyze refresh --all -server db summary rebuild sessions [--session-id ] +trajecta analyze reparse --trace-id +trajecta analyze scan --trace-id +trajecta analyze reanalyze --trace-id # 或 --session-id +trajecta analyze repair-usage --trace-id [--rewrite-cassette] +trajecta analyze backfill-exchanges [--dry-run] +trajecta analyze session --session-id +trajecta analyze batch --all --limit 1000 # 或 --trace-id/--request-id/--session-id/过滤器 +trajecta analyze refresh --all +trajecta db summary rebuild sessions [--session-id ] ``` `db summary rebuild` 当前只有 `sessions` 一个子命令（从 `logs` 重建 `session_summaries`，支持 `--session-id` 与 `--dry-run`）；`overview` 子命令已移除，`overview_metric_*` 两张表也不再被读写（Postgres 历史迁移仍会建表，除非运维手工 `DROP`）。 diff --git a/internal/appdb/appdb.go b/internal/appdb/appdb.go new file mode 100644 index 00000000..5b498086 --- /dev/null +++ b/internal/appdb/appdb.go @@ -0,0 +1,138 @@ +// Package appdb owns the config-driven access paths to the application +// database: which driver is in use, whether migrations are versioned SQL or +// startup schema initialization, and how to open the trace index and the auth +// store. +// +// Both binaries need this. The proxy process applies migrations before it opens +// its store, and the trajecta CLI opens the same database for its db, auth, +// audit and analyze commands. +package appdb + +import ( + "context" + "fmt" + "strings" + + "github.com/kingfs/Trajecta/internal/appdbmigrate" + "github.com/kingfs/Trajecta/internal/auth" + "github.com/kingfs/Trajecta/internal/config" + "github.com/kingfs/Trajecta/internal/store" +) + +// Auth migration and open hooks stay as variables so a test can observe which +// path the sqlite and postgres drivers take. +var ( + AuthMigrateDatabaseUp = auth.MigrateDatabaseUp + AuthOpenDatabase = auth.OpenDatabase +) + +// NormalizeDriver canonicalizes a database.driver value. An unset driver means +// Postgres; see store.normalizeDatabaseDriver for the same rule. +func NormalizeDriver(driver string) string { + driver = strings.ToLower(strings.TrimSpace(driver)) + switch driver { + case "": + return "postgres" + case "postgresql": + return "postgres" + default: + return driver + } +} + +// MigrationMode reports which migration contract a driver follows. +func MigrationMode(driver string) string { + switch NormalizeDriver(driver) { + case "postgres": + return "versioned-sql" + default: + return "schema-init" + } +} + +// MigrateUp applies the application migrations for cfg: versioned SQL for +// Postgres, startup schema initialization for SQLite. +func MigrateUp(cfg *config.Config, steps int) error { + if MigrationMode(cfg.DatabaseDriver()) == "versioned-sql" { + return appdbmigrate.MigrateUp(cfg.DatabaseDriver(), cfg.DatabaseDSN(), steps) + } + st, err := Initialize(cfg) + if err != nil { + return err + } + return st.Close() +} + +// OptimizeIndexes applies the non-transactional PostgreSQL index optimizations. +// SQLite reports a no-op result. +func OptimizeIndexes(cfg *config.Config) (appdbmigrate.IndexOptimizationResult, error) { + if MigrationMode(cfg.DatabaseDriver()) != "versioned-sql" { + return appdbmigrate.IndexOptimizationResult{ + Driver: NormalizeDriver(cfg.DatabaseDriver()), + Applied: false, + }, nil + } + return appdbmigrate.OptimizeIndexes(context.Background(), cfg.DatabaseDriver(), cfg.DatabaseDSN()) +} + +// Open migrates when cfg asks for it and returns the application store. +func Open(cfg *config.Config) (*store.Store, error) { + if cfg.DatabaseAutoMigrate() { + if err := MigrateUp(cfg, 0); err != nil { + return nil, err + } + } + return store.NewWithDatabaseOptions( + cfg.TraceOutputDir(), + cfg.DatabaseDriver(), + cfg.DatabaseDSN(), + cfg.DatabaseMaxOpenConns(), + cfg.DatabaseMaxIdleConns(), + store.DatabaseOptions{ + AutoMigrate: false, + UseSessionSummaryRead: cfg.DatabaseUseSessionSummaryRead(), + }, + ) +} + +// Initialize applies the startup schema without consulting auto_migrate. +func Initialize(cfg *config.Config) (*store.Store, error) { + return store.NewWithDatabase( + cfg.TraceOutputDir(), + cfg.DatabaseDriver(), + cfg.DatabaseDSN(), + cfg.DatabaseMaxOpenConns(), + cfg.DatabaseMaxIdleConns(), + ) +} + +// OpenAuthStore opens the auth tables, applying the SQLite auth migration +// first when auto_migrate is on. Postgres auth tables are owned by the +// application migration set, which the caller applies before opening. +func OpenAuthStore(cfg *config.Config) (*auth.Store, error) { + driver := NormalizeDriver(cfg.DatabaseDriver()) + switch driver { + case "sqlite": + if cfg.DatabaseAutoMigrate() { + if err := AuthMigrateDatabaseUp(driver, cfg.DatabaseDSN(), 0); err != nil { + return nil, fmt.Errorf("migrate database: %w", err) + } + } + case "postgres": + // Postgres auth tables are owned by the application migration set, which + // serve applies before opening the auth store. + default: + return nil, fmt.Errorf("auth store driver %q is not supported yet", driver) + } + + st, err := AuthOpenDatabase( + driver, + cfg.DatabaseDSN(), + cfg.DatabaseMaxOpenConns(), + cfg.DatabaseMaxIdleConns(), + ) + if err != nil { + return nil, err + } + return st, nil +} diff --git a/internal/appdb/appdb_test.go b/internal/appdb/appdb_test.go new file mode 100644 index 00000000..1ae82e43 --- /dev/null +++ b/internal/appdb/appdb_test.go @@ -0,0 +1,195 @@ +package appdb + +import ( + "context" + "database/sql" + "net/http" + "os" + "path/filepath" + "testing" + "time" + + "github.com/kingfs/Trajecta/internal/auth" + "github.com/kingfs/Trajecta/internal/config" + "github.com/kingfs/Trajecta/internal/store" + "github.com/kingfs/Trajecta/pkg/recordfile" + _ "modernc.org/sqlite" +) + +func TestOpenAuthStoreAutoMigrateSQLiteCreatesAuthSchemaAndInitUser(t *testing.T) { + dir := t.TempDir() + autoMigrate := true + cfg := &config.Config{} + cfg.Trace.OutputDir = dir + cfg.Database.Driver = "sqlite" + cfg.Database.DSN = filepath.Join(dir, "control.sqlite3") + cfg.Database.AutoMigrate = &autoMigrate + + st, err := OpenAuthStore(cfg) + if err != nil { + t.Fatalf("OpenAuthStore() error = %v", err) + } + defer st.Close() + + if _, err := st.CreateUser(context.Background(), "admin", "correct horse battery staple"); err != nil { + t.Fatalf("CreateUser() error = %v", err) + } + if err := st.VerifyPassword(context.Background(), "admin", "correct horse battery staple"); err != nil { + t.Fatalf("VerifyPassword() error = %v", err) + } +} + +func TestOpenAuthStoreAutoMigratePostgresSkipsAuthMigrator(t *testing.T) { + origMigrate := AuthMigrateDatabaseUp + origOpen := AuthOpenDatabase + t.Cleanup(func() { + AuthMigrateDatabaseUp = origMigrate + AuthOpenDatabase = origOpen + }) + + var migrated bool + var openedDriver string + var openedDSN string + var openedMaxOpen int + var openedMaxIdle int + AuthMigrateDatabaseUp = func(driver string, dsn string, steps int) error { + migrated = true + return nil + } + AuthOpenDatabase = func(driver string, dsn string, maxOpenConns int, maxIdleConns int) (*auth.Store, error) { + openedDriver = driver + openedDSN = dsn + openedMaxOpen = maxOpenConns + openedMaxIdle = maxIdleConns + return &auth.Store{}, nil + } + + autoMigrate := true + cfg := &config.Config{} + cfg.Database.Driver = "postgresql" + cfg.Database.DSN = "postgres://user:pass@example.invalid/trajecta?sslmode=disable" + cfg.Database.MaxOpenConns = 7 + cfg.Database.MaxIdleConns = 3 + cfg.Database.AutoMigrate = &autoMigrate + + st, err := OpenAuthStore(cfg) + if err != nil { + t.Fatalf("OpenAuthStore() error = %v", err) + } + defer st.Close() + + if migrated { + t.Fatalf("Postgres auth store called auth migrator; application migration should own auth tables") + } + if openedDriver != "postgres" || openedDSN != cfg.Database.DSN || openedMaxOpen != 7 || openedMaxIdle != 3 { + t.Fatalf("OpenDatabase args = driver=%q dsn=%q maxOpen=%d maxIdle=%d", openedDriver, openedDSN, openedMaxOpen, openedMaxIdle) + } +} + +func TestOpenApplicationDatabaseAutoMigrateFalseDoesNotCreateSchema(t *testing.T) { + t.Parallel() + + dir := t.TempDir() + dbPath := filepath.Join(dir, "trace_index.sqlite3") + autoMigrate := false + cfg := &config.Config{} + cfg.Trace.OutputDir = dir + cfg.Database.Driver = "sqlite" + cfg.Database.DSN = dbPath + cfg.Database.AutoMigrate = &autoMigrate + + st, err := Open(cfg) + if err != nil { + t.Fatalf("Open() error = %v", err) + } + defer st.Close() + + db, err := sql.Open("sqlite", dbPath) + if err != nil { + t.Fatalf("sql.Open() error = %v", err) + } + defer db.Close() + + var count int + if err := db.QueryRow(`SELECT count(*) FROM sqlite_master WHERE type = 'table' AND name = 'logs'`).Scan(&count); err != nil { + t.Fatalf("query sqlite_master error = %v", err) + } + if count != 0 { + t.Fatalf("logs table count = %d, want 0 with auto_migrate=false", count) + } +} + +func TestOpenApplicationDatabasePassesSessionSummaryReadConfig(t *testing.T) { + t.Parallel() + + dir := t.TempDir() + cfg := &config.Config{} + cfg.Trace.OutputDir = dir + cfg.Database.Driver = "sqlite" + cfg.Database.DSN = filepath.Join(dir, "trace_index.sqlite3") + useSummaryRead := true + cfg.Database.UseSessionSummaryRead = &useSummaryRead + + st, err := Open(cfg) + if err != nil { + t.Fatalf("Open() error = %v", err) + } + defer st.Close() + + writeCLISessionSummaryTestLog(t, st, dir, "config-summary-read.http", "sess-config-summary-read", time.Date(2026, 7, 3, 9, 0, 0, 0, time.UTC), http.StatusOK, 12) + // Settle the write path first: the poisoned value below is what proves the + // read came from the summary table, so nothing may rewrite it afterwards. + st.FlushDerivedRefresh() + db, err := sql.Open("sqlite", filepath.Join(dir, "trace_index.sqlite3")) + if err != nil { + t.Fatalf("sql.Open() error = %v", err) + } + defer db.Close() + if _, err := db.Exec(`UPDATE session_summaries SET request_count = 7 WHERE session_id = ?`, "sess-config-summary-read"); err != nil { + t.Fatalf("poison session_summaries error = %v", err) + } + page, err := st.ListSessionPage(1, 10, store.ListFilter{}) + if err != nil { + t.Fatalf("ListSessionPage() error = %v", err) + } + if len(page.Items) != 1 || page.Items[0].RequestCount != 7 { + t.Fatalf("ListSessionPage() = %+v, want summary read request_count 7", page.Items) + } +} + +// writeSessionSummaryTestLog seeds one recorded trace with a session id, so the +// session_summaries read path has something to return. +func writeCLISessionSummaryTestLog(t *testing.T, st *store.Store, dir string, name string, sessionID string, recordedAt time.Time, statusCode int, totalTokens int) { + t.Helper() + path := filepath.Join(dir, name) + if err := os.WriteFile(path, []byte("test"), 0o644); err != nil { + t.Fatalf("WriteFile(%q) error = %v", path, err) + } + header := recordfile.RecordHeader{ + Version: "LLM_PROXY_V3", + Meta: recordfile.MetaData{ + RequestID: name, + Time: recordedAt, + Model: "gpt-test", + Provider: "openai_compatible", + Operation: "responses", + Endpoint: "/v1/responses", + URL: "/v1/responses", + Method: "POST", + StatusCode: statusCode, + DurationMs: 100, + TTFTMs: 10, + ClientIP: "127.0.0.1", + ContentLength: 4, + }, + Usage: recordfile.UsageInfo{ + TotalTokens: totalTokens, + }, + } + if err := st.UpsertLogWithGrouping(path, header, store.GroupingInfo{ + SessionID: sessionID, + SessionSource: "header.session_id", + }); err != nil { + t.Fatalf("UpsertLogWithGrouping(%q) error = %v", path, err) + } +} diff --git a/cmd/server/analyze.go b/internal/cli/analyze.go similarity index 88% rename from cmd/server/analyze.go rename to internal/cli/analyze.go index d3ccc4f0..c2e8167d 100644 --- a/cmd/server/analyze.go +++ b/internal/cli/analyze.go @@ -1,4 +1,4 @@ -package main +package cli import ( "context" @@ -9,6 +9,7 @@ import ( "strings" "sync" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/reanalysis" "github.com/kingfs/Trajecta/internal/sessionanalysis" @@ -90,14 +91,14 @@ type analyzeSessionOptions struct { stdout io.Writer } -func newAnalyzeCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "analyze", Short: "Analyze recorded traces and rebuild derived observations", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } cmd.AddCommand(newAnalyzeReparseCommand(runtime)) @@ -111,7 +112,7 @@ func newAnalyzeCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeReparseCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeReparseCommand(runtime *Runtime) *cobra.Command { var traceID string cmd := &cobra.Command{ Use: "reparse", @@ -122,13 +123,13 @@ func newAnalyzeReparseCommand(runtime *cliRuntime) *cobra.Command { Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { if traceID == "" { - return cliUsageError("--trace-id is required", "trace-id") + return UsageError("--trace-id is required", "trace-id") } - return runCode(func() int { + return RunCode(func() int { return runAnalyzeReparse(analyzeReparseOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), traceID: traceID, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -138,7 +139,7 @@ func newAnalyzeReparseCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeScanCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeScanCommand(runtime *Runtime) *cobra.Command { var traceID string cmd := &cobra.Command{ Use: "scan", @@ -149,13 +150,13 @@ func newAnalyzeScanCommand(runtime *cliRuntime) *cobra.Command { Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { if traceID == "" { - return cliUsageError("--trace-id is required", "trace-id") + return UsageError("--trace-id is required", "trace-id") } - return runCode(func() int { + return RunCode(func() int { return runAnalyzeScan(analyzeScanOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), traceID: traceID, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -165,7 +166,7 @@ func newAnalyzeScanCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeRepairUsageCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeRepairUsageCommand(runtime *Runtime) *cobra.Command { var traceID string var rewriteCassette bool cmd := &cobra.Command{ @@ -177,14 +178,14 @@ func newAnalyzeRepairUsageCommand(runtime *cliRuntime) *cobra.Command { Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { if traceID == "" { - return cliUsageError("--trace-id is required", "trace-id") + return UsageError("--trace-id is required", "trace-id") } - return runCode(func() int { + return RunCode(func() int { return runAnalyzeRepairUsage(analyzeRepairUsageOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), traceID: traceID, rewriteCassette: rewriteCassette, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -195,7 +196,7 @@ func newAnalyzeRepairUsageCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeBackfillExchangesCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeBackfillExchangesCommand(runtime *Runtime) *cobra.Command { var dryRun bool cmd := &cobra.Command{ Use: "backfill-exchanges", @@ -204,11 +205,11 @@ func newAnalyzeBackfillExchangesCommand(runtime *cliRuntime) *cobra.Command { SilenceErrors: true, Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runAnalyzeBackfillExchanges(analyzeBackfillExchangesOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -218,7 +219,7 @@ func newAnalyzeBackfillExchangesCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeReanalyzeCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeReanalyzeCommand(runtime *Runtime) *cobra.Command { var traceID string var sessionID string var repairUsage bool @@ -233,20 +234,20 @@ func newAnalyzeReanalyzeCommand(runtime *cliRuntime) *cobra.Command { Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { if traceID == "" && sessionID == "" { - return cliUsageError("one of --trace-id or --session-id is required", "trace-id") + return UsageError("one of --trace-id or --session-id is required", "trace-id") } if traceID != "" && sessionID != "" { - return cliUsageError("--trace-id and --session-id are mutually exclusive", "trace-id") + return UsageError("--trace-id and --session-id are mutually exclusive", "trace-id") } - return runCode(func() int { + return RunCode(func() int { return runAnalyzeReanalyze(analyzeReanalyzeOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), traceID: traceID, sessionID: sessionID, repairUsage: repairUsage, reparse: reparse, scan: scan, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -260,7 +261,7 @@ func newAnalyzeReanalyzeCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeBatchCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeBatchCommand(runtime *Runtime) *cobra.Command { var opts analyzeBatchOptions cmd := &cobra.Command{ Use: "batch", @@ -270,24 +271,24 @@ func newAnalyzeBatchCommand(runtime *cliRuntime) *cobra.Command { SilenceErrors: true, Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - opts.configPath = runtime.configPath() - opts.format = runtime.outputFormat() + opts.configPath = runtime.ConfigPath() + opts.format = runtime.OutputFormat() opts.stdout = cmd.OutOrStdout() opts.limitSet = cmd.Flags().Changed("limit") if opts.rewriteCassette && !opts.repairUsage { - return cliUsageError("--rewrite-cassette requires --repair-usage", "rewrite-cassette") + return UsageError("--rewrite-cassette requires --repair-usage", "rewrite-cassette") } if opts.rewriteCassette && opts.enqueue { - return cliUsageError("--rewrite-cassette is only supported without --enqueue", "rewrite-cassette") + return UsageError("--rewrite-cassette is only supported without --enqueue", "rewrite-cassette") } if !opts.repairUsage && !opts.reparse && !opts.scan { opts.reparse = true opts.scan = true } if !opts.all && len(opts.traceIDs) == 0 && len(opts.requestIDs) == 0 && opts.sessionID == "" && !analyzeBatchHasFilter(opts) { - return cliUsageError("one of --all, --trace-id, --request-id, --session-id, or a filter is required", "all") + return UsageError("one of --all, --trace-id, --request-id, --session-id, or a filter is required", "all") } - return runCode(func() int { + return RunCode(func() int { return runAnalyzeBatch(opts) }) }, @@ -314,7 +315,7 @@ func newAnalyzeBatchCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeRefreshCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeRefreshCommand(runtime *Runtime) *cobra.Command { var opts analyzeBatchOptions cmd := &cobra.Command{ Use: "refresh", @@ -323,22 +324,22 @@ func newAnalyzeRefreshCommand(runtime *cliRuntime) *cobra.Command { SilenceErrors: true, Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - opts.configPath = runtime.configPath() - opts.format = runtime.outputFormat() + opts.configPath = runtime.ConfigPath() + opts.format = runtime.OutputFormat() opts.stdout = cmd.OutOrStdout() opts.limitSet = cmd.Flags().Changed("limit") opts.reparse = true opts.scan = true if opts.rewriteCassette && !opts.repairUsage { - return cliUsageError("--rewrite-cassette requires --repair-usage", "rewrite-cassette") + return UsageError("--rewrite-cassette requires --repair-usage", "rewrite-cassette") } if opts.rewriteCassette && opts.enqueue { - return cliUsageError("--rewrite-cassette is only supported without --enqueue", "rewrite-cassette") + return UsageError("--rewrite-cassette is only supported without --enqueue", "rewrite-cassette") } if !opts.all && len(opts.traceIDs) == 0 && len(opts.requestIDs) == 0 && opts.sessionID == "" && !analyzeBatchHasFilter(opts) { - return cliUsageError("one of --all, --trace-id, --request-id, --session-id, or a filter is required", "all") + return UsageError("one of --all, --trace-id, --request-id, --session-id, or a filter is required", "all") } - return runCode(func() int { + return RunCode(func() int { return runAnalyzeRefresh(opts) }) }, @@ -363,7 +364,7 @@ func newAnalyzeRefreshCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAnalyzeSessionCommand(runtime *cliRuntime) *cobra.Command { +func newAnalyzeSessionCommand(runtime *Runtime) *cobra.Command { var sessionID string cmd := &cobra.Command{ Use: "session", @@ -373,13 +374,13 @@ func newAnalyzeSessionCommand(runtime *cliRuntime) *cobra.Command { Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { if sessionID == "" { - return cliUsageError("--session-id is required", "session-id") + return UsageError("--session-id is required", "session-id") } - return runCode(func() int { + return RunCode(func() int { return runAnalyzeSession(analyzeSessionOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), sessionID: sessionID, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -395,7 +396,7 @@ func runAnalyzeReparse(opts analyzeReparseOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -417,7 +418,7 @@ func runAnalyzeReparse(opts analyzeReparseOptions) int { "stream_events": reanalysisResult.StreamEvents, "job_id": reanalysisResult.Job.ID, } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze reparse", output, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze reparse", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "reparsed trace %s with %s@%s (%d request nodes, %d response nodes, %d stream events)\n", opts.traceID, output["parser"], output["parser_version"], output["request_nodes"], output["response_nodes"], output["stream_events"]) return err @@ -434,7 +435,7 @@ func runAnalyzeRepairUsage(opts analyzeRepairUsageOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -458,7 +459,7 @@ func runAnalyzeRepairUsage(opts analyzeRepairUsageOptions) int { "total_tokens": result.Usage.After.TotalTokens, "job_id": result.Job.ID, } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze repair-usage", output, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze repair-usage", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "repaired usage for trace %s (total tokens %d, cassette rewrite %t)\n", opts.traceID, result.Usage.After.TotalTokens, result.Usage.CassetteRewrote) return err @@ -475,7 +476,7 @@ func runAnalyzeBackfillExchanges(opts analyzeBackfillExchangesOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -489,7 +490,7 @@ func runAnalyzeBackfillExchanges(opts analyzeBackfillExchangesOptions) int { slog.Error("Failed to backfill exchange metadata", "error", err) return 1 } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze backfill-exchanges", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze backfill-exchanges", result, func(w io.Writer) error { _, err := fmt.Fprintf(w, "backfilled exchange metadata: scanned=%d updated_model=%d updated_entry=%d legacy_or_unknown=%d missing_cassette=%d conflicts=%d dry_run=%t\n", result.Scanned, result.UpdatedModel, result.UpdatedEntry, result.LegacyOrUnknown, result.MissingCassette, result.Conflicts, result.DryRun) return err @@ -506,7 +507,7 @@ func runAnalyzeReanalyze(opts analyzeReanalyzeOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -545,7 +546,7 @@ func runAnalyzeReanalyze(opts analyzeReanalyzeOptions) int { output["parser"] = result.Observation.Parser output["parser_version"] = result.Observation.ParserVersion } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze reanalyze", output, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze reanalyze", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "reanalyzed trace %s with job %d (%s)\n", opts.traceID, result.Job.ID, result.Job.JobType) return err }); err != nil { @@ -568,7 +569,7 @@ func runAnalyzeReanalyze(opts analyzeReanalyzeOptions) int { "analysis_run_id": result.Session.AnalysisRunID, "finding_refs": result.Session.FindingRefs, } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze reanalyze", output, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze reanalyze", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "reanalyzed session %s (%d traces)\n", opts.sessionID, result.Session.TraceCount) return err }); err != nil { @@ -584,7 +585,7 @@ func runAnalyzeBatch(opts analyzeBatchOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -608,7 +609,7 @@ func runAnalyzeBatch(opts analyzeBatchOptions) int { "scan": opts.scan, "results": results, } - writeErr := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze batch", output, func(w io.Writer) error { + writeErr := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze batch", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "processed %d traces (completed=%d queued=%d failed=%d)\n", len(traceIDs), completed, enqueued, failed) return err }) @@ -725,7 +726,7 @@ func runAnalyzeRefresh(opts analyzeBatchOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -744,7 +745,7 @@ func runAnalyzeRefresh(opts analyzeBatchOptions) int { "status": job.Status, "enqueued": 1, } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze refresh", output, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze refresh", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "queued session refresh %s with job %d\n", opts.sessionID, job.ID) return err }); err != nil { @@ -779,7 +780,7 @@ func runAnalyzeRefresh(opts analyzeBatchOptions) int { "analysis_run_id": result.Session.AnalysisRunID, "finding_refs": result.Session.FindingRefs, } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze refresh", output, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze refresh", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "refreshed session %s (%d traces)\n", opts.sessionID, result.Session.TraceCount) return err }); err != nil { @@ -795,7 +796,7 @@ func runAnalyzeSession(opts analyzeSessionOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -849,7 +850,7 @@ func runAnalyzeSession(opts analyzeSessionOptions) int { "trace_count": len(output.TraceRefs), "finding_refs": len(output.FindingRefs), } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze session", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze session", result, func(w io.Writer) error { _, err := fmt.Fprintf(w, "analyzed session %s (%d traces, %d finding refs)\n", opts.sessionID, len(output.TraceRefs), len(output.FindingRefs)) return err }); err != nil { @@ -1015,7 +1016,7 @@ func runAnalyzeScan(opts analyzeScanOptions) int { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -1032,7 +1033,7 @@ func runAnalyzeScan(opts analyzeScanOptions) int { "findings": result.FindingCount, "job_id": result.Job.ID, } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "analyze scan", output, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "analyze scan", output, func(w io.Writer) error { _, err := fmt.Fprintf(w, "scanned trace %s (%d findings)\n", opts.traceID, result.FindingCount) return err }); err != nil { diff --git a/cmd/server/audit.go b/internal/cli/audit.go similarity index 92% rename from cmd/server/audit.go rename to internal/cli/audit.go index d3927439..d038093f 100644 --- a/cmd/server/audit.go +++ b/internal/cli/audit.go @@ -1,4 +1,4 @@ -package main +package cli import ( "context" @@ -10,6 +10,7 @@ import ( "strings" "time" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/config" responsesaudit "github.com/kingfs/Trajecta/internal/responses/audit" "github.com/spf13/cobra" @@ -256,14 +257,14 @@ type auditToolCallLifecycleSummary struct { ConversationID string `json:"conversation_id,omitempty"` } -func newAuditCommand(runtime *cliRuntime) *cobra.Command { +func newAuditCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "audit", Short: "Query stored audit traces", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } cmd.AddCommand(newAuditQueryCommand(runtime)) @@ -271,7 +272,7 @@ func newAuditCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAuditQueryCommand(runtime *cliRuntime) *cobra.Command { +func newAuditQueryCommand(runtime *Runtime) *cobra.Command { opts := auditQueryOptions{} cmd := &cobra.Command{ Use: "query", @@ -283,8 +284,8 @@ func newAuditQueryCommand(runtime *cliRuntime) *cobra.Command { "Use --list with conversation or client request selectors to return matching request audit summaries instead of the latest trace.", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - opts.configPath = runtime.configPath() - opts.format = runtime.outputFormat() + opts.configPath = runtime.ConfigPath() + opts.format = runtime.OutputFormat() opts.stdout = cmd.OutOrStdout() return runAuditQueryWithOptions(opts) }, @@ -303,7 +304,7 @@ func newAuditQueryCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAuditToolCallsCommand(runtime *cliRuntime) *cobra.Command { +func newAuditToolCallsCommand(runtime *Runtime) *cobra.Command { opts := auditToolCallAuditsOptions{} cmd := &cobra.Command{ Use: "tool-calls", @@ -313,8 +314,8 @@ func newAuditToolCallsCommand(runtime *cliRuntime) *cobra.Command { "Multiple selectors are combined with AND semantics. JSON output defaults to payload summaries only; use --include-payloads to include raw input_json, output_json, and metadata_json, which may contain sensitive data.", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - opts.configPath = runtime.configPath() - opts.format = runtime.outputFormat() + opts.configPath = runtime.ConfigPath() + opts.format = runtime.OutputFormat() opts.stdout = cmd.OutOrStdout() return runAuditToolCallAuditsWithOptions(opts) }, @@ -340,38 +341,38 @@ func runAuditQueryWithOptions(opts auditQueryOptions) error { conversationID := strings.TrimSpace(opts.conversationID) status, ok := responsesaudit.NormalizeRequestAuditStatus(opts.status) if !ok { - return cliUsageError(fmt.Sprintf("--status must be one of: %s", strings.Join(responsesaudit.RequestAuditStatusValues(), ", ")), "status") + return UsageError(fmt.Sprintf("--status must be one of: %s", strings.Join(responsesaudit.RequestAuditStatusValues(), ", ")), "status") } if status != "" && !opts.list { - return cliUsageError("--status can only be used with --list", "status") + return UsageError("--status can only be used with --list", "status") } operation, ok := responsesaudit.NormalizeRequestAuditOperation(opts.operation) if !ok { - return cliUsageError(fmt.Sprintf("--operation must be one of: %s", strings.Join(responsesaudit.RequestAuditOperationValues(), ", ")), "operation") + return UsageError(fmt.Sprintf("--operation must be one of: %s", strings.Join(responsesaudit.RequestAuditOperationValues(), ", ")), "operation") } if operation != "" && !opts.list { - return cliUsageError("--operation can only be used with --list", "operation") + return UsageError("--operation can only be used with --list", "operation") } if responseID == "" && requestAuditID == "" && clientRequestID == "" && conversationID == "" && (!opts.list || status == "" && operation == "") { - return cliUsageError("--response-id, --request-audit-id, --client-request-id, or --conversation-id is required; --list may also use --status or --operation", "response-id") + return UsageError("--response-id, --request-audit-id, --client-request-id, or --conversation-id is required; --list may also use --status or --operation", "response-id") } if opts.limit < 0 { - return cliUsageError("--limit must be greater than or equal to 0", "limit") + return UsageError("--limit must be greater than or equal to 0", "limit") } cfg, err := config.Load(opts.configPath) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "CONFIG_LOAD_FAILED", message: err.Error(), } } - st, err := openApplicationDatabase(cfg) + st, err := appdb.Open(cfg) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "STORE_OPEN_FAILED", message: err.Error(), } @@ -412,9 +413,9 @@ func runAuditQueryWithOptions(opts auditQueryOptions) error { Limit: opts.limit, }) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "RESPONSES_AUDIT_QUERY_FAILED", message: err.Error(), } @@ -425,13 +426,13 @@ func runAuditQueryWithOptions(opts auditQueryOptions) error { } result.Count = len(result.RequestAudits) result.Found = result.Count > 0 - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "audit.query", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "audit.query", result, func(w io.Writer) error { writeAuditQueryText(w, result) return nil }); err != nil { - return cliExitError{ - code: exitCodeInternal, - category: errorCategoryInternal, + return ExitError{ + code: ExitCodeInternal, + category: ErrorCategoryInternal, errCode: "OUTPUT_WRITE_FAILED", message: err.Error(), } @@ -448,18 +449,18 @@ func runAuditQueryWithOptions(opts auditQueryOptions) error { UpstreamExchangeLimit: opts.limit, }) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "RESPONSES_AUDIT_QUERY_FAILED", message: err.Error(), } } result.Found = found if !found { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "RESPONSES_AUDIT_NOT_FOUND", message: "responses audit trace not found", } @@ -481,13 +482,13 @@ func runAuditQueryWithOptions(opts auditQueryOptions) error { result.ToolCalls = append(result.ToolCalls, auditToolCallFromAudit(toolCall)) } } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "audit.query", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "audit.query", result, func(w io.Writer) error { writeAuditQueryText(w, result) return nil }); err != nil { - return cliExitError{ - code: exitCodeInternal, - category: errorCategoryInternal, + return ExitError{ + code: ExitCodeInternal, + category: ErrorCategoryInternal, errCode: "OUTPUT_WRITE_FAILED", message: err.Error(), } @@ -505,25 +506,25 @@ func runAuditToolCallAuditsWithOptions(opts auditToolCallAuditsOptions) error { executor := strings.TrimSpace(opts.executor) status, ok := responsesaudit.NormalizeToolCallAuditStatus(opts.status) if !ok { - return cliUsageError(fmt.Sprintf("--status must be one of: %s", strings.Join(responsesaudit.ToolCallAuditStatusValues(), ", ")), "status") + return UsageError(fmt.Sprintf("--status must be one of: %s", strings.Join(responsesaudit.ToolCallAuditStatusValues(), ", ")), "status") } if opts.limit < 0 { - return cliUsageError("--limit must be greater than or equal to 0", "limit") + return UsageError("--limit must be greater than or equal to 0", "limit") } cfg, err := config.Load(opts.configPath) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "CONFIG_LOAD_FAILED", message: err.Error(), } } - st, err := openApplicationDatabase(cfg) + st, err := appdb.Open(cfg) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "STORE_OPEN_FAILED", message: err.Error(), } @@ -565,9 +566,9 @@ func runAuditToolCallAuditsWithOptions(opts auditToolCallAuditsOptions) error { if opts.latestByCall { summaries, err := queryService.ListToolCallLifecycleSummaries(context.Background(), params) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "TOOL_CALL_AUDITS_QUERY_FAILED", message: err.Error(), } @@ -580,9 +581,9 @@ func runAuditToolCallAuditsWithOptions(opts auditToolCallAuditsOptions) error { } else { records, err := queryService.ListToolCallAudits(context.Background(), params) if err != nil { - return cliExitError{ - code: exitCodeAPI, - category: errorCategoryAPI, + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, errCode: "TOOL_CALL_AUDITS_QUERY_FAILED", message: err.Error(), } @@ -592,13 +593,13 @@ func runAuditToolCallAuditsWithOptions(opts auditToolCallAuditsOptions) error { } result.Count = len(result.ToolCallAudits) } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "audit.tool_calls", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "audit.tool_calls", result, func(w io.Writer) error { writeAuditToolCallAuditsText(w, result) return nil }); err != nil { - return cliExitError{ - code: exitCodeInternal, - category: errorCategoryInternal, + return ExitError{ + code: ExitCodeInternal, + category: ErrorCategoryInternal, errCode: "OUTPUT_WRITE_FAILED", message: err.Error(), } diff --git a/cmd/server/auth.go b/internal/cli/auth.go similarity index 92% rename from cmd/server/auth.go rename to internal/cli/auth.go index f874a327..2d8192ac 100644 --- a/cmd/server/auth.go +++ b/internal/cli/auth.go @@ -1,4 +1,4 @@ -package main +package cli import ( "context" @@ -10,6 +10,7 @@ import ( "strings" "time" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/appdbmigrate" "github.com/kingfs/Trajecta/internal/auth" "github.com/kingfs/Trajecta/internal/config" @@ -47,14 +48,14 @@ type authTokenOptions struct { stdout io.Writer } -func newAuthCommand(runtime *cliRuntime) *cobra.Command { +func newAuthCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "auth", Short: "Manage users, tokens, and auth database migrations", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } migrateCmd := &cobra.Command{ @@ -63,7 +64,7 @@ func newAuthCommand(runtime *cliRuntime) *cobra.Command { SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } migrateCmd.AddCommand(newAuthMigrateDirectionCommand(runtime, "up", "Apply auth database migrations")) @@ -76,7 +77,7 @@ func newAuthCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAuthMigrateDirectionCommand(runtime *cliRuntime, direction string, short string) *cobra.Command { +func newAuthMigrateDirectionCommand(runtime *Runtime, direction string, short string) *cobra.Command { var steps int var all bool var dryRun bool @@ -85,14 +86,14 @@ func newAuthMigrateDirectionCommand(runtime *cliRuntime, direction string, short Short: short, Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runAuthMigrateWithOptions(authMigrateOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), direction: direction, steps: steps, all: all, dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -106,19 +107,19 @@ func newAuthMigrateDirectionCommand(runtime *cliRuntime, direction string, short return cmd } -func newAuthMigrateStatusCommand(runtime *cliRuntime) *cobra.Command { +func newAuthMigrateStatusCommand(runtime *Runtime) *cobra.Command { var checkDB bool cmd := &cobra.Command{ Use: "status", Short: "Show auth database migration status", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runAuthMigrateWithOptions(authMigrateOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), direction: "status", checkDB: checkDB, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -128,7 +129,7 @@ func newAuthMigrateStatusCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAuthInitUserCommand(runtime *cliRuntime) *cobra.Command { +func newAuthInitUserCommand(runtime *Runtime) *cobra.Command { var username string var password string var dryRun bool @@ -137,13 +138,13 @@ func newAuthInitUserCommand(runtime *cliRuntime) *cobra.Command { Short: "Create the initial auth user", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runAuthInitUserWithOptions(authUserOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), username: username, password: password, dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -155,7 +156,7 @@ func newAuthInitUserCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAuthResetPasswordCommand(runtime *cliRuntime) *cobra.Command { +func newAuthResetPasswordCommand(runtime *Runtime) *cobra.Command { var username string var password string var dryRun bool @@ -164,13 +165,13 @@ func newAuthResetPasswordCommand(runtime *cliRuntime) *cobra.Command { Short: "Reset an auth user's password", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runAuthResetPasswordWithOptions(authUserOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), username: username, password: password, dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -182,7 +183,7 @@ func newAuthResetPasswordCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAuthCreateTokenCommand(runtime *cliRuntime) *cobra.Command { +func newAuthCreateTokenCommand(runtime *Runtime) *cobra.Command { var username string var name string var scope string @@ -193,15 +194,15 @@ func newAuthCreateTokenCommand(runtime *cliRuntime) *cobra.Command { Short: "Create an API token for an auth user", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runAuthCreateTokenWithOptions(authTokenOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), username: username, name: name, scope: scope, ttl: ttl, dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -229,7 +230,7 @@ func runAuthMigrateWithOptions(opts authMigrateOptions) int { } } if opts.dryRun { - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "auth.migrate."+opts.direction, result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "auth.migrate."+opts.direction, result, func(w io.Writer) error { fmt.Fprintf(w, "dry-run auth.migrate.%s: no changes will be applied\n", opts.direction) writeAuthMigrationReportText(w, result) return nil @@ -241,7 +242,7 @@ func runAuthMigrateWithOptions(opts authMigrateOptions) int { } switch opts.direction { case "status": - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "auth.migrate.status", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "auth.migrate.status", result, func(w io.Writer) error { fmt.Fprintf(w, "auth database migration status\n") writeAuthMigrationReportText(w, result) return nil @@ -297,7 +298,7 @@ func authMigrationReport(cfg *config.Config, direction string, steps int, all bo authMigrationCommandRole := "manage sqlite auth-owned migrations" const namespaceNote = "postgres auth tables are owned by the application schema_migrations namespace; an independent auth namespace has not been split yet" note := "sqlite auth migrations use the configured auth database path and embedded sqlite migration files" - if normalizeAuthStoreDriver(cfg.DatabaseDriver()) == "postgres" { + if appdb.NormalizeDriver(cfg.DatabaseDriver()) == "postgres" { source = "postgres-checked-in-sql" sourcePath = "ent/postgres-migrations" productionReady = true @@ -504,7 +505,7 @@ func runAuthInitUserWithOptions(opts authUserOptions) int { } slog.Info("User created", "driver", cfg.DatabaseDriver(), "username", opts.username) if opts.format == "json" { - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "auth.init-user", map[string]any{ + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "auth.init-user", map[string]any{ "mutated": true, "username": opts.username, }, nil); err != nil { @@ -538,7 +539,7 @@ func runAuthResetPasswordWithOptions(opts authUserOptions) int { } slog.Info("Password reset", "driver", cfg.DatabaseDriver(), "username", opts.username) if opts.format == "json" { - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "auth.reset-password", map[string]any{ + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "auth.reset-password", map[string]any{ "mutated": true, "username": opts.username, }, nil); err != nil { @@ -571,7 +572,7 @@ func runAuthCreateTokenWithOptions(opts authTokenOptions) int { return 1 } if opts.format == "json" { - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "auth.create-token", map[string]any{ + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "auth.create-token", map[string]any{ "mutated": true, "username": opts.username, "name": opts.name, @@ -597,7 +598,7 @@ func stdoutOrDefault(w io.Writer) io.Writer { func writeDryRunResult(stdout io.Writer, format string, command string, result map[string]any) int { if format == "json" { - if err := writeCLIResult(stdoutOrDefault(stdout), format, command, result, nil); err != nil { + if err := WriteResult(stdoutOrDefault(stdout), format, command, result, nil); err != nil { slog.Error("Write dry-run result failed", "error", err) return 1 } @@ -613,7 +614,7 @@ func openAuthStoreForCommand(configPath string) (*config.Config, *auth.Store, in slog.Error("Failed to load config", "path", configPath, "error", err) return nil, nil, 1 } - st, err := openAuthStoreWithAutoSchema(cfg) + st, err := appdb.OpenAuthStore(cfg) if err != nil { slog.Error("Open auth store failed", "error", err) return nil, nil, 1 diff --git a/cmd/server/cli_contract_test.go b/internal/cli/cli_contract_test.go similarity index 94% rename from cmd/server/cli_contract_test.go rename to internal/cli/cli_contract_test.go index 3e11fb23..206d2185 100644 --- a/cmd/server/cli_contract_test.go +++ b/internal/cli/cli_contract_test.go @@ -1,4 +1,4 @@ -package main +package cli import ( "encoding/json" @@ -62,15 +62,15 @@ func captureRunOutput(t *testing.T, args []string) (int, string, string) { func TestRunUnknownCommandExitsWithUsageCode(t *testing.T) { code, _, stderr := captureRunOutput(t, []string{"nosuchcommand"}) - if code != exitCodeUsage { - t.Fatalf("run(unknown command) = %d, want %d (stderr=%q)", code, exitCodeUsage, stderr) + if code != ExitCodeUsage { + t.Fatalf("run(unknown command) = %d, want %d (stderr=%q)", code, ExitCodeUsage, stderr) } } func TestRunUnknownCommandHonorsJSONFormat(t *testing.T) { code, _, stderr := captureRunOutput(t, []string{"--format", "json", "nosuchcommand"}) - if code != exitCodeUsage { - t.Fatalf("run(--format json unknown command) = %d, want %d (stderr=%q)", code, exitCodeUsage, stderr) + if code != ExitCodeUsage { + t.Fatalf("run(--format json unknown command) = %d, want %d (stderr=%q)", code, ExitCodeUsage, stderr) } envelope := decodeCLIEnvelope(t, stderr) if envelope.OK { @@ -119,8 +119,8 @@ trace: } code, _, stderr := captureRunOutput(t, []string{"-c", configPath, "db", "migrate", "down"}) - if code != exitCodeUsage { - t.Fatalf("run(db migrate down) = %d, want %d (stderr=%q)", code, exitCodeUsage, stderr) + if code != ExitCodeUsage { + t.Fatalf("run(db migrate down) = %d, want %d (stderr=%q)", code, ExitCodeUsage, stderr) } if count := strings.Count(stderr, appDBMigrateDownUnsupportedMessage); count != 1 { t.Fatalf("stderr mentions the unsupported message %d times, want 1:\n%s", count, stderr) @@ -147,8 +147,8 @@ trace: } code, _, stderr := captureRunOutput(t, []string{"-c", configPath, "--format", "json", "db", "migrate", "down"}) - if code != exitCodeUsage { - t.Fatalf("run(db migrate down --format json) = %d, want %d (stderr=%q)", code, exitCodeUsage, stderr) + if code != ExitCodeUsage { + t.Fatalf("run(db migrate down --format json) = %d, want %d (stderr=%q)", code, ExitCodeUsage, stderr) } // db migrate down never loads the database, so stderr must be the envelope // alone: the bare text line used to be printed before it. @@ -205,8 +205,8 @@ func TestResultWarningsAreHoistedIntoTheEnvelope(t *testing.T) { tc := tc t.Run(tc.name, func(t *testing.T) { t.Parallel() - if got := len(resultWarnings(tc.result)); got != tc.want { - t.Fatalf("resultWarnings() = %d warnings, want %d", got, tc.want) + if got := len(ResultWarnings(tc.result)); got != tc.want { + t.Fatalf("ResultWarnings() = %d warnings, want %d", got, tc.want) } }) } diff --git a/cmd/server/config.go b/internal/cli/config.go similarity index 97% rename from cmd/server/config.go rename to internal/cli/config.go index 6ca407d1..50b79ffb 100644 --- a/cmd/server/config.go +++ b/internal/cli/config.go @@ -1,4 +1,4 @@ -package main +package cli import ( "fmt" @@ -9,6 +9,7 @@ import ( "strconv" "strings" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/appdbmigrate" appconfig "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/redaction" @@ -223,30 +224,30 @@ type configInspectUpstreamsSourceSummary struct { Credentials string `json:"credentials"` } -func newConfigCommand(runtime *cliRuntime) *cobra.Command { +func newConfigCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "config", Short: "Inspect effective configuration", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } cmd.AddCommand(newConfigInspectCommand(runtime)) return cmd } -func newConfigInspectCommand(runtime *cliRuntime) *cobra.Command { +func newConfigInspectCommand(runtime *Runtime) *cobra.Command { return &cobra.Command{ Use: "inspect", Short: "Print a redacted effective configuration summary", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runConfigInspectWithOptions(configInspectOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -261,7 +262,7 @@ func runConfigInspectWithOptions(opts configInspectOptions) int { return 1 } result := buildConfigInspectResult(opts.configPath, cfg) - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "config.inspect", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "config.inspect", result, func(w io.Writer) error { writeConfigInspectText(w, result) return nil }); err != nil { @@ -296,9 +297,9 @@ func buildConfigInspectResult(configPath string, cfg *appconfig.Config) configIn DSN: redactConfigInspectDSN(cfg.DatabaseDSN()), AutoMigrate: cfg.DatabaseAutoMigrate(), UseSessionSummaryRead: cfg.DatabaseUseSessionSummaryRead(), - MigrationMode: appDBMigrationMode(cfg.DatabaseDriver()), + MigrationMode: appdb.MigrationMode(cfg.DatabaseDriver()), ProductionStorageDriver: appdbmigrate.ProductionStorageDriver, - ProductionReady: normalizeAuthStoreDriver(cfg.DatabaseDriver()) == appdbmigrate.ProductionStorageDriver, + ProductionReady: appdb.NormalizeDriver(cfg.DatabaseDriver()) == appdbmigrate.ProductionStorageDriver, StorageRole: configInspectDatabaseStorageRole(cfg.DatabaseDriver()), StorageContract: configInspectDatabaseStorageContract(cfg.DatabaseDriver()), }, @@ -632,7 +633,7 @@ func configInspectMCPPath(cfg *appconfig.Config) string { if cfg == nil || !cfg.MCP.Enabled { return "" } - normalized, err := normalizeMCPPath(cfg.MCP.Path) + normalized, err := appconfig.NormalizeMCPPath(cfg.MCP.Path) if err != nil { return strings.TrimSpace(cfg.MCP.Path) } @@ -736,14 +737,14 @@ func shouldRedactKey(key string) bool { } func configInspectDatabaseStorageRole(driver string) string { - if normalizeAuthStoreDriver(driver) == appdbmigrate.ProductionStorageDriver { + if appdb.NormalizeDriver(driver) == appdbmigrate.ProductionStorageDriver { return "production" } return appdbmigrate.SQLiteStorageRole } func configInspectDatabaseStorageContract(driver string) string { - if normalizeAuthStoreDriver(driver) == appdbmigrate.ProductionStorageDriver { + if appdb.NormalizeDriver(driver) == appdbmigrate.ProductionStorageDriver { return appdbmigrate.PostgresStorageContract } return appdbmigrate.SQLiteStorageContract diff --git a/cmd/server/config_test.go b/internal/cli/config_test.go similarity index 99% rename from cmd/server/config_test.go rename to internal/cli/config_test.go index 00a07e1f..b6c5483b 100644 --- a/cmd/server/config_test.go +++ b/internal/cli/config_test.go @@ -1,4 +1,4 @@ -package main +package cli import ( "bytes" diff --git a/cmd/server/db.go b/internal/cli/db.go similarity index 83% rename from cmd/server/db.go rename to internal/cli/db.go index e917dd2f..7cf238b5 100644 --- a/cmd/server/db.go +++ b/internal/cli/db.go @@ -1,13 +1,13 @@ -package main +package cli import ( - "context" "fmt" "io" "log/slog" "os" "strings" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/appdbmigrate" "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/store" @@ -43,14 +43,14 @@ type dbSummaryRebuildOptions struct { const appDBMigrateDownUnsupportedMessage = "db migrate down is unsupported for the production Postgres migration contract; restore from backup or use a reviewed manual migration plan" -func newDBCommand(runtime *cliRuntime) *cobra.Command { +func newDBCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "db", Short: "Manage application database migrations", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } migrateCmd := &cobra.Command{ @@ -59,7 +59,7 @@ func newDBCommand(runtime *cliRuntime) *cobra.Command { SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } migrateCmd.AddCommand(newAppDBMigrateDirectionCommand(runtime, "up", "Apply application database migrations")) @@ -72,14 +72,14 @@ func newDBCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newDBSummaryCommand(runtime *cliRuntime) *cobra.Command { +func newDBSummaryCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "summary", Short: "Maintain derived summary tables", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } rebuildCmd := &cobra.Command{ @@ -88,7 +88,7 @@ func newDBSummaryCommand(runtime *cliRuntime) *cobra.Command { SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } rebuildCmd.AddCommand(newDBSummaryRebuildSessionsCommand(runtime)) @@ -96,7 +96,7 @@ func newDBSummaryCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newDBSummaryRebuildSessionsCommand(runtime *cliRuntime) *cobra.Command { +func newDBSummaryRebuildSessionsCommand(runtime *Runtime) *cobra.Command { var dryRun bool var sessionID string cmd := &cobra.Command{ @@ -104,12 +104,12 @@ func newDBSummaryRebuildSessionsCommand(runtime *cliRuntime) *cobra.Command { Short: "Rebuild session_summaries from logs", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runDBSummaryRebuildSessionsWithOptions(dbSummaryRebuildOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), sessionID: sessionID, dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -120,7 +120,7 @@ func newDBSummaryRebuildSessionsCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newAppDBMigrateDirectionCommand(runtime *cliRuntime, direction string, short string) *cobra.Command { +func newAppDBMigrateDirectionCommand(runtime *Runtime, direction string, short string) *cobra.Command { var steps int var all bool var dryRun bool @@ -130,12 +130,12 @@ func newAppDBMigrateDirectionCommand(runtime *cliRuntime, direction string, shor Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { opts := appDBMigrateOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), direction: direction, steps: steps, all: all, dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), } code := runAppDBMigrateWithOptions(opts) @@ -143,15 +143,15 @@ func newAppDBMigrateDirectionCommand(runtime *cliRuntime, direction string, shor return nil } if direction == "down" && !dryRun { - return cliExitError{ - code: exitCodeUsage, - category: errorCategoryUsage, + return ExitError{ + code: ExitCodeUsage, + category: ErrorCategoryUsage, errCode: "UNSUPPORTED_DB_MIGRATE_DOWN", message: appDBMigrateDownUnsupportedMessage, field: "direction", } } - return cliExitErrorFromCode(code) + return ExitErrorFromCode(code) }, } cmd.Flags().IntVar(&steps, "step", 0, "Apply only N migration steps") @@ -162,7 +162,7 @@ func newAppDBMigrateDirectionCommand(runtime *cliRuntime, direction string, shor return cmd } -func newAppDBMigrateOptimizeIndexesCommand(runtime *cliRuntime) *cobra.Command { +func newAppDBMigrateOptimizeIndexesCommand(runtime *Runtime) *cobra.Command { var dryRun bool cmd := &cobra.Command{ Use: "optimize-indexes", @@ -170,36 +170,36 @@ func newAppDBMigrateOptimizeIndexesCommand(runtime *cliRuntime) *cobra.Command { Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { opts := appDBMigrateOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), direction: "optimize-indexes", dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), } code := runAppDBMigrateWithOptions(opts) if code == 0 { return nil } - return cliExitErrorFromCode(code) + return ExitErrorFromCode(code) }, } cmd.Flags().BoolVar(&dryRun, "dry-run", false, "Preview index optimizations without changing the database") return cmd } -func newAppDBMigrateStatusCommand(runtime *cliRuntime) *cobra.Command { +func newAppDBMigrateStatusCommand(runtime *Runtime) *cobra.Command { var checkDB bool cmd := &cobra.Command{ Use: "status", Short: "Show application database migration status", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runAppDBMigrateWithOptions(appDBMigrateOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), direction: "status", checkDB: checkDB, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -209,14 +209,14 @@ func newAppDBMigrateStatusCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newDBSecretCommand(runtime *cliRuntime) *cobra.Command { +func newDBSecretCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "secret", Short: "Inspect and back up local channel secret encryption keys", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } cmd.AddCommand(newDBSecretStatusCommand(runtime)) @@ -225,16 +225,16 @@ func newDBSecretCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newDBSecretStatusCommand(runtime *cliRuntime) *cobra.Command { +func newDBSecretStatusCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "status", Short: "Show local channel secret encryption key status", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runDBSecretStatusWithOptions(dbSecretOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -243,17 +243,17 @@ func newDBSecretStatusCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newDBSecretExportCommand(runtime *cliRuntime) *cobra.Command { +func newDBSecretExportCommand(runtime *Runtime) *cobra.Command { var outPath string cmd := &cobra.Command{ Use: "export", Short: "Export the local channel secret encryption key for backup", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runDBSecretExportWithOptions(dbSecretOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), outPath: outPath, }) @@ -264,17 +264,17 @@ func newDBSecretExportCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newDBSecretRotateCommand(runtime *cliRuntime) *cobra.Command { +func newDBSecretRotateCommand(runtime *Runtime) *cobra.Command { var yes bool cmd := &cobra.Command{ Use: "rotate", Short: "Rotate the local channel secret encryption key and re-encrypt channel secrets", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runDBSecretRotateWithOptions(dbSecretOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), yes: yes, }) @@ -308,7 +308,7 @@ func runAppDBMigrateWithOptions(opts appDBMigrateOptions) int { } switch opts.direction { case "status": - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.migrate.status", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.migrate.status", result, func(w io.Writer) error { fmt.Fprintf(w, "application database migration status\n") writeAppDBMigrationReportText(w, result) return nil @@ -318,12 +318,12 @@ func runAppDBMigrateWithOptions(opts appDBMigrateOptions) int { } return 0 case "up": - if err := migrateApplicationDatabaseUp(cfg, opts.steps); err != nil { + if err := appdb.MigrateUp(cfg, opts.steps); err != nil { slog.Error("Application database migration failed", "error", err) return 1 } result["mutated"] = true - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.migrate.up", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.migrate.up", result, func(w io.Writer) error { fmt.Fprintf(w, "application database schema migration applied\n") writeAppDBMigrationReportText(w, result) return nil @@ -333,7 +333,7 @@ func runAppDBMigrateWithOptions(opts appDBMigrateOptions) int { } return 0 case "optimize-indexes": - optimization, err := optimizeApplicationDatabaseIndexes(cfg) + optimization, err := appdb.OptimizeIndexes(cfg) if err != nil { slog.Error("Application database index optimization failed", "error", err) return 1 @@ -341,7 +341,7 @@ func runAppDBMigrateWithOptions(opts appDBMigrateOptions) int { result["mutated"] = optimization.Applied result["index_optimization_applied"] = optimization.Applied result["index_optimization_statement_count"] = len(optimization.Statements) - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.migrate.optimize-indexes", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.migrate.optimize-indexes", result, func(w io.Writer) error { if optimization.Applied { fmt.Fprintf(w, "application database index optimizations applied\n") } else { @@ -356,7 +356,7 @@ func runAppDBMigrateWithOptions(opts appDBMigrateOptions) int { return 0 case "down": // The caller (RunE) turns a non-zero code plus direction=down into a - // cliExitError carrying this message; printing here duplicated the line + // ExitError carrying this message; printing here duplicated the line // in text mode and prepended bare text to the JSON envelope. return 2 default: @@ -369,13 +369,13 @@ func appDBMigrationReport(cfg *config.Config, direction string, steps int, all b source := "sqlite-startup-schema-fallback" sourcePath := "internal/store raw DDL startup initialization" versioned := false - driver := normalizeAuthStoreDriver(cfg.DatabaseDriver()) + driver := appdb.NormalizeDriver(cfg.DatabaseDriver()) productionReady := false storageRole := appdbmigrate.SQLiteStorageRole storageContract := appdbmigrate.SQLiteStorageContract migrationAuthority := "internal/store raw DDL startup initialization" schemaStrategy := appdbmigrate.SQLiteSchemaStrategy - if appDBMigrationMode(driver) == "versioned-sql" { + if appdb.MigrationMode(driver) == "versioned-sql" { source = "postgres-checked-in-sql" sourcePath = "ent/postgres-migrations" versioned = true @@ -394,7 +394,7 @@ func appDBMigrationReport(cfg *config.Config, direction string, steps int, all b "steps": steps, "all": all, "database_namespace": "application", - "migration_mode": appDBMigrationMode(cfg.DatabaseDriver()), + "migration_mode": appdb.MigrationMode(cfg.DatabaseDriver()), "migration_source": source, "migration_source_path": sourcePath, "migration_authority": migrationAuthority, @@ -538,65 +538,6 @@ func writeAppDBMigrationReportText(w io.Writer, result map[string]any) { fmt.Fprintf(w, "auth_migration_scope: %s (%s)\n", result["auth_migration_scope"], result["auth_migration_command"]) } -func migrateApplicationDatabaseUp(cfg *config.Config, steps int) error { - if appDBMigrationMode(cfg.DatabaseDriver()) == "versioned-sql" { - return appdbmigrate.MigrateUp(cfg.DatabaseDriver(), cfg.DatabaseDSN(), steps) - } - st, err := initializeApplicationDatabase(cfg) - if err != nil { - return err - } - return st.Close() -} - -func optimizeApplicationDatabaseIndexes(cfg *config.Config) (appdbmigrate.IndexOptimizationResult, error) { - if appDBMigrationMode(cfg.DatabaseDriver()) != "versioned-sql" { - return appdbmigrate.IndexOptimizationResult{ - Driver: normalizeAuthStoreDriver(cfg.DatabaseDriver()), - Applied: false, - }, nil - } - return appdbmigrate.OptimizeIndexes(context.Background(), cfg.DatabaseDriver(), cfg.DatabaseDSN()) -} - -func appDBMigrationMode(driver string) string { - switch normalizeAuthStoreDriver(driver) { - case "postgres": - return "versioned-sql" - default: - return "schema-init" - } -} - -func openApplicationDatabase(cfg *config.Config) (*store.Store, error) { - if cfg.DatabaseAutoMigrate() { - if err := migrateApplicationDatabaseUp(cfg, 0); err != nil { - return nil, err - } - } - return store.NewWithDatabaseOptions( - cfg.TraceOutputDir(), - cfg.DatabaseDriver(), - cfg.DatabaseDSN(), - cfg.DatabaseMaxOpenConns(), - cfg.DatabaseMaxIdleConns(), - store.DatabaseOptions{ - AutoMigrate: false, - UseSessionSummaryRead: cfg.DatabaseUseSessionSummaryRead(), - }, - ) -} - -func initializeApplicationDatabase(cfg *config.Config) (*store.Store, error) { - return store.NewWithDatabase( - cfg.TraceOutputDir(), - cfg.DatabaseDriver(), - cfg.DatabaseDSN(), - cfg.DatabaseMaxOpenConns(), - cfg.DatabaseMaxIdleConns(), - ) -} - func runDBSummaryRebuildSessionsWithOptions(opts dbSummaryRebuildOptions) int { st, closeStore, code := openTraceStoreForCommand(opts.configPath) if code != 0 { @@ -646,7 +587,7 @@ func runDBSummaryRebuildSessionsWithOptions(opts dbSummaryRebuildOptions) int { result["rebuilt_all"] = true } result["mutated"] = true - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.summary.rebuild.sessions", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.summary.rebuild.sessions", result, func(w io.Writer) error { fmt.Fprintf(w, "session summaries rebuilt\n") writeDBSummaryRebuildSessionsText(w, result) return nil @@ -674,7 +615,7 @@ func runDBSecretStatusWithOptions(opts dbSecretOptions) int { } defer closeStore() status := st.SecretStatus() - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.status", status, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.status", status, func(w io.Writer) error { fmt.Fprintf(w, "mode: %s\n", status.Mode) fmt.Fprintf(w, "key_path: %s\n", status.KeyPath) fmt.Fprintf(w, "exists: %t\n", status.Exists) @@ -740,7 +681,7 @@ func runDBSecretExportWithOptions(opts dbSecretOptions) int { "key_path": status.KeyPath, "fingerprint": status.Fingerprint, } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.export", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.export", result, func(w io.Writer) error { fmt.Fprintf(w, "exported local secret key to %s\n", opts.outPath) fmt.Fprintf(w, "fingerprint: %s\n", status.Fingerprint) return nil @@ -757,7 +698,7 @@ func runDBSecretExportWithOptions(opts dbSecretOptions) int { "fingerprint": status.Fingerprint, "key": strings.TrimSpace(string(key)), } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.export", result, nil); err != nil { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.export", result, nil); err != nil { slog.Error("Write db secret export result failed", "error", err) return 1 } @@ -785,7 +726,7 @@ func runDBSecretRotateWithOptions(opts dbSecretOptions) int { slog.Error("Rotate local secret key failed", "error", err) return 1 } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.rotate", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "db.secret.rotate", result, func(w io.Writer) error { fmt.Fprintf(w, "rotated local secret key\n") fmt.Fprintf(w, "key_path: %s\n", result.KeyPath) fmt.Fprintf(w, "backup_path: %s\n", result.BackupPath) @@ -808,7 +749,7 @@ func openTraceStoreForCommand(configPath string) (*store.Store, func(), int) { slog.Error("Failed to load config", "path", configPath, "error", err) return nil, func() {}, 1 } - st, err := openApplicationDatabase(cfg) + st, err := appdb.Open(cfg) if err != nil { slog.Error("Open trace store failed", "error", err) return nil, func() {}, 1 diff --git a/cmd/server/doctor.go b/internal/cli/doctor.go similarity index 98% rename from cmd/server/doctor.go rename to internal/cli/doctor.go index 5a9e029d..b3805ac2 100644 --- a/cmd/server/doctor.go +++ b/internal/cli/doctor.go @@ -1,4 +1,4 @@ -package main +package cli import ( "context" @@ -15,6 +15,7 @@ import ( "strings" "time" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/appdbmigrate" appconfig "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/providerprobe" @@ -88,7 +89,7 @@ type doctorCheck struct { Detail map[string]any `json:"detail,omitempty"` } -func newDoctorCommand(runtime *cliRuntime) *cobra.Command { +func newDoctorCommand(runtime *Runtime) *cobra.Command { var checkDB bool var codexConfigPath string var probeProviders bool @@ -99,11 +100,11 @@ func newDoctorCommand(runtime *cliRuntime) *cobra.Command { Short: "Run offline startup diagnostics", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runDoctorWithOptions(doctorOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), codexConfigPath: codexConfigPath, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), checkDB: checkDB, probeProviders: probeProviders, @@ -123,7 +124,7 @@ func newDoctorCommand(runtime *cliRuntime) *cobra.Command { func runDoctorWithOptions(opts doctorOptions) int { result := buildDoctorResult(opts) - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "doctor", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "doctor", result, func(w io.Writer) error { writeDoctorText(w, result) return nil }); err != nil { @@ -231,7 +232,7 @@ func checkDoctorServerPort(cfg *appconfig.Config) doctorCheck { } func checkDoctorManagementConsistency(cfg *appconfig.Config) doctorCheck { - if err := validateServeConfig(cfg); err != nil { + if err := appconfig.ValidateServe(cfg); err != nil { return doctorCheck{ Name: "monitor_mcp.consistency", Status: doctorStatusFail, @@ -401,7 +402,7 @@ func canonicalDoctorClientEntrypoint(path string) string { } func doctorResponsesHTTPGuardAuthDetail(cfg *appconfig.Config) map[string]any { - driver := normalizeAuthStoreDriver(cfg.DatabaseDriver()) + driver := appdb.NormalizeDriver(cfg.DatabaseDriver()) supported := driver == "sqlite" || driver == "postgres" databasePathSource := "default" if strings.TrimSpace(cfg.Auth.DatabasePath) != "" { @@ -444,7 +445,7 @@ func doctorResponsesHTTPGuardManagement(cfg *appconfig.Config, normalizedPath st mcpPath := "" mcpPathOverlap := false if cfg.MCP.Enabled { - if normalized, err := normalizeMCPPath(cfg.MCP.Path); err == nil { + if normalized, err := appconfig.NormalizeMCPPath(cfg.MCP.Path); err == nil { mcpPath = normalized mcpPathOverlap = httpPathsOverlap(normalizedPath, mcpPath) } else { @@ -513,14 +514,14 @@ func checkDoctorResponsesDefaultModel(cfg *appconfig.Config) doctorCheck { } func checkDoctorResponsesStoreReadiness(cfg *appconfig.Config) doctorCheck { - effectiveDriver := normalizeAuthStoreDriver(cfg.DatabaseDriver()) + effectiveDriver := appdb.NormalizeDriver(cfg.DatabaseDriver()) detail := map[string]any{ "force_store": cfg.ResponsesForceStore(), "database_driver": effectiveDriver, "database_driver_raw": strings.TrimSpace(cfg.Database.Driver), "database_dsn": redactConfigInspectDSN(cfg.DatabaseDSN()), "database_auto_migrate": cfg.DatabaseAutoMigrate(), - "migration_mode": appDBMigrationMode(effectiveDriver), + "migration_mode": appdb.MigrationMode(effectiveDriver), "production_storage_driver": appdbmigrate.ProductionStorageDriver, "production_ready": effectiveDriver == appdbmigrate.ProductionStorageDriver, "storage_role": configInspectDatabaseStorageRole(effectiveDriver), @@ -567,7 +568,7 @@ var doctorResponsesAuditTables = []string{"request_audits", "execution_events", var doctorResponsesSettingsTables = []string{"app_settings"} func checkDoctorResponsesStoreHealth(cfg *appconfig.Config, checkDB bool) doctorCheck { - driver := normalizeAuthStoreDriver(cfg.DatabaseDriver()) + driver := appdb.NormalizeDriver(cfg.DatabaseDriver()) requiredTables := doctorResponsesRequiredStoreHealthTables() detail := map[string]any{ "force_store": cfg.ResponsesForceStore(), @@ -575,7 +576,7 @@ func checkDoctorResponsesStoreHealth(cfg *appconfig.Config, checkDB bool) doctor "database_driver_raw": strings.TrimSpace(cfg.Database.Driver), "database_dsn": redactConfigInspectDSN(cfg.DatabaseDSN()), "database_auto_migrate": cfg.DatabaseAutoMigrate(), - "migration_mode": appDBMigrationMode(driver), + "migration_mode": appdb.MigrationMode(driver), "production_storage_driver": appdbmigrate.ProductionStorageDriver, "production_ready": driver == appdbmigrate.ProductionStorageDriver, "storage_role": configInspectDatabaseStorageRole(driver), @@ -622,7 +623,7 @@ func doctorResponsesRequiredStoreHealthTables() []string { } func checkDoctorResponsesStoreTables(cfg *appconfig.Config, requiredTables []string) ([]string, error) { - driver := normalizeAuthStoreDriver(cfg.DatabaseDriver()) + driver := appdb.NormalizeDriver(cfg.DatabaseDriver()) switch driver { case "sqlite": return checkDoctorResponsesSQLiteTables(cfg.DatabaseDSN(), requiredTables) @@ -1059,7 +1060,7 @@ func checkDoctorProviderConfig(cfg *appconfig.Config) doctorCheck { } func checkDoctorAuthMigrationScope(cfg *appconfig.Config) doctorCheck { - driver := normalizeAuthStoreDriver(cfg.DatabaseDriver()) + driver := appdb.NormalizeDriver(cfg.DatabaseDriver()) shared := driver == appdbmigrate.ProductionStorageDriver effectiveNamespace := "auth" schemaAuthority := "sqlite_auth_embedded_migrations" diff --git a/cmd/server/doctor_test.go b/internal/cli/doctor_test.go similarity index 99% rename from cmd/server/doctor_test.go rename to internal/cli/doctor_test.go index 5dfcb56c..ac3e34c9 100644 --- a/cmd/server/doctor_test.go +++ b/internal/cli/doctor_test.go @@ -1,4 +1,4 @@ -package main +package cli import ( "bytes" diff --git a/cmd/server/main_test.go b/internal/cli/main_test.go similarity index 87% rename from cmd/server/main_test.go rename to internal/cli/main_test.go index 540e206d..19cf9de4 100644 --- a/cmd/server/main_test.go +++ b/internal/cli/main_test.go @@ -1,4 +1,4 @@ -package main +package cli import ( "bytes" @@ -24,12 +24,9 @@ import ( "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/observeworker" responsesaudit "github.com/kingfs/Trajecta/internal/responses/audit" - "github.com/kingfs/Trajecta/internal/router" "github.com/kingfs/Trajecta/internal/store" - "github.com/kingfs/Trajecta/internal/upstream" "github.com/kingfs/Trajecta/pkg/observe" "github.com/kingfs/Trajecta/pkg/recordfile" - "github.com/modelcontextprotocol/go-sdk/mcp" _ "modernc.org/sqlite" ) @@ -44,234 +41,28 @@ func reparseTraceForTest(t *testing.T, st *store.Store, traceID string) observe. return obs } -func TestLogResolvedUpstreamConfigIncludesRoutingDiagnostics(t *testing.T) { - var buf bytes.Buffer - logger := slog.New(slog.NewTextHandler(&buf, nil)) - prev := slog.Default() - slog.SetDefault(logger) - t.Cleanup(func() { - slog.SetDefault(prev) - }) - - logResolvedUpstreamConfig( - upstream.ResolvedUpstream{ - BaseURL: "https://generativelanguage.googleapis.com", - ProviderPreset: "google_genai", - ProtocolFamily: upstream.ProtocolFamilyGoogleGenAI, - RoutingProfile: upstream.RoutingProfileGoogleAIStudio, - }, - upstream.StartupDiagnostics{ - ConnectivityEndpoint: "/v1beta/models", - ConnectivityURL: "https://generativelanguage.googleapis.com/v1beta/models", - ModelRoutingHint: "model is selected in the request path", - }, - ) - - output := buf.String() - for _, want := range []string{ - "Resolved upstream config", - "provider_preset=google_genai", - "protocol_family=google_genai", - "routing_profile=google_ai_studio", - "connectivity_endpoint=/v1beta/models", - "connectivity_url=https://generativelanguage.googleapis.com/v1beta/models", - "model_routing_hint=\"model is selected in the request path\"", - } { - if !strings.Contains(output, want) { - t.Fatalf("log output = %q, want contain %q", output, want) - } - } -} - -func TestRootCommandHelpWorksWithConfigShortcut(t *testing.T) { - t.Parallel() - - cmd := newRootCommand() - var out bytes.Buffer - cmd.SetOut(&out) - cmd.SetErr(&out) - cmd.SetArgs([]string{"-c", "config.yaml", "--help"}) - if err := cmd.Execute(); err != nil { - t.Fatalf("Execute() error = %v", err) - } - output := out.String() - for _, want := range []string{ - "Postgres-first LLM gateway with local Responses runtime and record/replay", - "Available Commands:", - "-c, --config string", - } { - if !strings.Contains(output, want) { - t.Fatalf("help output = %q, want contain %q", output, want) - } - } -} - func TestCLIRuntimeReadsConfigFromEnv(t *testing.T) { t.Setenv("TRAJECTA_CONFIG", "env-config.yaml") - runtime := newCLIRuntime() - if got := runtime.configPath(); got != "env-config.yaml" { + runtime := NewRuntime() + if got := runtime.ConfigPath(); got != "env-config.yaml" { t.Fatalf("configPath() = %q, want env-config.yaml", got) } } -func TestRouterConfigFromChannelsUsesDatabaseTargets(t *testing.T) { - st, err := store.New(t.TempDir()) - if err != nil { - t.Fatalf("store.New() error = %v", err) - } - defer st.Close() - - if _, err := st.UpsertChannelConfig(store.ChannelConfigRecord{ - ID: "db-channel", - Name: "DB Channel", - BaseURL: "https://db.example.com/v1", - ProviderPreset: "openai", - HeadersJSON: "{}", - Enabled: true, - }); err != nil { - t.Fatalf("UpsertChannelConfig() error = %v", err) - } - if err := st.ReplaceChannelModels("db-channel", []store.ChannelModelRecord{ - {Model: "gpt-5", Source: "manual", Enabled: true}, - }); err != nil { - t.Fatalf("ReplaceChannelModels() error = %v", err) - } - - cfg := &config.Config{} - cfg.Upstream.BaseURL = "https://yaml.example.com/v1" - cfg.Upstream.ProviderPreset = "openai" - - routerCfg, source, err := routerConfigFromChannels(cfg, channel.NewService(st)) - if err != nil { - t.Fatalf("routerConfigFromChannels() error = %v", err) - } - if source != "database" { - t.Fatalf("source = %q, want database", source) - } - if len(routerCfg.Upstreams) != 1 || routerCfg.Upstreams[0].ID != "db-channel" { - t.Fatalf("routerCfg.Upstreams = %#v", routerCfg.Upstreams) - } - if routerCfg.Upstream.BaseURL != "" { - t.Fatalf("routerCfg.Upstream.BaseURL = %q, want empty", routerCfg.Upstream.BaseURL) - } -} - -func TestRouterConfigFromChannelsFallsBackToYAML(t *testing.T) { - st, err := store.New(t.TempDir()) - if err != nil { - t.Fatalf("store.New() error = %v", err) - } - defer st.Close() - - cfg := &config.Config{} - cfg.Upstream.BaseURL = "https://yaml.example.com/v1" - cfg.Upstream.ProviderPreset = "openai" - - routerCfg, source, err := routerConfigFromChannels(cfg, channel.NewService(st)) - if err != nil { - t.Fatalf("routerConfigFromChannels() error = %v", err) - } - if source != "yaml" { - t.Fatalf("source = %q, want yaml", source) - } - if routerCfg != cfg { - t.Fatalf("routerConfigFromChannels should return original cfg on YAML fallback") - } -} - -func TestRouterConfigFromChannelsKeepsYAMLWhenExplicitCredentialsExist(t *testing.T) { - st, err := store.New(t.TempDir()) - if err != nil { - t.Fatalf("store.New() error = %v", err) - } - defer st.Close() - - if _, err := st.UpsertChannelConfig(store.ChannelConfigRecord{ - ID: "db-channel", - Name: "DB Channel", - BaseURL: "https://db.example.com/v1", - ProviderPreset: "openai", - HeadersJSON: "{}", - Enabled: true, - }); err != nil { - t.Fatalf("UpsertChannelConfig() error = %v", err) - } - - cfg := &config.Config{} - cfg.Upstreams = []config.UpstreamTargetConfig{ - { - ID: "yaml-channel", - Upstream: config.UpstreamConfig{ - BaseURL: "https://yaml.example.com/v1", - ApiKey: "$env:OPENAI_TEST_KEY", - ProviderPreset: "openai", - }, - Credentials: []config.CredentialConfig{ - { - ID: "primary", - ApiKey: "$env:OPENAI_TEST_KEY", - }, - }, - }, - } - - routerCfg, source, err := routerConfigFromChannels(cfg, channel.NewService(st)) - if err != nil { - t.Fatalf("routerConfigFromChannels() error = %v", err) - } - if source != "yaml" { - t.Fatalf("source = %q, want yaml", source) - } - if routerCfg != cfg { - t.Fatalf("routerConfigFromChannels should return original cfg when YAML credentials are explicit") - } - if got := routerCfg.Upstreams[0].EffectiveCredentials()[0].ID; got != "primary" { - t.Fatalf("credential id = %q, want primary", got) - } -} - -func TestRootCommandRegistersBaseCommands(t *testing.T) { +func TestRootCommandRegistersManagementCommandGroups(t *testing.T) { t.Parallel() cmd := newRootCommand() - for _, want := range []string{"serve", "migrate", "db", "db secret", "db secret status", "db secret export", "db secret rotate", "config", "config inspect", "doctor", "provider", "provider probe", "provider probe-report", "provider probe-apply", "models", "models codex-config", "audit", "audit query", "audit tool-calls", "auth", "analyze", "analyze repair-usage", "analyze backfill-exchanges", "analyze reanalyze", "analyze batch", "analyze refresh", "version", "schema", "completion"} { + for _, want := range []string{"migrate", "db", "db secret", "db secret status", "db secret export", "db secret rotate", "config", "config inspect", "doctor", "provider", "provider probe", "provider probe-report", "provider probe-apply", "models", "models codex-config", "audit", "audit query", "audit tool-calls", "auth", "auth create-token", "analyze", "analyze repair-usage", "analyze backfill-exchanges", "analyze reanalyze", "analyze batch", "analyze refresh", "schema"} { parts := strings.Fields(want) found, _, err := cmd.Find(parts) - if err != nil || found.CommandPath() != cliName+" "+want { + if err != nil || found.CommandPath() != rootName+" "+want { t.Fatalf("root command missing %q: found=%v err=%v", want, found, err) } } } -func TestVersionCommandSupportsJSONEnvelope(t *testing.T) { - t.Parallel() - - cmd := newRootCommand() - var out bytes.Buffer - cmd.SetOut(&out) - cmd.SetErr(&out) - cmd.SetArgs([]string{"version", "--format", "json"}) - if err := cmd.Execute(); err != nil { - t.Fatalf("Execute() error = %v", err) - } - - var envelope struct { - OK bool `json:"ok"` - Command string `json:"command"` - Result struct { - Name string `json:"name"` - Version string `json:"version"` - } `json:"result"` - } - if err := json.Unmarshal(out.Bytes(), &envelope); err != nil { - t.Fatalf("json.Unmarshal() error = %v, output=%q", err, out.String()) - } - if !envelope.OK || envelope.Command != "version" || envelope.Result.Name != cliName || envelope.Result.Version == "" { - t.Fatalf("version envelope = %+v", envelope) - } -} - func TestSchemaCommandSupportsJSONEnvelopeForCommandPath(t *testing.T) { t.Parallel() @@ -306,7 +97,7 @@ func TestSchemaCommandSupportsJSONEnvelopeForCommandPath(t *testing.T) { if !envelope.OK || len(envelope.Result.Commands) != 1 { t.Fatalf("schema envelope = %+v", envelope) } - if envelope.Result.Commands[0].Path != cliName+" auth create-token" { + if envelope.Result.Commands[0].Path != rootName+" auth create-token" { t.Fatalf("schema path = %q", envelope.Result.Commands[0].Path) } if len(envelope.Result.Contracts.Formats) == 0 || envelope.Result.Contracts.Stdout == "" || envelope.Result.Contracts.Stderr == "" { @@ -1666,7 +1457,7 @@ func TestRootCommandRegistersModelsCodexConfig(t *testing.T) { if err != nil { t.Fatalf("Find(models codex-config) error = %v", err) } - if found == nil || found.CommandPath() != cliName+" models codex-config" { + if found == nil || found.CommandPath() != rootName+" models codex-config" { t.Fatalf("found command path = %q", found.CommandPath()) } } @@ -2547,76 +2338,6 @@ database: } } -func TestOpenAuthStoreAutoMigrateSQLiteCreatesAuthSchemaAndInitUser(t *testing.T) { - dir := t.TempDir() - autoMigrate := true - cfg := &config.Config{} - cfg.Trace.OutputDir = dir - cfg.Database.Driver = "sqlite" - cfg.Database.DSN = filepath.Join(dir, "control.sqlite3") - cfg.Database.AutoMigrate = &autoMigrate - - st, err := openAuthStore(cfg) - if err != nil { - t.Fatalf("openAuthStore() error = %v", err) - } - defer st.Close() - - if _, err := st.CreateUser(context.Background(), "admin", "correct horse battery staple"); err != nil { - t.Fatalf("CreateUser() error = %v", err) - } - if err := st.VerifyPassword(context.Background(), "admin", "correct horse battery staple"); err != nil { - t.Fatalf("VerifyPassword() error = %v", err) - } -} - -func TestOpenAuthStoreAutoMigratePostgresSkipsAuthMigrator(t *testing.T) { - origMigrate := authMigrateDatabaseUp - origOpen := authOpenDatabase - t.Cleanup(func() { - authMigrateDatabaseUp = origMigrate - authOpenDatabase = origOpen - }) - - var migrated bool - var openedDriver string - var openedDSN string - var openedMaxOpen int - var openedMaxIdle int - authMigrateDatabaseUp = func(driver string, dsn string, steps int) error { - migrated = true - return nil - } - authOpenDatabase = func(driver string, dsn string, maxOpenConns int, maxIdleConns int) (*auth.Store, error) { - openedDriver = driver - openedDSN = dsn - openedMaxOpen = maxOpenConns - openedMaxIdle = maxIdleConns - return &auth.Store{}, nil - } - - autoMigrate := true - cfg := &config.Config{} - cfg.Database.Driver = "postgresql" - cfg.Database.DSN = "postgres://user:pass@example.invalid/trajecta?sslmode=disable" - cfg.Database.MaxOpenConns = 7 - cfg.Database.MaxIdleConns = 3 - cfg.Database.AutoMigrate = &autoMigrate - - st, err := openAuthStore(cfg) - if err != nil { - t.Fatalf("openAuthStore() error = %v", err) - } - defer st.Close() - - if migrated { - t.Fatalf("Postgres auth store called auth migrator; application migration should own auth tables") - } - if openedDriver != "postgres" || openedDSN != cfg.Database.DSN || openedMaxOpen != 7 || openedMaxIdle != 3 { - t.Fatalf("OpenDatabase args = driver=%q dsn=%q maxOpen=%d maxIdle=%d", openedDriver, openedDSN, openedMaxOpen, openedMaxIdle) - } -} - func TestTopLevelMigrateAutoMigrateUsesApplicationDatabase(t *testing.T) { t.Parallel() @@ -2660,77 +2381,6 @@ database: } } -func TestOpenApplicationDatabaseAutoMigrateFalseDoesNotCreateSchema(t *testing.T) { - t.Parallel() - - dir := t.TempDir() - dbPath := filepath.Join(dir, "trace_index.sqlite3") - autoMigrate := false - cfg := &config.Config{} - cfg.Trace.OutputDir = dir - cfg.Database.Driver = "sqlite" - cfg.Database.DSN = dbPath - cfg.Database.AutoMigrate = &autoMigrate - - st, err := openApplicationDatabase(cfg) - if err != nil { - t.Fatalf("openApplicationDatabase() error = %v", err) - } - defer st.Close() - - db, err := sql.Open("sqlite", dbPath) - if err != nil { - t.Fatalf("sql.Open() error = %v", err) - } - defer db.Close() - - var count int - if err := db.QueryRow(`SELECT count(*) FROM sqlite_master WHERE type = 'table' AND name = 'logs'`).Scan(&count); err != nil { - t.Fatalf("query sqlite_master error = %v", err) - } - if count != 0 { - t.Fatalf("logs table count = %d, want 0 with auto_migrate=false", count) - } -} - -func TestOpenApplicationDatabasePassesSessionSummaryReadConfig(t *testing.T) { - t.Parallel() - - dir := t.TempDir() - cfg := &config.Config{} - cfg.Trace.OutputDir = dir - cfg.Database.Driver = "sqlite" - cfg.Database.DSN = filepath.Join(dir, "trace_index.sqlite3") - useSummaryRead := true - cfg.Database.UseSessionSummaryRead = &useSummaryRead - - st, err := openApplicationDatabase(cfg) - if err != nil { - t.Fatalf("openApplicationDatabase() error = %v", err) - } - defer st.Close() - - writeCLISessionSummaryTestLog(t, st, dir, "config-summary-read.http", "sess-config-summary-read", time.Date(2026, 7, 3, 9, 0, 0, 0, time.UTC), http.StatusOK, 12) - // Settle the write path first: the poisoned value below is what proves the - // read came from the summary table, so nothing may rewrite it afterwards. - st.FlushDerivedRefresh() - db, err := sql.Open("sqlite", filepath.Join(dir, "trace_index.sqlite3")) - if err != nil { - t.Fatalf("sql.Open() error = %v", err) - } - defer db.Close() - if _, err := db.Exec(`UPDATE session_summaries SET request_count = 7 WHERE session_id = ?`, "sess-config-summary-read"); err != nil { - t.Fatalf("poison session_summaries error = %v", err) - } - page, err := st.ListSessionPage(1, 10, store.ListFilter{}) - if err != nil { - t.Fatalf("ListSessionPage() error = %v", err) - } - if len(page.Items) != 1 || page.Items[0].RequestCount != 7 { - t.Fatalf("ListSessionPage() = %+v, want summary read request_count 7", page.Items) - } -} - func TestDBSummaryRebuildSessionsDryRunAndRebuildSQLite(t *testing.T) { t.Parallel() @@ -2768,7 +2418,7 @@ func TestDBSummaryRebuildSessionsDryRunAndRebuildSQLite(t *testing.T) { }); code != 0 { t.Fatalf("dry-run runDBSummaryRebuildSessionsWithOptions() = %d, output=%s", code, dryRunOut.String()) } - var dryEnvelope cliEnvelope + var dryEnvelope Envelope if err := json.Unmarshal(dryRunOut.Bytes(), &dryEnvelope); err != nil { t.Fatalf("json.Unmarshal(dry-run) error = %v; output=%s", err, dryRunOut.String()) } @@ -3647,9 +3297,9 @@ func TestAuditQueryCommandRequiresSelector(t *testing.T) { if err == nil { t.Fatal("Execute() error = nil, want selector usage error") } - var exit cliExitError - if !errors.As(err, &exit) || exit.code != exitCodeUsage { - t.Fatalf("Execute() error = %v, want usage cliExitError", err) + var exit ExitError + if !errors.As(err, &exit) || exit.code != ExitCodeUsage { + t.Fatalf("Execute() error = %v, want usage ExitError", err) } } @@ -3678,10 +3328,10 @@ func TestAuditQueryCommandReportsNotFound(t *testing.T) { }) err = cmd.Execute() if err == nil { - t.Fatal("Execute() error = nil, want not found cliExitError") + t.Fatal("Execute() error = nil, want not found ExitError") } - var exit cliExitError - if !errors.As(err, &exit) || exit.code != exitCodeAPI || exit.errCode != "RESPONSES_AUDIT_NOT_FOUND" { + var exit ExitError + if !errors.As(err, &exit) || exit.code != ExitCodeAPI || exit.errCode != "RESPONSES_AUDIT_NOT_FOUND" { t.Fatalf("Execute() error = %v, want RESPONSES_AUDIT_NOT_FOUND", err) } if strings.Contains(out.String(), "missing_client") { @@ -3952,79 +3602,6 @@ database: } } -func TestRunServeLogsActionableInvalidUpstreamConfig(t *testing.T) { - var buf bytes.Buffer - logger := slog.New(slog.NewTextHandler(&buf, nil)) - prev := slog.Default() - slog.SetDefault(logger) - t.Cleanup(func() { - slog.SetDefault(prev) - }) - - dir := t.TempDir() - configPath := filepath.Join(dir, "config.yaml") - configBody := []byte(strings.TrimSpace(` -database: - driver: "sqlite" -server: - port: "8080" -monitor: - port: "" -upstream: - base_url: "https://api.anthropic.com" - provider_preset: "anthropic" - protocol_family: "google_genai" -debug: - output_dir: "` + dir + `" - mask_key: false -`)) - if err := os.WriteFile(configPath, configBody, 0o644); err != nil { - t.Fatalf("WriteFile() error = %v", err) - } - - code := runServe([]string{"-c", configPath}) - if code != 1 { - t.Fatalf("runServe() = %d, want 1", code) - } - - output := buf.String() - for _, want := range []string{ - "Invalid upstream config", - `upstream.provider_preset=`, - `anthropic_messages`, - `google_genai`, - } { - if !strings.Contains(output, want) { - t.Fatalf("log output = %q, want contain %q", output, want) - } - } -} - -func TestValidateServeRouterConfigRequiresLocalResponsesServerBackend(t *testing.T) { - cfg := &config.Config{ - ResponsesServer: config.ResponsesServerConfig{}, - Upstreams: []config.UpstreamTargetConfig{ - { - ID: "anthropic-messages", - ModelDiscovery: router.ModelDiscoveryStaticOnly, - StaticModels: []string{"claude-sonnet-4-5"}, - Upstream: config.UpstreamConfig{ - BaseURL: "https://api.anthropic.com/v1", - ProviderPreset: "anthropic", - }, - }, - }, - } - - err := validateServeRouterConfig(cfg, cfg) - if err == nil { - t.Fatal("validateServeRouterConfig() error = nil, want local Responses server backend validation error") - } - if !strings.Contains(err.Error(), router.LocalResponsesServerBackendRequiredError) { - t.Fatalf("validateServeRouterConfig() error = %q, want contain %q", err.Error(), router.LocalResponsesServerBackendRequiredError) - } -} - func TestProviderProbeJSONReportsSuggestedAPISurface(t *testing.T) { upstreamServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if !strings.HasPrefix(r.Header.Get("Authorization"), "Bearer ") { @@ -4316,155 +3893,6 @@ database: } } -func TestStartupProviderProbeDefaultDisabledDoesNotProbe(t *testing.T) { - var calls int - upstreamServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - calls++ - http.NotFound(w, r) - })) - defer upstreamServer.Close() - - cfg := &config.Config{} - cfg.Upstreams = []config.UpstreamTargetConfig{ - { - ID: "local", - Upstream: config.UpstreamConfig{ - BaseURL: upstreamServer.URL + "/api/v1", - }, - }, - } - - if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { - t.Fatalf("applyStartupProviderProbeSuggestions() error = %v", err) - } - if calls != 0 { - t.Fatalf("startup probe made %d HTTP calls with default config, want 0", calls) - } - if cfg.Upstreams[0].Upstream.APIType != "" || cfg.Upstreams[0].Upstream.ProtocolFamily != "" { - t.Fatalf("upstream config was modified while disabled: %+v", cfg.Upstreams[0].Upstream) - } -} - -func TestStartupProviderProbeFillsMissingAPISurface(t *testing.T) { - upstreamServer := newOpenAIProbeTestServer(t) - defer upstreamServer.Close() - - cfg := &config.Config{} - cfg.ProviderProbe.StartupFill = true - cfg.Upstreams = []config.UpstreamTargetConfig{ - { - ID: "local", - Upstream: config.UpstreamConfig{ - BaseURL: upstreamServer.URL + "/api/v1", - }, - }, - } - - if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { - t.Fatalf("applyStartupProviderProbeSuggestions() error = %v", err) - } - got := cfg.Upstreams[0].Upstream - if got.ProtocolFamily != upstream.ProtocolFamilyOpenAICompatible { - t.Fatalf("ProtocolFamily = %q, want %q", got.ProtocolFamily, upstream.ProtocolFamilyOpenAICompatible) - } - if got.APIType != upstream.APITypeResponses { - t.Fatalf("APIType = %q, want %q", got.APIType, upstream.APITypeResponses) - } - if got.Capabilities.Responses == nil || !*got.Capabilities.Responses { - t.Fatalf("capabilities.responses = %v, want true", got.Capabilities.Responses) - } - if got.Capabilities.ChatCompletions == nil || !*got.Capabilities.ChatCompletions { - t.Fatalf("capabilities.chat_completions = %v, want true", got.Capabilities.ChatCompletions) - } - if got.Capabilities.Models == nil || !*got.Capabilities.Models { - t.Fatalf("capabilities.models = %v, want true", got.Capabilities.Models) - } -} - -func TestStartupProviderProbeFailureDoesNotBlockStartup(t *testing.T) { - upstreamServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - http.Error(w, "unavailable", http.StatusServiceUnavailable) - })) - defer upstreamServer.Close() - - cfg := &config.Config{} - cfg.ProviderProbe.StartupFill = true - cfg.Upstreams = []config.UpstreamTargetConfig{ - { - ID: "local", - Upstream: config.UpstreamConfig{ - BaseURL: upstreamServer.URL + "/api/v1", - }, - }, - } - - if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { - t.Fatalf("applyStartupProviderProbeSuggestions() error = %v, want nil best-effort startup", err) - } - got := cfg.Upstreams[0].Upstream - if got.APIType != "" || got.ProtocolFamily != "" || got.Capabilities.Responses != nil || got.Capabilities.ChatCompletions != nil { - t.Fatalf("failed startup probe modified config: %+v", got) - } -} - -func TestStartupProviderProbeDoesNotOverrideExplicitAPISurface(t *testing.T) { - upstreamServer := newOpenAIProbeTestServer(t) - defer upstreamServer.Close() - - responsesDisabled := false - cfg := &config.Config{} - cfg.ProviderProbe.StartupFill = true - cfg.Upstreams = []config.UpstreamTargetConfig{ - { - ID: "local", - Upstream: config.UpstreamConfig{ - BaseURL: upstreamServer.URL + "/api/v1", - APIType: upstream.APITypeChatCompletions, - ProtocolFamily: upstream.ProtocolFamilyAnthropicMessages, - Capabilities: config.UpstreamCapabilitiesConfig{ - Responses: &responsesDisabled, - }, - }, - }, - } - - if err := applyStartupProviderProbeSuggestions(context.Background(), cfg, upstreamServer.Client()); err != nil { - t.Fatalf("applyStartupProviderProbeSuggestions() error = %v", err) - } - got := cfg.Upstreams[0].Upstream - if got.ProtocolFamily != upstream.ProtocolFamilyAnthropicMessages { - t.Fatalf("ProtocolFamily = %q, want explicit %q", got.ProtocolFamily, upstream.ProtocolFamilyAnthropicMessages) - } - if got.APIType != upstream.APITypeChatCompletions { - t.Fatalf("APIType = %q, want explicit %q", got.APIType, upstream.APITypeChatCompletions) - } - if got.Capabilities.Responses == nil || *got.Capabilities.Responses { - t.Fatalf("capabilities.responses = %v, want explicit false", got.Capabilities.Responses) - } - if got.Capabilities.Models == nil || !*got.Capabilities.Models { - t.Fatalf("capabilities.models = %v, want missing field filled true", got.Capabilities.Models) - } -} - -func newOpenAIProbeTestServer(t *testing.T) *httptest.Server { - t.Helper() - return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - switch { - case r.Method == http.MethodGet && r.URL.Path == "/api/v1/models": - w.WriteHeader(http.StatusOK) - _, _ = w.Write([]byte(`{"data":[{"id":"gpt-test"}]}`)) - case r.Method == http.MethodPost && r.URL.Path == "/api/v1/chat/completions": - w.WriteHeader(http.StatusBadRequest) - _, _ = w.Write([]byte(`{"error":{"message":"model is required"}}`)) - case r.Method == http.MethodPost && r.URL.Path == "/api/v1/responses": - w.WriteHeader(http.StatusBadRequest) - _, _ = w.Write([]byte(`{"error":{"message":"input is required"}}`)) - default: - http.NotFound(w, r) - } - })) -} - func hasString(values []string, want string) bool { for _, value := range values { if value == want { @@ -5278,268 +4706,3 @@ debug: t.Fatalf("Stats() error = %v", err) } } - -func TestRunServeRejectsMCPWithoutMonitorPort(t *testing.T) { - var buf bytes.Buffer - logger := slog.New(slog.NewTextHandler(&buf, nil)) - prev := slog.Default() - slog.SetDefault(logger) - t.Cleanup(func() { - slog.SetDefault(prev) - }) - - dir := t.TempDir() - configPath := filepath.Join(dir, "config.yaml") - configBody := []byte(strings.TrimSpace(` -server: - port: "8080" -monitor: - port: "" -mcp: - enabled: true -upstream: - base_url: "https://api.openai.com/v1" -debug: - output_dir: "` + dir + `" - mask_key: false -`)) - if err := os.WriteFile(configPath, configBody, 0o644); err != nil { - t.Fatalf("WriteFile() error = %v", err) - } - - code := runServe([]string{"-c", configPath}) - if code != 1 { - t.Fatalf("runServe() = %d, want 1", code) - } - if output := buf.String(); !strings.Contains(output, "monitor.port is required when mcp.enabled=true") { - t.Fatalf("log output = %q, want monitor.port requirement", output) - } -} - -func TestRunServeRejectsUnknownDatabaseDriver(t *testing.T) { - var buf bytes.Buffer - logger := slog.New(slog.NewTextHandler(&buf, nil)) - prev := slog.Default() - slog.SetDefault(logger) - t.Cleanup(func() { - slog.SetDefault(prev) - }) - - dir := t.TempDir() - configPath := filepath.Join(dir, "config.yaml") - configBody := []byte(strings.TrimSpace(` -server: - port: "8080" -monitor: - port: "8081" -mcp: - enabled: true -database: - driver: "mysql" - dsn: "mysql://trajecta:secret@mysql:3306/trajecta" -upstream: - base_url: "https://api.openai.com/v1" -debug: - output_dir: "` + dir + `" - mask_key: false -`)) - if err := os.WriteFile(configPath, configBody, 0o644); err != nil { - t.Fatalf("WriteFile() error = %v", err) - } - - code := runServe([]string{"-c", configPath}) - if code != 1 { - t.Fatalf("runServe() = %d, want 1", code) - } - if output := buf.String(); !strings.Contains(output, "database.driver") || - !strings.Contains(output, "mysql") || - !strings.Contains(output, "is not supported yet; use sqlite or postgres") { - t.Fatalf("log output = %q, want unsupported database driver", output) - } -} - -func TestNewManagementMuxServesStreamableMCP(t *testing.T) { - t.Parallel() - - outputDir := t.TempDir() - st, err := store.New(outputDir) - if err != nil { - t.Fatalf("store.New() error = %v", err) - } - defer st.Close() - - cfg := &config.Config{} - cfg.Monitor.Port = "8081" - cfg.MCP.Enabled = true - cfg.MCP.Path = "/mcp" - - httpServer := httptest.NewServer(newManagementMux(st, nil, cfg)) - defer httpServer.Close() - - client := mcp.NewClient(&mcp.Implementation{Name: "test-client", Version: "v0.0.1"}, nil) - session, err := client.Connect(context.Background(), &mcp.StreamableClientTransport{ - Endpoint: httpServer.URL + "/mcp", - }, nil) - if err != nil { - t.Fatalf("client.Connect() error = %v", err) - } - defer session.Close() - if got := session.InitializeResult().ProtocolVersion; got != "2026-07-28" { - t.Fatalf("negotiated MCP protocol version = %q, want 2026-07-28", got) - } - - tools, err := session.ListTools(context.Background(), nil) - if err != nil { - t.Fatalf("session.ListTools() error = %v", err) - } - if len(tools.Tools) != 22 { - t.Fatalf("len(tools.Tools) = %d, want 22", len(tools.Tools)) - } -} - -func TestNewManagementMuxRejectsUnauthorizedMCP(t *testing.T) { - t.Parallel() - - outputDir := t.TempDir() - st, err := store.New(outputDir) - if err != nil { - t.Fatalf("store.New() error = %v", err) - } - defer st.Close() - - cfg := &config.Config{} - cfg.Monitor.Port = "8081" - cfg.MCP.Enabled = true - cfg.MCP.Path = "/mcp" - authStore := newTestAuthStore(t) - defer authStore.Close() - - httpServer := httptest.NewServer(newManagementMux(st, nil, cfg, authStore)) - defer httpServer.Close() - - client := mcp.NewClient(&mcp.Implementation{Name: "test-client", Version: "v0.0.1"}, nil) - _, err = client.Connect(context.Background(), &mcp.StreamableClientTransport{ - Endpoint: httpServer.URL + "/mcp", - }, nil) - if err == nil { - t.Fatalf("client.Connect() error = nil, want unauthorized error") - } - if !strings.Contains(err.Error(), "401") && !strings.Contains(strings.ToLower(err.Error()), "unauthorized") { - t.Fatalf("client.Connect() error = %v, want unauthorized", err) - } -} - -func TestNewManagementMuxServesAuthorizedStreamableMCP(t *testing.T) { - t.Parallel() - - outputDir := t.TempDir() - st, err := store.New(outputDir) - if err != nil { - t.Fatalf("store.New() error = %v", err) - } - defer st.Close() - - cfg := &config.Config{} - cfg.Monitor.Port = "8081" - cfg.MCP.Enabled = true - cfg.MCP.Path = "/mcp" - authStore := newTestAuthStore(t) - defer authStore.Close() - token, err := authStore.CreateToken(context.Background(), "admin", "mcp", auth.DefaultTokenScope, time.Hour) - if err != nil { - t.Fatalf("CreateToken() error = %v", err) - } - - httpServer := httptest.NewServer(newManagementMux(st, nil, cfg, authStore)) - defer httpServer.Close() - - client := mcp.NewClient(&mcp.Implementation{Name: "test-client", Version: "v0.0.1"}, nil) - session, err := client.Connect(context.Background(), &mcp.StreamableClientTransport{ - Endpoint: httpServer.URL + "/mcp", - HTTPClient: &http.Client{Transport: authTransport{Token: token.Token}}, - }, nil) - if err != nil { - t.Fatalf("client.Connect() error = %v", err) - } - defer session.Close() - - tools, err := session.ListTools(context.Background(), nil) - if err != nil { - t.Fatalf("session.ListTools() error = %v", err) - } - if len(tools.Tools) != 22 { - t.Fatalf("len(tools.Tools) = %d, want 22", len(tools.Tools)) - } -} - -func newTestAuthStore(t *testing.T) *auth.Store { - t.Helper() - dbPath := filepath.Join(t.TempDir(), "auth.sqlite3") - if err := auth.MigrateUp(dbPath, 0); err != nil { - t.Fatalf("auth.MigrateUp() error = %v", err) - } - authStore, err := auth.Open(dbPath) - if err != nil { - t.Fatalf("auth.Open() error = %v", err) - } - if _, err := authStore.CreateUser(context.Background(), "admin", "change-me-123"); err != nil { - _ = authStore.Close() - t.Fatalf("CreateUser() error = %v", err) - } - return authStore -} - -type authTransport struct { - Token string -} - -func (t authTransport) RoundTrip(req *http.Request) (*http.Response, error) { - clone := req.Clone(req.Context()) - clone.Header = req.Header.Clone() - clone.Header.Set("Authorization", "Bearer "+t.Token) - return http.DefaultTransport.RoundTrip(clone) -} - -func TestRouterConfigurationKeepsDisabledAndDeletedChannelsAcrossBootstrap(t *testing.T) { - for _, deleted := range []bool{false, true} { - t.Run(map[bool]string{false: "disabled", true: "deleted"}[deleted], func(t *testing.T) { - st, err := store.New(t.TempDir()) - if err != nil { - t.Fatal(err) - } - defer st.Close() - cfg := &config.Config{Upstream: config.UpstreamConfig{BaseURL: "https://yaml.example.invalid/v1", ProviderPreset: "openai"}} - svc := channel.NewService(st) - if _, err := svc.BootstrapFromConfig(cfg); err != nil { - t.Fatal(err) - } - channels, err := st.ListChannelConfigs() - if err != nil { - t.Fatal(err) - } - if len(channels) != 1 { - t.Fatalf("channels=%v", channels) - } - if deleted { - err = st.DeleteChannelConfig(channels[0].ID) - } else { - record := channels[0] - record.Enabled = false - _, err = st.UpsertChannelConfig(record) - } - if err != nil { - t.Fatal(err) - } - if imported, err := svc.BootstrapFromConfig(cfg); err != nil || imported != 0 { - t.Fatalf("bootstrap=%d %v", imported, err) - } - runtimeCfg, source, err := routerConfigFromChannels(cfg, svc) - if err != nil { - t.Fatal(err) - } - if source != "database" || len(runtimeCfg.EffectiveUpstreams()) != 0 { - t.Fatalf("source=%s targets=%v", source, runtimeCfg.EffectiveUpstreams()) - } - }) - } -} diff --git a/cmd/server/migrate.go b/internal/cli/migrate.go similarity index 88% rename from cmd/server/migrate.go rename to internal/cli/migrate.go index 7a4bbad3..b9f57fd9 100644 --- a/cmd/server/migrate.go +++ b/internal/cli/migrate.go @@ -1,10 +1,11 @@ -package main +package cli import ( "io" "log/slog" "os" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/migrate" "github.com/spf13/cobra" @@ -20,7 +21,7 @@ type migrateOptions struct { stdout io.Writer } -func newMigrateCommand(runtime *cliRuntime) *cobra.Command { +func newMigrateCommand(runtime *Runtime) *cobra.Command { var rewriteV2 bool var rebuildIndex bool var dryRun bool @@ -29,13 +30,13 @@ func newMigrateCommand(runtime *cliRuntime) *cobra.Command { Short: "Rewrite legacy cassettes and rebuild the trace metadata index", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runMigrateWithOptions(migrateOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), rewriteV2: rewriteV2, rebuildIndex: rebuildIndex, dryRun: dryRun, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -53,7 +54,7 @@ func runMigrate(args []string) int { rewriteV2 := fs.Bool("rewrite-v2", true, "Rewrite legacy V2 cassette files to V3") rebuildIndex := fs.Bool("rebuild-index", true, "Rebuild SQLite metadata index from cassette files") fs.SetOutput(os.Stderr) - if err := fs.Parse(normalizeLegacyFlagArgs(args)); err != nil { + if err := fs.Parse(NormalizeLegacyFlagArgs(args)); err != nil { return 2 } return runMigrateWithOptions(migrateOptions{ @@ -82,7 +83,7 @@ func runMigrateWithOptions(opts migrateOptions) int { "database_dsn": config.RedactDSN(cfg.DatabaseDSN()), }) } - traceStore, err := openApplicationDatabase(cfg) + traceStore, err := appdb.Open(cfg) if err != nil { slog.Error("Failed to initialize trace store", "error", err) return 1 @@ -108,7 +109,7 @@ func runMigrateWithOptions(opts migrateOptions) int { "indexed_rows", result.RebuiltIndexRows, ) if opts.format == "json" { - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "migrate", map[string]any{ + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "migrate", map[string]any{ "mutated": opts.rewriteV2 || opts.rebuildIndex, "output_dir": cfg.TraceOutputDir(), "scanned_files": result.ScannedFiles, diff --git a/cmd/server/models.go b/internal/cli/models.go similarity index 99% rename from cmd/server/models.go rename to internal/cli/models.go index 20f91055..c06c1f5d 100644 --- a/cmd/server/models.go +++ b/internal/cli/models.go @@ -1,4 +1,4 @@ -package main +package cli import ( "database/sql" @@ -209,21 +209,21 @@ type modelsCodexLocalFieldStatus struct { Actual string `json:"actual,omitempty"` } -func newModelsCommand(runtime *cliRuntime) *cobra.Command { +func newModelsCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "models", Short: "Inspect model-related local configuration", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } cmd.AddCommand(newModelsCodexConfigCommand(runtime)) return cmd } -func newModelsCodexConfigCommand(runtime *cliRuntime) *cobra.Command { +func newModelsCodexConfigCommand(runtime *Runtime) *cobra.Command { var codexConfigPath string var checkDB bool cmd := &cobra.Command{ @@ -231,12 +231,12 @@ func newModelsCodexConfigCommand(runtime *cliRuntime) *cobra.Command { Short: "Generate a Codex profile TOML suggestion from local Responses model profiles", Args: cobra.ExactArgs(1), RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runModelsCodexConfigWithOptions(modelsCodexConfigOptions{ - configPath: runtime.configPath(), + configPath: runtime.ConfigPath(), codexConfigPath: codexConfigPath, checkDB: checkDB, - format: runtime.outputFormat(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), model: args[0], }) @@ -255,7 +255,7 @@ func runModelsCodexConfigWithOptions(opts modelsCodexConfigOptions) int { return 1 } result := buildModelsCodexConfigResult(cfg, opts.model, opts.codexConfigPath, opts.checkDB) - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, codexConfigCommand, result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, codexConfigCommand, result, func(w io.Writer) error { writeModelsCodexConfigText(w, result) return nil }); err != nil { diff --git a/cmd/server/provider.go b/internal/cli/provider.go similarity index 78% rename from cmd/server/provider.go rename to internal/cli/provider.go index 70416e22..538c505c 100644 --- a/cmd/server/provider.go +++ b/internal/cli/provider.go @@ -1,4 +1,4 @@ -package main +package cli import ( "context" @@ -10,6 +10,7 @@ import ( "strings" "time" + "github.com/kingfs/Trajecta/internal/appdb" "github.com/kingfs/Trajecta/internal/channel" "github.com/kingfs/Trajecta/internal/config" "github.com/kingfs/Trajecta/internal/providerprobe" @@ -25,14 +26,14 @@ type providerProbeOptions struct { command string } -func newProviderCommand(runtime *cliRuntime) *cobra.Command { +func newProviderCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "provider", Short: "Inspect configured upstream providers", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } cmd.AddCommand(newProviderProbeCommand(runtime)) @@ -41,7 +42,7 @@ func newProviderCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newProviderProbeCommand(runtime *cliRuntime) *cobra.Command { +func newProviderProbeCommand(runtime *Runtime) *cobra.Command { var id string var timeout time.Duration cmd := &cobra.Command{ @@ -49,10 +50,10 @@ func newProviderProbeCommand(runtime *cliRuntime) *cobra.Command { Short: "Probe configured upstream provider API surfaces", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runProviderProbeWithOptions(providerProbeOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), id: id, timeout: timeout, @@ -66,7 +67,7 @@ func newProviderProbeCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newProviderProbeApplyCommand(runtime *cliRuntime) *cobra.Command { +func newProviderProbeApplyCommand(runtime *Runtime) *cobra.Command { var id string var timeout time.Duration cmd := &cobra.Command{ @@ -74,10 +75,10 @@ func newProviderProbeApplyCommand(runtime *cliRuntime) *cobra.Command { Short: "Apply provider probe suggestions to managed channels", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runProviderProbeApplyWithOptions(providerProbeOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), id: id, timeout: timeout, @@ -91,7 +92,7 @@ func newProviderProbeApplyCommand(runtime *cliRuntime) *cobra.Command { return cmd } -func newProviderProbeReportCommand(runtime *cliRuntime) *cobra.Command { +func newProviderProbeReportCommand(runtime *Runtime) *cobra.Command { var id string var timeout time.Duration cmd := &cobra.Command{ @@ -99,10 +100,10 @@ func newProviderProbeReportCommand(runtime *cliRuntime) *cobra.Command { Short: "Generate a read-only batch provider probe report for configured upstreams", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runProviderProbeWithOptions(providerProbeOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), id: id, timeout: timeout, @@ -143,7 +144,7 @@ func runProviderProbeWithOptions(opts providerProbeOptions) int { if command == "" { command = "provider.probe" } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, command, result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, command, result, func(w io.Writer) error { writeProviderProbeText(w, result) return nil }); err != nil { @@ -165,7 +166,7 @@ func providerProbeTargets(cfg config.Config, id string) ([]providerprobe.ProbeTa if id != "" && targetID != id { continue } - probeTarget := providerProbeTargetFromConfig(target) + probeTarget := providerprobe.TargetFromConfig(target) probeTarget.TargetSource = "upstream" if strings.TrimSpace(probeTarget.BaseURL) == "" { continue @@ -181,47 +182,13 @@ func providerProbeTargets(cfg config.Config, id string) ([]providerprobe.ProbeTa return targets, nil } -func providerProbeTargetFromConfig(target config.UpstreamTargetConfig) providerprobe.ProbeTarget { - upstream := target.Upstream - apiKey := strings.TrimSpace(upstream.ApiKey) - headers := cloneProviderProbeHeaders(upstream.Headers) - for _, credential := range target.EffectiveCredentials() { - if credential.Enabled != nil && !*credential.Enabled { - continue - } - if apiKey == "" { - apiKey = strings.TrimSpace(credential.ApiKey) - } - for key, value := range credential.Headers { - headers[key] = value - } - break - } - return providerprobe.ProbeTarget{ - ProviderID: strings.TrimSpace(target.ID), - BaseURL: strings.TrimSpace(upstream.BaseURL), - APIKey: apiKey, - Headers: headers, - SpecifiedAPIType: upstream.APIType, - SpecifiedProtocolFamily: upstream.ProtocolFamily, - } -} - -func cloneProviderProbeHeaders(in map[string]string) map[string]string { - out := map[string]string{} - for key, value := range in { - out[key] = value - } - return out -} - func runProviderProbeApplyWithOptions(opts providerProbeOptions) int { cfg, err := config.Load(opts.configPath) if err != nil { slog.Error("Failed to load config", "path", opts.configPath, "error", err) return 1 } - st, err := openApplicationDatabase(cfg) + st, err := appdb.Open(cfg) if err != nil { slog.Error("Open application store failed", "error", err) return 1 @@ -248,7 +215,7 @@ func runProviderProbeApplyWithOptions(opts providerProbeOptions) int { if command == "" { command = "provider.probe_apply" } - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, command, result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, command, result, func(w io.Writer) error { writeProviderProbeApplyText(w, result) return nil }); err != nil { diff --git a/internal/cli/register.go b/internal/cli/register.go new file mode 100644 index 00000000..7b5fcca1 --- /dev/null +++ b/internal/cli/register.go @@ -0,0 +1,34 @@ +package cli + +import "github.com/spf13/cobra" + +// Register adds every server-management command group to root. +// +// The caller owns the root command and its persistent --config/--format flags; +// rt must be bound to those flags so the groups resolve the same values. The +// per-binary commands (version, completion) and the runtime command (serve) are +// deliberately not registered here. +func Register(root *cobra.Command, rt *Runtime, opts Options) { + if root == nil { + return + } + if rt == nil { + rt = NewRuntime() + } + rt.name = opts.Name + rt.version = opts.Version + + root.AddCommand( + newAnalyzeCommand(rt), + newMigrateCommand(rt), + newDBCommand(rt), + newConfigCommand(rt), + newDoctorCommand(rt), + newProviderCommand(rt), + newModelsCommand(rt), + newToolsCommand(rt), + newAuditCommand(rt), + newAuthCommand(rt), + newSchemaCommand(rt, root), + ) +} diff --git a/internal/cli/root.go b/internal/cli/root.go new file mode 100644 index 00000000..3ce3b254 --- /dev/null +++ b/internal/cli/root.go @@ -0,0 +1,612 @@ +// Package cli holds the Trajecta command-line contract and the +// server-management command groups. +// +// The contract is the exit-code table, the JSON envelope, the viper-backed +// Runtime that resolves --config and --format, and the ERROR log recorder that +// lets a command report the cause it only logged. Both binaries share it. +// +// The command groups are the operations that never need the proxy process: +// analyze, audit, auth, config, db, doctor, migrate, models, provider, schema +// and tools. The trajecta binary registers all of them through Register; the +// server binary imports this package for the contract only and registers none. +package cli + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "log/slog" + "os" + "reflect" + "strings" + "sync" + + "github.com/spf13/cobra" + "github.com/spf13/pflag" + "github.com/spf13/viper" +) + +// Version is the CLI version reported by the version and schema envelopes. +// Each binary assigns it from its own build metadata before it builds the +// command tree. +var Version = "dev" + +// Options describes the binary embedding the command groups. +type Options struct { + // Name labels the command tree; it drives the schema payload. + Name string + // Version overrides the package Version for the schema payload. + Version string +} + +// Runtime carries the resolved --config and --format values plus the helpers the +// command groups share. +type Runtime struct { + settings *viper.Viper + name string + version string + + // resultWritten records that a command already emitted its own envelope, so + // the top-level failure path does not print a second one. + resultWritten bool +} + +// NewRuntime returns a Runtime backed by its own viper instance, with the +// TRAJECTA_ environment prefix already wired up. +func NewRuntime() *Runtime { + settings := viper.New() + settings.SetEnvPrefix("TRAJECTA") + settings.SetEnvKeyReplacer(strings.NewReplacer("-", "_")) + settings.AutomaticEnv() + return &Runtime{settings: settings} +} + +// RuntimeFromViper returns a Runtime that reads --config and --format from an +// existing viper instance, so an embedding binary keeps its own flag +// definitions. +func RuntimeFromViper(settings *viper.Viper) *Runtime { + if settings == nil { + return NewRuntime() + } + return &Runtime{settings: settings} +} + +// BindPFlag connects a persistent flag to the runtime setting of the same name. +func (rt *Runtime) BindPFlag(key string, flag *pflag.Flag) { + if rt == nil || rt.settings == nil || flag == nil { + return + } + if err := rt.settings.BindPFlag(key, flag); err != nil { + panic(err) + } +} + +// Name is the command-tree name used by the schema payload. +func (rt *Runtime) Name() string { + if rt == nil || rt.name == "" { + return "trajecta" + } + return rt.name +} + +// Version is the version used by the schema payload. +func (rt *Runtime) Version() string { + if rt == nil || rt.version == "" { + return Version + } + return rt.version +} + +// ConfigPath is the resolved --config value. +func (rt *Runtime) ConfigPath() string { + if rt == nil || rt.settings == nil { + return "config.yaml" + } + if path := rt.settings.GetString("config"); path != "" { + return path + } + return "config.yaml" +} + +// OutputFormat is the resolved --format value. +func (rt *Runtime) OutputFormat() string { + if rt == nil || rt.settings == nil { + return "text" + } + if format := rt.settings.GetString("format"); format != "" { + return format + } + return "text" +} + +// MarkResultWritten records that a command emitted its own result envelope, +// which may already carry ok=false and its own warnings. +func (rt *Runtime) MarkResultWritten() { + if rt != nil { + rt.resultWritten = true + } +} + +// ResultWritten reports whether a command already emitted its own envelope. +func (rt *Runtime) ResultWritten() bool { + return rt != nil && rt.resultWritten +} + +// ValidateOutputFormat rejects any --format other than text or json. +func (rt *Runtime) ValidateOutputFormat() error { + switch rt.OutputFormat() { + case "text", "json": + return nil + default: + return ExitError{ + code: ExitCodeUsage, + category: ErrorCategoryUsage, + errCode: "INVALID_OUTPUT_FORMAT", + message: "--format must be one of: text, json", + field: "format", + } + } +} + +// Execute runs one command tree under the shared top-level contract: the ERROR +// log recorder that surfaces a logged cause in the JSON envelope, the legacy +// single-dash flag normalization, and the exit-code classification. It is the +// single place the binaries hand control to cobra. +func Execute(args []string, root *cobra.Command, rt *Runtime) int { + logger := slog.New(slog.NewTextHandler(os.Stderr, nil)) + slog.SetDefault(logger) + + root.SetArgs(NormalizeLegacyFlagArgs(args)) + root.SetOut(os.Stdout) + root.SetErr(os.Stderr) + recorder := &ErrorRecorder{} + slog.SetDefault(slog.New(NewLogHandler(os.Stderr, recorder))) + + if err := root.Execute(); err != nil { + exit := ExitErrorFromErr(err) + // Commands that only log their failure and return a bare code (the + // trace store, config loading) would otherwise collapse into + // COMMAND_FAILED/"command failed"; surface the logged cause instead. + if recorder.Has() && fallback(exit.message, "command failed") == "command failed" { + exit.message = recorder.Message() + } + // cobra fails before the persistent --format flag is bound when the + // command itself is unknown, so read it from argv as a fallback. + format := rt.OutputFormat() + if format != "json" { + if fromArgs := outputFormatFromArgs(args); fromArgs != "" { + format = fromArgs + } + } + // A command that wrote an ok=false envelope already reported the failure; + // only the text form still needs the cause on stderr. + if rt.ResultWritten() { + if format != "json" { + fmt.Fprintln(root.ErrOrStderr(), "error: "+exit.Error()) + } + return exit.code + } + WriteError(root.ErrOrStderr(), format, exit) + return exit.code + } + return 0 +} + +// outputFormatFromArgs finds an explicit --format value in the raw argv, for the +// failure paths where cobra never bound the flag. +func outputFormatFromArgs(args []string) string { + for i, arg := range args { + if arg == "--format" && i+1 < len(args) { + return args[i+1] + } + if value, ok := strings.CutPrefix(arg, "--format="); ok { + return value + } + } + return "" +} + +// ExitErrorFromErr maps a command error to the documented exit contract: +// usage/validation errors exit 3, internal failures exit 8. +func ExitErrorFromErr(err error) ExitError { + var exit ExitError + if errors.As(err, &exit) { + return exit + } + message := err.Error() + if IsUsageError(err) { + return UsageError(message, "") + } + return ExitError{ + code: ExitCodeInternal, + category: ErrorCategoryInternal, + errCode: "INTERNAL_ERROR", + message: message, + } +} + +// IsUsageError reports whether a bare cobra error is really a usage error. +func IsUsageError(err error) bool { + if err == nil { + return false + } + message := err.Error() + for _, prefix := range []string{ + "unknown command ", + "unknown flag: ", + "unknown shorthand flag: ", + "required flag(s) ", + "invalid argument ", + "accepts at most ", + "accepts between ", + "accepts ", + } { + if strings.HasPrefix(message, prefix) { + return true + } + } + return false +} + +// ErrorRecorder keeps the most recent ERROR log line so a command that only +// logs its failure can still report the real cause in the JSON envelope. +type ErrorRecorder struct { + mu sync.Mutex + lastText string +} + +func (r *ErrorRecorder) record(message string) { + r.mu.Lock() + defer r.mu.Unlock() + r.lastText = message +} + +// Has reports whether an ERROR line was recorded. +func (r *ErrorRecorder) Has() bool { + r.mu.Lock() + defer r.mu.Unlock() + return strings.TrimSpace(r.lastText) != "" +} + +// Message is the most recent recorded ERROR line. +func (r *ErrorRecorder) Message() string { + r.mu.Lock() + defer r.mu.Unlock() + return r.lastText +} + +// LogHandler writes the usual text log and mirrors ERROR records into the +// recorder. +type LogHandler struct { + primary slog.Handler + recorder *ErrorRecorder +} + +// NewLogHandler returns the CLI's slog handler. +func NewLogHandler(w io.Writer, recorder *ErrorRecorder) slog.Handler { + return &LogHandler{ + primary: slog.NewTextHandler(w, nil), + recorder: recorder, + } +} + +func (h *LogHandler) Enabled(ctx context.Context, level slog.Level) bool { + return h.primary.Enabled(ctx, level) +} + +func (h *LogHandler) Handle(ctx context.Context, record slog.Record) error { + if record.Level >= slog.LevelError { + // The `error` attribute carries the real cause; the remaining attributes + // are context that the human-readable log line already shows. + var cause string + var context []string + record.Attrs(func(attr slog.Attr) bool { + if attr.Key == "error" { + if cause == "" { + cause = attr.Value.String() + } + return true + } + context = append(context, fmt.Sprintf("%s=%v", attr.Key, attr.Value.Any())) + return true + }) + message := record.Message + if cause != "" { + message = message + ": " + cause + } else if len(context) > 0 { + message = message + ": " + strings.Join(context, " ") + } + h.recorder.record(message) + } + return h.primary.Handle(ctx, record) +} + +func (h *LogHandler) WithAttrs(attrs []slog.Attr) slog.Handler { + return &LogHandler{primary: h.primary.WithAttrs(attrs), recorder: h.recorder} +} + +func (h *LogHandler) WithGroup(name string) slog.Handler { + return &LogHandler{primary: h.primary.WithGroup(name), recorder: h.recorder} +} + +// RunCode turns a command's integer exit code into the matching error, so a +// RunE body can hand a `func() int` helper straight back to cobra. +func RunCode(run func() int) error { + if code := run(); code != 0 { + return ExitErrorFromCode(code) + } + return nil +} + +// UsageError builds the documented exit-3 error. +func UsageError(message string, field string) ExitError { + return ExitError{ + code: ExitCodeUsage, + category: ErrorCategoryUsage, + errCode: "USAGE_ERROR", + message: message, + field: field, + } +} + +// APIError builds the documented exit-1 error: the command ran but the +// operation failed. +func APIError(message string) ExitError { + return ExitError{ + code: ExitCodeAPI, + category: ErrorCategoryAPI, + errCode: "COMMAND_FAILED", + message: message, + } +} + +// RequireSubcommand renders help and exits 3 when a command group is invoked +// without a subcommand. +func RequireSubcommand(cmd *cobra.Command) error { + if cmd.Flag("format") != nil && cmd.Flag("format").Value.String() == "json" { + return UsageError(fmt.Sprintf("%s requires a subcommand", cmd.CommandPath()), "command") + } + if err := cmd.Help(); err != nil { + return ExitError{ + code: ExitCodeInternal, + category: ErrorCategoryInternal, + errCode: "HELP_RENDER_FAILED", + message: err.Error(), + } + } + return ExitErrorFromCode(ExitCodeUsage) +} + +// NormalizeLegacyFlagArgs rewrites a legacy single-dash long flag (-config) into +// the double-dash form cobra expects. +func NormalizeLegacyFlagArgs(args []string) []string { + normalized := make([]string, len(args)) + for i, arg := range args { + if strings.HasPrefix(arg, "-") && !strings.HasPrefix(arg, "--") && len(arg) > 2 { + normalized[i] = "-" + arg + continue + } + normalized[i] = arg + } + return normalized +} + +// ExitError is the CLI's own error: it carries the process exit code and the +// envelope fields. +type ExitError struct { + code int + category string + errCode string + message string + field string +} + +func (e ExitError) Error() string { + if e.message != "" { + return e.message + } + return fmt.Sprintf("exit %d", e.code) +} + +// Exit codes and error categories follow the documented contract. +const ( + ExitCodeAPI = 1 + ExitCodeUsage = 3 + ExitCodeInternal = 8 + ErrorCategoryAPI = "api" + ErrorCategoryUsage = "usage" + ErrorCategoryInternal = "internal" +) + +// ExitErrorFromCode maps a bare integer exit code onto the contract. +func ExitErrorFromCode(code int) ExitError { + switch code { + case 2, ExitCodeUsage: + return ExitError{ + code: ExitCodeUsage, + category: ErrorCategoryUsage, + errCode: "USAGE_ERROR", + message: "command usage is invalid", + } + case ExitCodeInternal: + return ExitError{ + code: ExitCodeInternal, + category: ErrorCategoryInternal, + errCode: "INTERNAL_ERROR", + message: "command failed with an internal error", + } + default: + return ExitError{ + code: code, + category: ErrorCategoryAPI, + errCode: "COMMAND_FAILED", + message: "command failed", + } + } +} + +// Envelope is the JSON result/error document both binaries emit. +type Envelope struct { + OK bool `json:"ok"` + Command string `json:"command,omitempty"` + Result any `json:"result,omitempty"` + Warnings []string `json:"warnings"` + Error *ErrorDetail `json:"error,omitempty"` +} + +// ErrorDetail is the machine-readable error inside an Envelope. +type ErrorDetail struct { + Code string `json:"code"` + Category string `json:"category"` + Message string `json:"message"` + Field string `json:"field,omitempty"` + Retryable bool `json:"retryable"` + SafeToRetry bool `json:"safe_to_retry"` +} + +// WriteResult writes the text form or a success envelope. +func WriteResult(w io.Writer, format string, command string, result any, text func(io.Writer) error) error { + return WriteResultWithWarnings(w, format, command, result, ResultWarnings(result), text) +} + +// WriteResultWithWarnings writes the text form or a JSON envelope. The +// envelope-level warnings mirror the ones the text form prints, so a caller that +// only reads envelope.warnings does not miss them. +func WriteResultWithWarnings(w io.Writer, format string, command string, result any, warnings []string, text func(io.Writer) error) error { + return WriteResultWithState(w, format, command, true, result, warnings, text) +} + +// WriteResultWithState writes the text form or a JSON envelope whose ok field +// reflects the command's own verdict (a check that found problems reports +// ok=false while still exiting non-zero through the returned error). +func WriteResultWithState(w io.Writer, format string, command string, ok bool, result any, warnings []string, text func(io.Writer) error) error { + if format != "json" { + if text == nil { + return nil + } + return text(w) + } + if warnings == nil { + warnings = []string{} + } + return WriteJSON(w, Envelope{ + OK: ok, + Command: command, + Result: result, + Warnings: warnings, + }) +} + +// ResultWarnings collects the warnings a command attached to its own result, so +// the envelope carries them whether they are a []string field or a map entry. +func ResultWarnings(result any) []string { + switch value := result.(type) { + case map[string]any: + return StringSlice(value["warnings"]) + case []map[string]any: + var out []string + for _, item := range value { + out = append(out, StringSlice(item["warnings"])...) + } + return out + } + return warningsFromStruct(result) +} + +// StringSlice coerces the usual JSON-ish shapes into a []string. +func StringSlice(value any) []string { + switch typed := value.(type) { + case []string: + return typed + case []any: + out := make([]string, 0, len(typed)) + for _, item := range typed { + if text, ok := item.(string); ok && text != "" { + out = append(out, text) + } + } + return out + case string: + if typed == "" { + return nil + } + return []string{typed} + } + return nil +} + +// warningsFromStruct reads a top-level `warnings` []string field, plus a +// `reports[].warnings` list, through reflection so the helper stays dependency +// free and works for the report structs the provider commands return. +func warningsFromStruct(result any) []string { + value := reflect.ValueOf(result) + for value.Kind() == reflect.Pointer || value.Kind() == reflect.Interface { + if value.IsNil() { + return nil + } + value = value.Elem() + } + if value.Kind() != reflect.Struct { + return nil + } + var out []string + if field := value.FieldByName("Warnings"); field.IsValid() && field.Kind() == reflect.Slice { + out = append(out, StringSlice(field.Interface())...) + } + if field := value.FieldByName("Reports"); field.IsValid() && field.Kind() == reflect.Slice { + for i := 0; i < field.Len(); i++ { + item := field.Index(i) + for item.Kind() == reflect.Pointer { + if item.IsNil() { + break + } + item = item.Elem() + } + if item.Kind() != reflect.Struct { + continue + } + if warnings := item.FieldByName("Warnings"); warnings.IsValid() && warnings.Kind() == reflect.Slice { + out = append(out, StringSlice(warnings.Interface())...) + } + } + } + return out +} + +// WriteError renders a failure in the selected format. +func WriteError(w io.Writer, format string, exit ExitError) { + if format != "json" { + if exit.message != "" { + fmt.Fprintln(w, exit.message) + } + return + } + _ = WriteJSON(w, Envelope{ + OK: false, + Warnings: []string{}, + Error: &ErrorDetail{ + Code: fallback(exit.errCode, "COMMAND_FAILED"), + Category: fallback(exit.category, ErrorCategoryAPI), + Message: fallback(exit.message, "command failed"), + Field: exit.field, + Retryable: exit.category != ErrorCategoryUsage, + SafeToRetry: exit.category != ErrorCategoryUsage, + }, + }) +} + +// WriteJSON writes an indented JSON document. +func WriteJSON(w io.Writer, value any) error { + encoder := json.NewEncoder(w) + encoder.SetIndent("", " ") + return encoder.Encode(value) +} + +func fallback(value string, fallbackValue string) string { + if value != "" { + return value + } + return fallbackValue +} diff --git a/cmd/server/schema.go b/internal/cli/schema.go similarity index 86% rename from cmd/server/schema.go rename to internal/cli/schema.go index 142bff33..7debde69 100644 --- a/cmd/server/schema.go +++ b/internal/cli/schema.go @@ -1,4 +1,4 @@ -package main +package cli import ( "fmt" @@ -26,8 +26,8 @@ type cliSchemaContracts struct { } type cliErrorEnvelope struct { - OK bool `json:"ok"` - Error cliError `json:"error"` + OK bool `json:"ok"` + Error ErrorDetail `json:"error"` } type cliCommandSchema struct { @@ -49,7 +49,7 @@ type cliFlagSchema struct { Values []string `json:"values,omitempty"` } -func newSchemaCommand(runtime *cliRuntime, root *cobra.Command) *cobra.Command { +func newSchemaCommand(runtime *Runtime, root *cobra.Command) *cobra.Command { return &cobra.Command{ Use: "schema [command...]", Short: "Print machine-readable CLI schema for agents and automation", @@ -63,8 +63,8 @@ func newSchemaCommand(runtime *cliRuntime, root *cobra.Command) *cobra.Command { return err } } - schema := buildCLISchema(root, target) - return writeCLIResult(cmd.OutOrStdout(), runtime.outputFormat(), "schema", schema, func(w io.Writer) error { + schema := buildCLISchema(runtime, root, target) + return WriteResult(cmd.OutOrStdout(), runtime.OutputFormat(), "schema", schema, func(w io.Writer) error { for _, item := range schema.Commands { if _, err := fmt.Fprintf(w, "%s\t%s\n", item.Path, item.Short); err != nil { return err @@ -86,9 +86,9 @@ func findCommandByPath(root *cobra.Command, path string) (*cobra.Command, error) } target, _, err := root.Find(parts) if err != nil || target == nil { - return nil, cliExitError{ - code: exitCodeUsage, - category: errorCategoryUsage, + return nil, ExitError{ + code: ExitCodeUsage, + category: ErrorCategoryUsage, errCode: "UNKNOWN_COMMAND", message: fmt.Sprintf("unknown command %q", path), field: "command", @@ -97,26 +97,26 @@ func findCommandByPath(root *cobra.Command, path string) (*cobra.Command, error) return target, nil } -func buildCLISchema(root *cobra.Command, target *cobra.Command) cliSchema { +func buildCLISchema(runtime *Runtime, root *cobra.Command, target *cobra.Command) cliSchema { return cliSchema{ - Name: cliName, - Version: Version, + Name: runtime.Name(), + Version: runtime.Version(), Description: root.Short, Contracts: cliSchemaContracts{ Formats: []string{"text", "json"}, ExitCodes: map[string]int{ "success": 0, - "api_error": exitCodeAPI, - "usage_or_validation": exitCodeUsage, - "internal_error": exitCodeInternal, + "api_error": ExitCodeAPI, + "usage_or_validation": ExitCodeUsage, + "internal_error": ExitCodeInternal, }, Stdout: "primary command result only", Stderr: "logs, diagnostics, and structured error envelopes on failure", ErrorEnvelope: cliErrorEnvelope{ OK: false, - Error: cliError{ + Error: ErrorDetail{ Code: "USAGE_ERROR", - Category: errorCategoryUsage, + Category: ErrorCategoryUsage, Message: "example error message", Field: "flag_or_argument", Retryable: false, diff --git a/internal/cli/testroot_test.go b/internal/cli/testroot_test.go new file mode 100644 index 00000000..5af19865 --- /dev/null +++ b/internal/cli/testroot_test.go @@ -0,0 +1,43 @@ +package cli + +import ( + "github.com/spf13/cobra" +) + +// rootName is the command-tree name of the test root. +const rootName = "trajecta" + +// testOpsRuntime is the shared runtime of the most recently built test root. The +// run helper hands it to Execute so the tests exercise the same top-level error +// path the trajecta binary uses. +var testOpsRuntime *Runtime + +// newRootCommand builds the command tree under test: the shared --config and +// --format flags plus every registered server-management group. +// +// The version and completion commands belong to the embedding binary, not to +// this package, so they are absent here; the binaries test those themselves. +func newRootCommand() *cobra.Command { + root := &cobra.Command{ + Use: "trajecta", + Short: "Trajecta command-line tool", + SilenceUsage: true, + SilenceErrors: true, + } + root.PersistentFlags().StringP("config", "c", "config.yaml", "path to the configuration file") + root.PersistentFlags().String("format", "text", "output format: text or json") + + runtime := NewRuntime() + runtime.BindPFlag("config", root.PersistentFlags().Lookup("config")) + runtime.BindPFlag("format", root.PersistentFlags().Lookup("format")) + testOpsRuntime = runtime + Register(root, runtime, Options{Name: "trajecta", Version: Version}) + return root +} + +// run executes the test command tree under the shared top-level contract and +// returns the process exit code. +func run(args []string) int { + cmd := newRootCommand() + return Execute(args, cmd, testOpsRuntime) +} diff --git a/cmd/server/tools.go b/internal/cli/tools.go similarity index 96% rename from cmd/server/tools.go rename to internal/cli/tools.go index 1b7d2bbd..cbe5d93d 100644 --- a/cmd/server/tools.go +++ b/internal/cli/tools.go @@ -1,4 +1,4 @@ -package main +package cli import ( "fmt" @@ -80,30 +80,30 @@ type toolsStatusMCPServer struct { Enabled bool `json:"enabled"` } -func newToolsCommand(runtime *cliRuntime) *cobra.Command { +func newToolsCommand(runtime *Runtime) *cobra.Command { cmd := &cobra.Command{ Use: "tools", Short: "Inspect hosted tool readiness", SilenceUsage: true, SilenceErrors: true, RunE: func(cmd *cobra.Command, args []string) error { - return requireSubcommand(cmd) + return RequireSubcommand(cmd) }, } cmd.AddCommand(newToolsStatusCommand(runtime)) return cmd } -func newToolsStatusCommand(runtime *cliRuntime) *cobra.Command { +func newToolsStatusCommand(runtime *Runtime) *cobra.Command { return &cobra.Command{ Use: "status", Short: "Print offline hosted tool readiness for integration tests", Args: cobra.NoArgs, RunE: func(cmd *cobra.Command, args []string) error { - return runCode(func() int { + return RunCode(func() int { return runToolsStatusWithOptions(toolsStatusOptions{ - configPath: runtime.configPath(), - format: runtime.outputFormat(), + configPath: runtime.ConfigPath(), + format: runtime.OutputFormat(), stdout: cmd.OutOrStdout(), }) }) @@ -118,7 +118,7 @@ func runToolsStatusWithOptions(opts toolsStatusOptions) int { return 1 } result := buildToolsStatusResult(opts.configPath, cfg) - if err := writeCLIResult(stdoutOrDefault(opts.stdout), opts.format, "tools.status", result, func(w io.Writer) error { + if err := WriteResult(stdoutOrDefault(opts.stdout), opts.format, "tools.status", result, func(w io.Writer) error { writeToolsStatusText(w, result) return nil }); err != nil { diff --git a/cmd/server/tools_test.go b/internal/cli/tools_test.go similarity index 98% rename from cmd/server/tools_test.go rename to internal/cli/tools_test.go index 610f8dbd..7bbdcfd1 100644 --- a/cmd/server/tools_test.go +++ b/internal/cli/tools_test.go @@ -1,4 +1,4 @@ -package main +package cli import ( "bytes" @@ -20,7 +20,7 @@ func TestRootCommandRegistersToolsStatus(t *testing.T) { cmd := newRootCommand() found, _, err := cmd.Find([]string{"tools", "status"}) - if err != nil || found.CommandPath() != cliName+" tools status" { + if err != nil || found.CommandPath() != rootName+" tools status" { t.Fatalf("found command path = %q err=%v", found.CommandPath(), err) } } diff --git a/cmd/server/tzdata.go b/internal/cli/tzdata.go similarity index 97% rename from cmd/server/tzdata.go rename to internal/cli/tzdata.go index 7f55af33..91ecc614 100644 --- a/cmd/server/tzdata.go +++ b/internal/cli/tzdata.go @@ -1,4 +1,4 @@ -package main +package cli // The container runtime is a bare alpine image that has no /usr/share/zoneinfo // unless the tzdata package is installed, and Go reports a missing zone as a diff --git a/internal/config/serve.go b/internal/config/serve.go new file mode 100644 index 00000000..aef2493c --- /dev/null +++ b/internal/config/serve.go @@ -0,0 +1,62 @@ +package config + +import ( + "errors" + "fmt" + "strings" +) + +// NormalizeMCPPath canonicalizes the Monitor/MCP endpoint path: an empty value +// means /mcp, a missing leading slash is added, and a trailing slash is +// dropped. It is shared by the serve preflight, the management mux and the +// config inspection and doctor commands. +func NormalizeMCPPath(path string) (string, error) { + path = strings.TrimSpace(path) + if path == "" { + return "/mcp", nil + } + if path == "/" { + return "", errors.New("mcp.path must not be /") + } + if !strings.HasPrefix(path, "/") { + path = "/" + path + } + path = strings.TrimRight(path, "/") + if path == "" { + return "", errors.New("mcp.path must not be empty") + } + return path, nil +} + +// EffectiveMCPPath is the mounted MCP path, or "" when MCP is disabled. It +// panics on an invalid non-empty path, which ValidateServe rejects first. +func EffectiveMCPPath(cfg *Config) string { + if cfg == nil || !cfg.MCP.Enabled { + return "" + } + normalized, err := NormalizeMCPPath(cfg.MCP.Path) + if err != nil { + panic(err) + } + return normalized +} + +// ValidateServe reports the configuration problems that stop the proxy from +// starting: an unsupported database driver, and an MCP surface without a +// Monitor port or with an unusable path. +func ValidateServe(cfg *Config) error { + switch driver := cfg.DatabaseDriver(); driver { + case "sqlite", "postgres", "postgresql": + default: + return fmt.Errorf("database.driver %q is not supported yet; use sqlite or postgres", driver) + } + if cfg.MCP.Enabled && cfg.Monitor.Port == "" { + return errors.New("monitor.port is required when mcp.enabled=true") + } + if cfg.MCP.Enabled { + if _, err := NormalizeMCPPath(cfg.MCP.Path); err != nil { + return err + } + } + return nil +} diff --git a/internal/providerprobe/target.go b/internal/providerprobe/target.go new file mode 100644 index 00000000..04300114 --- /dev/null +++ b/internal/providerprobe/target.go @@ -0,0 +1,49 @@ +package providerprobe + +import ( + "strings" + + "github.com/kingfs/Trajecta/internal/config" +) + +// TargetFromConfig converts one configured upstream target into a probe target, +// folding in the first enabled credential. +// +// Both the provider CLI commands and the serve startup probe go through this, so +// a CLI probe and a startup probe always see the same base URL, credential and +// declared API surface. +func TargetFromConfig(target config.UpstreamTargetConfig) ProbeTarget { + upstreamCfg := target.Upstream + apiKey := strings.TrimSpace(upstreamCfg.ApiKey) + headers := CloneHeaders(upstreamCfg.Headers) + for _, credential := range target.EffectiveCredentials() { + if credential.Enabled != nil && !*credential.Enabled { + continue + } + if apiKey == "" { + apiKey = strings.TrimSpace(credential.ApiKey) + } + for key, value := range credential.Headers { + headers[key] = value + } + break + } + return ProbeTarget{ + ProviderID: strings.TrimSpace(target.ID), + BaseURL: strings.TrimSpace(upstreamCfg.BaseURL), + APIKey: apiKey, + Headers: headers, + SpecifiedAPIType: upstreamCfg.APIType, + SpecifiedProtocolFamily: upstreamCfg.ProtocolFamily, + } +} + +// CloneHeaders copies a header map so a probe never mutates the configuration it +// was handed. +func CloneHeaders(in map[string]string) map[string]string { + out := map[string]string{} + for key, value := range in { + out[key] = value + } + return out +}