Skip to content

feat(scan): support JSON as alternative EvaluationLog output format - #816

Open
yvonnedevlinrh wants to merge 1 commit into
complytime:mainfrom
yvonnedevlinrh:opsx/json-evallog
Open

feat(scan): support JSON as alternative EvaluationLog output format#816
yvonnedevlinrh wants to merge 1 commit into
complytime:mainfrom
yvonnedevlinrh:opsx/json-evallog

Conversation

@yvonnedevlinrh

Copy link
Copy Markdown
Contributor

Summary

Adds --log-format yaml|json flag and COMPLYTIME_LOG_FORMAT environment variable to complyctl scan, allowing users to select JSON as an alternative EvaluationLog serialization format. YAML remains the default.

Closes #795

Motivation

Automation pipelines that consume EvaluationLog output often require JSON for downstream tooling (jq, structured logging, API ingestion). Today, an extra YAML-to-JSON conversion step is needed. This change eliminates that step by producing JSON natively.

Changes

Constants (internal/complytime/consts.go)

  • EvalLogFormatEnvVar = "COMPLYTIME_LOG_FORMAT" — env var name constant
  • EvalLogFormatYAML = "yaml" / EvalLogFormatJSON = "json" — named format constants

Output Package (internal/output/evaluator.go)

  • Shadow structs (serializableEvaluationLog, serializableControlEvaluation, serializableAssessmentLog) gain json: struct tags alongside existing yaml: tags, using kebab-case keys matching upstream go-gemara (e.g., json:"assessment-logs", json:"steps-executed")
  • Write(outDir, logFormat string) — updated signature; branches on logFormat: "json" uses json.MarshalIndent with 2-space indentation and .json extension; otherwise YAML path unchanged
  • Added encoding/json and internal/complytime imports

CLI Integration (cmd/complyctl/cli/scan.go)

  • scanOptions.logFormat field
  • --log-format flag (default "yaml") with help text referencing env var
  • Shell completion returning ["yaml", "json"]
  • Env var override: reads COMPLYTIME_LOG_FORMAT when flag not explicitly set (flag takes precedence)
  • validateLogFormat() — case-sensitive validation rejecting anything other than "yaml" or "json"
  • logFormat plumbed through: scanPolicy()runScanAndReport()processScanOutput()writeScanReports()eval.Write()

Tests

  • internal/output/evaluator_test.go: 3 new tests — JSON output with structural round-trip validation, JSON kebab-case field names with negative PascalCase/camelCase assertions, YAML default behavior
  • cmd/complyctl/cli/cli_test.go: 7 new tests — validateLogFormat valid/invalid/constants, env var applied/precedence/invalid/empty
  • Updated 5 existing Write() calls and 2 processScanOutput() calls for new signatures

Documentation

  • CHANGELOG.md — entry under ### Added
  • AGENTS.md — json-evallog added to Recent Changes

OpenSpec Artifacts

  • openspec/changes/json-evallog/ — proposal, design (5 decisions), spec (4 requirements, 11 scenarios), and tasks

Verification

  • go test -race -coverprofile=coverage.out ./... — all pass
  • golangci-lint run ./... — 0 issues
  • go build -mod=vendor ./... — clean
  • Review council: Adversary (APPROVE), Architect (APPROVE), Testing (APPROVE after fixes)

Usage

# JSON output (flag)
complyctl scan --log-format json

# JSON output (env var)
COMPLYTIME_LOG_FORMAT=json complyctl scan

# Flag takes precedence
COMPLYTIME_LOG_FORMAT=json complyctl scan --log-format yaml  # produces YAML

# Default (unchanged)
complyctl scan  # produces YAML as before

@yvonnedevlinrh
yvonnedevlinrh requested a review from a team as a code owner August 14, 2026 15:10
@yvonnedevlinrh yvonnedevlinrh self-assigned this Aug 14, 2026
@yvonnedevlinrh yvonnedevlinrh added the llm_assisted Filed or drafted with LLM assistance label Aug 14, 2026
Add --log-format yaml|json flag and COMPLYTIME_LOG_FORMAT env var to
select the EvaluationLog serialization format. YAML remains the
default. JSON output uses 2-space indentation with kebab-case field
names matching the upstream go-gemara schema.

Implementation:
- EvalLogFormatEnvVar, EvalLogFormatYAML, EvalLogFormatJSON constants
  in internal/complytime/consts.go
- Shadow structs in internal/output/evaluator.go gain json: tags
  alongside existing yaml: tags (kebab-case matching go-gemara)
- Write(outDir, logFormat string) branches on format using
  json.MarshalIndent for the JSON path
- scanOptions.logFormat field, flag registration with shell
  completion, env var override (flag precedence), and
  validateLogFormat() in cmd/complyctl/cli/scan.go
- Tests for JSON output, field names, YAML default, format
  validation, and env var precedence

Includes OpenSpec artifacts in openspec/changes/json-evallog/.

Closes complytime#795

Assisted-by: OpenCode (claude-opus-4-6)
Signed-off-by: Yvonne Devlin <ydevlin@redhat.com>
@yvonnedevlinrh
yvonnedevlinrh requested review from gxmiranda and hbraswelrh and removed request for cdaniels255 and marcusburghardt August 14, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

llm_assisted Filed or drafted with LLM assistance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

scan: support JSON as an alternative EvaluationLog output format

1 participant