From 78c313a729e7cd3301a5e958092e51d5e1fa1307 Mon Sep 17 00:00:00 2001 From: Walker Lockard Date: Tue, 8 Sep 2026 15:09:38 -0700 Subject: [PATCH 1/6] fix: retain sanitized factory execution transcripts --- .github/workflows/guide-draft.yml | 9 +++++ factory/Dockerfile | 2 + factory/README.md | 43 +++++++++++++++++++++ factory/scripts/build-diagnostics.sh | 1 + factory/scripts/build-transcript.sh | 48 ++++++++++++++++++++++++ factory/scripts/container-entrypoint.sh | 9 ++++- factory/scripts/run-kit.sh | 3 +- factory/scripts/transcript.jq | 38 +++++++++++++++++++ factory/tests/test-container.sh | 16 +++++++- factory/tests/test-coordinator.sh | 24 +++++++++--- factory/tests/test-export-boundary.sh | 1 + factory/tests/test-transcript.sh | 50 +++++++++++++++++++++++++ 12 files changed, 236 insertions(+), 8 deletions(-) create mode 100755 factory/scripts/build-transcript.sh create mode 100644 factory/scripts/transcript.jq create mode 100644 factory/tests/test-transcript.sh diff --git a/.github/workflows/guide-draft.yml b/.github/workflows/guide-draft.yml index 1899eef..fcdac11 100644 --- a/.github/workflows/guide-draft.yml +++ b/.github/workflows/guide-draft.yml @@ -122,6 +122,15 @@ jobs: "$RUNNER_TEMP/export" cp "$RUNNER_TEMP/export/run-report.json" "$RUNNER_TEMP/run-report.json" + - name: Upload sanitized execution transcript + if: always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure') + uses: actions/upload-artifact@v4 + with: + name: guide-factory-transcript-${{ github.run_id }}-${{ github.run_attempt }} + path: ${{ runner.temp }}/export/execution-transcript.json + retention-days: 7 + if-no-files-found: ignore + - name: Upload safe factory diagnostics if: always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure') uses: actions/upload-artifact@v4 diff --git a/factory/Dockerfile b/factory/Dockerfile index c9df313..bb4183e 100644 --- a/factory/Dockerfile +++ b/factory/Dockerfile @@ -22,6 +22,8 @@ RUN file=kit-v${KIT_VERSION}-x86_64-unknown-linux-gnu.tar.gz \ COPY --from=lint-builder /out/lint-guide /usr/local/bin/lint-guide COPY factory/scripts/validate-report.sh /usr/local/bin/validate-report COPY factory/scripts/project-kit-events.sh /usr/local/bin/project-kit-events +COPY factory/scripts/build-transcript.sh /usr/local/bin/build-transcript +COPY factory/scripts/transcript.jq /usr/local/bin/transcript.jq COPY factory/scripts/build-diagnostics.sh /usr/local/bin/build-diagnostics COPY factory/scripts/validate-diagnostics.sh /usr/local/bin/validate-diagnostics COPY factory/scripts/container-entrypoint.sh /usr/local/bin/factory-entrypoint diff --git a/factory/README.md b/factory/README.md index c7d44dc..eeb43d6 100644 --- a/factory/README.md +++ b/factory/README.md @@ -22,3 +22,46 @@ payloads. If runtime-event parsing fails, rejected events are discarded; the bundle still retains independently validated failure metadata and the outer Kit invocation status. The workflow log records the parser failure. If bundle validation itself fails closed, the workflow safely skips the missing artifact. + +## Sanitized execution transcript + +For a completed Kit invocation (success or failure), Actions separately uploads +`export/execution-transcript.json` as +`guide-factory-transcript-RUN_ID-RUN_ATTEMPT`, retained for **seven days** under +repository artifact access controls. The upload names only that file, never an +export glob or raw session directory. Stale transcript exports are removed before +startup. Capture runs immediately after Kit exits, independently of runtime-event +projection and before report validation, so parser errors and factory-reported +failure do not discard it. Capture failure is nonfatal and removes the export; +abrupt runner/container loss or cancellation is not guaranteed to produce it. +There is **no automatic retry** and no change to lint behavior. + +This is **not a full conversational transcript**. It is an allowlisted structural +summary of persisted `.kit/sessions/w-*/*.jsonl` records from parent and subagent +sessions: anonymous per-file session references, per-session call references, +recognized role/part kinds and tool names (`unknown` otherwise), result error +flags, and safe integer exit codes (0–255). Shell-shaped results nested inside +compose output are summarized too. Stdout/stderr retain only nonempty presence, +JSON validity and top-level array/object counts—not their values. These fields +can help distinguish a command failure from malformed JSON without revealing its +content; they do not prove a particular linter ran or succeeded. + +No prompts, text, reasoning, arguments, original identifiers, URLs, metadata, +stdout/stderr contents, or unknown fields are copied. Kit's PascalCase parts and +Text/Structured tool outputs are supported; replacement records are summarized +as encountered, not reconstructed into a canonical conversation. Replacements +may repeat events. Compose spill previews/files, tool-output Parts/Files and +unrecognized formats may omit data; no referenced artifact paths are followed. +Malformed lines produce a fixed marker, retaining preceding valid events even +with a truncated tail. Session file names are not exported; references do not +encode parent-child relationships or global chronological order. + +Capture reads at most 64 regular files, 1 MiB per file and 8 MiB total; session +symlinks and symlinked directories below the supplied Kit home are skipped or +rejected. It retains at most 4,096 events, 16 shell-shaped results per tool result, +and a 2 MiB final artifact. Source/event truncation sets `limited`; unsupported +fields and per-result omissions need not do so. Sanitization/size validation +must succeed before the transcript becomes world-readable (`0644`), allowing the +non-root Actions host to read a root-owned container export. Diagnostics likewise +receive `0644` only after their existing strict validator succeeds. Neither +artifact contains raw session logs; do not upload those as a fallback. diff --git a/factory/scripts/build-diagnostics.sh b/factory/scripts/build-diagnostics.sh index f932f63..d7f1a09 100755 --- a/factory/scripts/build-diagnostics.sh +++ b/factory/scripts/build-diagnostics.sh @@ -149,6 +149,7 @@ if ! jq -n \ fi "$DIAGNOSTICS_VALIDATOR" "$temporary" >/dev/null 2>&1 || exit 1 +chmod 0644 "$temporary" 2>/dev/null || exit 1 mv -f -- "$temporary" "$output" 2>/dev/null || exit 1 temporary= succeeded=true diff --git a/factory/scripts/build-transcript.sh b/factory/scripts/build-transcript.sh new file mode 100755 index 0000000..204bf19 --- /dev/null +++ b/factory/scripts/build-transcript.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# Only structural data is exported. Never print source paths or jq errors. +set -euo pipefail +[[ $# == 2 ]] || exit 1 +home=$1 +output=$2 +SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +filter="$SCRIPT_DIR/transcript.jq" +[[ -d $(dirname "$output") && ! -d $output ]] || exit 1 +rm -f -- "$output" +tmp=$(mktemp -d) +trap 'rm -rf -- "$tmp"' EXIT +sessions="$home/.kit/sessions" +# Fixed-depth globs, rejecting every symlink component below the supplied home. +[[ ! -L $home && ! -L $home/.kit && ! -L $sessions ]] || exit 1 +count=0 +bytes=0 +limited=false +: >"$tmp/events" +shopt -s nullglob +for dir in "$sessions"/w-*; do + [[ -d $dir && ! -L $dir ]] || continue + for file in "$dir"/*.jsonl; do + [[ -f $file && ! -L $file ]] || continue + if ((count >= 64 || bytes >= 8388608)); then limited=true; break 2; fi + count=$((count + 1)) + # Bound input even for an unterminated or enormous line. + head -c 1048576 -- "$file" >"$tmp/source" 2>/dev/null || exit 1 + size=$(wc -c <"$tmp/source") + bytes=$((bytes + size)) + if ((size >= 1048576)); then limited=true; fi + jq -Rnc --argjson session "$count" -f "$filter" <"$tmp/source" >>"$tmp/events" 2>/dev/null || exit 1 + done +done +jq -sc --argjson sessions "$count" --argjson limited "$limited" ' + {schema_version:1,kind:"guide_factory_transcript",sessions:$sessions, + limited:($limited or length > 4096),events:.[0:4096]} +' "$tmp/events" >"$tmp/safe" 2>/dev/null +[[ $(wc -c <"$tmp/safe") -le 2097152 ]] || exit 1 +# Defense in depth: no unrecognized key or string may cross the export boundary. +jq -e ' + def keysafe: IN("schema_version","kind","sessions","limited","events","session_ref","call_ref","role","part","tool","is_error","shell_results","exit_code","stdout","stderr","present","json_valid","json_count"); + all(.. | objects | keys[]; keysafe) and + all(.. | strings; IN("guide_factory_transcript","System","Developer","User","Assistant","Tool","Context","Notification","unknown","Text","Media","File","Structured","Reasoning","ToolCall","ToolResult","Custom","malformed","shell","compose","tool","tool_search","subagent","prompt","fork","close","skill","docs","edit","a2a","auth","subagents")) and + all(.. | numbers; floor == . and . >= 0 and . <= 8388608) +' "$tmp/safe" >/dev/null 2>&1 || exit 1 +chmod 0644 "$tmp/safe" +mv -f -- "$tmp/safe" "$output" diff --git a/factory/scripts/container-entrypoint.sh b/factory/scripts/container-entrypoint.sh index f47a798..61e2c13 100755 --- a/factory/scripts/container-entrypoint.sh +++ b/factory/scripts/container-entrypoint.sh @@ -8,11 +8,13 @@ EXPORT_ROOT=${FACTORY_EXPORT_ROOT:-/export} KIT_HOME=${FACTORY_KIT_HOME:-/tmp/kit-home} REPORT_VALIDATOR=${FACTORY_REPORT_VALIDATOR:-/usr/local/bin/validate-report} EVENT_PROJECTOR=${FACTORY_EVENT_PROJECTOR:-/usr/local/bin/project-kit-events} +TRANSCRIPT_BUILDER=${FACTORY_TRANSCRIPT_BUILDER:-/usr/local/bin/build-transcript} DIAGNOSTICS_BUILDER=${FACTORY_DIAGNOSTICS_BUILDER:-/usr/local/bin/build-diagnostics} mkdir -p "$EXPORT_ROOT" rm -rf "$EXPORT_ROOT/guide" "$EXPORT_ROOT/run-report.json" \ - "$EXPORT_ROOT/kit-error-summary.json" "$EXPORT_ROOT/factory-diagnostics.json" + "$EXPORT_ROOT/kit-error-summary.json" "$EXPORT_ROOT/factory-diagnostics.json" \ + "$EXPORT_ROOT/execution-transcript.json" test -r "$INPUT_ROOT/issue.json" test -r "$INPUT_ROOT/catalog.json" test -r "$REPO_ROOT/factory/coordinator.md" @@ -45,6 +47,11 @@ if KIT_RUNTIME_EVENTS=1 "$KIT_BIN" prompt \ else kit_status=$? fi +# Persist safe session structure before consuming any projector/report results. +if ! "$TRANSCRIPT_BUILDER" "$KIT_HOME" "$EXPORT_ROOT/execution-transcript.json" 2>/dev/null; then + rm -f -- "$EXPORT_ROOT/execution-transcript.json" + printf '%s\n' 'factory: sanitized transcript unavailable' >&2 +fi if wait "$projector_pid"; then projector_status=0 else diff --git a/factory/scripts/run-kit.sh b/factory/scripts/run-kit.sh index 12976cb..af68e75 100755 --- a/factory/scripts/run-kit.sh +++ b/factory/scripts/run-kit.sh @@ -12,7 +12,8 @@ export_dir=$3 mkdir -p "$export_dir" export_dir="$(realpath "$export_dir")" rm -rf "$export_dir/guide" "$export_dir/run-report.json" \ - "$export_dir/kit-error-summary.json" "$export_dir/factory-diagnostics.json" + "$export_dir/kit-error-summary.json" "$export_dir/factory-diagnostics.json" \ + "$export_dir/execution-transcript.json" [[ -r "$issue_json" ]] || { printf 'issue JSON is not readable: %s\n' "$issue_json" >&2; exit 2; } [[ -r "$catalog_json" ]] || { printf 'catalog JSON is not readable: %s\n' "$catalog_json" >&2; exit 2; } : "${OPENROUTER_API_KEY:?OPENROUTER_API_KEY is required}" diff --git a/factory/scripts/transcript.jq b/factory/scripts/transcript.jq new file mode 100644 index 0000000..73ad82d --- /dev/null +++ b/factory/scripts/transcript.jq @@ -0,0 +1,38 @@ +# Matches Kit session Record / agentkit-core externally tagged Part and ToolOutput. +def toolname: if IN("shell","compose","tool","tool_search","subagent","prompt","fork","close","skill","docs","edit","a2a","auth","subagents") then . else "unknown" end; +def role: if IN("System","Developer","User","Assistant","Tool","Context","Notification") then . else "unknown" end; +def stream: + if type != "string" then {present:false,json_valid:false,json_count:null} + else {present:(length > 0)} + + (try (fromjson | {json_valid:true,json_count:(if type == "array" or type == "object" then length else null end)}) + catch {json_valid:false,json_count:null}) end; +def shells: + (if .Structured != null then .Structured elif (.Text | type) == "string" then (try (.Text | fromjson) catch null) else null end) + | [.. | objects | select(has("exit_code") and has("stdout") and has("stderr")) | + {exit_code:(.exit_code | if type == "number" then if floor == . and . >= 0 and . <= 255 then . else null end else null end), + stdout:(.stdout | stream),stderr:(.stderr | stream)}][0:16]; +def parts: + if type != "object" then {role:"unknown",part:"malformed"} + elif (.schema_version | IN(1,2,3)) and + ((.item | type) == "object" or (.replacement | type) == "array") then + (if .item != null then [.item] else .replacement end)[] | + . as $item | if (.parts | type) != "array" then {role:"unknown",part:"malformed"} + else .parts[] | . as $part | + {role:($item.kind | role)} + + (if type != "object" or length != 1 then {part:"unknown"} + else keys[0] as $k | + {part:($k | if IN("Text","Media","File","Structured","Reasoning","ToolCall","ToolResult","Custom") then . else "unknown" end)} + + (if $k == "ToolCall" then {id:$part.ToolCall.id,tool:($part.ToolCall.name | toolname)} + elif $k == "ToolResult" then {id:$part.ToolResult.call_id,is_error:($part.ToolResult.is_error | if type == "boolean" then . else null end),shell_results:($part.ToolResult.output | shells)} + else {} end) end) + end + else {role:"unknown",part:"malformed"} end; +reduce (inputs | (try fromjson catch null) | (try parts catch {role:"unknown",part:"malformed"})) as $event + ({calls:{},next:0,events:[]}; + if (.events | length) >= 4097 then . else + (if ($event.id | type) == "string" then + if .calls[$event.id] == null then .next += 1 | .calls[$event.id] = .next else . end + else . end) | + .events += [($event | del(.id)) + {session_ref:$session} + + (if ($event.id | type) == "string" then {call_ref:.calls[$event.id]} else {} end)] end) +| .events[] diff --git a/factory/tests/test-container.sh b/factory/tests/test-container.sh index c7d6886..8750be3 100755 --- a/factory/tests/test-container.sh +++ b/factory/tests/test-container.sh @@ -6,6 +6,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" source "$ROOT/factory/tests/test-helper.sh" TMP="$(mktemp -d)" export TMP +export FACTORY_TRANSCRIPT_BUILDER="$ROOT/factory/scripts/build-transcript.sh" export FACTORY_EVENT_PROJECTOR="$ROOT/factory/scripts/project-kit-events.sh" export FACTORY_DIAGNOSTICS_BUILDER="$ROOT/factory/scripts/build-diagnostics.sh" trap 'rm -rf "$TMP"; exit 130' INT TERM @@ -122,6 +123,7 @@ test_startup_failures_remove_stale_diagnostics() { host_export="$TMP/stale-host-export" mkdir -p "$host_export" printf '%s\n' 'SECRET_STALE_HOST_DIAGNOSTIC' >"$host_export/factory-diagnostics.json" + touch "$host_export/execution-transcript.json" host_status=0 OPENROUTER_API_KEY=or-test "$ROOT/factory/scripts/run-kit.sh" \ @@ -130,6 +132,7 @@ test_startup_failures_remove_stale_diagnostics() { assert_eq 2 "$host_status" test ! -e "$host_export/factory-diagnostics.json" \ || fail 'run-kit retained stale diagnostics after startup failure' + test ! -e "$host_export/execution-transcript.json" || fail 'stale host transcript' repo="$TMP/stale-container-repo" input="$TMP/stale-container-input" @@ -138,6 +141,7 @@ test_startup_failures_remove_stale_diagnostics() { printf 'assignment\n' >"$repo/factory/coordinator.md" printf '%s\n' 'SECRET_STALE_CONTAINER_DIAGNOSTIC' \ >"$container_export/factory-diagnostics.json" + touch "$container_export/execution-transcript.json" container_status=0 FACTORY_REPO_ROOT="$repo" FACTORY_INPUT_ROOT="$input" \ @@ -147,6 +151,7 @@ test_startup_failures_remove_stale_diagnostics() { "$ROOT/factory/scripts/container-entrypoint.sh" >/dev/null 2>&1 \ || container_status=$? assert_eq 1 "$container_status" + test ! -e "$container_export/execution-transcript.json" || fail 'stale container transcript' test ! -e "$container_export/factory-diagnostics.json" \ || fail 'entrypoint retained stale diagnostics after startup failure' } @@ -436,6 +441,8 @@ test_entrypoint_exports_only_selected_guide_with_mocked_kit() { #!/usr/bin/env bash set -euo pipefail [[ "$1" == prompt ]] +mkdir -p "$HOME/.kit/sessions/w-success" +printf '%s\n' '{"schema_version":3,"item":{"kind":"Assistant","parts":[{"Text":{"text":"SECRET_SUCCESS"}}]}}' >"$HOME/.kit/sessions/w-success/test.jsonl" mkdir -p "$FACTORY_WORKSPACE_ROOT/guides/acme" printf 'guide\n' >"$FACTORY_WORKSPACE_ROOT/guides/acme/research.md" printf 'ignore\n' >"$FACTORY_WORKSPACE_ROOT/not-exported.txt" @@ -462,7 +469,9 @@ MOCK test -f "$export_root/guide/meta.yaml" test -f "$export_root/run-report.json" test ! -e "$export_root/not-exported.txt" - assert_eq "3" "$(find "$export_root" -type f | wc -l | tr -d ' ')" + jq -e '.sessions == 1 and .events[0].part == "Text"' "$export_root/execution-transcript.json" >/dev/null || fail 'successful Kit lost transcript' + ! grep -q SECRET "$export_root/execution-transcript.json" || fail 'successful transcript leaked text' + assert_eq "4" "$(find "$export_root" -type f | wc -l | tr -d ' ')" } test_entrypoint_rejects_invalid_report() { @@ -566,6 +575,8 @@ test_entrypoint_handles_invalid_projection_and_missing_report() { workspace="$TMP/runtime-workspace-$kind"; export_root="$TMP/runtime-export-$kind" cat >"$fake_kit" <<'MOCK' #!/usr/bin/env bash +mkdir -p "$HOME/.kit/sessions/w-test" +printf '%s\n' '{"schema_version":3,"session_id":"SECRET_SESSION","generation":1,"item":{"kind":"Assistant","parts":[{"Text":{"text":"SECRET_TEXT"}}]}}' >"$HOME/.kit/sessions/w-test/session.jsonl" marker=$(printf '\001kit-runtime\001') printf '%s%s\n' "$marker" "${RUNTIME_BAD_LINE}" >&2 printf '%s\n' 'ordinary diagnostic after bad event' >&2 @@ -587,6 +598,8 @@ MOCK "$ROOT/factory/scripts/container-entrypoint.sh" >/dev/null 2>"$TMP/runtime-$kind.err"; then fail "entrypoint accepted $kind runtime event" fi + jq -e '.sessions == 1 and .events[0].part == "Text"' "$export_root/execution-transcript.json" >/dev/null || fail 'parser failure lost transcript' + ! grep -q SECRET "$export_root/execution-transcript.json" || fail 'container transcript leaked secret' "$ROOT/factory/scripts/validate-diagnostics.sh" "$export_root/factory-diagnostics.json" >/dev/null \ || fail "entrypoint lost safe fallback for $kind runtime event" jq -e '.stage == "kit_prompt" and (.events | length) == 2 and all(.events[]; .tool == "kit_prompt")' \ @@ -623,6 +636,7 @@ MOCK "$ROOT/factory/scripts/validate-diagnostics.sh" "$export_root/factory-diagnostics.json" >/dev/null jq -e '.stage == "factory_outcome" and .classification == "factory_reported_failure" and .report.outcome == "failed" and (.events | length) == 2' \ "$export_root/factory-diagnostics.json" >/dev/null || fail 'lost failed-report fallback' + test -r "$export_root/execution-transcript.json" || fail 'factory failure lost transcript' test -f "$export_root/run-report.json" || fail 'lost failed report' test ! -e "$export_root/guide" || fail 'exported failed guide' if grep -q 'SECRET_' "$export_root/factory-diagnostics.json"; then diff --git a/factory/tests/test-coordinator.sh b/factory/tests/test-coordinator.sh index f430135..a934e83 100755 --- a/factory/tests/test-coordinator.sh +++ b/factory/tests/test-coordinator.sh @@ -194,10 +194,10 @@ assert_step_contains() { } upload_step_block() { - local workflow=$1 - awk ' - $0 == " - name: Upload safe factory diagnostics" { found=1 } - found && /^ - name: / && $0 != " - name: Upload safe factory diagnostics" { exit } + local workflow=$1 name=${2:-Upload safe factory diagnostics} + awk -v target=" - name: $name" ' + $0 == target { found=1 } + found && /^ - name: / && $0 != target { exit } found { print } ' "$workflow" } @@ -254,8 +254,22 @@ assert_upload_contract() { fail 'upload step contains a multiline or nested field value' return 1 fi + block="$(upload_step_block "$workflow" 'Upload sanitized execution transcript')" + [[ -n "$block" ]] || { fail 'missing sanitized transcript upload'; return 1; } + assert_upload_field_equals "$block" ' ' if "always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure')" || return 1 + assert_upload_field_equals "$block" ' ' uses 'actions/upload-artifact@v4' || return 1 + # shellcheck disable=SC2016 + assert_upload_field_equals "$block" ' ' name 'guide-factory-transcript-${{ github.run_id }}-${{ github.run_attempt }}' || return 1 + # shellcheck disable=SC2016 + assert_upload_field_equals "$block" ' ' path '${{ runner.temp }}/export/execution-transcript.json' || return 1 + assert_upload_field_equals "$block" ' ' retention-days '7' || return 1 + assert_upload_field_equals "$block" ' ' if-no-files-found ignore || return 1 + if grep -Eq '^ [^[:space:]]' <<<"$block"; then + fail 'transcript upload contains a multiline or nested field value' + return 1 + fi upload_count="$(count_upload_artifact_actions "$workflow")" - assert_eq '1' "$upload_count" || return 1 + assert_eq '2' "$upload_count" || return 1 } steps_with() { diff --git a/factory/tests/test-export-boundary.sh b/factory/tests/test-export-boundary.sh index fdb4ae4..0d532bb 100755 --- a/factory/tests/test-export-boundary.sh +++ b/factory/tests/test-export-boundary.sh @@ -6,6 +6,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" source "$ROOT/factory/tests/test-helper.sh" TMP="$(mktemp -d)" export TMP +export FACTORY_TRANSCRIPT_BUILDER="$ROOT/factory/scripts/build-transcript.sh" trap 'rm -rf "$TMP"; exit 130' INT TERM test_launcher_canonicalizes_paths_and_mounts_gitless_snapshot() { diff --git a/factory/tests/test-transcript.sh b/factory/tests/test-transcript.sh new file mode 100644 index 0000000..90af2fe --- /dev/null +++ b/factory/tests/test-transcript.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +set -euo pipefail +ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd) +# shellcheck disable=SC1091 +source "$ROOT/factory/tests/test-helper.sh" +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +mkdir -p "$TMP/home/.kit/sessions/w-parent" "$TMP/home/.kit/sessions/w-child" +cat >"$TMP/home/.kit/sessions/w-parent/parent.jsonl" <<'JSON' +{"schema_version":3,"session_id":"SECRET_SESSION","generation":1,"item":{"kind":"Assistant","parts":[{"Text":{"text":"SECRET_PROMPT"}},{"Reasoning":{"text":"SECRET_REASON"}},{"ToolCall":{"id":"SECRET_CALL","name":"compose","input":{"script":"SECRET_ARGUMENT"},"metadata":{"secret":"SECRET_META"}}}]}} +{"schema_version":3,"session_id":"SECRET_SESSION","generation":1,"item":{"kind":"Tool","parts":[{"ToolResult":{"call_id":"SECRET_CALL","is_error":false,"output":{"Structured":{"nested":{"exit_code":1,"stdout":"{\"errors\":[\"SECRET_LINT\"]}","stderr":"SECRET_STDERR","secret":"SECRET_UNKNOWN"}}}}}]}} +{"schema_version":3,"session_id":"SECRET_SESSION","generation":2,"replacement":[{"kind":"User","parts":[{"Custom":{"secret":"SECRET_CUSTOM"}},{"ToolCall":{"id":"SECRET_OTHER","name":"SECRET_TOOL","input":{}}}]}]} +{"truncated":"SECRET_TAIL +JSON +cp "$TMP/home/.kit/sessions/w-parent/parent.jsonl" "$TMP/home/.kit/sessions/w-child/child.jsonl" +ln -s "$TMP/home/.kit/sessions/w-parent" "$TMP/home/.kit/sessions/w-link" +ln -s "$TMP/home/.kit/sessions/w-parent/parent.jsonl" "$TMP/home/.kit/sessions/w-child/link.jsonl" +BUILD="$ROOT/factory/scripts/build-transcript.sh" +bash "$BUILD" "$TMP/home" "$TMP/transcript.json" +! grep -q SECRET "$TMP/transcript.json" || fail 'transcript leaked a canary' +jq -e '.sessions == 2 and ([.events[] | select(.part == "malformed")] | length) == 2 and any(.events[]; .tool == "unknown") and any(.events[]; .shell_results[0].exit_code == 1 and .shell_results[0].stdout.json_valid == true and .shell_results[0].stdout.json_count == 1) and ([.events[] | select(.part == "ToolResult") | .call_ref] == [1,1])' "$TMP/transcript.json" >/dev/null +[[ -n $(find "$TMP/transcript.json" -perm 0644 -print) ]] || fail 'transcript not host readable' +bash "$ROOT/factory/scripts/build-diagnostics.sh" docker_build 1 - - - "$TMP/diagnostics.json" +[[ -n $(find "$TMP/diagnostics.json" -perm 0644 -print) ]] || fail 'validated diagnostics not host readable' +printf 'PASS sanitized transcript\n' + +# Text-wrapped JSON is a real ToolOutput variant; never copy its payload. +mkdir -p "$TMP/text/.kit/sessions/w-test" +jq -nc '{schema_version:3,item:{kind:"Tool",parts:[{ToolResult:{call_id:"SECRET",is_error:true,output:{Text:({exit_code:2,stdout:"[1,2]",stderr:"SECRET"}|tojson)}}}]}}' >"$TMP/text/.kit/sessions/w-test/test.jsonl" +printf '%s\n' '{"schema_version":3,"item":{"parts":[{"ToolResult":{"output":7}}]}}' >>"$TMP/text/.kit/sessions/w-test/test.jsonl" +bash "$BUILD" "$TMP/text" "$TMP/text.json" +jq -e '.events[0].shell_results[0].stdout.json_count == 2 and .events[1].part == "malformed"' "$TMP/text.json" >/dev/null +! grep -q SECRET "$TMP/text.json" || fail 'text output leaked a canary' +# Source prefix truncation is flagged, not copied or treated as complete. +head -c 1100000 /dev/zero | tr '\0' x >"$TMP/text/.kit/sessions/w-test/large.jsonl" +bash "$BUILD" "$TMP/text" "$TMP/limited.json" +jq -e '.limited == true and any(.events[]; .part == "malformed")' "$TMP/limited.json" >/dev/null +# A symlinked session root cannot be traversed, and stale output is removed. +mv "$TMP/text/.kit/sessions" "$TMP/real-sessions" +ln -s "$TMP/real-sessions" "$TMP/text/.kit/sessions" +if bash "$BUILD" "$TMP/text" "$TMP/text.json"; then fail 'followed session symlink'; fi +test ! -e "$TMP/text.json" + +# Workflow upload is explicit, independent, and short-lived. +workflow="$ROOT/.github/workflows/guide-draft.yml" +sed -n '/name: Upload sanitized execution transcript/,/name: Upload safe factory diagnostics/p' "$workflow" >"$TMP/upload" +grep -Fq "if: always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure')" "$TMP/upload" +# shellcheck disable=SC2016 +grep -Fq 'path: ${{ runner.temp }}/export/execution-transcript.json' "$TMP/upload" +grep -Fq 'retention-days: 7' "$TMP/upload" From 1789e2eb2a9908ef2bc4cad723adb9fd9d73e097 Mon Sep 17 00:00:00 2001 From: Walker Lockard Date: Thu, 17 Sep 2026 09:45:57 -0700 Subject: [PATCH 2/6] docs: document Salesforce app setup methods and MCP endpoints --- guides/salesforce/external.md | 134 +++++++++++++++++++--- guides/salesforce/meta.yaml | 106 +++++++++++++++++- guides/salesforce/research.md | 196 ++++++++++++++++++++++++++++----- guides/salesforce/speakeasy.md | 10 +- 4 files changed, 404 insertions(+), 42 deletions(-) diff --git a/guides/salesforce/external.md b/guides/salesforce/external.md index 9eef89f..aaad57f 100644 --- a/guides/salesforce/external.md +++ b/guides/salesforce/external.md @@ -4,9 +4,28 @@ setup_version: 1 # Connect Salesforce to the Speakeasy AI Control Plane -Use Salesforce System Administrator credentials for an API-enabled production or sandbox org where Hosted MCP Servers are available. Salesforce documents availability for Enterprise Edition and above. You need authority to create an **External Client App** and enable Hosted MCP Servers. +Use Salesforce System Administrator credentials for an API-enabled production or sandbox org where Hosted MCP Servers are available. Salesforce documents availability for Enterprise Edition and above. You need authority to install the Speakeasy application or create an **External Client App**, and enable Hosted MCP Servers. -Sign in to the Salesforce org that will expose its records. Create the credentials in that same org. This guide does not cover scratch orgs. +Sign in to the Salesforce org you want to connect. Install or create the app in that same org. This guide does not cover scratch orgs. For a lower-edition org, confirm Hosted MCP availability in [Salesforce Setup](#open-salesforce-setup) before beginning either method. + +Choose one method: + +- **Method 1 — Speakeasy application:** [install the application](#install-speakeasy-application), then [contact Speakeasy support](#contact-speakeasy-support) to finish OAuth. Do not follow the own-app credential steps. +- **Method 2 — Your own External Client App:** begin at [Open Salesforce Setup](#open-salesforce-setup), then create the app and copy its **Consumer Key**. + +Both methods require an approved endpoint and administrator activation of the [selected MCP server](#enable-sobject-server). For method 1, coordinate activation and OAuth sequencing with Speakeasy support. The endpoint list documents Salesforce URLs, not live-tested Speakeasy compatibility. + +### Install Speakeasy's Salesforce application {#install-speakeasy-application} + +**Method 1 only.** Install Speakeasy's Salesforce application into the intended org using [login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2](https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2). + + + +### Contact Speakeasy support to finish OAuth {#contact-speakeasy-support} + +After installation, **contact Speakeasy support to finish OAuth setup**. Installation alone does not complete OAuth setup. Coordinate the selected endpoint, org type, and server activation with support; do not create another app or apply the own-app credential instructions below to the installed package. + + ### Open Salesforce Setup {#open-salesforce-setup} @@ -23,6 +42,8 @@ For a lower-edition org: ### Start an External Client App {#start-external-client-app} +**Method 2 only.** Follow this step through [Copy the Consumer Key](#copy-consumer-key) only if you are creating your own app. + 1. In **Quick Find**, enter `external client`. 2. Select **External Client App Manager**. 3. Select **New External Client App**. @@ -70,34 +91,119 @@ The app can take up to 30 minutes to become operational. If attaching it immedia 3. Complete the Salesforce verification prompt if it appears. 4. Copy **Consumer Key**. You will use it as the Speakeasy **Client ID**. +Do not copy the **Consumer Secret** for this path. + -### Enable the selected SObject server {#enable-sobject-server} +### Enable the selected MCP server {#enable-sobject-server} -Choose the least-privileged server that meets the team's needs: +**Both methods.** Choose the least-privileged server that meets the team's needs. For method 1, coordinate this step with Speakeasy support: - `sobject-reads` allows discovery, query, search, and relationship traversal without changing records. - `sobject-mutations` allows reading, creating, and updating records without deleting them. - `sobject-deletes` allows identifying and deleting records without creating or updating them. - `sobject-all` allows creating, reading, updating, deleting, querying, and searching records. -If the ticket does not specify the team's approved read, write, or delete requirements, obtain the server choice from the application or cloud security owner. +- Data 360 (`data360`) can change customer-data configuration as well as query data. It requires a Data 360 license and API v66.0 or later, with **Manage Data 360** for configuration or **View Data 360** for read-only operations. +- Headless 360 (Beta), API ID `platform/headless-360`, provides broad Setup and platform operations, not read-only record access. It is available starting July 2026 under Beta Services Terms and requires API v67.0 or later, an External Client App with `mcp_api`, and an OAuth client. +- Tableau Next, API ID `analytics/tableau-next`, provides semantic-model and analytics access. Confirm that the target org has the required Tableau Next capabilities before selecting it. + +If the ticket does not specify the team's approved records, data-platform, admin, or analytics requirements, obtain the server choice from the application or cloud security owner. 1. Return to **Setup**. 2. In **Quick Find**, enter `MCP Servers`. 3. Select **MCP Servers** under **API Catalog**. 4. Find the server whose API ID matches the approved choice. -5. Use the available control to enable that server. -6. Record its URL from the table below, using the production or sandbox form that matches the org. +5. Use the available control to enable that server. For Headless 360, find `headless-360` and select **Activate**. +6. Record its URL from the list below, using the production or sandbox form that matches the org. 7. Wait up to two minutes for the server to become active. -| Server | Production URL | Sandbox URL | -| --- | --- | --- | -| `sobject-reads` | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads` | -| `sobject-mutations` | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations` | -| `sobject-deletes` | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes` | -| `sobject-all` | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-all` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all` | +Data 360's sandbox URL places `/sandbox` after `/data`, unlike the platform and analytics endpoints. Copy the exact URL for your server and org type. + +**SObject Reads — production** + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads +``` + +**SObject Reads — sandbox** + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads +``` + +**SObject Mutations — production** + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations +``` + +**SObject Mutations — sandbox** + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations +``` + +**SObject Deletes — production** + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes +``` + +**SObject Deletes — sandbox** + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes +``` + +**SObject All — production** + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-all +``` + +**SObject All — sandbox** + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all +``` + +**Data 360 — production** + +``` +https://api.salesforce.com/platform/mcp/v1/data/data360 +``` + +**Data 360 — sandbox** + +``` +https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360 +``` + +**Headless 360 (Beta) — production** + +``` +https://api.salesforce.com/platform/mcp/v1/platform/headless-360 +``` + +**Headless 360 (Beta) — sandbox** + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360 +``` + +**Tableau Next — production** + +``` +https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next +``` + +**Tableau Next — sandbox** + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next +``` If the connection fails with valid credentials, confirm that the selected server is enabled, the URL matches the server and org type, and the org has API access. - + diff --git a/guides/salesforce/meta.yaml b/guides/salesforce/meta.yaml index de546a6..83a78ae 100644 --- a/guides/salesforce/meta.yaml +++ b/guides/salesforce/meta.yaml @@ -2,11 +2,15 @@ schema_version: 1 slug: salesforce title: Salesforce -summary: Connect to Salesforce records through hosted SObject MCP servers with selectable read, write, and delete boundaries. +summary: Connect to Salesforce hosted MCP servers using the Speakeasy application with a required support handoff or your own External Client App, selecting the appropriate endpoint and org type. aliases: - com.pulsemcp.mirror/gram-salesforce credential_setup: options: + - id: speakeasy-application + kind: oauth + client_registration: manual + upstream_setup: provider-steps - id: oauth-client kind: oauth client_registration: manual @@ -18,7 +22,7 @@ credential_setup: - external.md#copy-consumer-key requirements: - id: salesforce-admin - description: Salesforce System Administrator access to an API-enabled org where Hosted MCP Servers are available, with authority to create an External Client App and enable servers + description: Salesforce System Administrator access to an API-enabled org where Hosted MCP Servers are available, with authority to install the Speakeasy application or create an External Client App, and enable the selected servers documentation: external: external.md speakeasy: speakeasy.md @@ -29,41 +33,85 @@ remotes: transport: streamable-http authentication: - oauth-client + - speakeasy-application - id: sobject-reads-sandbox url: https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads transport: streamable-http authentication: - oauth-client + - speakeasy-application - id: sobject-mutations-production url: https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations transport: streamable-http authentication: - oauth-client + - speakeasy-application - id: sobject-mutations-sandbox url: https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations transport: streamable-http authentication: - oauth-client + - speakeasy-application - id: sobject-deletes-production url: https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes transport: streamable-http authentication: - oauth-client + - speakeasy-application - id: sobject-deletes-sandbox url: https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes transport: streamable-http authentication: - oauth-client + - speakeasy-application - id: sobject-all-production url: https://api.salesforce.com/platform/mcp/v1/platform/sobject-all transport: streamable-http authentication: - oauth-client + - speakeasy-application - id: sobject-all-sandbox url: https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all transport: streamable-http authentication: - oauth-client + - speakeasy-application + - id: data360-production + url: https://api.salesforce.com/platform/mcp/v1/data/data360 + transport: streamable-http + authentication: + - oauth-client + - speakeasy-application + - id: data360-sandbox + url: https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360 + transport: streamable-http + authentication: + - oauth-client + - speakeasy-application + - id: headless-360-production + url: https://api.salesforce.com/platform/mcp/v1/platform/headless-360 + transport: streamable-http + authentication: + - oauth-client + - speakeasy-application + - id: headless-360-sandbox + url: https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360 + transport: streamable-http + authentication: + - oauth-client + - speakeasy-application + - id: tableau-next-production + url: https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next + transport: streamable-http + authentication: + - oauth-client + - speakeasy-application + - id: tableau-next-sandbox + url: https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next + transport: streamable-http + authentication: + - oauth-client + - speakeasy-application provenance: - source: provider-documentation locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/hosted-mcp-servers-overview.html @@ -175,3 +223,57 @@ provenance: name: Speakeasy setup canonical section classification: official observed_at: "2026-08-06T23:23:14Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/setup-overview.html + name: Set Up Your Org + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/create-external-client-app.html + name: Create an External Client App + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/hosted-mcp-servers-overview.html + name: Salesforce Hosted MCP Servers + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/client-connection-overview.html + name: Connecting an MCP Client + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/servers-reference.html + name: Standard MCP Servers Reference + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/activate-mcp-servers.html + name: Activate MCP Servers + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/references/reference/data360-mcp.html + name: Data 360 MCP Server + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/references/reference/headless-360-mcp.html + name: Headless 360 MCP Server (Beta) + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/references/reference/tableau-next.html + name: Tableau Next + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: provider-documentation + locator: https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/postman.html + name: Configure Postman + classification: official + observed_at: "2026-09-17T16:15:46Z" + - source: operator-instruction + locator: https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2 + name: Speakeasy Salesforce application installation and mandatory support OAuth handoff + observed_at: "2026-09-17T16:15:46Z" diff --git a/guides/salesforce/research.md b/guides/salesforce/research.md index 031815a..b1c36ab 100644 --- a/guides/salesforce/research.md +++ b/guides/salesforce/research.md @@ -1,18 +1,21 @@ --- research_version: 1 slug: salesforce -researched_at: 2026-08-06T23:23:14Z +researched_at: 2026-09-17T16:15:46Z --- # Salesforce — Research Dossier ## Server facts -- **Guide scope:** Salesforce publishes several Hosted MCP Servers. This Guide - covers the four standard SObject servers because they share one documented - browser setup and have fixed, fully documented URLs. Product-specific and - custom servers are not interchangeable with these URLs and can carry product - licenses or org-specific names. +- **Guide scope:** two alternative OAuth setup methods: install Speakeasy's + Salesforce application and contact Speakeasy support, or create your own + External Client App. Cover all four standard SObject endpoints plus the + currently cataloged Data 360, Headless 360 (Beta), and Tableau Next endpoints. + Select the endpoint and org type explicitly; SObject Reads is not a universal + URL. Custom servers and legacy product endpoints are outside this refresh. + Sources: operator instruction and current `servers-reference.html`, observed + on the refresh date recorded below. - **Production MCP Servers:** - SObject Reads: `https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads` @@ -31,7 +34,7 @@ researched_at: 2026-08-06T23:23:14Z - **Transport:** `streamable-http`. Salesforce's Postman setup explicitly selects **HTTP**, not STDIO, for these remote URLs. The endpoint implements protected-resource discovery and returns `401 Unauthorized` without OAuth. -- **Authentication:** per-user OAuth 2.0 Authorization Code with PKCE. An +- **Authentication:** per-user OAuth 2.0 Authorization Code with PKCE. For the self-created method, an administrator manually registers an **External Client App** and the MCP client uses its **Consumer Key** as the client ID. Salesforce says Connected Apps are not supported for Hosted MCP authentication. The documented public @@ -77,13 +80,44 @@ researched_at: 2026-08-06T23:23:14Z installing that package in the scratch org, but the Hosted MCP documentation does not provide an executable browser workflow for those packaging steps. +## Current endpoint catalog and setup gates + +Sources observed on the refresh date: `servers-reference.html`, the three +product reference pages listed under Refresh provenance, and `postman.html`. +The four SObject URLs above are retained from the earlier endpoint-specific +research; the current catalog reconfirms all four server identities and access +boundaries, and Postman reconfirms the platform production/sandbox form. + +| Server | Production URL | Sandbox URL | Setup-relevant boundary / gate | +| --- | --- | --- | --- | +| Data 360 | `https://api.salesforce.com/platform/mcp/v1/data/data360` | `https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360` | Data 360 license; API v66.0+; `Manage Data 360` for configuration, `View Data 360` for read-only operations. Can change customer-data configuration, not merely query SQL. Calls count against underlying Connect API limits and Flex Credit usage. | +| Headless 360 (Beta) | `https://api.salesforce.com/platform/mcp/v1/platform/headless-360` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360` | Beta, available starting July 2026 under Beta Services Terms; API v67.0+; ECA with `mcp_api` and OAuth client. Broad Setup/platform operations, not a read-only records endpoint. | +| Tableau Next | `https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next` | `https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next` | Semantic-model and analytics access. The reference does not specify a license SKU or named permission set; do not invent one or promise access without the org's Tableau Next capabilities. | + +Data 360's documented sandbox position is **after `/data`**, unlike the +platform and analytics endpoints. Preserve each literal URL; do not normalize +it using the generic Postman template. Product references also label these +sandbox URLs for scratch orgs, but this guide retains its production/sandbox +scope because scratch-org app creation needs a separate packaging workflow. +Data 360's page says it replaces a previous server, now called Data 360 Legacy; +that legacy endpoint is not researched or offered here. The current general +catalog describes Data 360 narrowly as querying data; prefer the dedicated +reference's broader capability and permission requirements. + +All standard servers enforce user-level field security, object permissions, +and sharing. Choose only approved capabilities. System Administrator or +equivalent permission is needed to create an ECA (`setup-overview.html`); +server activation requires an administrator (`activate-mcp-servers.html`). +These are not grants of data access to every connecting user. + ## Credential flow -Who acts: a Salesforce System Administrator. Salesforce's Hosted MCP docs +Who acts: a Salesforce System Administrator (or equivalent permissions for +creating the ECA, per the current setup overview). Salesforce's Hosted MCP docs require an administrator to enable servers, and Salesforce's External Client App documentation states that a Salesforce administrator creates the app. -What gets created: one local **External Client App** with OAuth enabled. The +For method 2, what gets created: one local **External Client App** with OAuth enabled. The candidate Speakeasy configuration uses the generated **Consumer Key** as **Client ID** and leaves **Client Secret (optional)** empty. Salesforce documents that Consumer Key-only PKCE configuration for Postman and Cursor, @@ -108,8 +142,47 @@ authorization, the user should log out of other Salesforce orgs, sign in to the target org in the default browser, and keep that browser open. This is a connection-time user action, not an administrator credential-creation step. +### Setup-method decision + +**Method 1 — Speakeasy application:** follow {#install-speakeasy-application} +then {#contact-speakeasy-support}. This method ends in a mandatory support +handoff; do not tell the admin to create another app, copy an unknown client +ID/secret, or apply the self-created app's security settings to this package. + +**Method 2 — Your own External Client App:** follow the existing Setup, +app-creation, OAuth, and Consumer Key steps below. Both methods require an +approved endpoint and administrator activation of the selected server. +The package method's exact activation/OAuth sequencing is not supplied; +coordinate it with support rather than inventing a completed connection. + +Operator provenance for method 1: explicit task instruction, observed on the +refresh date; installer locator below. This is not independently verified +package contents, installation UI, org compatibility, or OAuth behavior. + ## Console walkthrough +### Install Speakeasy's Salesforce application {#install-speakeasy-application} + +- Method 1 only: as an administrator, install Speakeasy's Salesforce application + into the intended org using the operator-supplied URL: + `https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2`. +- The operator supplied this exact URL; no sandbox installer alternative, + installation audience, approval sequence, package password, or package + contents were supplied or independently verified. Do not invent them. +- Values copied: none specified. +- Screenshot exception: no verified installation screen is available; use the + exact installation link rather than a fabricated UI description. + +### Contact Speakeasy support to finish OAuth {#contact-speakeasy-support} + +- After installation, administrators **must contact Speakeasy support to finish + OAuth setup**. Installation alone does not complete OAuth setup. +- No support URL, channel, credential exchange, SLA, or follow-up console steps + were supplied. State the mandatory handoff plainly; do not invent them and + do not request or publish secret values. +- Screenshot exception: this is a support handoff, not a documented console UI. +- Provenance for both method-1 steps: operator instruction, refresh date. + ### Open Salesforce Setup {#open-salesforce-setup} - Entry: sign in to the Salesforce org that will expose its records. At the top @@ -126,6 +199,8 @@ connection-time user action, not an administrator credential-creation step. ### Start an External Client App {#start-external-client-app} +- Method 2 only; do not repeat this flow after installing the Speakeasy app. + - From **Setup**, enter `external client` in **Quick Find**, then select **External Client App Manager**. - Click **New External Client App**. @@ -193,7 +268,7 @@ connection-time user action, not an administrator credential-creation step. - Screenshot exception: the credential is sensitive and the screen adds no setup information beyond the exact label. Do not capture the key. -### Enable the selected SObject server {#enable-sobject-server} +### Enable the selected MCP server {#enable-sobject-server} - Return to **Setup**. In **Quick Find**, enter `MCP Servers`, then select **MCP Servers** under **API Catalog**. @@ -201,35 +276,52 @@ connection-time user action, not an administrator credential-creation step. Salesforce's current activation page says to toggle needed servers on, but does not publish the exact list-row names or the toggle's label or state. Use the selected server's confirmed API ID to distinguish among - `sobject-reads`, `sobject-mutations`, `sobject-deletes`, and `sobject-all`; + `sobject-reads`, `sobject-mutations`, `sobject-deletes`, `sobject-all`, + `data360`, `platform/headless-360`, and `analytics/tableau-next`; do not infer additional UI labels from those IDs. -- If the ticket does not specify the team's approved read, write, or delete - requirements, obtain the server choice from the application or cloud +- If the ticket does not specify the team's approved records, data-platform, + admin, or analytics requirements, obtain the server choice from the application or cloud security owner before enabling one. -- Match the selected server to the remote URL in **Server facts**, and match +- Match the selected server to the remote URL in **Server facts** or **Current endpoint catalog and setup gates**, and match the URL variant to the org type (production versus sandbox). - Wait up to two minutes for the server to become active. - Recovery: if the client returns a connection failure with valid OAuth, confirm that the exact server is enabled and that the URL uses the correct production or sandbox form. Also confirm the org has API access. - Screenshot note: **MCP Servers** under **API Catalog**, showing the available - server list and the control used to enable the chosen SObject server. The + server list and the control used to enable the chosen MCP server. The capture pass must record the rendered row and control labels rather than assuming labels from the server API IDs. +For Headless 360, the dedicated reference supplies a more specific activation +transition: **Setup** > **Quick Find**: `MCP Servers` > **MCP Servers** > find +`headless-360` > **Activate**. Other standard servers use the general activation +page's toggle instructions. The existing {#enable-sobject-server} anchor is +retained for the shared endpoint-selection/activation step, even when the +chosen server is not an SObject server. Source: Headless 360 reference and +activation page, refresh date. + ## Speakeasy setup +The manual credential-entry skeleton below applies to **method 2 only**. +For method 1, the administrator must contact Speakeasy support to finish OAuth; +no public self-service credential-entry procedure is established for this +package. Metadata models its OAuth registration as manual (not DCR), with no +invented credential fields. Do not derive a package Client ID or secret from +the self-created-app instructions. + + Per-guide values rendered into the canonical `doctrine/speakeasy-setup.md` skeleton: - Provider: Salesforce. -- Remote URL: the one production or sandbox SObject URL selected in - {#enable-sobject-server}; all eight supported choices are in Metadata. +- Remote URL: the production or sandbox URL selected in + {#enable-sobject-server}; all fourteen cataloged choices are in Metadata. - Transport: `streamable-http`; the **Transport** field is read-only. - Add-server path: use **Custom remote server** only. The operator forced `speakeasy_add_server: custom-remote` because the catalog mapping is - unreliable or unsuitable for this Guide's selection among eight distinct - production and sandbox SObject URLs. Pasting the selected URL preserves the + unreliable or unsuitable for this Guide's selection among distinct + production and sandbox URLs. Pasting the selected URL preserves the server and org-type choice made in {#enable-sobject-server}. Do not render a catalog path or a catalog-presence open question. - Authentication Option: OAuth with a manually registered client. @@ -256,7 +348,7 @@ In the Speakeasy AI Control Plane sidebar, under **Connect**, select **Sources**, then click **Add Source**. Choose **Custom remote server**. On the **Add a custom remote MCP server** -page, paste the selected SObject URL into **Remote MCP server URL** and click +page, paste the selected MCP URL into **Remote MCP server URL** and click **Add server**. This creates the hosted MCP server and opens its **Overview** page. @@ -288,6 +380,34 @@ the Client ID. This guide covers setup only. For anything beyond it — billing, tool behavior, limits — see [Salesforce's MCP documentation](https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/hosted-mcp-servers-overview.html). +## Research limitations + +- This refresh read ten unique public primary pages, starting with the supplied + setup overview and necessary setup/client/catalog references. No authenticated + org, app installation, OAuth exchange, or compatibility test was performed. +- Tableau Next entitlement details are absent from the fetched reference; + Data 360 Legacy and custom-server creation are outside this bounded research. + Endpoint listings document provider URLs, not tested Speakeasy compatibility + or availability in the Speakeasy MCP Catalog. +- The operator resolved the setup-method scope: publish both alternatives. + Missing package internals are not a reason to block the explicitly authorized + install-plus-support route; its mandatory handoff is the documented outcome. +- OAuth details for method 2: Postman documents production authorization/token + URLs `https://login.salesforce.com/services/oauth2/authorize` and + `https://login.salesforce.com/services/oauth2/token`; sandbox uses + `https://test.salesforce.com/services/oauth2/authorize` and + `https://test.salesforce.com/services/oauth2/token`. It uses Authorization + Code with PKCE, SHA-256, scopes `mcp_api refresh_token`, Consumer Key as + Client ID, and blank Client Secret. These are provider reference facts, not + additional verified Speakeasy input fields (source: `postman.html`, refresh date). +- The current ECA page separately offers optional production hardening: + requiring a secret for web-based clients, permission-set preauthorization, + IP restrictions, and token/session policies. Do not silently enable these or + claim the baseline secretless flow satisfies every organization's policies. + Policy-specific Speakeasy compatibility is untested. The setup overview's + broad PKCE instruction does not override the ECA page's precise baseline + Security checkbox instructions retained above. + ## Open questions - The Hosted MCP activation page says to toggle servers on but does not publish @@ -330,12 +450,16 @@ limits — see [Salesforce's MCP documentation](https://developer.salesforce.com - **Machine-readable indexes:** prior research successfully reached `https://developer.salesforce.com/docs/llms.txt` and its linked Hosted MCP index at `https://developer.salesforce.com/docs/llms-hosted-mcp-servers.txt`; - they enumerated the guide and reference pages used below. During this - refresh, the developer documentation and index requests returned HTTP 403, - so sound prior findings were retained rather than re-inferred from snippets. + they enumerated the guide and reference pages used below. During the August + research, developer documentation and index requests returned HTTP 403. + The September refresh successfully read the ten primary pages listed below; + older facts not revisited retain their original observation date. `https://help.salesforce.com/llms.txt` had previously timed out. -Provenance records below use `2026-08-06T23:23:14Z` for this refresh. +### Retained August provenance + +Records below retain observation date `2026-08-06T23:23:14Z`; endpoint +observations are historical, not tests rerun in September. - `https://developer.salesforce.com/docs/llms-hosted-mcp-servers.txt` — machine-readable Hosted MCP source inventory; backs documentation-property @@ -399,7 +523,7 @@ Provenance records below use `2026-08-06T23:23:14Z` for this refresh. — live endpoint observation; backs protected resource URL and advertised `mcp_api` / `refresh_token` scopes. - All eight URLs in **Server facts** — live unauthenticated endpoint - observations returned HTTP 401 on this refresh, backing the URLs' existence + observations returned HTTP 401 in the August research, backing the URLs' existence and OAuth protection. The protected-resource metadata request for production SObject Reads returned HTTP 200 and advertised `mcp_api` and `refresh_token`. - `doctrine/speakeasy-setup.md` — observed `2026-08-06T23:23:14Z`; backs the @@ -409,3 +533,25 @@ Provenance records below use `2026-08-06T23:23:14Z` for this refresh. `salesforce` — observed `2026-08-06T23:23:14Z`; backs the decision not to render or investigate a catalog path because the Guide-level `speakeasy_add_server: custom-remote` override controls path selection. + +### Refresh provenance + +Observed at `2026-09-17T16:15:46Z` (current system date, 2026-09-17). Every +new or revalidated fact above cites these locators by page name; retained +August-only facts and probes keep their earlier provenance. + +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/setup-overview.html` — Set Up Your Org; backs setup prerequisites, ECA, scopes, PKCE/JWT overview. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/create-external-client-app.html` — Create an External Client App; backs baseline OAuth settings, creation flow, propagation and optional production hardening. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/hosted-mcp-servers-overview.html` — Salesforce Hosted MCP Servers; backs per-user OAuth and platform/product scope. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/client-connection-overview.html` — Connecting an MCP Client; backs ECA requirement, no Connected Apps, tested-client list excluding Speakeasy. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/servers-reference.html` — Standard MCP Servers Reference; backs seven current standard server families and user permissions. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/activate-mcp-servers.html` — Activate MCP Servers; backs administrator activation navigation, default disabled, two-minute wait. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/references/reference/data360-mcp.html` — Data 360 MCP Server; backs exact URL forms, license/API/permission gates, broader capabilities and usage accounting. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/references/reference/headless-360-mcp.html` — Headless 360 MCP Server (Beta); backs exact URL forms, beta/API prerequisites and Activate action. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/references/reference/tableau-next.html` — Tableau Next; backs exact URL forms, semantic analytics scope and absence of entitlement details. +- `https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/postman.html` — Configure Postman; backs HTTP transport, platform URL forms, authorization/token URLs, PKCE and credential mapping. +- Operator instruction in this delegated task — backs the two-method scope, + Speakeasy application identity, exact installation locator + `https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2`, + and mandatory contact with Speakeasy support to finish OAuth. Not a claim + that the package installation or OAuth flow was independently tested. diff --git a/guides/salesforce/speakeasy.md b/guides/salesforce/speakeasy.md index 62e319f..e11ae2e 100644 --- a/guides/salesforce/speakeasy.md +++ b/guides/salesforce/speakeasy.md @@ -1,11 +1,15 @@ # Speakeasy setup +**Method 1 — Speakeasy application:** after [installing the application](external.md#install-speakeasy-application), you must [contact Speakeasy support to finish OAuth](external.md#contact-speakeasy-support). Coordinate server activation and the add-server step below with support. Do not use the manual credential instructions for this method. + +**Method 2 — Your own External Client App:** after creating the app and enabling the selected server, follow both steps below. + ### Add the server in Speakeasy {#add-server-in-speakeasy} 1. In the Speakeasy AI Control Plane sidebar, under **Connect**, select **Sources**. 2. Select **Add Source**. 3. Choose **Custom remote server**. -4. On the **Add a custom remote MCP server** page, paste the URL recorded in [Enable the selected SObject server](external.md#enable-sobject-server) into **Remote MCP server URL**. +4. On the **Add a custom remote MCP server** page, paste the URL recorded in [Enable the selected MCP server](external.md#enable-sobject-server) into **Remote MCP server URL**. 5. Select **Add server**. This creates the hosted MCP server and opens its **Overview** page. @@ -14,6 +18,10 @@ This creates the hosted MCP server and opens its **Overview** page. ### Connect your credentials {#connect-speakeasy-credentials} +**Method 1 — Speakeasy application:** finish OAuth through the mandatory [Speakeasy support handoff](external.md#contact-speakeasy-support), not the fields below. + +**Method 2 — Your own External Client App only:** + 1. From the server's **Overview**, open **Settings**. 2. Under **Authentication**, select **Configure Manually**. 3. In the **Attach Remote Identity Provider** sheet, set **Client Type** to **Manual**. From 29b01037de15454df53a36ef0d17c81a0e638326 Mon Sep 17 00:00:00 2001 From: Walker Lockard Date: Thu, 17 Sep 2026 11:11:48 -0700 Subject: [PATCH 3/6] Revert "fix: retain sanitized factory execution transcripts" This reverts commit 78c313a729e7cd3301a5e958092e51d5e1fa1307. --- .github/workflows/guide-draft.yml | 9 ----- factory/Dockerfile | 2 - factory/README.md | 43 --------------------- factory/scripts/build-diagnostics.sh | 1 - factory/scripts/build-transcript.sh | 48 ------------------------ factory/scripts/container-entrypoint.sh | 9 +---- factory/scripts/run-kit.sh | 3 +- factory/scripts/transcript.jq | 38 ------------------- factory/tests/test-container.sh | 16 +------- factory/tests/test-coordinator.sh | 24 +++--------- factory/tests/test-export-boundary.sh | 1 - factory/tests/test-transcript.sh | 50 ------------------------- 12 files changed, 8 insertions(+), 236 deletions(-) delete mode 100755 factory/scripts/build-transcript.sh delete mode 100644 factory/scripts/transcript.jq delete mode 100644 factory/tests/test-transcript.sh diff --git a/.github/workflows/guide-draft.yml b/.github/workflows/guide-draft.yml index fcdac11..1899eef 100644 --- a/.github/workflows/guide-draft.yml +++ b/.github/workflows/guide-draft.yml @@ -122,15 +122,6 @@ jobs: "$RUNNER_TEMP/export" cp "$RUNNER_TEMP/export/run-report.json" "$RUNNER_TEMP/run-report.json" - - name: Upload sanitized execution transcript - if: always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure') - uses: actions/upload-artifact@v4 - with: - name: guide-factory-transcript-${{ github.run_id }}-${{ github.run_attempt }} - path: ${{ runner.temp }}/export/execution-transcript.json - retention-days: 7 - if-no-files-found: ignore - - name: Upload safe factory diagnostics if: always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure') uses: actions/upload-artifact@v4 diff --git a/factory/Dockerfile b/factory/Dockerfile index bb4183e..c9df313 100644 --- a/factory/Dockerfile +++ b/factory/Dockerfile @@ -22,8 +22,6 @@ RUN file=kit-v${KIT_VERSION}-x86_64-unknown-linux-gnu.tar.gz \ COPY --from=lint-builder /out/lint-guide /usr/local/bin/lint-guide COPY factory/scripts/validate-report.sh /usr/local/bin/validate-report COPY factory/scripts/project-kit-events.sh /usr/local/bin/project-kit-events -COPY factory/scripts/build-transcript.sh /usr/local/bin/build-transcript -COPY factory/scripts/transcript.jq /usr/local/bin/transcript.jq COPY factory/scripts/build-diagnostics.sh /usr/local/bin/build-diagnostics COPY factory/scripts/validate-diagnostics.sh /usr/local/bin/validate-diagnostics COPY factory/scripts/container-entrypoint.sh /usr/local/bin/factory-entrypoint diff --git a/factory/README.md b/factory/README.md index eeb43d6..c7d44dc 100644 --- a/factory/README.md +++ b/factory/README.md @@ -22,46 +22,3 @@ payloads. If runtime-event parsing fails, rejected events are discarded; the bundle still retains independently validated failure metadata and the outer Kit invocation status. The workflow log records the parser failure. If bundle validation itself fails closed, the workflow safely skips the missing artifact. - -## Sanitized execution transcript - -For a completed Kit invocation (success or failure), Actions separately uploads -`export/execution-transcript.json` as -`guide-factory-transcript-RUN_ID-RUN_ATTEMPT`, retained for **seven days** under -repository artifact access controls. The upload names only that file, never an -export glob or raw session directory. Stale transcript exports are removed before -startup. Capture runs immediately after Kit exits, independently of runtime-event -projection and before report validation, so parser errors and factory-reported -failure do not discard it. Capture failure is nonfatal and removes the export; -abrupt runner/container loss or cancellation is not guaranteed to produce it. -There is **no automatic retry** and no change to lint behavior. - -This is **not a full conversational transcript**. It is an allowlisted structural -summary of persisted `.kit/sessions/w-*/*.jsonl` records from parent and subagent -sessions: anonymous per-file session references, per-session call references, -recognized role/part kinds and tool names (`unknown` otherwise), result error -flags, and safe integer exit codes (0–255). Shell-shaped results nested inside -compose output are summarized too. Stdout/stderr retain only nonempty presence, -JSON validity and top-level array/object counts—not their values. These fields -can help distinguish a command failure from malformed JSON without revealing its -content; they do not prove a particular linter ran or succeeded. - -No prompts, text, reasoning, arguments, original identifiers, URLs, metadata, -stdout/stderr contents, or unknown fields are copied. Kit's PascalCase parts and -Text/Structured tool outputs are supported; replacement records are summarized -as encountered, not reconstructed into a canonical conversation. Replacements -may repeat events. Compose spill previews/files, tool-output Parts/Files and -unrecognized formats may omit data; no referenced artifact paths are followed. -Malformed lines produce a fixed marker, retaining preceding valid events even -with a truncated tail. Session file names are not exported; references do not -encode parent-child relationships or global chronological order. - -Capture reads at most 64 regular files, 1 MiB per file and 8 MiB total; session -symlinks and symlinked directories below the supplied Kit home are skipped or -rejected. It retains at most 4,096 events, 16 shell-shaped results per tool result, -and a 2 MiB final artifact. Source/event truncation sets `limited`; unsupported -fields and per-result omissions need not do so. Sanitization/size validation -must succeed before the transcript becomes world-readable (`0644`), allowing the -non-root Actions host to read a root-owned container export. Diagnostics likewise -receive `0644` only after their existing strict validator succeeds. Neither -artifact contains raw session logs; do not upload those as a fallback. diff --git a/factory/scripts/build-diagnostics.sh b/factory/scripts/build-diagnostics.sh index d7f1a09..f932f63 100755 --- a/factory/scripts/build-diagnostics.sh +++ b/factory/scripts/build-diagnostics.sh @@ -149,7 +149,6 @@ if ! jq -n \ fi "$DIAGNOSTICS_VALIDATOR" "$temporary" >/dev/null 2>&1 || exit 1 -chmod 0644 "$temporary" 2>/dev/null || exit 1 mv -f -- "$temporary" "$output" 2>/dev/null || exit 1 temporary= succeeded=true diff --git a/factory/scripts/build-transcript.sh b/factory/scripts/build-transcript.sh deleted file mode 100755 index 204bf19..0000000 --- a/factory/scripts/build-transcript.sh +++ /dev/null @@ -1,48 +0,0 @@ -#!/usr/bin/env bash -# Only structural data is exported. Never print source paths or jq errors. -set -euo pipefail -[[ $# == 2 ]] || exit 1 -home=$1 -output=$2 -SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) -filter="$SCRIPT_DIR/transcript.jq" -[[ -d $(dirname "$output") && ! -d $output ]] || exit 1 -rm -f -- "$output" -tmp=$(mktemp -d) -trap 'rm -rf -- "$tmp"' EXIT -sessions="$home/.kit/sessions" -# Fixed-depth globs, rejecting every symlink component below the supplied home. -[[ ! -L $home && ! -L $home/.kit && ! -L $sessions ]] || exit 1 -count=0 -bytes=0 -limited=false -: >"$tmp/events" -shopt -s nullglob -for dir in "$sessions"/w-*; do - [[ -d $dir && ! -L $dir ]] || continue - for file in "$dir"/*.jsonl; do - [[ -f $file && ! -L $file ]] || continue - if ((count >= 64 || bytes >= 8388608)); then limited=true; break 2; fi - count=$((count + 1)) - # Bound input even for an unterminated or enormous line. - head -c 1048576 -- "$file" >"$tmp/source" 2>/dev/null || exit 1 - size=$(wc -c <"$tmp/source") - bytes=$((bytes + size)) - if ((size >= 1048576)); then limited=true; fi - jq -Rnc --argjson session "$count" -f "$filter" <"$tmp/source" >>"$tmp/events" 2>/dev/null || exit 1 - done -done -jq -sc --argjson sessions "$count" --argjson limited "$limited" ' - {schema_version:1,kind:"guide_factory_transcript",sessions:$sessions, - limited:($limited or length > 4096),events:.[0:4096]} -' "$tmp/events" >"$tmp/safe" 2>/dev/null -[[ $(wc -c <"$tmp/safe") -le 2097152 ]] || exit 1 -# Defense in depth: no unrecognized key or string may cross the export boundary. -jq -e ' - def keysafe: IN("schema_version","kind","sessions","limited","events","session_ref","call_ref","role","part","tool","is_error","shell_results","exit_code","stdout","stderr","present","json_valid","json_count"); - all(.. | objects | keys[]; keysafe) and - all(.. | strings; IN("guide_factory_transcript","System","Developer","User","Assistant","Tool","Context","Notification","unknown","Text","Media","File","Structured","Reasoning","ToolCall","ToolResult","Custom","malformed","shell","compose","tool","tool_search","subagent","prompt","fork","close","skill","docs","edit","a2a","auth","subagents")) and - all(.. | numbers; floor == . and . >= 0 and . <= 8388608) -' "$tmp/safe" >/dev/null 2>&1 || exit 1 -chmod 0644 "$tmp/safe" -mv -f -- "$tmp/safe" "$output" diff --git a/factory/scripts/container-entrypoint.sh b/factory/scripts/container-entrypoint.sh index 61e2c13..f47a798 100755 --- a/factory/scripts/container-entrypoint.sh +++ b/factory/scripts/container-entrypoint.sh @@ -8,13 +8,11 @@ EXPORT_ROOT=${FACTORY_EXPORT_ROOT:-/export} KIT_HOME=${FACTORY_KIT_HOME:-/tmp/kit-home} REPORT_VALIDATOR=${FACTORY_REPORT_VALIDATOR:-/usr/local/bin/validate-report} EVENT_PROJECTOR=${FACTORY_EVENT_PROJECTOR:-/usr/local/bin/project-kit-events} -TRANSCRIPT_BUILDER=${FACTORY_TRANSCRIPT_BUILDER:-/usr/local/bin/build-transcript} DIAGNOSTICS_BUILDER=${FACTORY_DIAGNOSTICS_BUILDER:-/usr/local/bin/build-diagnostics} mkdir -p "$EXPORT_ROOT" rm -rf "$EXPORT_ROOT/guide" "$EXPORT_ROOT/run-report.json" \ - "$EXPORT_ROOT/kit-error-summary.json" "$EXPORT_ROOT/factory-diagnostics.json" \ - "$EXPORT_ROOT/execution-transcript.json" + "$EXPORT_ROOT/kit-error-summary.json" "$EXPORT_ROOT/factory-diagnostics.json" test -r "$INPUT_ROOT/issue.json" test -r "$INPUT_ROOT/catalog.json" test -r "$REPO_ROOT/factory/coordinator.md" @@ -47,11 +45,6 @@ if KIT_RUNTIME_EVENTS=1 "$KIT_BIN" prompt \ else kit_status=$? fi -# Persist safe session structure before consuming any projector/report results. -if ! "$TRANSCRIPT_BUILDER" "$KIT_HOME" "$EXPORT_ROOT/execution-transcript.json" 2>/dev/null; then - rm -f -- "$EXPORT_ROOT/execution-transcript.json" - printf '%s\n' 'factory: sanitized transcript unavailable' >&2 -fi if wait "$projector_pid"; then projector_status=0 else diff --git a/factory/scripts/run-kit.sh b/factory/scripts/run-kit.sh index af68e75..12976cb 100755 --- a/factory/scripts/run-kit.sh +++ b/factory/scripts/run-kit.sh @@ -12,8 +12,7 @@ export_dir=$3 mkdir -p "$export_dir" export_dir="$(realpath "$export_dir")" rm -rf "$export_dir/guide" "$export_dir/run-report.json" \ - "$export_dir/kit-error-summary.json" "$export_dir/factory-diagnostics.json" \ - "$export_dir/execution-transcript.json" + "$export_dir/kit-error-summary.json" "$export_dir/factory-diagnostics.json" [[ -r "$issue_json" ]] || { printf 'issue JSON is not readable: %s\n' "$issue_json" >&2; exit 2; } [[ -r "$catalog_json" ]] || { printf 'catalog JSON is not readable: %s\n' "$catalog_json" >&2; exit 2; } : "${OPENROUTER_API_KEY:?OPENROUTER_API_KEY is required}" diff --git a/factory/scripts/transcript.jq b/factory/scripts/transcript.jq deleted file mode 100644 index 73ad82d..0000000 --- a/factory/scripts/transcript.jq +++ /dev/null @@ -1,38 +0,0 @@ -# Matches Kit session Record / agentkit-core externally tagged Part and ToolOutput. -def toolname: if IN("shell","compose","tool","tool_search","subagent","prompt","fork","close","skill","docs","edit","a2a","auth","subagents") then . else "unknown" end; -def role: if IN("System","Developer","User","Assistant","Tool","Context","Notification") then . else "unknown" end; -def stream: - if type != "string" then {present:false,json_valid:false,json_count:null} - else {present:(length > 0)} + - (try (fromjson | {json_valid:true,json_count:(if type == "array" or type == "object" then length else null end)}) - catch {json_valid:false,json_count:null}) end; -def shells: - (if .Structured != null then .Structured elif (.Text | type) == "string" then (try (.Text | fromjson) catch null) else null end) - | [.. | objects | select(has("exit_code") and has("stdout") and has("stderr")) | - {exit_code:(.exit_code | if type == "number" then if floor == . and . >= 0 and . <= 255 then . else null end else null end), - stdout:(.stdout | stream),stderr:(.stderr | stream)}][0:16]; -def parts: - if type != "object" then {role:"unknown",part:"malformed"} - elif (.schema_version | IN(1,2,3)) and - ((.item | type) == "object" or (.replacement | type) == "array") then - (if .item != null then [.item] else .replacement end)[] | - . as $item | if (.parts | type) != "array" then {role:"unknown",part:"malformed"} - else .parts[] | . as $part | - {role:($item.kind | role)} + - (if type != "object" or length != 1 then {part:"unknown"} - else keys[0] as $k | - {part:($k | if IN("Text","Media","File","Structured","Reasoning","ToolCall","ToolResult","Custom") then . else "unknown" end)} + - (if $k == "ToolCall" then {id:$part.ToolCall.id,tool:($part.ToolCall.name | toolname)} - elif $k == "ToolResult" then {id:$part.ToolResult.call_id,is_error:($part.ToolResult.is_error | if type == "boolean" then . else null end),shell_results:($part.ToolResult.output | shells)} - else {} end) end) - end - else {role:"unknown",part:"malformed"} end; -reduce (inputs | (try fromjson catch null) | (try parts catch {role:"unknown",part:"malformed"})) as $event - ({calls:{},next:0,events:[]}; - if (.events | length) >= 4097 then . else - (if ($event.id | type) == "string" then - if .calls[$event.id] == null then .next += 1 | .calls[$event.id] = .next else . end - else . end) | - .events += [($event | del(.id)) + {session_ref:$session} + - (if ($event.id | type) == "string" then {call_ref:.calls[$event.id]} else {} end)] end) -| .events[] diff --git a/factory/tests/test-container.sh b/factory/tests/test-container.sh index 8750be3..c7d6886 100755 --- a/factory/tests/test-container.sh +++ b/factory/tests/test-container.sh @@ -6,7 +6,6 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" source "$ROOT/factory/tests/test-helper.sh" TMP="$(mktemp -d)" export TMP -export FACTORY_TRANSCRIPT_BUILDER="$ROOT/factory/scripts/build-transcript.sh" export FACTORY_EVENT_PROJECTOR="$ROOT/factory/scripts/project-kit-events.sh" export FACTORY_DIAGNOSTICS_BUILDER="$ROOT/factory/scripts/build-diagnostics.sh" trap 'rm -rf "$TMP"; exit 130' INT TERM @@ -123,7 +122,6 @@ test_startup_failures_remove_stale_diagnostics() { host_export="$TMP/stale-host-export" mkdir -p "$host_export" printf '%s\n' 'SECRET_STALE_HOST_DIAGNOSTIC' >"$host_export/factory-diagnostics.json" - touch "$host_export/execution-transcript.json" host_status=0 OPENROUTER_API_KEY=or-test "$ROOT/factory/scripts/run-kit.sh" \ @@ -132,7 +130,6 @@ test_startup_failures_remove_stale_diagnostics() { assert_eq 2 "$host_status" test ! -e "$host_export/factory-diagnostics.json" \ || fail 'run-kit retained stale diagnostics after startup failure' - test ! -e "$host_export/execution-transcript.json" || fail 'stale host transcript' repo="$TMP/stale-container-repo" input="$TMP/stale-container-input" @@ -141,7 +138,6 @@ test_startup_failures_remove_stale_diagnostics() { printf 'assignment\n' >"$repo/factory/coordinator.md" printf '%s\n' 'SECRET_STALE_CONTAINER_DIAGNOSTIC' \ >"$container_export/factory-diagnostics.json" - touch "$container_export/execution-transcript.json" container_status=0 FACTORY_REPO_ROOT="$repo" FACTORY_INPUT_ROOT="$input" \ @@ -151,7 +147,6 @@ test_startup_failures_remove_stale_diagnostics() { "$ROOT/factory/scripts/container-entrypoint.sh" >/dev/null 2>&1 \ || container_status=$? assert_eq 1 "$container_status" - test ! -e "$container_export/execution-transcript.json" || fail 'stale container transcript' test ! -e "$container_export/factory-diagnostics.json" \ || fail 'entrypoint retained stale diagnostics after startup failure' } @@ -441,8 +436,6 @@ test_entrypoint_exports_only_selected_guide_with_mocked_kit() { #!/usr/bin/env bash set -euo pipefail [[ "$1" == prompt ]] -mkdir -p "$HOME/.kit/sessions/w-success" -printf '%s\n' '{"schema_version":3,"item":{"kind":"Assistant","parts":[{"Text":{"text":"SECRET_SUCCESS"}}]}}' >"$HOME/.kit/sessions/w-success/test.jsonl" mkdir -p "$FACTORY_WORKSPACE_ROOT/guides/acme" printf 'guide\n' >"$FACTORY_WORKSPACE_ROOT/guides/acme/research.md" printf 'ignore\n' >"$FACTORY_WORKSPACE_ROOT/not-exported.txt" @@ -469,9 +462,7 @@ MOCK test -f "$export_root/guide/meta.yaml" test -f "$export_root/run-report.json" test ! -e "$export_root/not-exported.txt" - jq -e '.sessions == 1 and .events[0].part == "Text"' "$export_root/execution-transcript.json" >/dev/null || fail 'successful Kit lost transcript' - ! grep -q SECRET "$export_root/execution-transcript.json" || fail 'successful transcript leaked text' - assert_eq "4" "$(find "$export_root" -type f | wc -l | tr -d ' ')" + assert_eq "3" "$(find "$export_root" -type f | wc -l | tr -d ' ')" } test_entrypoint_rejects_invalid_report() { @@ -575,8 +566,6 @@ test_entrypoint_handles_invalid_projection_and_missing_report() { workspace="$TMP/runtime-workspace-$kind"; export_root="$TMP/runtime-export-$kind" cat >"$fake_kit" <<'MOCK' #!/usr/bin/env bash -mkdir -p "$HOME/.kit/sessions/w-test" -printf '%s\n' '{"schema_version":3,"session_id":"SECRET_SESSION","generation":1,"item":{"kind":"Assistant","parts":[{"Text":{"text":"SECRET_TEXT"}}]}}' >"$HOME/.kit/sessions/w-test/session.jsonl" marker=$(printf '\001kit-runtime\001') printf '%s%s\n' "$marker" "${RUNTIME_BAD_LINE}" >&2 printf '%s\n' 'ordinary diagnostic after bad event' >&2 @@ -598,8 +587,6 @@ MOCK "$ROOT/factory/scripts/container-entrypoint.sh" >/dev/null 2>"$TMP/runtime-$kind.err"; then fail "entrypoint accepted $kind runtime event" fi - jq -e '.sessions == 1 and .events[0].part == "Text"' "$export_root/execution-transcript.json" >/dev/null || fail 'parser failure lost transcript' - ! grep -q SECRET "$export_root/execution-transcript.json" || fail 'container transcript leaked secret' "$ROOT/factory/scripts/validate-diagnostics.sh" "$export_root/factory-diagnostics.json" >/dev/null \ || fail "entrypoint lost safe fallback for $kind runtime event" jq -e '.stage == "kit_prompt" and (.events | length) == 2 and all(.events[]; .tool == "kit_prompt")' \ @@ -636,7 +623,6 @@ MOCK "$ROOT/factory/scripts/validate-diagnostics.sh" "$export_root/factory-diagnostics.json" >/dev/null jq -e '.stage == "factory_outcome" and .classification == "factory_reported_failure" and .report.outcome == "failed" and (.events | length) == 2' \ "$export_root/factory-diagnostics.json" >/dev/null || fail 'lost failed-report fallback' - test -r "$export_root/execution-transcript.json" || fail 'factory failure lost transcript' test -f "$export_root/run-report.json" || fail 'lost failed report' test ! -e "$export_root/guide" || fail 'exported failed guide' if grep -q 'SECRET_' "$export_root/factory-diagnostics.json"; then diff --git a/factory/tests/test-coordinator.sh b/factory/tests/test-coordinator.sh index a934e83..f430135 100755 --- a/factory/tests/test-coordinator.sh +++ b/factory/tests/test-coordinator.sh @@ -194,10 +194,10 @@ assert_step_contains() { } upload_step_block() { - local workflow=$1 name=${2:-Upload safe factory diagnostics} - awk -v target=" - name: $name" ' - $0 == target { found=1 } - found && /^ - name: / && $0 != target { exit } + local workflow=$1 + awk ' + $0 == " - name: Upload safe factory diagnostics" { found=1 } + found && /^ - name: / && $0 != " - name: Upload safe factory diagnostics" { exit } found { print } ' "$workflow" } @@ -254,22 +254,8 @@ assert_upload_contract() { fail 'upload step contains a multiline or nested field value' return 1 fi - block="$(upload_step_block "$workflow" 'Upload sanitized execution transcript')" - [[ -n "$block" ]] || { fail 'missing sanitized transcript upload'; return 1; } - assert_upload_field_equals "$block" ' ' if "always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure')" || return 1 - assert_upload_field_equals "$block" ' ' uses 'actions/upload-artifact@v4' || return 1 - # shellcheck disable=SC2016 - assert_upload_field_equals "$block" ' ' name 'guide-factory-transcript-${{ github.run_id }}-${{ github.run_attempt }}' || return 1 - # shellcheck disable=SC2016 - assert_upload_field_equals "$block" ' ' path '${{ runner.temp }}/export/execution-transcript.json' || return 1 - assert_upload_field_equals "$block" ' ' retention-days '7' || return 1 - assert_upload_field_equals "$block" ' ' if-no-files-found ignore || return 1 - if grep -Eq '^ [^[:space:]]' <<<"$block"; then - fail 'transcript upload contains a multiline or nested field value' - return 1 - fi upload_count="$(count_upload_artifact_actions "$workflow")" - assert_eq '2' "$upload_count" || return 1 + assert_eq '1' "$upload_count" || return 1 } steps_with() { diff --git a/factory/tests/test-export-boundary.sh b/factory/tests/test-export-boundary.sh index 0d532bb..fdb4ae4 100755 --- a/factory/tests/test-export-boundary.sh +++ b/factory/tests/test-export-boundary.sh @@ -6,7 +6,6 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" source "$ROOT/factory/tests/test-helper.sh" TMP="$(mktemp -d)" export TMP -export FACTORY_TRANSCRIPT_BUILDER="$ROOT/factory/scripts/build-transcript.sh" trap 'rm -rf "$TMP"; exit 130' INT TERM test_launcher_canonicalizes_paths_and_mounts_gitless_snapshot() { diff --git a/factory/tests/test-transcript.sh b/factory/tests/test-transcript.sh deleted file mode 100644 index 90af2fe..0000000 --- a/factory/tests/test-transcript.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail -ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd) -# shellcheck disable=SC1091 -source "$ROOT/factory/tests/test-helper.sh" -TMP=$(mktemp -d) -trap 'rm -rf "$TMP"' EXIT -mkdir -p "$TMP/home/.kit/sessions/w-parent" "$TMP/home/.kit/sessions/w-child" -cat >"$TMP/home/.kit/sessions/w-parent/parent.jsonl" <<'JSON' -{"schema_version":3,"session_id":"SECRET_SESSION","generation":1,"item":{"kind":"Assistant","parts":[{"Text":{"text":"SECRET_PROMPT"}},{"Reasoning":{"text":"SECRET_REASON"}},{"ToolCall":{"id":"SECRET_CALL","name":"compose","input":{"script":"SECRET_ARGUMENT"},"metadata":{"secret":"SECRET_META"}}}]}} -{"schema_version":3,"session_id":"SECRET_SESSION","generation":1,"item":{"kind":"Tool","parts":[{"ToolResult":{"call_id":"SECRET_CALL","is_error":false,"output":{"Structured":{"nested":{"exit_code":1,"stdout":"{\"errors\":[\"SECRET_LINT\"]}","stderr":"SECRET_STDERR","secret":"SECRET_UNKNOWN"}}}}}]}} -{"schema_version":3,"session_id":"SECRET_SESSION","generation":2,"replacement":[{"kind":"User","parts":[{"Custom":{"secret":"SECRET_CUSTOM"}},{"ToolCall":{"id":"SECRET_OTHER","name":"SECRET_TOOL","input":{}}}]}]} -{"truncated":"SECRET_TAIL -JSON -cp "$TMP/home/.kit/sessions/w-parent/parent.jsonl" "$TMP/home/.kit/sessions/w-child/child.jsonl" -ln -s "$TMP/home/.kit/sessions/w-parent" "$TMP/home/.kit/sessions/w-link" -ln -s "$TMP/home/.kit/sessions/w-parent/parent.jsonl" "$TMP/home/.kit/sessions/w-child/link.jsonl" -BUILD="$ROOT/factory/scripts/build-transcript.sh" -bash "$BUILD" "$TMP/home" "$TMP/transcript.json" -! grep -q SECRET "$TMP/transcript.json" || fail 'transcript leaked a canary' -jq -e '.sessions == 2 and ([.events[] | select(.part == "malformed")] | length) == 2 and any(.events[]; .tool == "unknown") and any(.events[]; .shell_results[0].exit_code == 1 and .shell_results[0].stdout.json_valid == true and .shell_results[0].stdout.json_count == 1) and ([.events[] | select(.part == "ToolResult") | .call_ref] == [1,1])' "$TMP/transcript.json" >/dev/null -[[ -n $(find "$TMP/transcript.json" -perm 0644 -print) ]] || fail 'transcript not host readable' -bash "$ROOT/factory/scripts/build-diagnostics.sh" docker_build 1 - - - "$TMP/diagnostics.json" -[[ -n $(find "$TMP/diagnostics.json" -perm 0644 -print) ]] || fail 'validated diagnostics not host readable' -printf 'PASS sanitized transcript\n' - -# Text-wrapped JSON is a real ToolOutput variant; never copy its payload. -mkdir -p "$TMP/text/.kit/sessions/w-test" -jq -nc '{schema_version:3,item:{kind:"Tool",parts:[{ToolResult:{call_id:"SECRET",is_error:true,output:{Text:({exit_code:2,stdout:"[1,2]",stderr:"SECRET"}|tojson)}}}]}}' >"$TMP/text/.kit/sessions/w-test/test.jsonl" -printf '%s\n' '{"schema_version":3,"item":{"parts":[{"ToolResult":{"output":7}}]}}' >>"$TMP/text/.kit/sessions/w-test/test.jsonl" -bash "$BUILD" "$TMP/text" "$TMP/text.json" -jq -e '.events[0].shell_results[0].stdout.json_count == 2 and .events[1].part == "malformed"' "$TMP/text.json" >/dev/null -! grep -q SECRET "$TMP/text.json" || fail 'text output leaked a canary' -# Source prefix truncation is flagged, not copied or treated as complete. -head -c 1100000 /dev/zero | tr '\0' x >"$TMP/text/.kit/sessions/w-test/large.jsonl" -bash "$BUILD" "$TMP/text" "$TMP/limited.json" -jq -e '.limited == true and any(.events[]; .part == "malformed")' "$TMP/limited.json" >/dev/null -# A symlinked session root cannot be traversed, and stale output is removed. -mv "$TMP/text/.kit/sessions" "$TMP/real-sessions" -ln -s "$TMP/real-sessions" "$TMP/text/.kit/sessions" -if bash "$BUILD" "$TMP/text" "$TMP/text.json"; then fail 'followed session symlink'; fi -test ! -e "$TMP/text.json" - -# Workflow upload is explicit, independent, and short-lived. -workflow="$ROOT/.github/workflows/guide-draft.yml" -sed -n '/name: Upload sanitized execution transcript/,/name: Upload safe factory diagnostics/p' "$workflow" >"$TMP/upload" -grep -Fq "if: always() && !cancelled() && (steps.kit.outcome == 'success' || steps.kit.outcome == 'failure')" "$TMP/upload" -# shellcheck disable=SC2016 -grep -Fq 'path: ${{ runner.temp }}/export/execution-transcript.json' "$TMP/upload" -grep -Fq 'retention-days: 7' "$TMP/upload" From abb0164803ce4dfc0afa32d8f86fd5f8770a31f4 Mon Sep 17 00:00:00 2001 From: Walker Lockard Date: Thu, 17 Sep 2026 11:21:34 -0700 Subject: [PATCH 4/6] docs: simplify Salesforce guide for administrators --- guides/salesforce/external.md | 154 ++++++++------------------------- guides/salesforce/speakeasy.md | 14 +-- 2 files changed, 41 insertions(+), 127 deletions(-) diff --git a/guides/salesforce/external.md b/guides/salesforce/external.md index aaad57f..44809aa 100644 --- a/guides/salesforce/external.md +++ b/guides/salesforce/external.md @@ -6,44 +6,45 @@ setup_version: 1 Use Salesforce System Administrator credentials for an API-enabled production or sandbox org where Hosted MCP Servers are available. Salesforce documents availability for Enterprise Edition and above. You need authority to install the Speakeasy application or create an **External Client App**, and enable Hosted MCP Servers. -Sign in to the Salesforce org you want to connect. Install or create the app in that same org. This guide does not cover scratch orgs. For a lower-edition org, confirm Hosted MCP availability in [Salesforce Setup](#open-salesforce-setup) before beginning either method. +Sign in to the Salesforce org you want to connect. Install or create the app in that same org. This guide does not cover scratch orgs. For a lower-edition org, confirm Hosted MCP availability in [Salesforce Setup](#open-salesforce-setup) before starting either path. -Choose one method: +### Open Salesforce Setup {#open-salesforce-setup} + +1. At the top of any Salesforce page, select the setup gear icon. +2. Select **Setup**. + +For a lower-edition org: + +1. In **Quick Find**, enter `MCP Servers`. +2. Select **MCP Servers** under **API Catalog**. +3. Confirm that Hosted MCP Servers are available in the target org before continuing. -- **Method 1 — Speakeasy application:** [install the application](#install-speakeasy-application), then [contact Speakeasy support](#contact-speakeasy-support) to finish OAuth. Do not follow the own-app credential steps. -- **Method 2 — Your own External Client App:** begin at [Open Salesforce Setup](#open-salesforce-setup), then create the app and copy its **Consumer Key**. + -Both methods require an approved endpoint and administrator activation of the [selected MCP server](#enable-sobject-server). For method 1, coordinate activation and OAuth sequencing with Speakeasy support. The endpoint list documents Salesforce URLs, not live-tested Speakeasy compatibility. +Choose one path: **use Speakeasy's Salesforce app** and finish OAuth with support, or **create your own Salesforce app** and connect it yourself. + +## Use Speakeasy's Salesforce app ### Install Speakeasy's Salesforce application {#install-speakeasy-application} -**Method 1 only.** Install Speakeasy's Salesforce application into the intended org using [login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2](https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2). +Install Speakeasy's Salesforce application into the intended org using [the Speakeasy Salesforce app installation page](https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2). ### Contact Speakeasy support to finish OAuth {#contact-speakeasy-support} -After installation, **contact Speakeasy support to finish OAuth setup**. Installation alone does not complete OAuth setup. Coordinate the selected endpoint, org type, and server activation with support; do not create another app or apply the own-app credential instructions below to the installed package. +After installation, **contact Speakeasy support to finish OAuth setup**. Installation alone does not complete OAuth. Coordinate your production or sandbox endpoint and [server activation](#enable-sobject-server) with support. Your next step is with support—not the app-creation walkthrough below. -### Open Salesforce Setup {#open-salesforce-setup} +## Create your own Salesforce app -1. At the top of any Salesforce page, select the setup gear icon. -2. Select **Setup**. +Before creating the app, choose a server in the [endpoint reference](#endpoint-reference) and confirm its prerequisites. Then create an **External Client App** in the org you want to connect and enable that server. -For a lower-edition org: - -1. In **Quick Find**, enter `MCP Servers`. -2. Select **MCP Servers** under **API Catalog**. -3. Confirm that Hosted MCP Servers are available in the target org before continuing. - - +This path uses the app's **Consumer Key** without a client secret. Salesforce documents this configuration for compatible public clients; it has not been verified with the Speakeasy AI Control Plane. ### Start an External Client App {#start-external-client-app} -**Method 2 only.** Follow this step through [Copy the Consumer Key](#copy-consumer-key) only if you are creating your own app. - 1. In **Quick Find**, enter `external client`. 2. Select **External Client App Manager**. 3. Select **New External Client App**. @@ -80,7 +81,7 @@ For a lower-edition org: Select **Create**. -The app can take up to 30 minutes to become operational. If attaching it immediately fails even though the settings are correct, wait for that window before changing the configuration. +The app can take up to 30 minutes to become operational. If attachment fails immediately, allow that window before retrying. @@ -97,113 +98,32 @@ Do not copy the **Consumer Secret** for this path. ### Enable the selected MCP server {#enable-sobject-server} -**Both methods.** Choose the least-privileged server that meets the team's needs. For method 1, coordinate this step with Speakeasy support: - -- `sobject-reads` allows discovery, query, search, and relationship traversal without changing records. -- `sobject-mutations` allows reading, creating, and updating records without deleting them. -- `sobject-deletes` allows identifying and deleting records without creating or updating them. -- `sobject-all` allows creating, reading, updating, deleting, querying, and searching records. - -- Data 360 (`data360`) can change customer-data configuration as well as query data. It requires a Data 360 license and API v66.0 or later, with **Manage Data 360** for configuration or **View Data 360** for read-only operations. -- Headless 360 (Beta), API ID `platform/headless-360`, provides broad Setup and platform operations, not read-only record access. It is available starting July 2026 under Beta Services Terms and requires API v67.0 or later, an External Client App with `mcp_api`, and an OAuth client. -- Tableau Next, API ID `analytics/tableau-next`, provides semantic-model and analytics access. Confirm that the target org has the required Tableau Next capabilities before selecting it. - -If the ticket does not specify the team's approved records, data-platform, admin, or analytics requirements, obtain the server choice from the application or cloud security owner. +Choose the least-privileged server that meets your team's needs using the endpoint reference below. If you are unsure which capabilities are approved, ask the application or cloud security owner before enabling a server. 1. Return to **Setup**. 2. In **Quick Find**, enter `MCP Servers`. 3. Select **MCP Servers** under **API Catalog**. 4. Find the server whose API ID matches the approved choice. 5. Use the available control to enable that server. For Headless 360, find `headless-360` and select **Activate**. -6. Record its URL from the list below, using the production or sandbox form that matches the org. +6. Record its URL from the endpoint reference below, using the production or sandbox form that matches the org. 7. Wait up to two minutes for the server to become active. -Data 360's sandbox URL places `/sandbox` after `/data`, unlike the platform and analytics endpoints. Copy the exact URL for your server and org type. - -**SObject Reads — production** - -``` -https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads -``` - -**SObject Reads — sandbox** - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads -``` - -**SObject Mutations — production** - -``` -https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations -``` - -**SObject Mutations — sandbox** - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations -``` - -**SObject Deletes — production** - -``` -https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes -``` - -**SObject Deletes — sandbox** - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes -``` - -**SObject All — production** - -``` -https://api.salesforce.com/platform/mcp/v1/platform/sobject-all -``` - -**SObject All — sandbox** - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all -``` - -**Data 360 — production** - -``` -https://api.salesforce.com/platform/mcp/v1/data/data360 -``` - -**Data 360 — sandbox** - -``` -https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360 -``` - -**Headless 360 (Beta) — production** - -``` -https://api.salesforce.com/platform/mcp/v1/platform/headless-360 -``` - -**Headless 360 (Beta) — sandbox** - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360 -``` - -**Tableau Next — production** + -``` -https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next -``` +If you created your own app, continue to [Speakeasy setup](speakeasy.md#add-server-in-speakeasy) with your selected URL and **Consumer Key**. If you installed Speakeasy's app, continue with Speakeasy support. -**Tableau Next — sandbox** +## Endpoint reference -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next -``` +Use the exact URL for your server and org type. Data 360 places `/sandbox` after `/data`; the other servers place it after `/v1`. -If the connection fails with valid credentials, confirm that the selected server is enabled, the URL matches the server and org type, and the org has API access. +| Server / API ID | Production URL | Sandbox URL | Access and prerequisites | +| --- | --- | --- | --- | +| SObject Reads (`sobject-reads`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads` | Discovery, query, search, and relationship traversal; no record changes. | +| SObject Mutations (`sobject-mutations`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations` | Read, create, and update records; no deletes. | +| SObject Deletes (`sobject-deletes`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes` | Identify and delete records; no creates or updates. | +| SObject All (`sobject-all`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-all` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all` | Create, read, update, delete, query, and search records. | +| Data 360 (`data360`) | `https://api.salesforce.com/platform/mcp/v1/data/data360` | `https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360` | Query data and change customer-data configuration. Requires a Data 360 license, API v66.0+, and **Manage Data 360** for configuration or **View Data 360** for read-only operations. | +| Headless 360 (Beta) (`platform/headless-360`) | `https://api.salesforce.com/platform/mcp/v1/platform/headless-360` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360` | Broad Setup and platform operations, not read-only record access. Available starting July 2026 under Beta Services Terms. Requires API v67.0+, an External Client App with `mcp_api`, and an OAuth client. | +| Tableau Next (`analytics/tableau-next`) | `https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next` | `https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next` | Semantic-model and analytics access. Confirm the org has the required Tableau Next capabilities. | - +Calls remain subject to the signed-in user's field-level security, object permissions, and sharing rules. If the connection fails with valid credentials, confirm that the selected server is enabled, the URL matches the server and org type, and the org has API access. diff --git a/guides/salesforce/speakeasy.md b/guides/salesforce/speakeasy.md index e11ae2e..ea7c520 100644 --- a/guides/salesforce/speakeasy.md +++ b/guides/salesforce/speakeasy.md @@ -1,8 +1,6 @@ # Speakeasy setup -**Method 1 — Speakeasy application:** after [installing the application](external.md#install-speakeasy-application), you must [contact Speakeasy support to finish OAuth](external.md#contact-speakeasy-support). Coordinate server activation and the add-server step below with support. Do not use the manual credential instructions for this method. - -**Method 2 — Your own External Client App:** after creating the app and enabling the selected server, follow both steps below. +Follow these steps after [creating your own Salesforce app](external.md#create-your-own-salesforce-app) and [enabling your selected MCP server](external.md#enable-sobject-server). If you installed Speakeasy's Salesforce app instead, [contact Speakeasy support to finish OAuth](external.md#contact-speakeasy-support). ### Add the server in Speakeasy {#add-server-in-speakeasy} @@ -18,9 +16,7 @@ This creates the hosted MCP server and opens its **Overview** page. ### Connect your credentials {#connect-speakeasy-credentials} -**Method 1 — Speakeasy application:** finish OAuth through the mandatory [Speakeasy support handoff](external.md#contact-speakeasy-support), not the fields below. - -**Method 2 — Your own External Client App only:** +If attachment still fails after the app's 30-minute activation window, stop and escalate; do not change the OAuth settings. 1. From the server's **Overview**, open **Settings**. 2. Under **Authentication**, select **Configure Manually**. @@ -28,15 +24,13 @@ This creates the hosted MCP server and opens its **Overview** page. The sheet shows the **Redirect URI** with a copy button. It is the callback URL registered in Salesforce as `{{ gram.oauth.callback_url }}`. -The Consumer Key-only mapping below is unverified. Salesforce documents it for compatible public clients but not for the Speakeasy AI Control Plane. - 4. Paste the [**Consumer Key**](external.md#copy-consumer-key) into **Client ID**. 5. Leave **Client Secret (optional)** empty. 6. Select **Attach Identity Provider**. 7. Confirm that the sheet's **Redirect URI** matches the `{{ gram.oauth.callback_url }}` value registered in [**Callback URL**](external.md#configure-oauth-settings). -If attaching still fails after Salesforce's documented 30-minute app propagation window, stop and escalate instead of changing the candidate configuration. - +These steps configure the server and attach its identity provider; they do not verify Salesforce authorization or a working connection. + This guide covers setup only. For anything beyond it — billing, tool behavior, limits — see [Salesforce's MCP documentation](https://developer.salesforce.com/docs/platform/hosted-mcp-servers/guide/hosted-mcp-servers-overview.html). From 0029f41fb8f0691714369d5a7c3ed8778878ad21 Mon Sep 17 00:00:00 2001 From: Walker Lockard Date: Thu, 17 Sep 2026 11:30:09 -0700 Subject: [PATCH 5/6] docs: make Salesforce setup URLs copyable --- guides/salesforce/external.md | 126 +++++++++++++++++++++++++++++++--- 1 file changed, 116 insertions(+), 10 deletions(-) diff --git a/guides/salesforce/external.md b/guides/salesforce/external.md index 44809aa..7579a38 100644 --- a/guides/salesforce/external.md +++ b/guides/salesforce/external.md @@ -27,7 +27,11 @@ Choose one path: **use Speakeasy's Salesforce app** and finish OAuth with suppor ### Install Speakeasy's Salesforce application {#install-speakeasy-application} -Install Speakeasy's Salesforce application into the intended org using [the Speakeasy Salesforce app installation page](https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2). +Open this installation URL to install Speakeasy's Salesforce application into the intended org: + +``` +https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA2 +``` @@ -116,14 +120,116 @@ If you created your own app, continue to [Speakeasy setup](speakeasy.md#add-serv Use the exact URL for your server and org type. Data 360 places `/sandbox` after `/data`; the other servers place it after `/v1`. -| Server / API ID | Production URL | Sandbox URL | Access and prerequisites | -| --- | --- | --- | --- | -| SObject Reads (`sobject-reads`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads` | Discovery, query, search, and relationship traversal; no record changes. | -| SObject Mutations (`sobject-mutations`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations` | Read, create, and update records; no deletes. | -| SObject Deletes (`sobject-deletes`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes` | Identify and delete records; no creates or updates. | -| SObject All (`sobject-all`) | `https://api.salesforce.com/platform/mcp/v1/platform/sobject-all` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all` | Create, read, update, delete, query, and search records. | -| Data 360 (`data360`) | `https://api.salesforce.com/platform/mcp/v1/data/data360` | `https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360` | Query data and change customer-data configuration. Requires a Data 360 license, API v66.0+, and **Manage Data 360** for configuration or **View Data 360** for read-only operations. | -| Headless 360 (Beta) (`platform/headless-360`) | `https://api.salesforce.com/platform/mcp/v1/platform/headless-360` | `https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360` | Broad Setup and platform operations, not read-only record access. Available starting July 2026 under Beta Services Terms. Requires API v67.0+, an External Client App with `mcp_api`, and an OAuth client. | -| Tableau Next (`analytics/tableau-next`) | `https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next` | `https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next` | Semantic-model and analytics access. Confirm the org has the required Tableau Next capabilities. | +**SObject Reads (sobject-reads)** + +Discovery, query, search, and relationship traversal; no record changes. + +Production: + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads +``` + +Sandbox: + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads +``` + +**SObject Mutations (sobject-mutations)** + +Read, create, and update records; no deletes. + +Production: + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations +``` + +Sandbox: + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations +``` + +**SObject Deletes (sobject-deletes)** + +Identify and delete records; no creates or updates. + +Production: + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes +``` + +Sandbox: + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes +``` + +**SObject All (sobject-all)** + +Create, read, update, delete, query, and search records. + +Production: + +``` +https://api.salesforce.com/platform/mcp/v1/platform/sobject-all +``` + +Sandbox: + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all +``` + +**Data 360 (data360)** + +Query data and change customer-data configuration. Requires a Data 360 license, API v66.0+, and **Manage Data 360** for configuration or **View Data 360** for read-only operations. + +Production: + +``` +https://api.salesforce.com/platform/mcp/v1/data/data360 +``` + +Sandbox: + +``` +https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360 +``` + +**Headless 360 (Beta) (platform/headless-360)** + +Broad Setup and platform operations, not read-only record access. Available starting July 2026 under Beta Services Terms. Requires API v67.0+, an External Client App with `mcp_api`, and an OAuth client. + +Production: + +``` +https://api.salesforce.com/platform/mcp/v1/platform/headless-360 +``` + +Sandbox: + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360 +``` + +**Tableau Next (analytics/tableau-next)** + +Semantic-model and analytics access. Confirm the org has the required Tableau Next capabilities. + +Production: + +``` +https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next +``` + +Sandbox: + +``` +https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next +``` Calls remain subject to the signed-in user's field-level security, object permissions, and sharing rules. If the connection fails with valid credentials, confirm that the selected server is enabled, the URL matches the server and org type, and the org has API access. From 54838e64ce0aaa185673857174dd7ec0319ff350 Mon Sep 17 00:00:00 2001 From: Walker Lockard Date: Thu, 17 Sep 2026 11:35:44 -0700 Subject: [PATCH 6/6] docs: show only production Salesforce endpoint URLs --- guides/salesforce/external.md | 66 +++-------------------------------- 1 file changed, 5 insertions(+), 61 deletions(-) diff --git a/guides/salesforce/external.md b/guides/salesforce/external.md index 7579a38..031efb6 100644 --- a/guides/salesforce/external.md +++ b/guides/salesforce/external.md @@ -4,7 +4,7 @@ setup_version: 1 # Connect Salesforce to the Speakeasy AI Control Plane -Use Salesforce System Administrator credentials for an API-enabled production or sandbox org where Hosted MCP Servers are available. Salesforce documents availability for Enterprise Edition and above. You need authority to install the Speakeasy application or create an **External Client App**, and enable Hosted MCP Servers. +Use Salesforce System Administrator credentials for an API-enabled production org where Hosted MCP Servers are available. Salesforce documents availability for Enterprise Edition and above. You need authority to install the Speakeasy application or create an **External Client App**, and enable Hosted MCP Servers. Sign in to the Salesforce org you want to connect. Install or create the app in that same org. This guide does not cover scratch orgs. For a lower-edition org, confirm Hosted MCP availability in [Salesforce Setup](#open-salesforce-setup) before starting either path. @@ -37,7 +37,7 @@ https://login.salesforce.com/packaging/installPackage.apexp?p0=04tdM000000cNGXQA ### Contact Speakeasy support to finish OAuth {#contact-speakeasy-support} -After installation, **contact Speakeasy support to finish OAuth setup**. Installation alone does not complete OAuth. Coordinate your production or sandbox endpoint and [server activation](#enable-sobject-server) with support. Your next step is with support—not the app-creation walkthrough below. +After installation, **contact Speakeasy support to finish OAuth setup**. Installation alone does not complete OAuth. Coordinate your selected endpoint and [server activation](#enable-sobject-server) with support. Your next step is with support—not the app-creation walkthrough below. @@ -109,7 +109,7 @@ Choose the least-privileged server that meets your team's needs using the endpoi 3. Select **MCP Servers** under **API Catalog**. 4. Find the server whose API ID matches the approved choice. 5. Use the available control to enable that server. For Headless 360, find `headless-360` and select **Activate**. -6. Record its URL from the endpoint reference below, using the production or sandbox form that matches the org. +6. Record its URL from the endpoint reference below. 7. Wait up to two minutes for the server to become active. @@ -118,118 +118,62 @@ If you created your own app, continue to [Speakeasy setup](speakeasy.md#add-serv ## Endpoint reference -Use the exact URL for your server and org type. Data 360 places `/sandbox` after `/data`; the other servers place it after `/v1`. +The following endpoints are for production orgs. Copy the URL for your selected server. **SObject Reads (sobject-reads)** Discovery, query, search, and relationship traversal; no record changes. -Production: - ``` https://api.salesforce.com/platform/mcp/v1/platform/sobject-reads ``` -Sandbox: - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-reads -``` - **SObject Mutations (sobject-mutations)** Read, create, and update records; no deletes. -Production: - ``` https://api.salesforce.com/platform/mcp/v1/platform/sobject-mutations ``` -Sandbox: - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-mutations -``` - **SObject Deletes (sobject-deletes)** Identify and delete records; no creates or updates. -Production: - ``` https://api.salesforce.com/platform/mcp/v1/platform/sobject-deletes ``` -Sandbox: - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-deletes -``` - **SObject All (sobject-all)** Create, read, update, delete, query, and search records. -Production: - ``` https://api.salesforce.com/platform/mcp/v1/platform/sobject-all ``` -Sandbox: - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/sobject-all -``` - **Data 360 (data360)** Query data and change customer-data configuration. Requires a Data 360 license, API v66.0+, and **Manage Data 360** for configuration or **View Data 360** for read-only operations. -Production: - ``` https://api.salesforce.com/platform/mcp/v1/data/data360 ``` -Sandbox: - -``` -https://api.salesforce.com/platform/mcp/v1/data/sandbox/data360 -``` - **Headless 360 (Beta) (platform/headless-360)** Broad Setup and platform operations, not read-only record access. Available starting July 2026 under Beta Services Terms. Requires API v67.0+, an External Client App with `mcp_api`, and an OAuth client. -Production: - ``` https://api.salesforce.com/platform/mcp/v1/platform/headless-360 ``` -Sandbox: - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/platform/headless-360 -``` - **Tableau Next (analytics/tableau-next)** Semantic-model and analytics access. Confirm the org has the required Tableau Next capabilities. -Production: - ``` https://api.salesforce.com/platform/mcp/v1/analytics/tableau-next ``` -Sandbox: - -``` -https://api.salesforce.com/platform/mcp/v1/sandbox/analytics/tableau-next -``` - -Calls remain subject to the signed-in user's field-level security, object permissions, and sharing rules. If the connection fails with valid credentials, confirm that the selected server is enabled, the URL matches the server and org type, and the org has API access. +Calls remain subject to the signed-in user's field-level security, object permissions, and sharing rules. If the connection fails with valid credentials, confirm that the selected server is enabled, the URL matches the selected server, and the org has API access.