From 03724be1cbf4e7566289a8aeedc8a9f63ccaa1cc Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 24 Aug 2026 17:48:21 +0000 Subject: [PATCH 1/6] Centralize engine default domain sets Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --- .github/aw/network.md | 15 ++ docs/src/content/docs/reference/network.md | 15 ++ pkg/workflow/data/ecosystem_domains.json | 11 - pkg/workflow/domains.go | 221 ++++++++++++--------- pkg/workflow/domains_test.go | 19 +- 5 files changed, 177 insertions(+), 104 deletions(-) diff --git a/.github/aw/network.md b/.github/aw/network.md index eb366d188bf..d02788e7dd4 100644 --- a/.github/aw/network.md +++ b/.github/aw/network.md @@ -87,6 +87,21 @@ Keywords expanding to curated domain lists: | `python-native` | Python native build deps | Native toolchain mirrors for building Python packages from source | | `copilot-vendor` | Copilot plan-specific APIs / telemetry | `api.business.githubcopilot.com`, `api.enterprise.githubcopilot.com`, `api.individual.githubcopilot.com`, `telemetry.enterprise.githubcopilot.com` | +## Automatic Engine Domain Sets + +Each engine automatically receives the domain set it requires in addition to +`network.allowed`. These named sets are maintained by the compiler for analysis +and reporting; they are not valid `network.allowed` identifiers. + +| Engine set | Included domains | +|---|---| +| `copilot` | `api.github.com`, `api.githubcopilot.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com` | +| `claude` | Anthropic APIs, GitHub transport, certificate/OCSP services, Ubuntu package metadata, Playwright downloads, and `host.docker.internal` | +| `codex` | `172.30.0.1`, `api.github.com`, `api.openai.com`, `chatgpt.com`, `github.com`, `host.docker.internal`, `openai.com` | +| `gemini` | `*.googleapis.com`, `generativelanguage.googleapis.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com` | +| `pi` | `api.githubcopilot.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com`; provider-scoped models replace the API host with the selected provider endpoint | +| `threat-detection` | Copilot API and telemetry hosts, `api.github.com`, `github.com`, `host.docker.internal`, and `registry.npmjs.org` for read-only lockfile validation | + ## Invalid Shorthands These look like ecosystem identifiers but are **not recognised** — using them causes a **compile-time error**: diff --git a/docs/src/content/docs/reference/network.md b/docs/src/content/docs/reference/network.md index be26109878c..13d16e19d5a 100644 --- a/docs/src/content/docs/reference/network.md +++ b/docs/src/content/docs/reference/network.md @@ -116,6 +116,21 @@ Mix ecosystem identifiers with specific domains for fine-grained control: | `swift` | Swift packages (`swift.org`, `cocoapods.org`) | | `zig` | Zig packages (`ziglang.org`) | +## Automatic Engine Domain Sets + +Each engine automatically receives the domain set it requires in addition to +`network.allowed`. These named sets are maintained by the compiler for analysis +and reporting; they are not valid `network.allowed` identifiers. + +| Engine set | Included domains | +|---|---| +| `copilot` | `api.github.com`, `api.githubcopilot.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com` | +| `claude` | Anthropic APIs, GitHub transport, certificate/OCSP services, Ubuntu package metadata, Playwright downloads, and `host.docker.internal` | +| `codex` | `172.30.0.1`, `api.github.com`, `api.openai.com`, `chatgpt.com`, `github.com`, `host.docker.internal`, `openai.com` | +| `gemini` | `*.googleapis.com`, `generativelanguage.googleapis.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com` | +| `pi` | `api.githubcopilot.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com`; provider-scoped models replace the API host with the selected provider endpoint | +| `threat-detection` | Copilot API and telemetry hosts, `api.github.com`, `github.com`, `host.docker.internal`, and `registry.npmjs.org` for read-only lockfile validation | + ### Ecosystem Identifier Validation Single-word entries in `network.allowed` that match the ecosystem identifier pattern (`[a-z][a-z0-9-]*`) are validated against the known ecosystem list at compile time. An unrecognized identifier produces a compilation error with the full list of valid options: diff --git a/pkg/workflow/data/ecosystem_domains.json b/pkg/workflow/data/ecosystem_domains.json index 09c573e2bb1..4675bd44ead 100644 --- a/pkg/workflow/data/ecosystem_domains.json +++ b/pkg/workflow/data/ecosystem_domains.json @@ -257,16 +257,5 @@ "scala": ["repo.scala-sbt.org", "scala-ci.typesafe.com", "repo.typesafe.com", "jitpack.io", "dl.bintray.com", "scala.jfrog.io"], "swift": ["download.swift.org", "swift.org", "cocoapods.org", "cdn.cocoapods.org"], "terraform": ["releases.hashicorp.com", "apt.releases.hashicorp.com", "yum.releases.hashicorp.com", "registry.terraform.io"], - "threat-detection": [ - "api.business.githubcopilot.com", - "api.enterprise.githubcopilot.com", - "api.github.com", - "api.githubcopilot.com", - "api.individual.githubcopilot.com", - "github.com", - "host.docker.internal", - "registry.npmjs.org", - "telemetry.enterprise.githubcopilot.com" - ], "zig": ["ziglang.org", "pkg.machengine.org", "deps.files.ghostty.org"] } diff --git a/pkg/workflow/domains.go b/pkg/workflow/domains.go index 1baa9c3cab0..ee47a627fad 100644 --- a/pkg/workflow/domains.go +++ b/pkg/workflow/domains.go @@ -65,6 +65,124 @@ func getLoadedEcosystemDomains() map[string][]string { // to an engine default and that test starts failing, the domain belongs behind an explicit // ecosystem/runtime opt-in instead, not in the unconditional default list. +// engineDefaultDomainSets centralizes every unconditional engine allow-list. These +// sets are intentionally distinct from user-selectable ecosystem domain lists: the +// compiler includes them automatically for the matching engine, rather than allowing +// users to select them through network.allowed. +// +// Engine default domain lists intentionally exclude package registries (npm, PyPI, +// and similar). See the package-registry invariant above. +var engineDefaultDomainSets = map[string][]string{ + "claude": { + "*.githubusercontent.com", + "anthropic.com", + "api.anthropic.com", + "api.github.com", + "api.snapcraft.io", + "archive.ubuntu.com", + "azure.archive.ubuntu.com", + "cdn.playwright.dev", + "codeload.github.com", + "crl.geotrust.com", + "crl.globalsign.com", + "crl.identrust.com", + "crl.sectigo.com", + "crl.thawte.com", + "crl.usertrust.com", + "crl.verisign.com", + "crl3.digicert.com", + "crl4.digicert.com", + "crls.ssl.com", + "ghcr.io", + "github-cloud.githubusercontent.com", + "github-cloud.s3.amazonaws.com", + "github.com", + "host.docker.internal", + "json-schema.org", + "json.schemastore.org", + "keyserver.ubuntu.com", + "lfs.github.com", + "objects.githubusercontent.com", + "ocsp.digicert.com", + "ocsp.geotrust.com", + "ocsp.globalsign.com", + "ocsp.identrust.com", + "ocsp.sectigo.com", + "ocsp.ssl.com", + "ocsp.thawte.com", + "ocsp.usertrust.com", + "ocsp.verisign.com", + "packagecloud.io", + "packages.cloud.google.com", + "packages.microsoft.com", + "playwright.download.prss.microsoft.com", + "ppa.launchpad.net", + "raw.githubusercontent.com", + "s.symcb.com", + "s.symcd.com", + "security.ubuntu.com", + "sentry.io", + "statsig.anthropic.com", + "ts-crl.ws.symantec.com", + "ts-ocsp.ws.symantec.com", + }, + "codex": { + "172.30.0.1", // AWF gateway IP - Codex resolves host.docker.internal to this IP for Rust DNS compatibility + "api.github.com", // Codex startup performs GitHub plugin sync requests against the GitHub API + "api.openai.com", + "chatgpt.com", // Codex CLI connects to chatgpt.com (and subdomains e.g. ab.chatgpt.com) for auth/telemetry + "github.com", // Codex startup accesses GitHub-hosted plugin metadata pages + "host.docker.internal", + "openai.com", + }, + "copilot": { + "api.github.com", + "api.githubcopilot.com", + "github.com", + "host.docker.internal", + "raw.githubusercontent.com", + }, + "gemini": { + "*.googleapis.com", + "generativelanguage.googleapis.com", + "github.com", + "host.docker.internal", + "raw.githubusercontent.com", + }, + "pi": { + "api.githubcopilot.com", // Default provider (Copilot routing) + "host.docker.internal", + "github.com", + "raw.githubusercontent.com", + }, + "pi-base": { + "host.docker.internal", // MCP gateway / API proxy access + "github.com", + "raw.githubusercontent.com", + }, + "threat-detection": { + "api.business.githubcopilot.com", + "api.enterprise.githubcopilot.com", + "api.github.com", + "api.githubcopilot.com", + "api.individual.githubcopilot.com", + "github.com", + "host.docker.internal", + "registry.npmjs.org", + "telemetry.enterprise.githubcopilot.com", + }, +} + +// GetEngineDefaultDomainSets returns copies of the named engine domain sets for +// analysis and reporting. Engine domain sets are not valid network.allowed values. +func GetEngineDefaultDomainSets() map[string][]string { + sets := make(map[string][]string, len(engineDefaultDomainSets)) + for name, domains := range engineDefaultDomainSets { + sets[name] = append([]string(nil), domains...) + } + return sets +} + // CopilotDefaultDomains are the default domains required for GitHub Copilot CLI authentication and operation. // // This list is limited to the shared gateway/GitHub transport baseline: the MCP/API gateway @@ -72,97 +190,21 @@ func getLoadedEcosystemDomains() map[string][]string { // inference. Plan-specific Copilot API hosts (business/enterprise/individual) and Copilot // telemetry are *not* part of the default set: agents route inference through the AWF api-proxy, // so those vendor hosts require an explicit `network: { allowed: [copilot-vendor] }` opt-in. -var CopilotDefaultDomains = []string{ - "api.github.com", - "api.githubcopilot.com", - "github.com", - "host.docker.internal", - "raw.githubusercontent.com", -} +var CopilotDefaultDomains = engineDefaultDomainSets["copilot"] -// CodexDefaultDomains are the minimal default domains required for Codex CLI operation -var CodexDefaultDomains = []string{ - "172.30.0.1", // AWF gateway IP - Codex resolves host.docker.internal to this IP for Rust DNS compatibility - "api.github.com", // Codex startup performs GitHub plugin sync requests against the GitHub API - "api.openai.com", - "chatgpt.com", // Codex CLI connects to chatgpt.com (and subdomains e.g. ab.chatgpt.com) for auth/telemetry - "github.com", // Codex startup accesses GitHub-hosted plugin metadata pages - "host.docker.internal", - "openai.com", -} +// CodexDefaultDomains are the minimal default domains required for Codex CLI operation. +var CodexDefaultDomains = engineDefaultDomainSets["codex"] -// ClaudeDefaultDomains are the default domains required for Claude Code CLI authentication and operation -var ClaudeDefaultDomains = []string{ - "*.githubusercontent.com", - "anthropic.com", - "api.anthropic.com", - "api.github.com", - "api.snapcraft.io", - "archive.ubuntu.com", - "azure.archive.ubuntu.com", - "cdn.playwright.dev", - "codeload.github.com", - "crl.geotrust.com", - "crl.globalsign.com", - "crl.identrust.com", - "crl.sectigo.com", - "crl.thawte.com", - "crl.usertrust.com", - "crl.verisign.com", - "crl3.digicert.com", - "crl4.digicert.com", - "crls.ssl.com", - "ghcr.io", - "github-cloud.githubusercontent.com", - "github-cloud.s3.amazonaws.com", - "github.com", - "host.docker.internal", - "json-schema.org", - "json.schemastore.org", - "keyserver.ubuntu.com", - "lfs.github.com", - "objects.githubusercontent.com", - "ocsp.digicert.com", - "ocsp.geotrust.com", - "ocsp.globalsign.com", - "ocsp.identrust.com", - "ocsp.sectigo.com", - "ocsp.ssl.com", - "ocsp.thawte.com", - "ocsp.usertrust.com", - "ocsp.verisign.com", - "packagecloud.io", - "packages.cloud.google.com", - "packages.microsoft.com", - "playwright.download.prss.microsoft.com", - "ppa.launchpad.net", - "raw.githubusercontent.com", - "s.symcb.com", - "s.symcd.com", - "security.ubuntu.com", - "sentry.io", - "statsig.anthropic.com", - "ts-crl.ws.symantec.com", - "ts-ocsp.ws.symantec.com", -} +// ClaudeDefaultDomains are the default domains required for Claude Code CLI authentication and operation. +var ClaudeDefaultDomains = engineDefaultDomainSets["claude"] // GeminiDefaultDomains are the default domains required for Google Gemini CLI authentication and operation. -var GeminiDefaultDomains = []string{ - "*.googleapis.com", - "generativelanguage.googleapis.com", - "github.com", - "host.docker.internal", - "raw.githubusercontent.com", -} +var GeminiDefaultDomains = engineDefaultDomainSets["gemini"] // PiBaseDefaultDomains are the base domains required for the Pi CLI to operate, // independent of the chosen LLM provider. When a model uses provider/model format, // provider-specific API domains are added on top via GetDefaultDomainsForEngine. -var PiBaseDefaultDomains = []string{ - "host.docker.internal", // MCP gateway / API proxy access - "github.com", - "raw.githubusercontent.com", -} +var PiBaseDefaultDomains = engineDefaultDomainSets["pi-base"] // piProviderDomains maps provider prefixes to their API domains. // Covers the same set of providers that Pi can route through via the AWF LLM gateway. @@ -180,12 +222,7 @@ var piProviderDomains = map[string]string{ // PiDefaultDomains are the static default domains for backward compatibility when // no model provider prefix is given. When a provider/model format is used, the // dynamic path (GetDefaultDomainsForEngine) resolves provider-specific domains instead. -var PiDefaultDomains = []string{ - "api.githubcopilot.com", // Default provider (Copilot routing) - "host.docker.internal", - "github.com", - "raw.githubusercontent.com", -} +var PiDefaultDomains = engineDefaultDomainSets["pi"] // extractProviderFromModel parses "provider/model" format and returns the // lowercase provider prefix. Returns ("", nil) when no model is given or the @@ -752,8 +789,8 @@ func GetAllowedDomainsForEngine(engine constants.EngineName, network *NetworkPer } // GetThreatDetectionAllowedDomains returns the minimal set of domains allowed for a Copilot -// detection run. It loads the "threat-detection" ecosystem from ecosystem_domains.json, which -// includes the Copilot API endpoints needed for read-only threat analysis plus registry.npmjs.org +// detection run. The "threat-detection" engine domain set includes the Copilot API endpoints +// needed for read-only threat analysis plus registry.npmjs.org // for read-only npm package validation (e.g. verifying lockfile integrity hashes). It intentionally // excludes raw.githubusercontent.com (not needed when MCP servers are disabled and the CLI binary // is pre-installed). npm registry access is read-only metadata lookup only — installs are not @@ -761,7 +798,7 @@ func GetAllowedDomainsForEngine(engine constants.EngineName, network *NetworkPer // Any additional user-specified network.allowed entries are merged in (typically empty for detection). // Returns a deduplicated, sorted, comma-separated string suitable for AWF's --allow-domains flag. func GetThreatDetectionAllowedDomains(network *NetworkPermissions) string { - detectionDomains := getEcosystemDomains("threat-detection") + detectionDomains := engineDefaultDomainSets["threat-detection"] // Pass nil tools and runtimes: detection runs with no npm/runtime ecosystem, so // ecosystem domain expansion is intentionally skipped. return mergeDomainsWithNetworkToolsAndRuntimes(detectionDomains, network, nil, nil) diff --git a/pkg/workflow/domains_test.go b/pkg/workflow/domains_test.go index e6f9b65e10e..1621342fcbc 100644 --- a/pkg/workflow/domains_test.go +++ b/pkg/workflow/domains_test.go @@ -409,7 +409,7 @@ func TestCopilotVendorDomainsRequireOptIn(t *testing.T) { } func TestThreatDetectionDomains(t *testing.T) { - detectionDomains := getEcosystemDomains("threat-detection") + detectionDomains := GetEngineDefaultDomainSets()["threat-detection"] // Detection domains must include every required Copilot API domain requiredDomains := []string{ @@ -423,6 +423,7 @@ func TestThreatDetectionDomains(t *testing.T) { "registry.npmjs.org", "telemetry.enterprise.githubcopilot.com", } + detectionMap := make(map[string]bool) for _, d := range detectionDomains { detectionMap[d] = true @@ -444,6 +445,22 @@ func TestThreatDetectionDomains(t *testing.T) { "threat-detection ecosystem should have exactly %d entries", len(requiredDomains)) } +func TestGetEngineDefaultDomainSets(t *testing.T) { + sets := GetEngineDefaultDomainSets() + + assert.Equal(t, CopilotDefaultDomains, sets["copilot"]) + assert.Equal(t, ClaudeDefaultDomains, sets["claude"]) + assert.Equal(t, CodexDefaultDomains, sets["codex"]) + assert.Equal(t, GeminiDefaultDomains, sets["gemini"]) + assert.Equal(t, PiBaseDefaultDomains, sets["pi-base"]) + assert.Equal(t, PiDefaultDomains, sets["pi"]) + assert.NotEmpty(t, sets["threat-detection"]) + + sets["copilot"][0] = "modified.example.com" + assert.NotEqual(t, "modified.example.com", CopilotDefaultDomains[0], + "callers must not be able to modify registered domain sets") +} + func TestGetThreatDetectionAllowedDomains(t *testing.T) { // With empty network permissions the result equals the sorted detection domains result := GetThreatDetectionAllowedDomains(&NetworkPermissions{Allowed: []string{}}) From 7dc9cba8329d3749b2771b8d9691f4e7c0bc5e0b Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 24 Aug 2026 17:51:17 +0000 Subject: [PATCH 2/6] Protect registered engine domain sets Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --- pkg/workflow/domains.go | 18 +++++++++++------- pkg/workflow/domains_test.go | 8 ++++++++ 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/pkg/workflow/domains.go b/pkg/workflow/domains.go index ee47a627fad..ef06ee8ce4f 100644 --- a/pkg/workflow/domains.go +++ b/pkg/workflow/domains.go @@ -178,11 +178,15 @@ var engineDefaultDomainSets = map[string][]string{ func GetEngineDefaultDomainSets() map[string][]string { sets := make(map[string][]string, len(engineDefaultDomainSets)) for name, domains := range engineDefaultDomainSets { - sets[name] = append([]string(nil), domains...) + sets[name] = copyEngineDefaultDomainSet(domains) } return sets } +func copyEngineDefaultDomainSet(domains []string) []string { + return append([]string(nil), domains...) +} + // CopilotDefaultDomains are the default domains required for GitHub Copilot CLI authentication and operation. // // This list is limited to the shared gateway/GitHub transport baseline: the MCP/API gateway @@ -190,21 +194,21 @@ func GetEngineDefaultDomainSets() map[string][]string { // inference. Plan-specific Copilot API hosts (business/enterprise/individual) and Copilot // telemetry are *not* part of the default set: agents route inference through the AWF api-proxy, // so those vendor hosts require an explicit `network: { allowed: [copilot-vendor] }` opt-in. -var CopilotDefaultDomains = engineDefaultDomainSets["copilot"] +var CopilotDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["copilot"]) // CodexDefaultDomains are the minimal default domains required for Codex CLI operation. -var CodexDefaultDomains = engineDefaultDomainSets["codex"] +var CodexDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["codex"]) // ClaudeDefaultDomains are the default domains required for Claude Code CLI authentication and operation. -var ClaudeDefaultDomains = engineDefaultDomainSets["claude"] +var ClaudeDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["claude"]) // GeminiDefaultDomains are the default domains required for Google Gemini CLI authentication and operation. -var GeminiDefaultDomains = engineDefaultDomainSets["gemini"] +var GeminiDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["gemini"]) // PiBaseDefaultDomains are the base domains required for the Pi CLI to operate, // independent of the chosen LLM provider. When a model uses provider/model format, // provider-specific API domains are added on top via GetDefaultDomainsForEngine. -var PiBaseDefaultDomains = engineDefaultDomainSets["pi-base"] +var PiBaseDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["pi-base"]) // piProviderDomains maps provider prefixes to their API domains. // Covers the same set of providers that Pi can route through via the AWF LLM gateway. @@ -222,7 +226,7 @@ var piProviderDomains = map[string]string{ // PiDefaultDomains are the static default domains for backward compatibility when // no model provider prefix is given. When a provider/model format is used, the // dynamic path (GetDefaultDomainsForEngine) resolves provider-specific domains instead. -var PiDefaultDomains = engineDefaultDomainSets["pi"] +var PiDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["pi"]) // extractProviderFromModel parses "provider/model" format and returns the // lowercase provider prefix. Returns ("", nil) when no model is given or the diff --git a/pkg/workflow/domains_test.go b/pkg/workflow/domains_test.go index 1621342fcbc..d0bae77e171 100644 --- a/pkg/workflow/domains_test.go +++ b/pkg/workflow/domains_test.go @@ -459,6 +459,14 @@ func TestGetEngineDefaultDomainSets(t *testing.T) { sets["copilot"][0] = "modified.example.com" assert.NotEqual(t, "modified.example.com", CopilotDefaultDomains[0], "callers must not be able to modify registered domain sets") + + original := CopilotDefaultDomains[0] + t.Cleanup(func() { + CopilotDefaultDomains[0] = original + }) + CopilotDefaultDomains[0] = "modified.example.com" + assert.Equal(t, original, GetEngineDefaultDomainSets()["copilot"][0], + "exported compatibility variables must not modify registered domain sets") } func TestGetThreatDetectionAllowedDomains(t *testing.T) { From 3cd8e226d584535639b536a8be66dd0d068d20b1 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 24 Aug 2026 18:10:56 +0000 Subject: [PATCH 3/6] docs(adr): add draft ADR-55461 for centralizing engine default domain sets Co-Authored-By: Claude Sonnet 4.6 --- ...1-centralize-engine-default-domain-sets.md | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 docs/adr/55461-centralize-engine-default-domain-sets.md diff --git a/docs/adr/55461-centralize-engine-default-domain-sets.md b/docs/adr/55461-centralize-engine-default-domain-sets.md new file mode 100644 index 00000000000..6add95f2a6d --- /dev/null +++ b/docs/adr/55461-centralize-engine-default-domain-sets.md @@ -0,0 +1,46 @@ +# ADR-55461: Centralize Engine Default Domain Sets + +**Date**: 2026-08-24 +**Status**: Draft +**Deciders**: Unknown + +--- + +### Context + +Engine-specific unconditional domain allow-lists were scattered across multiple separate static Go variables (`CopilotDefaultDomains`, `CodexDefaultDomains`, `ClaudeDefaultDomains`, `GeminiDefaultDomains`, `PiDefaultDomains`, `PiBaseDefaultDomains`) and an `ecosystem_domains.json` file that also served as the user-configurable ecosystem registry. This made it impossible to enumerate or inspect the full set of engine defaults in one place, and it conflated engine-internal allow-lists (automatically injected by the compiler) with user-selectable ecosystem identifiers (explicitly opted into via `network.allowed`). Specifically, the threat-detection allow-list being stored in the ecosystem JSON meant users could theoretically select it as a `network.allowed` entry, which was never the intended behavior. + +### Decision + +We will centralize all engine-specific unconditional domain allow-lists into a single unexported package-level map (`engineDefaultDomainSets`) and expose a copy-returning public accessor (`GetEngineDefaultDomainSets()`) for analysis and reporting. Existing exported compatibility variables (`CopilotDefaultDomains`, etc.) will derive their values from this registry at initialization time via a `copyEngineDefaultDomainSet` helper. The threat-detection allow-list will be removed from `ecosystem_domains.json` and moved exclusively into the new registry, preventing it from being user-selectable. + +### Alternatives Considered + +#### Alternative 1: Keep separate static variables, add an aggregation function + +Maintain each engine's domain list as its own `var` but introduce a function that aggregates them into a map for reporting. This avoids any initialization-time dependency between the registry and the compatibility variables, but leaves the domain lists distributed across the file. It does not solve the fundamental issue of drift between copies and fails to provide the single source of truth needed for the compiler to automatically reference them. + +#### Alternative 2: Store all engine domain lists in a JSON/YAML configuration file + +Move all engine domain lists to a structured data file (similar to `ecosystem_domains.json`) and load them at runtime. This would make the lists editable without recompilation and easily inspectable, but introduces parse-at-startup overhead, a potential initialization failure path, and blurs the distinction between code-managed engine defaults and data-managed ecosystem lists. The type safety and inline documentation benefits of Go code are also lost. + +### Consequences + +#### Positive +- Single authoritative source (`engineDefaultDomainSets`) for all engine allow-lists prevents content drift between the registry and the exported compatibility variables. +- `GetEngineDefaultDomainSets()` enables programmatic enumeration of all engine domain sets for analysis, reporting, and the new documentation tables in both network reference files. +- Threat-detection domains are correctly separated from user-selectable ecosystem identifiers, eliminating the path for users to select `threat-detection` as a `network.allowed` value. +- Immutability is enforced: `GetEngineDefaultDomainSets()` returns deep copies, and exported variables are initialized from copies, so external callers cannot corrupt the registry. + +#### Negative +- `engineDefaultDomainSets` is a mutable package-level variable (not a constant), so code in the same package could modify it at runtime; tests must guard against this. +- Exported compatibility variables (`CopilotDefaultDomains`, etc.) now hold a snapshot from package initialization. Code that modifies these variables directly (e.g., in tests) will not affect what `GetEngineDefaultDomainSets()` returns, creating a subtle two-source-of-truth scenario within the package. +- Any future engine whose domain list needs dynamic construction cannot be expressed as a plain `[]string` literal in the map and will require refactoring the registry structure. + +#### Neutral +- The PR adds `GetEngineDefaultDomainSets()` as a new public API surface. Future callers may depend on it, so the set of keys and the copy semantics become a stability commitment. +- Documentation tables for engine domain sets are now automatically derivable from the registry, but the two network reference files (`docs/src/content/docs/reference/network.md` and `.github/aw/network.md`) are still updated manually — there is no automated sync between the registry and the docs. + +--- + +*ADR created by [adr-writer agent]. Review and finalize before changing status from Draft to Accepted.* From a099902a176ddfcdf17645dc124991b61a2c7f87 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 24 Aug 2026 18:55:22 +0000 Subject: [PATCH 4/6] Centralize workflow domain data in JSON Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --- pkg/workflow/data/ecosystem_domains.json | 599 +++++++++++++---------- pkg/workflow/domains.go | 189 ++----- pkg/workflow/domains_test.go | 10 + 3 files changed, 396 insertions(+), 402 deletions(-) diff --git a/pkg/workflow/data/ecosystem_domains.json b/pkg/workflow/data/ecosystem_domains.json index 4675bd44ead..6ff002e83c2 100644 --- a/pkg/workflow/data/ecosystem_domains.json +++ b/pkg/workflow/data/ecosystem_domains.json @@ -1,261 +1,342 @@ { - "bazel": ["releases.bazel.build", "mirror.bazel.build", "bcr.bazel.build", "blog.bazel.build"], - "chrome": ["*.google.com", "*.googleapis.com", "*.gvt1.com"], - "clojure": ["repo.clojars.org", "clojars.org"], - "containers": ["ghcr.io", "registry.hub.docker.com", "*.docker.io", "*.docker.com", "production.cloudflare.docker.com", "dl.k8s.io", "pkgs.k8s.io", "quay.io", "mcr.microsoft.com", "gcr.io", "auth.docker.io"], - "copilot-vendor": ["api.business.githubcopilot.com", "api.enterprise.githubcopilot.com", "api.individual.githubcopilot.com", "telemetry.enterprise.githubcopilot.com"], - "dart": ["pub.dev", "pub.dartlang.org", "storage.googleapis.com"], - "defaults": [ - "crl3.digicert.com", - "crl4.digicert.com", - "ocsp.digicert.com", - "ts-crl.ws.symantec.com", - "ts-ocsp.ws.symantec.com", - "crl.geotrust.com", - "ocsp.geotrust.com", - "crl.thawte.com", - "ocsp.thawte.com", - "crl.verisign.com", - "ocsp.verisign.com", - "crl.globalsign.com", - "ocsp.globalsign.com", - "crls.ssl.com", - "ocsp.ssl.com", - "crl.identrust.com", - "ocsp.identrust.com", - "crl.sectigo.com", - "ocsp.sectigo.com", - "crl.usertrust.com", - "ocsp.usertrust.com", - "s.symcb.com", - "s.symcd.com", - "json-schema.org", - "json.schemastore.org", - "archive.ubuntu.com", - "security.ubuntu.com", - "ppa.launchpad.net", - "keyserver.ubuntu.com", - "azure.archive.ubuntu.com", - "api.snapcraft.io", - "packagecloud.io", - "packages.cloud.google.com", - "packages.microsoft.com", - "www.googleapis.com" - ], - "deno": ["deno.land", "jsr.io", "googleapis.deno.dev", "fresh.deno.dev"], - "dev-tools": [ - "app.renovatebot.com", - "appveyor.com", - "badgen.net", - "circleci.com", - "codacy.com", - "codeclimate.com", - "codecov.io", - "coveralls.io", - "deepsource.io", - "drone.io", - "img.shields.io", - "readthedocs.io", - "readthedocs.org", - "renovatebot.com", - "semaphoreci.com", - "shields.io", - "snyk.io", - "sonarcloud.io", - "sonarqube.com", - "travis-ci.com" - ], - "dotnet": [ - "nuget.org", - "dist.nuget.org", - "api.nuget.org", - "nuget.pkg.github.com", - "dotnet.microsoft.com", - "pkgs.dev.azure.com", - "builds.dotnet.microsoft.com", - "dotnetcli.blob.core.windows.net", - "nugetregistryv2prod.blob.core.windows.net", - "azuresearch-usnc.nuget.org", - "azuresearch-ussc.nuget.org", - "dc.services.visualstudio.com", - "dot.net", - "ci.dot.net", - "www.microsoft.com", - "oneocsp.microsoft.com", - "*.vsblob.vsassets.io" - ], - "elixir": ["hex.pm", "repo.hex.pm", "builds.hex.pm", "cdn.hex.pm", "fastly.hex.pm"], - "fonts": ["fonts.googleapis.com", "fonts.gstatic.com"], - "github": [ - "*.githubusercontent.com", - "codeload.github.com", - "docs.github.com", - "github-cloud.githubusercontent.com", - "github-cloud.s3.amazonaws.com", - "github.blog", - "github.com", - "github.githubassets.com", - "lfs.github.com", - "objects.githubusercontent.com", - "patchdiff.githubusercontent.com", - "patch-diff.githubusercontent.com", - "raw.githubusercontent.com" - ], - "github-actions": [ - "productionresultssa0.blob.core.windows.net", - "productionresultssa1.blob.core.windows.net", - "productionresultssa2.blob.core.windows.net", - "productionresultssa3.blob.core.windows.net", - "productionresultssa4.blob.core.windows.net", - "productionresultssa5.blob.core.windows.net", - "productionresultssa6.blob.core.windows.net", - "productionresultssa7.blob.core.windows.net", - "productionresultssa8.blob.core.windows.net", - "productionresultssa9.blob.core.windows.net", - "productionresultssa10.blob.core.windows.net", - "productionresultssa11.blob.core.windows.net", - "productionresultssa12.blob.core.windows.net", - "productionresultssa13.blob.core.windows.net", - "productionresultssa14.blob.core.windows.net", - "productionresultssa15.blob.core.windows.net", - "productionresultssa16.blob.core.windows.net", - "productionresultssa17.blob.core.windows.net", - "productionresultssa18.blob.core.windows.net", - "productionresultssa19.blob.core.windows.net" - ], - "gh-aw": ["codeload.github.com", "github.com", "github.github.com", "raw.githubusercontent.com"], - "go": ["go.dev", "golang.org", "proxy.golang.org", "sum.golang.org", "pkg.go.dev", "goproxy.io", "storage.googleapis.com"], - "haskell": ["haskell.org", "*.hackage.haskell.org", "get-ghcup.haskell.org", "downloads.haskell.org"], - "java": [ - "www.java.com", - "jdk.java.net", - "api.adoptium.net", - "adoptium.net", - "repo.maven.apache.org", - "maven.apache.org", - "repo1.maven.org", - "maven.pkg.github.com", - "maven.oracle.com", - "repo.spring.io", - "gradle.org", - "services.gradle.org", - "plugins.gradle.org", - "plugins-artifacts.gradle.org", - "repo.grails.org", - "download.eclipse.org", - "download.oracle.com", - "jcenter.bintray.com", - "dlcdn.apache.org", - "archive.apache.org", - "download.java.net", - "api.foojay.io", - "cdn.azul.com", - "central.sonatype.com", - "maven.google.com", - "dl.google.com", - "repo.gradle.org", - "downloads.gradle-dn.com", - "develocity.apache.org", - "scans-in.gradle.com", - "ge.spockframework.org", - "*.gradle-enterprise.cloud", - "maven-central.storage-download.googleapis.com", - "repository.apache.org" - ], - "julia": ["pkg.julialang.org", "*.pkg.julialang.org", "julialang.org", "julialang-s3.julialang.org", "storage.julialang.net"], - "kotlin": ["download.jetbrains.com", "ge.jetbrains.com", "packages.jetbrains.team", "kotlin.bintray.com", "maven.pkg.jetbrains.space"], - "latex": ["ctan.org", "mirror.ctan.org", "mirrors.ctan.org", "tug.org", "www.tug.org", "ftp.tug.org", "latex-project.org", "www.latex-project.org", "miktex.org", "packages.miktex.org"], - "lean": ["elan.lean-lang.org", "lean-lang.org", "leanprover.github.io", "reservoir.lean-lang.org", "static.lean-lang.org"], - "linux-distros": [ - "deb.debian.org", - "security.debian.org", - "keyring.debian.org", - "packages.debian.org", - "debian.map.fastlydns.net", - "apt.llvm.org", - "dl.fedoraproject.org", - "mirrors.fedoraproject.org", - "download.fedoraproject.org", - "mirror.centos.org", - "vault.centos.org", - "dl-cdn.alpinelinux.org", - "pkg.alpinelinux.org", - "mirror.archlinux.org", - "archlinux.org", - "download.opensuse.org", - "cdn.redhat.com" - ], - "local": ["127.0.0.1", "::1", "localhost"], - "lua": ["luarocks.org", "www.luarocks.org"], - "node": [ - "npmjs.org", - "npmjs.com", - "www.npmjs.com", - "www.npmjs.org", - "registry.npmjs.com", - "registry.npmjs.org", - "skimdb.npmjs.com", - "npm.pkg.github.com", - "api.npms.io", - "nodejs.org", - "yarnpkg.com", - "registry.yarnpkg.com", - "repo.yarnpkg.com", - "deb.nodesource.com", - "get.pnpm.io", - "bun.sh", - "deno.land", - "jsr.io", - "registry.bower.io", - "esm.sh", - "googleapis.deno.dev", - "googlechromelabs.github.io", - "storage.googleapis.com", - "cdn.jsdelivr.net", - "telemetry.vercel.com" - ], - "node-cdns": ["cdn.jsdelivr.net", "data.jsdelivr.com", "code.jquery.com", "cdn.sheetjs.com"], - "ocaml": ["opam.ocaml.org", "ocaml.org", "erratique.ch"], - "perl": ["cpan.org", "www.cpan.org", "metacpan.org", "cpan.metacpan.org"], - "php": ["repo.packagist.org", "packagist.org", "getcomposer.org", "bitbucket.org"], - "playwright": ["playwright.download.prss.microsoft.com", "cdn.playwright.dev"], - "powershell": ["powershellgallery.com", "www.powershellgallery.com"], - "python": [ - "pypi.python.org", - "pypi.org", - "pip.pypa.io", - "*.pythonhosted.org", - "files.pythonhosted.org", - "bootstrap.pypa.io", - "conda.binstar.org", - "conda.anaconda.org", - "binstar.org", - "anaconda.org", - "repo.continuum.io", - "repo.anaconda.com" - ], - "python-native": [ - "pypi.python.org", - "pypi.org", - "pip.pypa.io", - "*.pythonhosted.org", - "files.pythonhosted.org", - "bootstrap.pypa.io", - "conda.binstar.org", - "conda.anaconda.org", - "binstar.org", - "anaconda.org", - "repo.continuum.io", - "repo.anaconda.com", - "crates.io", - "index.crates.io", - "static.crates.io" - ], - "r": ["cloud.r-project.org", "cran.r-project.org", "cran.rstudio.com", "r-project.org"], - "ruby": ["rubygems.org", "api.rubygems.org", "rubygems.pkg.github.com", "bundler.rubygems.org", "gems.rubyforge.org", "gems.rubyonrails.org", "index.rubygems.org", "cache.ruby-lang.org", "*.rvm.io"], - "rust": ["crates.io", "index.crates.io", "static.crates.io", "sh.rustup.rs", "static.rust-lang.org"], - "scala": ["repo.scala-sbt.org", "scala-ci.typesafe.com", "repo.typesafe.com", "jitpack.io", "dl.bintray.com", "scala.jfrog.io"], - "swift": ["download.swift.org", "swift.org", "cocoapods.org", "cdn.cocoapods.org"], - "terraform": ["releases.hashicorp.com", "apt.releases.hashicorp.com", "yum.releases.hashicorp.com", "registry.terraform.io"], - "zig": ["ziglang.org", "pkg.machengine.org", "deps.files.ghostty.org"] + "ecosystems": { + "bazel": ["releases.bazel.build", "mirror.bazel.build", "bcr.bazel.build", "blog.bazel.build"], + "chrome": ["*.google.com", "*.googleapis.com", "*.gvt1.com"], + "clojure": ["repo.clojars.org", "clojars.org"], + "containers": ["ghcr.io", "registry.hub.docker.com", "*.docker.io", "*.docker.com", "production.cloudflare.docker.com", "dl.k8s.io", "pkgs.k8s.io", "quay.io", "mcr.microsoft.com", "gcr.io", "auth.docker.io"], + "copilot-vendor": ["api.business.githubcopilot.com", "api.enterprise.githubcopilot.com", "api.individual.githubcopilot.com", "telemetry.enterprise.githubcopilot.com"], + "dart": ["pub.dev", "pub.dartlang.org", "storage.googleapis.com"], + "defaults": [ + "crl3.digicert.com", + "crl4.digicert.com", + "ocsp.digicert.com", + "ts-crl.ws.symantec.com", + "ts-ocsp.ws.symantec.com", + "crl.geotrust.com", + "ocsp.geotrust.com", + "crl.thawte.com", + "ocsp.thawte.com", + "crl.verisign.com", + "ocsp.verisign.com", + "crl.globalsign.com", + "ocsp.globalsign.com", + "crls.ssl.com", + "ocsp.ssl.com", + "crl.identrust.com", + "ocsp.identrust.com", + "crl.sectigo.com", + "ocsp.sectigo.com", + "crl.usertrust.com", + "ocsp.usertrust.com", + "s.symcb.com", + "s.symcd.com", + "json-schema.org", + "json.schemastore.org", + "archive.ubuntu.com", + "security.ubuntu.com", + "ppa.launchpad.net", + "keyserver.ubuntu.com", + "azure.archive.ubuntu.com", + "api.snapcraft.io", + "packagecloud.io", + "packages.cloud.google.com", + "packages.microsoft.com", + "www.googleapis.com" + ], + "deno": ["deno.land", "jsr.io", "googleapis.deno.dev", "fresh.deno.dev"], + "dev-tools": [ + "app.renovatebot.com", + "appveyor.com", + "badgen.net", + "circleci.com", + "codacy.com", + "codeclimate.com", + "codecov.io", + "coveralls.io", + "deepsource.io", + "drone.io", + "img.shields.io", + "readthedocs.io", + "readthedocs.org", + "renovatebot.com", + "semaphoreci.com", + "shields.io", + "snyk.io", + "sonarcloud.io", + "sonarqube.com", + "travis-ci.com" + ], + "dotnet": [ + "nuget.org", + "dist.nuget.org", + "api.nuget.org", + "nuget.pkg.github.com", + "dotnet.microsoft.com", + "pkgs.dev.azure.com", + "builds.dotnet.microsoft.com", + "dotnetcli.blob.core.windows.net", + "nugetregistryv2prod.blob.core.windows.net", + "azuresearch-usnc.nuget.org", + "azuresearch-ussc.nuget.org", + "dc.services.visualstudio.com", + "dot.net", + "ci.dot.net", + "www.microsoft.com", + "oneocsp.microsoft.com", + "*.vsblob.vsassets.io" + ], + "elixir": ["hex.pm", "repo.hex.pm", "builds.hex.pm", "cdn.hex.pm", "fastly.hex.pm"], + "fonts": ["fonts.googleapis.com", "fonts.gstatic.com"], + "github": [ + "*.githubusercontent.com", + "codeload.github.com", + "docs.github.com", + "github-cloud.githubusercontent.com", + "github-cloud.s3.amazonaws.com", + "github.blog", + "github.com", + "github.githubassets.com", + "lfs.github.com", + "objects.githubusercontent.com", + "patchdiff.githubusercontent.com", + "patch-diff.githubusercontent.com", + "raw.githubusercontent.com" + ], + "github-actions": [ + "productionresultssa0.blob.core.windows.net", + "productionresultssa1.blob.core.windows.net", + "productionresultssa2.blob.core.windows.net", + "productionresultssa3.blob.core.windows.net", + "productionresultssa4.blob.core.windows.net", + "productionresultssa5.blob.core.windows.net", + "productionresultssa6.blob.core.windows.net", + "productionresultssa7.blob.core.windows.net", + "productionresultssa8.blob.core.windows.net", + "productionresultssa9.blob.core.windows.net", + "productionresultssa10.blob.core.windows.net", + "productionresultssa11.blob.core.windows.net", + "productionresultssa12.blob.core.windows.net", + "productionresultssa13.blob.core.windows.net", + "productionresultssa14.blob.core.windows.net", + "productionresultssa15.blob.core.windows.net", + "productionresultssa16.blob.core.windows.net", + "productionresultssa17.blob.core.windows.net", + "productionresultssa18.blob.core.windows.net", + "productionresultssa19.blob.core.windows.net" + ], + "gh-aw": ["codeload.github.com", "github.com", "github.github.com", "raw.githubusercontent.com"], + "go": ["go.dev", "golang.org", "proxy.golang.org", "sum.golang.org", "pkg.go.dev", "goproxy.io", "storage.googleapis.com"], + "haskell": ["haskell.org", "*.hackage.haskell.org", "get-ghcup.haskell.org", "downloads.haskell.org"], + "java": [ + "www.java.com", + "jdk.java.net", + "api.adoptium.net", + "adoptium.net", + "repo.maven.apache.org", + "maven.apache.org", + "repo1.maven.org", + "maven.pkg.github.com", + "maven.oracle.com", + "repo.spring.io", + "gradle.org", + "services.gradle.org", + "plugins.gradle.org", + "plugins-artifacts.gradle.org", + "repo.grails.org", + "download.eclipse.org", + "download.oracle.com", + "jcenter.bintray.com", + "dlcdn.apache.org", + "archive.apache.org", + "download.java.net", + "api.foojay.io", + "cdn.azul.com", + "central.sonatype.com", + "maven.google.com", + "dl.google.com", + "repo.gradle.org", + "downloads.gradle-dn.com", + "develocity.apache.org", + "scans-in.gradle.com", + "ge.spockframework.org", + "*.gradle-enterprise.cloud", + "maven-central.storage-download.googleapis.com", + "repository.apache.org" + ], + "julia": ["pkg.julialang.org", "*.pkg.julialang.org", "julialang.org", "julialang-s3.julialang.org", "storage.julialang.net"], + "kotlin": ["download.jetbrains.com", "ge.jetbrains.com", "packages.jetbrains.team", "kotlin.bintray.com", "maven.pkg.jetbrains.space"], + "latex": ["ctan.org", "mirror.ctan.org", "mirrors.ctan.org", "tug.org", "www.tug.org", "ftp.tug.org", "latex-project.org", "www.latex-project.org", "miktex.org", "packages.miktex.org"], + "lean": ["elan.lean-lang.org", "lean-lang.org", "leanprover.github.io", "reservoir.lean-lang.org", "static.lean-lang.org"], + "linux-distros": [ + "deb.debian.org", + "security.debian.org", + "keyring.debian.org", + "packages.debian.org", + "debian.map.fastlydns.net", + "apt.llvm.org", + "dl.fedoraproject.org", + "mirrors.fedoraproject.org", + "download.fedoraproject.org", + "mirror.centos.org", + "vault.centos.org", + "dl-cdn.alpinelinux.org", + "pkg.alpinelinux.org", + "mirror.archlinux.org", + "archlinux.org", + "download.opensuse.org", + "cdn.redhat.com" + ], + "local": ["127.0.0.1", "::1", "localhost"], + "lua": ["luarocks.org", "www.luarocks.org"], + "node": [ + "npmjs.org", + "npmjs.com", + "www.npmjs.com", + "www.npmjs.org", + "registry.npmjs.com", + "registry.npmjs.org", + "skimdb.npmjs.com", + "npm.pkg.github.com", + "api.npms.io", + "nodejs.org", + "yarnpkg.com", + "registry.yarnpkg.com", + "repo.yarnpkg.com", + "deb.nodesource.com", + "get.pnpm.io", + "bun.sh", + "deno.land", + "jsr.io", + "registry.bower.io", + "esm.sh", + "googleapis.deno.dev", + "googlechromelabs.github.io", + "storage.googleapis.com", + "cdn.jsdelivr.net", + "telemetry.vercel.com" + ], + "node-cdns": ["cdn.jsdelivr.net", "data.jsdelivr.com", "code.jquery.com", "cdn.sheetjs.com"], + "ocaml": ["opam.ocaml.org", "ocaml.org", "erratique.ch"], + "perl": ["cpan.org", "www.cpan.org", "metacpan.org", "cpan.metacpan.org"], + "php": ["repo.packagist.org", "packagist.org", "getcomposer.org", "bitbucket.org"], + "playwright": ["playwright.download.prss.microsoft.com", "cdn.playwright.dev"], + "powershell": ["powershellgallery.com", "www.powershellgallery.com"], + "python": [ + "pypi.python.org", + "pypi.org", + "pip.pypa.io", + "*.pythonhosted.org", + "files.pythonhosted.org", + "bootstrap.pypa.io", + "conda.binstar.org", + "conda.anaconda.org", + "binstar.org", + "anaconda.org", + "repo.continuum.io", + "repo.anaconda.com" + ], + "python-native": [ + "pypi.python.org", + "pypi.org", + "pip.pypa.io", + "*.pythonhosted.org", + "files.pythonhosted.org", + "bootstrap.pypa.io", + "conda.binstar.org", + "conda.anaconda.org", + "binstar.org", + "anaconda.org", + "repo.continuum.io", + "repo.anaconda.com", + "crates.io", + "index.crates.io", + "static.crates.io" + ], + "r": ["cloud.r-project.org", "cran.r-project.org", "cran.rstudio.com", "r-project.org"], + "ruby": ["rubygems.org", "api.rubygems.org", "rubygems.pkg.github.com", "bundler.rubygems.org", "gems.rubyforge.org", "gems.rubyonrails.org", "index.rubygems.org", "cache.ruby-lang.org", "*.rvm.io"], + "rust": ["crates.io", "index.crates.io", "static.crates.io", "sh.rustup.rs", "static.rust-lang.org"], + "scala": ["repo.scala-sbt.org", "scala-ci.typesafe.com", "repo.typesafe.com", "jitpack.io", "dl.bintray.com", "scala.jfrog.io"], + "swift": ["download.swift.org", "swift.org", "cocoapods.org", "cdn.cocoapods.org"], + "terraform": ["releases.hashicorp.com", "apt.releases.hashicorp.com", "yum.releases.hashicorp.com", "registry.terraform.io"], + "zig": ["ziglang.org", "pkg.machengine.org", "deps.files.ghostty.org"] + }, + "engine-defaults": { + "claude": [ + "*.githubusercontent.com", + "anthropic.com", + "api.anthropic.com", + "api.github.com", + "api.snapcraft.io", + "archive.ubuntu.com", + "azure.archive.ubuntu.com", + "cdn.playwright.dev", + "codeload.github.com", + "crl.geotrust.com", + "crl.globalsign.com", + "crl.identrust.com", + "crl.sectigo.com", + "crl.thawte.com", + "crl.usertrust.com", + "crl.verisign.com", + "crl3.digicert.com", + "crl4.digicert.com", + "crls.ssl.com", + "ghcr.io", + "github-cloud.githubusercontent.com", + "github-cloud.s3.amazonaws.com", + "github.com", + "host.docker.internal", + "json-schema.org", + "json.schemastore.org", + "keyserver.ubuntu.com", + "lfs.github.com", + "objects.githubusercontent.com", + "ocsp.digicert.com", + "ocsp.geotrust.com", + "ocsp.globalsign.com", + "ocsp.identrust.com", + "ocsp.sectigo.com", + "ocsp.ssl.com", + "ocsp.thawte.com", + "ocsp.usertrust.com", + "ocsp.verisign.com", + "packagecloud.io", + "packages.cloud.google.com", + "packages.microsoft.com", + "playwright.download.prss.microsoft.com", + "ppa.launchpad.net", + "raw.githubusercontent.com", + "s.symcb.com", + "s.symcd.com", + "security.ubuntu.com", + "sentry.io", + "statsig.anthropic.com", + "ts-crl.ws.symantec.com", + "ts-ocsp.ws.symantec.com" + ], + "codex": ["172.30.0.1", "api.github.com", "api.openai.com", "chatgpt.com", "github.com", "host.docker.internal", "openai.com"], + "copilot": ["api.github.com", "api.githubcopilot.com", "github.com", "host.docker.internal", "raw.githubusercontent.com"], + "gemini": ["*.googleapis.com", "generativelanguage.googleapis.com", "github.com", "host.docker.internal", "raw.githubusercontent.com"], + "pi": ["api.githubcopilot.com", "host.docker.internal", "github.com", "raw.githubusercontent.com"], + "pi-base": ["host.docker.internal", "github.com", "raw.githubusercontent.com"], + "threat-detection": [ + "api.business.githubcopilot.com", + "api.enterprise.githubcopilot.com", + "api.github.com", + "api.githubcopilot.com", + "api.individual.githubcopilot.com", + "github.com", + "host.docker.internal", + "registry.npmjs.org", + "telemetry.enterprise.githubcopilot.com" + ] + }, + "pi-provider-domains": { + "copilot": "api.githubcopilot.com", + "github-copilot": "api.githubcopilot.com", + "anthropic": "api.anthropic.com", + "openai": "api.openai.com", + "codex": "api.openai.com" + }, + "sanitization-defaults": ["github.com", "localhost"] } diff --git a/pkg/workflow/domains.go b/pkg/workflow/domains.go index ef06ee8ce4f..0b9328d26dc 100644 --- a/pkg/workflow/domains.go +++ b/pkg/workflow/domains.go @@ -17,32 +17,47 @@ import ( var domainsLog = logger.New("workflow:domains") //go:embed data/ecosystem_domains.json -var ecosystemDomainsJSON []byte +var domainSetsJSON []byte -var loadEcosystemDomains = sync.OnceValues(func() (map[string][]string, error) { - domainsLog.Print("Loading ecosystem domains from embedded JSON") +type domainSets struct { + Ecosystems map[string][]string `json:"ecosystems"` + EngineDefaults map[string][]string `json:"engine-defaults"` + PiProviderDomains map[string]string `json:"pi-provider-domains"` + SanitizationDefaults []string `json:"sanitization-defaults"` +} + +var loadDomainSets = sync.OnceValues(func() (domainSets, error) { + domainsLog.Print("Loading domain sets from embedded JSON") - ecosystemDomains := make(map[string][]string) - if err := json.Unmarshal(ecosystemDomainsJSON, &ecosystemDomains); err != nil { - return nil, fmt.Errorf("failed to load ecosystem domains from JSON: %w", err) + var sets domainSets + if err := json.Unmarshal(domainSetsJSON, &sets); err != nil { + return domainSets{}, fmt.Errorf("failed to load domain sets from JSON: %w", err) } - // Pre-sort all domain lists once so getEcosystemDomains only needs to copy, not sort. - for key := range ecosystemDomains { - sort.Strings(ecosystemDomains[key]) + // Pre-sort all domain lists once so lookup functions only need to copy, not sort. + for key := range sets.Ecosystems { + sort.Strings(sets.Ecosystems[key]) } + for key := range sets.EngineDefaults { + sort.Strings(sets.EngineDefaults[key]) + } + sort.Strings(sets.SanitizationDefaults) - domainsLog.Printf("Loaded %d ecosystem categories", len(ecosystemDomains)) - return ecosystemDomains, nil + domainsLog.Printf("Loaded %d ecosystem categories and %d engine default domain sets", len(sets.Ecosystems), len(sets.EngineDefaults)) + return sets, nil }) -func getLoadedEcosystemDomains() map[string][]string { - ecosystemDomains, err := loadEcosystemDomains() +func getLoadedDomainSets() domainSets { + sets, err := loadDomainSets() if err != nil { - domainsLog.Printf("Failed to load ecosystem domains: %v", err) - return map[string][]string{} + domainsLog.Printf("Failed to load domain sets: %v", err) + return domainSets{} } - return ecosystemDomains + return sets +} + +func getLoadedEcosystemDomains() map[string][]string { + return getLoadedDomainSets().Ecosystems } // Engine default domain lists intentionally exclude package registries (npm, PyPI, ...). @@ -71,107 +86,9 @@ func getLoadedEcosystemDomains() map[string][]string { // users to select them through network.allowed. // // Engine default domain lists intentionally exclude package registries (npm, PyPI, -// and similar). See the package-registry invariant above. -var engineDefaultDomainSets = map[string][]string{ - "claude": { - "*.githubusercontent.com", - "anthropic.com", - "api.anthropic.com", - "api.github.com", - "api.snapcraft.io", - "archive.ubuntu.com", - "azure.archive.ubuntu.com", - "cdn.playwright.dev", - "codeload.github.com", - "crl.geotrust.com", - "crl.globalsign.com", - "crl.identrust.com", - "crl.sectigo.com", - "crl.thawte.com", - "crl.usertrust.com", - "crl.verisign.com", - "crl3.digicert.com", - "crl4.digicert.com", - "crls.ssl.com", - "ghcr.io", - "github-cloud.githubusercontent.com", - "github-cloud.s3.amazonaws.com", - "github.com", - "host.docker.internal", - "json-schema.org", - "json.schemastore.org", - "keyserver.ubuntu.com", - "lfs.github.com", - "objects.githubusercontent.com", - "ocsp.digicert.com", - "ocsp.geotrust.com", - "ocsp.globalsign.com", - "ocsp.identrust.com", - "ocsp.sectigo.com", - "ocsp.ssl.com", - "ocsp.thawte.com", - "ocsp.usertrust.com", - "ocsp.verisign.com", - "packagecloud.io", - "packages.cloud.google.com", - "packages.microsoft.com", - "playwright.download.prss.microsoft.com", - "ppa.launchpad.net", - "raw.githubusercontent.com", - "s.symcb.com", - "s.symcd.com", - "security.ubuntu.com", - "sentry.io", - "statsig.anthropic.com", - "ts-crl.ws.symantec.com", - "ts-ocsp.ws.symantec.com", - }, - "codex": { - "172.30.0.1", // AWF gateway IP - Codex resolves host.docker.internal to this IP for Rust DNS compatibility - "api.github.com", // Codex startup performs GitHub plugin sync requests against the GitHub API - "api.openai.com", - "chatgpt.com", // Codex CLI connects to chatgpt.com (and subdomains e.g. ab.chatgpt.com) for auth/telemetry - "github.com", // Codex startup accesses GitHub-hosted plugin metadata pages - "host.docker.internal", - "openai.com", - }, - "copilot": { - "api.github.com", - "api.githubcopilot.com", - "github.com", - "host.docker.internal", - "raw.githubusercontent.com", - }, - "gemini": { - "*.googleapis.com", - "generativelanguage.googleapis.com", - "github.com", - "host.docker.internal", - "raw.githubusercontent.com", - }, - "pi": { - "api.githubcopilot.com", // Default provider (Copilot routing) - "host.docker.internal", - "github.com", - "raw.githubusercontent.com", - }, - "pi-base": { - "host.docker.internal", // MCP gateway / API proxy access - "github.com", - "raw.githubusercontent.com", - }, - "threat-detection": { - "api.business.githubcopilot.com", - "api.enterprise.githubcopilot.com", - "api.github.com", - "api.githubcopilot.com", - "api.individual.githubcopilot.com", - "github.com", - "host.docker.internal", - "registry.npmjs.org", - "telemetry.enterprise.githubcopilot.com", - }, -} +// and similar). The threat-detection set is the exception because the Copilot +// threat detector uses the npm registry. See the package-registry invariant above. +var engineDefaultDomainSets = getLoadedDomainSets().EngineDefaults // GetEngineDefaultDomainSets returns copies of the named engine domain sets for // analysis and reporting. Engine domain sets are not valid network.allowed values. @@ -210,18 +127,12 @@ var GeminiDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["g // provider-specific API domains are added on top via GetDefaultDomainsForEngine. var PiBaseDefaultDomains = copyEngineDefaultDomainSet(engineDefaultDomainSets["pi-base"]) -// piProviderDomains maps provider prefixes to their API domains. -// Covers the same set of providers that Pi can route through via the AWF LLM gateway. +// piProviderDomains maps provider prefixes to their API domains. It covers the +// same set of providers that Pi can route through via the AWF LLM gateway. // Note: "google" is intentionally omitted — Pi backend resolution only supports // copilot, anthropic, openai, and codex; adding google here without backend // support would produce an inconsistent routing configuration. -var piProviderDomains = map[string]string{ - "copilot": "api.githubcopilot.com", - "github-copilot": "api.githubcopilot.com", - "anthropic": "api.anthropic.com", - "openai": "api.openai.com", - "codex": "api.openai.com", -} +var piProviderDomains = getLoadedDomainSets().PiProviderDomains // PiDefaultDomains are the static default domains for backward compatibility when // no model provider prefix is given. When a provider/model format is used, the @@ -274,13 +185,6 @@ func getPiDefaultDomains(model string) ([]string, error) { return domains, nil } -// PlaywrightDomains are the domains required for Playwright browser downloads -// These domains are needed when Playwright MCP server initializes in the Docker container -var PlaywrightDomains = []string{ - "cdn.playwright.dev", - "playwright.download.prss.microsoft.com", -} - // compoundEcosystems defines ecosystem identifiers that expand to the union of multiple // component ecosystems. These are resolved at lookup time, so they stay in sync with // any future changes to the component ecosystems. @@ -308,7 +212,7 @@ func getEcosystemDomains(category string) []string { return result } - ecosystemDomains := getLoadedEcosystemDomains() + ecosystemDomains := getLoadedDomainSets().Ecosystems domains, exists := ecosystemDomains[category] if !exists { return []string{} @@ -533,7 +437,7 @@ func GetDomainEcosystem(domain string) string { // Fall back to any ecosystems not in the priority list, sorted for determinism remaining := make([]string, 0) - for ecosystem := range getLoadedEcosystemDomains() { + for ecosystem := range getLoadedDomainSets().Ecosystems { if _, ok := checked[ecosystem]; !ok { remaining = append(remaining, ecosystem) } @@ -626,8 +530,9 @@ func extractPlaywrightDomains(tools map[string]any) []string { // Check if Playwright tool is configured if _, hasPlaywright := tools["playwright"]; hasPlaywright { - domainsLog.Printf("Detected Playwright tool, adding %d domains for browser downloads", len(PlaywrightDomains)) - return PlaywrightDomains + domains := getEcosystemDomains("playwright") + domainsLog.Printf("Detected Playwright tool, adding %d domains for browser downloads", len(domains)) + return domains } return []string{} @@ -1034,7 +939,7 @@ func expandAllowedDomains(entries []string) []string { // computeExpandedAllowedDomainsForSanitization computes the allowed domains for URL sanitization, // unioning the engine/network base set with the safe-outputs.allowed-domains entries. -// It always includes "localhost" and "github.com" in the result. +// It always includes the sanitization defaults in the result. // The allowed-domains entries support ecosystem identifiers (same syntax as network.allowed). // Returns an error if the engine's model is malformed (e.g. a leading slash). func (c *Compiler) computeExpandedAllowedDomainsForSanitization(data *WorkflowData) (string, error) { @@ -1063,11 +968,9 @@ func (c *Compiler) computeExpandedAllowedDomainsForSanitization(data *WorkflowDa } } - // Always allow localhost (for local development URL references) - domainMap["localhost"] = struct{}{} - - // Always allow github.com (GitHub page of the current repo) - domainMap["github.com"] = struct{}{} + for _, domain := range getLoadedDomainSets().SanitizationDefaults { + domainMap[domain] = struct{}{} + } // Produce a sorted, comma-separated result return strings.Join(sliceutil.SortedKeys(domainMap), ","), nil diff --git a/pkg/workflow/domains_test.go b/pkg/workflow/domains_test.go index d0bae77e171..1c282e08223 100644 --- a/pkg/workflow/domains_test.go +++ b/pkg/workflow/domains_test.go @@ -469,6 +469,16 @@ func TestGetEngineDefaultDomainSets(t *testing.T) { "exported compatibility variables must not modify registered domain sets") } +func TestEmbeddedDomainSets(t *testing.T) { + sets := getLoadedDomainSets() + + assert.Contains(t, sets.Ecosystems, "defaults") + assert.Contains(t, sets.EngineDefaults, "copilot") + assert.Equal(t, "api.githubcopilot.com", sets.PiProviderDomains["copilot"]) + assert.Equal(t, []string{"github.com", "localhost"}, sets.SanitizationDefaults) + assert.NotContains(t, sets.Ecosystems, "threat-detection") +} + func TestGetThreatDetectionAllowedDomains(t *testing.T) { // With empty network permissions the result equals the sorted detection domains result := GetThreatDetectionAllowedDomains(&NetworkPermissions{Allowed: []string{}}) From 03a3bcb1a697f06ca59418ca84c2b02bdba7b99d Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 24 Aug 2026 19:10:33 +0000 Subject: [PATCH 5/6] Preserve threat detection network alias Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com> --- .github/aw/network.md | 7 ++++-- docs/src/content/docs/reference/network.md | 7 ++++-- pkg/workflow/data/ecosystem_domains.json | 11 +++++++++ pkg/workflow/domains.go | 26 +++++++++++++--------- pkg/workflow/domains_test.go | 18 +++++++++++---- 5 files changed, 50 insertions(+), 19 deletions(-) diff --git a/.github/aw/network.md b/.github/aw/network.md index d02788e7dd4..330b1c12c18 100644 --- a/.github/aw/network.md +++ b/.github/aw/network.md @@ -86,12 +86,14 @@ Keywords expanding to curated domain lists: | `lean` | Lean theorem prover | `lean-lang.org`, `elan.lean-lang.org`, `reservoir.lean-lang.org` | | `python-native` | Python native build deps | Native toolchain mirrors for building Python packages from source | | `copilot-vendor` | Copilot plan-specific APIs / telemetry | `api.business.githubcopilot.com`, `api.enterprise.githubcopilot.com`, `api.individual.githubcopilot.com`, `telemetry.enterprise.githubcopilot.com` | +| `threat-detection` | Compatibility alias for Copilot threat detection | Copilot API/telemetry hosts, GitHub API/web, `host.docker.internal`, `registry.npmjs.org` | ## Automatic Engine Domain Sets Each engine automatically receives the domain set it requires in addition to `network.allowed`. These named sets are maintained by the compiler for analysis -and reporting; they are not valid `network.allowed` identifiers. +and reporting; they are not valid `network.allowed` identifiers, except for +the legacy `threat-detection` compatibility alias listed above. | Engine set | Included domains | |---|---| @@ -100,7 +102,8 @@ and reporting; they are not valid `network.allowed` identifiers. | `codex` | `172.30.0.1`, `api.github.com`, `api.openai.com`, `chatgpt.com`, `github.com`, `host.docker.internal`, `openai.com` | | `gemini` | `*.googleapis.com`, `generativelanguage.googleapis.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com` | | `pi` | `api.githubcopilot.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com`; provider-scoped models replace the API host with the selected provider endpoint | -| `threat-detection` | Copilot API and telemetry hosts, `api.github.com`, `github.com`, `host.docker.internal`, and `registry.npmjs.org` for read-only lockfile validation | +| `pi-base` | `github.com`, `host.docker.internal`, `raw.githubusercontent.com`; applied as the provider-independent baseline before a provider prefix is resolved | +| `threat-detection` | Applied automatically only to Copilot threat-detection runs: Copilot API and telemetry hosts, `api.github.com`, `github.com`, `host.docker.internal`, and `registry.npmjs.org` for read-only lockfile validation. External Claude, Codex, Gemini, and other detection runs use their own engine defaults. | ## Invalid Shorthands diff --git a/docs/src/content/docs/reference/network.md b/docs/src/content/docs/reference/network.md index 13d16e19d5a..b4ad8b4a389 100644 --- a/docs/src/content/docs/reference/network.md +++ b/docs/src/content/docs/reference/network.md @@ -89,6 +89,7 @@ Mix ecosystem identifiers with specific domains for fine-grained control: | `bazel` | Bazel build system (`releases.bazel.build`, `bcr.bazel.build`) | | `clojure` | Clojure packages (`clojars.org`) | | `copilot-vendor` | Plan-specific Copilot API hosts (`api.business.githubcopilot.com`, `api.enterprise.githubcopilot.com`, `api.individual.githubcopilot.com`) and Copilot telemetry (`telemetry.enterprise.githubcopilot.com`) — not enabled by default, since agents route inference through the firewall gateway | +| `threat-detection` | Compatibility alias for Copilot threat-detection network access (`api.githubcopilot.com`, Copilot plan APIs, telemetry, `api.github.com`, `github.com`, `host.docker.internal`, `registry.npmjs.org`) | | `dart` | Dart/Flutter packages (`pub.dev`, `storage.googleapis.com`) | | `deno` | Deno runtime (`deno.land`, `jsr.io`, `googleapis.deno.dev`) | | `dotnet` | NuGet packages and .NET SDK | @@ -120,7 +121,8 @@ Mix ecosystem identifiers with specific domains for fine-grained control: Each engine automatically receives the domain set it requires in addition to `network.allowed`. These named sets are maintained by the compiler for analysis -and reporting; they are not valid `network.allowed` identifiers. +and reporting; they are not valid `network.allowed` identifiers, except for +the legacy `threat-detection` compatibility alias listed above. | Engine set | Included domains | |---|---| @@ -129,7 +131,8 @@ and reporting; they are not valid `network.allowed` identifiers. | `codex` | `172.30.0.1`, `api.github.com`, `api.openai.com`, `chatgpt.com`, `github.com`, `host.docker.internal`, `openai.com` | | `gemini` | `*.googleapis.com`, `generativelanguage.googleapis.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com` | | `pi` | `api.githubcopilot.com`, `github.com`, `host.docker.internal`, `raw.githubusercontent.com`; provider-scoped models replace the API host with the selected provider endpoint | -| `threat-detection` | Copilot API and telemetry hosts, `api.github.com`, `github.com`, `host.docker.internal`, and `registry.npmjs.org` for read-only lockfile validation | +| `pi-base` | `github.com`, `host.docker.internal`, `raw.githubusercontent.com`; applied as the provider-independent baseline before a provider prefix is resolved | +| `threat-detection` | Applied automatically only to Copilot threat-detection runs: Copilot API and telemetry hosts, `api.github.com`, `github.com`, `host.docker.internal`, and `registry.npmjs.org` for read-only lockfile validation. External Claude, Codex, Gemini, and other detection runs use their own engine defaults. | ### Ecosystem Identifier Validation diff --git a/pkg/workflow/data/ecosystem_domains.json b/pkg/workflow/data/ecosystem_domains.json index 6ff002e83c2..b6f395f3b39 100644 --- a/pkg/workflow/data/ecosystem_domains.json +++ b/pkg/workflow/data/ecosystem_domains.json @@ -258,6 +258,17 @@ "scala": ["repo.scala-sbt.org", "scala-ci.typesafe.com", "repo.typesafe.com", "jitpack.io", "dl.bintray.com", "scala.jfrog.io"], "swift": ["download.swift.org", "swift.org", "cocoapods.org", "cdn.cocoapods.org"], "terraform": ["releases.hashicorp.com", "apt.releases.hashicorp.com", "yum.releases.hashicorp.com", "registry.terraform.io"], + "threat-detection": [ + "api.business.githubcopilot.com", + "api.enterprise.githubcopilot.com", + "api.github.com", + "api.githubcopilot.com", + "api.individual.githubcopilot.com", + "github.com", + "host.docker.internal", + "registry.npmjs.org", + "telemetry.enterprise.githubcopilot.com" + ], "zig": ["ziglang.org", "pkg.machengine.org", "deps.files.ghostty.org"] }, "engine-defaults": { diff --git a/pkg/workflow/domains.go b/pkg/workflow/domains.go index 0b9328d26dc..9ebe4f58e3f 100644 --- a/pkg/workflow/domains.go +++ b/pkg/workflow/domains.go @@ -60,7 +60,7 @@ func getLoadedEcosystemDomains() map[string][]string { return getLoadedDomainSets().Ecosystems } -// Engine default domain lists intentionally exclude package registries (npm, PyPI, ...). +// Runtime engine default domain lists intentionally exclude package registries (npm, PyPI, ...). // // Engine CLIs and SDKs are installed by dedicated GitHub Actions steps that run on the // runner *before* the AWF-wrapped agent step, so package registries are not needed inside @@ -73,21 +73,25 @@ func getLoadedEcosystemDomains() map[string][]string { // documented behavior that package ecosystems require explicit opt-in // (`network: { allowed: [node] }`, `[python]`, or a matching `runtimes:` entry). // -// This invariant is enforced by TestEngineDefaultDomainsDoNotOverlapEcosystems in -// domains_package_registry_test.go, which fails if any engine default domain list below -// overlaps with the full "node" or "python" ecosystem domain sets in data/ecosystem_domains.json -// — not just the registries known when this comment was written. If you need to add a domain -// to an engine default and that test starts failing, the domain belongs behind an explicit -// ecosystem/runtime opt-in instead, not in the unconditional default list. +// This invariant is enforced for runtime engine defaults by +// TestEngineDefaultDomainsDoNotOverlapEcosystems in domains_package_registry_test.go, which +// fails if those default domain lists overlap with the full "node" or "python" ecosystem +// domain sets in data/ecosystem_domains.json — not just the registries known when this comment +// was written. If you need to add a package-registry domain to a runtime engine default and +// that test starts failing, the domain belongs behind an explicit ecosystem/runtime opt-in +// instead, not in the unconditional default list. Copilot threat detection is the exception: +// its dedicated detection allow-list includes registry.npmjs.org for read-only lockfile +// validation, and that list is not part of normal agent engine defaults. // engineDefaultDomainSets centralizes every unconditional engine allow-list. These // sets are intentionally distinct from user-selectable ecosystem domain lists: the // compiler includes them automatically for the matching engine, rather than allowing // users to select them through network.allowed. // -// Engine default domain lists intentionally exclude package registries (npm, PyPI, -// and similar). The threat-detection set is the exception because the Copilot -// threat detector uses the npm registry. See the package-registry invariant above. +// Runtime engine default domain lists intentionally exclude package registries +// (npm, PyPI, and similar). The threat-detection set is the exception because +// Copilot threat detection uses the npm registry. See the package-registry +// invariant above. var engineDefaultDomainSets = getLoadedDomainSets().EngineDefaults // GetEngineDefaultDomainSets returns copies of the named engine domain sets for @@ -707,7 +711,7 @@ func GetAllowedDomainsForEngine(engine constants.EngineName, network *NetworkPer // Any additional user-specified network.allowed entries are merged in (typically empty for detection). // Returns a deduplicated, sorted, comma-separated string suitable for AWF's --allow-domains flag. func GetThreatDetectionAllowedDomains(network *NetworkPermissions) string { - detectionDomains := engineDefaultDomainSets["threat-detection"] + detectionDomains := copyEngineDefaultDomainSet(engineDefaultDomainSets["threat-detection"]) // Pass nil tools and runtimes: detection runs with no npm/runtime ecosystem, so // ecosystem domain expansion is intentionally skipped. return mergeDomainsWithNetworkToolsAndRuntimes(detectionDomains, network, nil, nil) diff --git a/pkg/workflow/domains_test.go b/pkg/workflow/domains_test.go index 1c282e08223..2e9f01fb98b 100644 --- a/pkg/workflow/domains_test.go +++ b/pkg/workflow/domains_test.go @@ -429,7 +429,7 @@ func TestThreatDetectionDomains(t *testing.T) { detectionMap[d] = true } for _, required := range requiredDomains { - assert.True(t, detectionMap[required], "Required domain %q not found in threat-detection ecosystem", required) + assert.True(t, detectionMap[required], "Required domain %q not found in threat-detection domain set", required) } // Detection domains must NOT include the domains excluded for supply-chain reduction @@ -437,12 +437,20 @@ func TestThreatDetectionDomains(t *testing.T) { "raw.githubusercontent.com", } for _, excluded := range excludedDomains { - assert.False(t, detectionMap[excluded], "Domain %q should not be in threat-detection ecosystem (excluded to reduce supply chain surface)", excluded) + assert.False(t, detectionMap[excluded], "Domain %q should not be in threat-detection domain set (excluded to reduce supply chain surface)", excluded) } // Verify exact count — no silent additions assert.Len(t, detectionDomains, len(requiredDomains), - "threat-detection ecosystem should have exactly %d entries", len(requiredDomains)) + "threat-detection domain set should have exactly %d entries", len(requiredDomains)) +} + +func TestThreatDetectionNetworkAllowedCompatibilityAlias(t *testing.T) { + engineDefaults := GetEngineDefaultDomainSets()["threat-detection"] + expanded := GetAllowedDomains(&NetworkPermissions{Allowed: []string{"threat-detection"}}) + + assert.Equal(t, engineDefaults, expanded, + "legacy network.allowed threat-detection alias must expand to the Copilot detection domain set") } func TestGetEngineDefaultDomainSets(t *testing.T) { @@ -473,10 +481,12 @@ func TestEmbeddedDomainSets(t *testing.T) { sets := getLoadedDomainSets() assert.Contains(t, sets.Ecosystems, "defaults") + assert.Contains(t, sets.Ecosystems, "threat-detection") assert.Contains(t, sets.EngineDefaults, "copilot") assert.Equal(t, "api.githubcopilot.com", sets.PiProviderDomains["copilot"]) assert.Equal(t, []string{"github.com", "localhost"}, sets.SanitizationDefaults) - assert.NotContains(t, sets.Ecosystems, "threat-detection") + assert.Equal(t, sets.EngineDefaults["threat-detection"], sets.Ecosystems["threat-detection"], + "legacy network.allowed threat-detection alias must stay in sync with Copilot detection defaults") } func TestGetThreatDetectionAllowedDomains(t *testing.T) { From 5e44cf7a7fb46d1ec7632aa80bf2adc3abf55676 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Mon, 24 Aug 2026 20:30:31 +0000 Subject: [PATCH 6/6] Finalize domain registry ADR Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com> --- ...61-centralize-engine-default-domain-sets.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/adr/55461-centralize-engine-default-domain-sets.md b/docs/adr/55461-centralize-engine-default-domain-sets.md index 6add95f2a6d..47298ddf071 100644 --- a/docs/adr/55461-centralize-engine-default-domain-sets.md +++ b/docs/adr/55461-centralize-engine-default-domain-sets.md @@ -1,18 +1,18 @@ # ADR-55461: Centralize Engine Default Domain Sets **Date**: 2026-08-24 -**Status**: Draft +**Status**: Accepted **Deciders**: Unknown --- ### Context -Engine-specific unconditional domain allow-lists were scattered across multiple separate static Go variables (`CopilotDefaultDomains`, `CodexDefaultDomains`, `ClaudeDefaultDomains`, `GeminiDefaultDomains`, `PiDefaultDomains`, `PiBaseDefaultDomains`) and an `ecosystem_domains.json` file that also served as the user-configurable ecosystem registry. This made it impossible to enumerate or inspect the full set of engine defaults in one place, and it conflated engine-internal allow-lists (automatically injected by the compiler) with user-selectable ecosystem identifiers (explicitly opted into via `network.allowed`). Specifically, the threat-detection allow-list being stored in the ecosystem JSON meant users could theoretically select it as a `network.allowed` entry, which was never the intended behavior. +Engine-specific unconditional domain allow-lists were scattered across multiple separate static Go variables (`CopilotDefaultDomains`, `CodexDefaultDomains`, `ClaudeDefaultDomains`, `GeminiDefaultDomains`, `PiDefaultDomains`, `PiBaseDefaultDomains`) and an `ecosystem_domains.json` file that also served as the user-configurable ecosystem registry. This made it impossible to enumerate or inspect the full set of engine defaults in one place, and it conflated engine-internal allow-lists (automatically injected by the compiler) with user-selectable ecosystem identifiers (explicitly opted into via `network.allowed`). Specifically, the threat-detection allow-list was only represented as an ecosystem entry, obscuring its role as an internal Copilot detection default. ### Decision -We will centralize all engine-specific unconditional domain allow-lists into a single unexported package-level map (`engineDefaultDomainSets`) and expose a copy-returning public accessor (`GetEngineDefaultDomainSets()`) for analysis and reporting. Existing exported compatibility variables (`CopilotDefaultDomains`, etc.) will derive their values from this registry at initialization time via a `copyEngineDefaultDomainSet` helper. The threat-detection allow-list will be removed from `ecosystem_domains.json` and moved exclusively into the new registry, preventing it from being user-selectable. +We will centralize all domain allow-list data in the embedded `ecosystem_domains.json` file and load it once with `sync.OnceValues`. Engine-specific unconditional domain allow-lists are exposed in Go through an unexported package-level map (`engineDefaultDomainSets`) and a copy-returning public accessor (`GetEngineDefaultDomainSets()`) for analysis and reporting. Existing exported compatibility variables (`CopilotDefaultDomains`, etc.) derive their values from this registry at initialization time via a `copyEngineDefaultDomainSet` helper. The threat-detection allow-list will live in the engine-default registry while the legacy `network.allowed: [threat-detection]` ecosystem alias is retained as a compatibility path. ### Alternatives Considered @@ -20,22 +20,22 @@ We will centralize all engine-specific unconditional domain allow-lists into a s Maintain each engine's domain list as its own `var` but introduce a function that aggregates them into a map for reporting. This avoids any initialization-time dependency between the registry and the compatibility variables, but leaves the domain lists distributed across the file. It does not solve the fundamental issue of drift between copies and fails to provide the single source of truth needed for the compiler to automatically reference them. -#### Alternative 2: Store all engine domain lists in a JSON/YAML configuration file +#### Alternative 2: Keep engine domain lists in Go code -Move all engine domain lists to a structured data file (similar to `ecosystem_domains.json`) and load them at runtime. This would make the lists editable without recompilation and easily inspectable, but introduces parse-at-startup overhead, a potential initialization failure path, and blurs the distinction between code-managed engine defaults and data-managed ecosystem lists. The type safety and inline documentation benefits of Go code are also lost. +Keep the engine domain lists as Go literals and only aggregate them in code. This preserves inline comments next to the data and avoids JSON parsing at package initialization time, but leaves the maintainer workflow split between code and data files and makes it harder to inspect all domain sets together. ### Consequences #### Positive -- Single authoritative source (`engineDefaultDomainSets`) for all engine allow-lists prevents content drift between the registry and the exported compatibility variables. +- Single authoritative embedded data source for all ecosystem and engine allow-lists prevents content drift between the registry and the exported compatibility variables. - `GetEngineDefaultDomainSets()` enables programmatic enumeration of all engine domain sets for analysis, reporting, and the new documentation tables in both network reference files. -- Threat-detection domains are correctly separated from user-selectable ecosystem identifiers, eliminating the path for users to select `threat-detection` as a `network.allowed` value. +- Threat-detection domains are represented as an engine-default set while preserving the legacy `network.allowed: [threat-detection]` compatibility alias. - Immutability is enforced: `GetEngineDefaultDomainSets()` returns deep copies, and exported variables are initialized from copies, so external callers cannot corrupt the registry. #### Negative - `engineDefaultDomainSets` is a mutable package-level variable (not a constant), so code in the same package could modify it at runtime; tests must guard against this. - Exported compatibility variables (`CopilotDefaultDomains`, etc.) now hold a snapshot from package initialization. Code that modifies these variables directly (e.g., in tests) will not affect what `GetEngineDefaultDomainSets()` returns, creating a subtle two-source-of-truth scenario within the package. -- Any future engine whose domain list needs dynamic construction cannot be expressed as a plain `[]string` literal in the map and will require refactoring the registry structure. +- Any future engine whose domain list needs dynamic construction cannot be fully expressed as a plain JSON array and will require refactoring the registry structure. #### Neutral - The PR adds `GetEngineDefaultDomainSets()` as a new public API surface. Future callers may depend on it, so the set of keys and the copy semantics become a stability commitment. @@ -43,4 +43,4 @@ Move all engine domain lists to a structured data file (similar to `ecosystem_do --- -*ADR created by [adr-writer agent]. Review and finalize before changing status from Draft to Accepted.* +*ADR finalized after implementation and compatibility review.*