Test: SDK-based `permissions` via managed settings

[joshspicer]

Refs: #327187, #327383

• anyOS Don Jayamanne (@DonJayamanne)
• anyOS Vijay Upadya (@vijayupadya)
• anyOS roblourens

Complexity: 5

Authors: joshspicer

Create Issue

---

The Copilot SDK that we adopt via AHP now enforces permissions, a generic and cross-surface way for enterprises to provide fine-grained controls for various tool calls.

The following selectors are currently supported

Shell
Read
Edit
Domain

Setup

Use the file-based managed settings setup from #323147. Configure Shell rules for deny, ask, and allow, then start a fresh Agents session with Allow All enabled.

This is an example managed-settings.json. Modify and start a new session for it to take effect.

{
  "permissions": {
    "allow": [
      "Shell(echo allowed *)",
      "Shell(echo conflict *)",
      "Read(/managed-allow/**)",
      "Edit(/managed-allow/**)",
      "Domain(api.github.com)"
    ],
    "ask": [
      "Shell(echo ask *)",
      "Shell(echo conflict *)",
      "Read(/managed-ask/**)",
      "Edit(/managed-ask/**)", 
      "Domain(github.com)"
    ],
    "deny": [
      "Shell(echo denied *)",
      "Shell(echo conflict *)",
      "Read(~/.ssh/**)",
      "Edit(/managed-deny/**)",
      "Read(//tmp/test-file.txt)", 
      "Domain(raw.githubusercontent.com)"
    ]
  }
}

Use this one-liner on mac to install the config above

sudo mkdir -p "/Library/Application Support/GitHubCopilot" && printf '%s\n' '{"permissions":{"allow":["Shell(echo managed-allowed *)","Read(/managed-allow/**)","Edit(/managed-allow/**)","Domain(api.github.com)"],"ask":["Shell(echo managed-ask *)","Read(/managed-ask/**)","Edit(/managed-ask/**)","Domain(github.com)"],"deny":["Shell(echo managed-denied *)","Read(~/.ssh/**)","Edit(/managed-deny/**)","Domain(raw.githubusercontent.com)"]}}' | sudo tee "/Library/Application Support/GitHubCopilot/managed-settings.json" >/dev/null

Prefix	Resolution	Example
/	Relative to the workspace root	Read(/src/**)
./	Relative to the current working directory	Edit(./generated/**)
~/	Relative to the current user’s home directory	Read(~/.ssh/**)
//	Relative to the filesystem root	Edit(//etc/**)

Manually create the example files above (from Read/Edit) manually so that they exist! Note that there is a known bug with the workspace root being correctly detected from VS Code - #329955

TIP: You can confirm that the file has been loaded by the copilot SDK with > Developer: Policy Diagnostics and look at the Agent Runtime Resolution section.

Task

Please exercise these new permissions in any way you can think of. We are looking for edge cases in the new permissions model, so please get creative! Some basic things to test inclue:

• Confirm that the precedence (deny > ask > allow > default ask) is enforced
• Confirm that 'ask' prompts you for approval each time
• Confirm that deny fully denies you (no option to approve, explained in tool response from SDK)
• Confirm that 'allow' will allow without prompts (NOTE: VS Code client settings must ALSO be enabled to allow auto approval)
• Confirm that everything else defaults to 'ask' when this file is present.
• 'echo conflict ' should be denied due to precedence rules

Please also try these commands in the Copilot CLI, who should read the exact same file and behave the exact same way (same implementation).

⚠️ FOR ANY ISSUES, PLEASE > Developer: Export Agent Host Debug Logs... and send to Josh

Examples

Some example screenshots are below. Note that the client has 'Allow all' selected, but this config is enforced in the SDK and therefore continues to enforce the allow/ask/deny pattern.

deny file read

ask Domain

deny domain

ask terminal command

deny terminal command

Known issues

• SDK permissions: Workspace path is not detected correctly #329955

[DonJayamanne]

joshspicer Fairly complex and managed to get the help of Copilot to help with a few simple scenarios and all worked.

[roblourens]

The / and // thing is pretty confusing, is that standard?

[joshspicer]

The / and // thing is pretty confusing, is that standard?

Yea, the idea is to align Edit() and Read() with Claude Code configs, e.g https://code.claude.com/docs/en/permissions#read-and-edit

cc/ Harald Kirschner (@digitarald)

[joshspicer]

Follow-up from re-running this TPI on August 26: reopened #329955 because the current Agent Host bundle was still on runtime 1.0.81-0, while the workspace-root Read/Edit fix first shipped in 1.0.81-1. The Edit(/managed-deny/**) scenario prompted and could be approved instead of being denied.

The pending runtime adoption is #332803. Re-run the workspace-root allow/deny cases after that bump lands before considering the deployment verified.