Repository navigation
Conversation
| ### What this agent must always do | ||
|
|
||
| - Add an `Assisted-by: AGENT_NAME:MODEL_VERSION` git trailer to every commit containing AI-assisted content. | ||
| - Add an `AI-Assisted-By: MODEL (REASONING_EFFORT)` git trailer to every commit containing AI-assisted content, for example `AI-Assisted-By: Claude Opus 4.5 (medium)`. Ask the contributor for the reasoning effort if you do not know it. Do not add your own agent name unless the contributor asks for it. |
There was a problem hiding this comment.
I think the policy is also interested in the harness used, but I agree that it's confusing that we're saying AGENT_NAME when we mean HARNESS_NAME
| - Add an `AI-Assisted-By: MODEL (REASONING_EFFORT)` git trailer to every commit containing AI-assisted content, for example `AI-Assisted-By: Claude Opus 4.5 (medium)`. Ask the contributor for the reasoning effort if you do not know it. Do not add your own agent name unless the contributor asks for it. | |
| - Add an `AI-Assisted-By: HARNESS:MODEL (REASONING_EFFORT)` git trailer to every commit containing AI-assisted content, for example `AI-Assisted-By: Claude code:Claude Opus 4.5 (medium)`. Ask the contributor for the reasoning effort if you do not know it. |
There was a problem hiding this comment.
This is my own interpretation, I might be wrong the maintainers can correct me on it
49ca625 to
b610600
Compare
There was a problem hiding this comment.
I'd currently veto this change and would like to stick to what we have.
AI- adds no extra value, it just makes the trailer more specific and limited to AI use. Linux kernel discusses dropping it completely which I think is not a good idea given the EU AI Act.
The trailers goal is to know about the IF and WHAT, not the "HOW"
So in its current form it is generic enough to also server other attributions outside of AI use.
In a sense this is somewhat meta data, so changing the format is not desired.
b610600 to
56c1f58
Compare
|
Thanks @AndyScherzinger. I've cut this down: the key and the The effort is the "how", but it changes the "what": the same model gives measurably different results depending on it. The agent mostly tells attackers which tool runs on a contributor's machine, so I'd make it optional. The 26 commits with I'd keep the model required, even though the kernel dropped it in August. With the same agent, SWE-agent solved 18.0% of SWE-bench Lite with GPT-4 Turbo and 13.0% with Claude 3 Opus, and the share of hallucinated package names was about four times higher on average for open-source than for commercial models. If problems accumulate with a model, or one quantization of an open-weight model, the trailer lets us find those commits and ask for more, such as tests. On the AI Act: the trailer stays, but the Act doesn't require it from contributors. Art. 50 puts machine-readable marking on the providers of AI systems. Deployers only have to disclose AI-generated text if it is published to inform the public on matters of public interest, like news, and not even then if a human reviewed it and takes editorial responsibility. A commit or PR description isn't that kind of text, and its author reviews it anyway. |
Ask for the reasoning effort wherever the tool offers a setting and make the agent name optional, so the trailer reads `Assisted-by: MODEL_VERSION (REASONING_EFFORT)` or `Assisted-by: AGENT_NAME:MODEL_VERSION (REASONING_EFFORT)`. Existing commits with AGENT_NAME:MODEL_VERSION stay as they are. For an open-weight model run locally, the model version includes the quantization. The agent mostly describes the contributor's setup, and a public record of which agent each contributor runs helps attackers more than reviewers. The model, its quantization and the reasoning effort change what the contribution looks like, so they tell a reviewer more. Mention AI-Assisted-By: for content an LLM generated; the AI Policy check has recognised it since nextcloud#742. Replace qwen3-coder-32b in the examples, which does not exist, with qwen3-coder-30b-a3b-instruct-q4_k_m. Assisted-by: Claude Opus 5.5 (xhigh) Signed-off-by: MiMoHo <37556964+MiMoHo@users.noreply.github.com>
56c1f58 to
264548d
Compare
The policy asks for
Assisted-by: AGENT_NAME:MODEL_VERSION. This PR keeps the key and theAGENT_NAME:MODEL_VERSIONpart and changes two things: the reasoning effort is added wherever the tool offers a setting, and the agent name becomes optional.Tools without an effort setting leave it out, for example
Assisted-by: Copilot:gpt-4o. Existing commits such asAssisted-by: ClaudeCode:claude-sonnet-4-6don't need to be rewritten, and the AI Policy check doesn't change.Assisted-by:keeps covering any kind of help. For content an LLM generated,AI-Assisted-By:says so explicitly; the check has recognised it since #742.The model stays required, because it is the part a reviewer can act on. With the same agent, SWE-agent resolved 18.0% of the SWE-bench Lite issues with GPT-4 Turbo and 13.0% with Claude 3 Opus, and its authors found Llama 3 and DeepSeek Coder "subpar" in that setting. The typical mistakes differ too: across 16 code models, the share of hallucinated package names averaged at least 5.2% for commercial and 21.7% for open-source models (Spracklen et al., USENIX Security 2025), which matters for every PR that adds a dependency. If problems pile up with one model, the trailer lets maintainers find the affected commits with
git log --grep, look at them again and ask PRs made with that model for more, such as tests. For an open-weight model running locally, the policy now counts the quantization as part of the model version, for exampleqwen3-coder-30b-a3b-instruct-q4_k_m. 8-bit formats lose little or nothing (Kurtic et al., ACL 2025), but Llama 3 still degrades noticeably at very low bit widths (Huang et al., Visual Intelligence 2024).The reasoning effort tells a reviewer more than the agent does, because the same model gives measurably different results depending on how much it is allowed to think. Anthropic reports that Opus 4.5 at medium effort matches Sonnet 4.5's best score on SWE-bench Verified and only pulls ahead, by 4.3 points, at its highest effort. Peer-reviewed work points the same way. On problems a smaller model can partly solve, extra compute at inference time can beat a 14 times larger model (Snell et al., ICLR 2025), and a 7B model with a better search strategy consistently outperformed a 34B model on the MATH benchmark (Wu et al., ICLR 2025). The model name alone is only half the information.
The agent, on the other hand, mostly describes the contributor's setup, and that is what attackers care about. Coding agents run with access to the shell, the source code and often credentials. In the Nx supply chain attack, malicious packages used locally installed AI command line tools to search machines for secrets, and Invariant Labs showed how a crafted GitHub issue can take over a coding agent. A permanent, public record of which agent each GitHub user runs makes it easier to pick targets. Making it optional costs reviewers little: in nextcloud/server, the 26 commits from the last twelve months that use
AI-Assisted-By:already name only the model.Some objections I expect:
Assisted-by:andAGENT_NAME:MODEL_VERSIONstay; the effort is added and the agent becomes optional.Assisted-by: LLM, because naming models gives proprietary vendors free advertising. I see the model as a fact about how a change was made, the same way the bug report template asks for the server and PHP version. The policy's examples already name open-weight models, Devstral and Qwen3-Coder, and Nextcloud's own Ethical AI Rating asks whether a model is freely available. That question needs the model name.Assisted-by: Copilot:gpt-4o.I also replaced
qwen3-coder-32bin the examples. I couldn't find a Qwen3-Coder model with 32B parameters, so the example now uses Qwen3-Coder-30B-A3B-Instruct with a 4-bit quantization.Like #816, this PR reduces what contributors have to make public about themselves: there their real name, here the tools they run. Each of the two can be merged on its own.
🤖 AI (if applicable)
The model helped me draft this change. I reviewed and edited everything. The commit carries the same line as a trailer.
Assisted-by: Claude Opus 5.5 (xhigh)