Update contributor guidelines regarding AI spam, reviews, code by Jefffrey · Pull Request #24510 · apache/datafusion

Jefffrey · 2026-08-20T07:49:38Z

From the discussion here:

Policy on uptick of LLM PRs from new contributors #24413

Updating our contributing guidelines, specifically to target cases where we see spam PRs from AI origins especially from new contributors. Aiming to set some rules/guidelines around this, and also hopefully make any agents involved at least reconsider before spamming

Jefffrey · 2026-08-20T07:49:58Z

Just an initial draft, can probably tighten up and or expand on some of the wording

codecov-commenter · 2026-08-20T08:15:44Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.62%. Comparing base (52e964c) to head (393dd0f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24510      +/-   ##
==========================================
- Coverage   82.62%   82.62%   -0.01%     
==========================================
  Files        1147     1147              
  Lines      445238   445238              
  Branches   445238   445238              
==========================================
- Hits       367878   367863      -15     
- Misses      55040    55051      +11     
- Partials    22320    22324       +4

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Rich-T-kid

This all make sense to me. I agree with @saadtajwar point about making a distinction between AI involvement and blind AI slop

alamb · 2026-08-25T10:56:39Z

 general it is both polite and will help avoid unnecessary duplication of work if
 you leave a note on an issue when you start working on it.

+If there is already a recent/active PR for an issue you plan to work on, please


maybe we can also suggest they help with the existing PR (e.g. help review it, work with the other contributor to get the PR to follow the guidelines / go through the review process)

alamb · 2026-08-25T10:59:21Z

+## Before starting work
+
+Before you start work on an issue, you MUST follow the instructions in
+[Open Contribution and Assigning tickets](docs/source/contributor-guide/index.md#open-contribution-and-assigning-tickets). You must ensure duplicate work is not being created.


We could perhaps soften this language "Please ensure duplicate work is not being created..."

Jefffrey · 2026-08-30T13:02:59Z

(i aim to revisit this soon and update with suggestions 👍)

…idelines

alamb · 2026-10-02T19:29:04Z

I am going to try and revive / update this, given it came up with a conversation with @neilconway today and apache/arrow-rs#11209 (comment) with @Jefffrey

alamb · 2026-10-02T20:25:43Z

I took the liberty of pushing a bunch of commits to this branch to refine the contributor guide for AI contributions and spam

alamb

I am obviously being biased here but I think these look good now. We should wait for others in the community to weigh in

2010YOUY01

Took a read, and I think it's a great improvement.

Maybe we can make ai contribution section a top-level doc page, like open a new section next to Introduction:

2010YOUY01 · 2026-10-03T00:53:14Z

+DataFusion has the following policy for AI-assisted PRs:
+
+- We welcome AI-assisted PRs from anyone. We do not welcome unreviewed "AI dumps" (defined below).
+- The PR author should have personally read the entire PR they submit, and **understand the core ideas** behind the implementation **end-to-end**. Authors should be ready to justify and help reviewers understand the design and code during review.


I feel “understand the core idea” is a bit vague now, and we could make the expectation more concrete. I also think setting a higher bar for PRs makes it easier to make progress during review.

Perhaps

“Understand the PR” means more than being able to follow the diff. It means: - Could reproduce the implementation without relying on AI. - Understand how the change fits into the surrounding architecture. - Can judge whether the design adds only necessary complexity and is maintainable long term.

Could reproduce the implementation without relying on AI

I might push back on this point as its a bit vague to enforce; does it mean you can write the PR after you got knowledge from working with the LLM, for example? Or just you mainly used LLM as a shortcut for the ideas you had in your brain

I know there are cases where LLMs can help iterate on an idea and identify edge cases, etc. so it can be confusing if this "disqualifies" the PR so to speak

I might push back on this point as its a bit vague to enforce;

I agree this might not convey the idea clearly. I think we can agree that a PR should be opened with sufficient understanding, but “understanding” itself is still a bit vague.

In practice, I see quite a few PRs where the contributor's understanding isn't deep enough when the PR is opened. That makes review much harder, and sometimes the review still effectively turns into the reviewer driving the AI through the contributor.

I'm not sure what the best way is to define the bar for “enough understanding/confidence to open a PR.” “Being able to reimplement it manually” seems like one concrete test for that bar, rather than the principle itself. And I agree edge cases finding should be excluded.

i do feel the "Authors should be ready to justify and help reviewers understand the design and code during review" should hopefully cover this; and if it does turn into reviewer driving the LLM then it'll fall into the part where we dont want them to just paste LLM output

kumarUjjawal · 2026-10-03T07:11:41Z

Maybe we can make ai contribution section a top-level doc page, like open a new section next to Introduction:

Agree!

neilconway · 2026-10-03T13:07:30Z

+If there is an existing PR for an issue you plan to work on, please review that
+PR before opening a new one. Duplicate, unacknowledged PRs consume valuable


Should we say "if there is an existing open PR"? There's a lot of older issues with one or more stale, long-closed PRs from prior attempts to work on them.

agreed, good to be explicit here 👍

Jefffrey

thanks for picking this up again 🙇

it looks pretty good 🙏

Jefffrey · 2026-10-03T15:09:35Z

+If you want to work on an issue which is not already assigned to someone and has
+no comment indicating someone is already working on it, you can assign the issue
+to yourself by submitting a single word comment `take`. However, if you are unable
+to make progress please unassign the issue by commenting a single word `untake`.


perhaps a side discussion, but something to note is even arrow (main repo) has turned off their take action:

GH-51232: [CI] Remove the "take" comment bot for self-assigning issues arrow#51233

it could be worth exploring as a case could be made it sometimes can stifle discussion if its too easy for someone to just come in an 'take' an issue (though i havent been keeping an eye on datafusion recently so im not sure if this is a concern)

Jefffrey · 2026-10-03T15:14:53Z

+DataFusion has the following policy for AI-assisted PRs:
+
+- We welcome AI-assisted PRs from anyone. We do not welcome unreviewed "AI dumps" (defined below).
+- The PR author should have personally read the entire PR they submit, and **understand the core ideas** behind the implementation **end-to-end**. Authors should be ready to justify and help reviewers understand the design and code during review.


Could reproduce the implementation without relying on AI

I might push back on this point as its a bit vague to enforce; does it mean you can write the PR after you got knowledge from working with the LLM, for example? Or just you mainly used LLM as a shortcut for the ideas you had in your brain

I know there are cases where LLMs can help iterate on an idea and identify edge cases, etc. so it can be confusing if this "disqualifies" the PR so to speak

Jefffrey · 2026-10-03T15:15:15Z

+If there is an existing PR for an issue you plan to work on, please review that
+PR before opening a new one. Duplicate, unacknowledged PRs consume valuable


agreed, good to be explicit here 👍

Jefffrey · 2026-10-03T15:17:53Z

+
+The same policy applies to review discussion as to the code itself: reviewers
+want to talk to **you**, not to your AI tool. Please do not paste an AI-generated
+response to a review comment verbatim or have your agent respond to


im not sure how often we see it here, but sometimes people might use it for translation. we could put a point where we allow it but we expect it to just translate. we could technically ask them to deepL it, but LLMs can be better at localizing and making it read a bit easier. the main point however is it should still represent the original message, and not fluff it up

Update contributor guidelines regarding AI spam

4f55df9

github-actions Bot added the documentation Improvements or additions to documentation label Aug 20, 2026

saadtajwar reviewed Aug 20, 2026

View reviewed changes

Comment thread docs/source/contributor-guide/index.md Outdated

Rich-T-kid approved these changes Aug 20, 2026

View reviewed changes

alamb reviewed Aug 25, 2026

View reviewed changes

Merge remote-tracking branch 'apache/main' into update-contributor-gu…

5c47448

…idelines

fixup merge

b423e16

alamb mentioned this pull request Oct 2, 2026

ci: Enforce PR limit to 3 for non-committers #25973

Merged

alamb added 3 commits October 2, 2026 16:03

updates

681d0e2

refine

81fd398

refine

3954921

alamb marked this pull request as ready for review October 2, 2026 20:23

refine

2f9c6e0

alamb approved these changes Oct 2, 2026

View reviewed changes

note about reviews

908c18d

alamb changed the title ~~Update contributor guidelines regarding AI spam~~ Update contributor guidelines regarding AI spam, reviews, code Oct 2, 2026

spelling

4bdb15c

alamb mentioned this pull request Oct 2, 2026

Discussion: guidelines for LLM-generated PR reviews #25317

Open

Merge branch 'main' into update-contributor-guidelines

393dd0f

2010YOUY01 approved these changes Oct 3, 2026

View reviewed changes

neilconway approved these changes Oct 3, 2026

View reviewed changes

Jefffrey commented Oct 3, 2026

View reviewed changes

		If there is an existing PR for an issue you plan to work on, please review that
		PR before opening a new one. Duplicate, unacknowledged PRs consume valuable

Conversation

Jefffrey commented Aug 20, 2026 • edited by alamb Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jefffrey commented Aug 20, 2026

Uh oh!

codecov-commenter commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Rich-T-kid left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jefffrey commented Aug 30, 2026

Uh oh!

alamb commented Oct 2, 2026

Uh oh!

alamb commented Oct 2, 2026

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

2010YOUY01 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kumarUjjawal commented Oct 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jefffrey left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Jefffrey commented Aug 20, 2026 •

edited by alamb

Loading

codecov-commenter commented Aug 20, 2026 •

edited

Loading