Skip to content

Update contributor guidelines regarding AI spam, reviews, code #24510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Jefffrey wants to merge 10 commits into
base: main
Choose a base branch
from
+114 −39
Open

Update contributor guidelines regarding AI spam, reviews, code #24510

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
4f55df9
Update contributor guidelines regarding AI spam
Jefffrey Aug 20, 2026
5c47448
Merge remote-tracking branch 'apache/main' into update-contributor-gu…
alamb Oct 1, 2026
b423e16
fixup merge
alamb Oct 2, 2026
681d0e2
updates
alamb Oct 2, 2026
81fd398
refine
alamb Oct 2, 2026
3954921
refine
alamb Oct 2, 2026
2f9c6e0
refine
alamb Oct 2, 2026
908c18d
note about reviews
alamb Oct 2, 2026
4bdb15c
spelling
alamb Oct 2, 2026
393dd0f
Merge branch 'main' into update-contributor-guidelines
alamb Oct 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,19 @@

- [Quick Start Setup](docs/source/contributor-guide/development_environment.md#quick-start)
- [Testing Quick Start](docs/source/contributor-guide/testing.md#testing-quick-start)
- [Open Contribution and Assigning tickets](docs/source/contributor-guide/index.md#open-contribution-and-assigning-tickets)
- [Choosing What Kind of Test to Write](docs/source/contributor-guide/testing.md#choosing-what-kind-of-test-to-write)
- [Before Submitting a PR](docs/source/contributor-guide/index.md#before-submitting-a-pr)
- [Reviewing Pull Requests](docs/source/contributor-guide/pr_review.md)
- [Contributor Guide](docs/source/contributor-guide/index.md)
- [Architecture Guide](docs/source/contributor-guide/architecture.md)

## Before Starting Work

Before you start work on an issue, you MUST follow the instructions in
[Open Contribution and Assigning tickets](docs/source/contributor-guide/index.md#open-contribution-and-assigning-tickets). You must ensure duplicate work is not being created.

@alamb alamb Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could perhaps soften this language "Please ensure duplicate work is not being created..."

Do not create multiple PRs if this is your first time contributing.

## Before Committing

Before committing any changes, you MUST follow the instructions in
Expand Down
140 changes: 101 additions & 39 deletions docs/source/contributor-guide/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,24 +54,29 @@ Contributors drive the project forward based on their own priorities and
interests and thus you are free to work on any issue that interests you.

If someone is already working on an issue that you want or need but hasn't
been able to finish it yet, you should feel free to work on it as well. In
general it is both polite and will help avoid unnecessary duplication of work if
you leave a note on an issue when you start working on it.
been able to finish it yet, feel free to help them out.

If you want to work on an issue which is not already assigned to someone else
and there are no comment indicating that someone is already working on that
issue then you can assign the issue to yourself by submitting a single word
comment `take`. This will assign the issue to yourself. However, if you are
unable to make progress you should unassign the issue by commenting a single
word `untake`.
If there is an existing PR for an issue you plan to work on, please review that
PR before opening a new one. Duplicate, unacknowledged PRs consume valuable
Comment on lines +59 to +60

@neilconway neilconway Oct 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we say "if there is an existing open PR"? There's a lot of older issues with one or more stale, long-closed PRs from prior attempts to work on them.

@Jefffrey Jefffrey Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, good to be explicit here 👍

reviewer time and we may close them. If there is an existing PR, please identify
it in the PR description and explain why you are opening a new one and not
helping with the previous one. In general it is both polite and will help avoid
unnecessary duplication of work if you also leave a note on an issue when you
start working on it.

If you want to work on an issue which is not already assigned to someone and has
no comment indicating someone is already working on it, you can assign the issue
to yourself by submitting a single word comment `take`. However, if you are unable
to make progress please unassign the issue by commenting a single word `untake`.

@Jefffrey Jefffrey Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps a side discussion, but something to note is even arrow (main repo) has turned off their take action:

it could be worth exploring as a case could be made it sometimes can stifle discussion if its too easy for someone to just come in an 'take' an issue (though i havent been keeping an eye on datafusion recently so im not sure if this is a concern)


# Developer's guide

## Pull Request Overview

We welcome pull requests (PRs) from anyone in the community.

DataFusion is a rapidly evolving project and we try to review and merge PRs quickly.
DataFusion is a rapidly evolving project and we try to review and merge PRs
quickly.

Review bandwidth is currently our most limited resource, and we highly encourage reviews by the broader community. If you are waiting for your PR to be reviewed, consider helping review other PRs that are waiting. Such review both helps the reviewer to learn the codebase and become more expert, as well as helps identify issues in the PR (such as lack of test coverage), that can be addressed and make future reviews faster and more efficient.

Expand All @@ -95,7 +100,7 @@ When possible, we recommend splitting your contributions into multiple smaller f

1. The PR is more likely to be reviewed quickly -- our reviewers struggle to find the contiguous time needed to review large PRs.
2. The PR discussions tend to be more focused and less likely to get lost among several different threads.
3. It is often easier to accept and act on feedback when it comes early on in a small change, before a particular approach has been polished too much.
3. It is often easier to accept and act on feedback when it comes early in a small change, before a particular approach has been polished too much.

If you are concerned that a larger design will be lost in a string of small PRs, creating a large draft PR that shows how they all work together can help.

Expand Down Expand Up @@ -129,6 +134,87 @@ Please ensure your PR follows the [testing guide](testing.md). In particular:
[Choosing What Kind of Test to Write](testing.md#choosing-what-kind-of-test-to-write).
- Run any relevant commands from the [testing quick start](testing.md#testing-quick-start).

## AI-Assisted contributions

DataFusion has the following policy for AI-assisted PRs:

- We welcome AI-assisted PRs from anyone. We do not welcome unreviewed "AI dumps" (defined below).
- The PR author should have personally read the entire PR they submit, and **understand the core ideas** behind the implementation **end-to-end**. Authors should be ready to justify and help reviewers understand the design and code during review.

@2010YOUY01 2010YOUY01 Oct 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel “understand the core idea” is a bit vague now, and we could make the expectation more concrete. I also think setting a higher bar for PRs makes it easier to make progress during review.

Perhaps

“Understand the PR” means more than being able to follow the diff. It means:

- Could reproduce the implementation without relying on AI.
- Understand how the change fits into the surrounding architecture.
- Can judge whether the design adds only necessary complexity and is maintainable long term.

@Jefffrey Jefffrey Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could reproduce the implementation without relying on AI

I might push back on this point as its a bit vague to enforce; does it mean you can write the PR after you got knowledge from working with the LLM, for example? Or just you mainly used LLM as a shortcut for the ideas you had in your brain

I know there are cases where LLMs can help iterate on an idea and identify edge cases, etc. so it can be confusing if this "disqualifies" the PR so to speak

@2010YOUY01 2010YOUY01 Oct 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might push back on this point as its a bit vague to enforce;

I agree this might not convey the idea clearly. I think we can agree that a PR should be opened with sufficient understanding, but “understanding” itself is still a bit vague.

In practice, I see quite a few PRs where the contributor's understanding isn't deep enough when the PR is opened. That makes review much harder, and sometimes the review still effectively turns into the reviewer driving the AI through the contributor.

I'm not sure what the best way is to define the bar for “enough understanding/confidence to open a PR.” “Being able to reimplement it manually” seems like one concrete test for that bar, rather than the principle itself. And I agree edge cases finding should be excluded.

@Jefffrey Jefffrey Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i do feel the "Authors should be ready to justify and help reviewers understand the design and code during review" should hopefully cover this; and if it does turn into reviewer driving the LLM then it'll fall into the part where we dont want them to just paste LLM output

@2010YOUY01 2010YOUY01 Oct 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i do feel the "Authors should be ready to justify and help reviewers understand the design and code during review" should hopefully cover this;

100%, this is a good practical test for 'enough undersantinding for the PR'

I think it also worth a separate documentation section for 'how to write good PR/Issue description', I'll give it a try later.

- **Call out unknowns and assumptions**. It's okay to not fully understand some bits of AI-generated code. Please point these cases out so we can work together to clear up any concerns.

### What is an "AI dump" and why it is not helpful

An "AI dump" is a PR, or a series of PRs, consisting largely of AI-generated
code and descriptions that the author has not reviewed and does not understand.
The code may even be correct. The problem is that all the work of understanding
falls on the reviewer.

Code review serves two purposes:

1. Finish the intended task.
2. Share knowledge between authors and reviewers, as a long-term investment in
the project. For this reason, even if someone familiar with the codebase
could finish a task more quickly by themselves, we are still happy to help
a new contributor work on it.

An AI dump meets neither purpose. Maintainers could finish the task faster by
running the AI tool themselves, and an author who acts only as a pass-through
proxy for the tool learns little from the review.

Reviewing capacity for the project is **very limited**, so PRs that appear to be
AI dumps may not get reviewed, and may eventually be closed.

Multiple PRs created in a short amount of time, especially by a first-time
contributor, that in our judgment show a lack of understanding or author
engagement may be treated as spam and closed. One high quality PR that you work
with maintainers to merge is far more valuable to you and the project than ten
PRs you have your agent generate and submit for you.

### Responding to review comments

The same policy applies to review discussion as to the code itself: reviewers
want to talk to **you**, not to your AI tool. Please do not paste an AI-generated
response to a review comment verbatim or have your agent respond to

@Jefffrey Jefffrey Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not sure how often we see it here, but sometimes people might use it for translation. we could put a point where we allow it but we expect it to just translate. we could technically ask them to deepL it, but LLMs can be better at localizing and making it read a bit easier. the main point however is it should still represent the original message, and not fluff it up

reviewer comments. Some signs that a reply is an unreviewed AI dump:

- It summarizes the diff rather than answering the question that was asked.
- It lists the commands that were run locally (e.g. `cargo fmt`, `cargo test`)
and whether they passed. This is not useful to reviewers because CI already
runs these checks.
- It contains statements that don't make sense in context, such as claiming
tests could not be run because `cargo` is not installed.

For example, see [this review thread in arrow-rs][arrow-rs-review-example] where
the reviewer asked a design question, and received several replies that
described what had changed and which commands had been run, rather than an
answer to the question.

The point of code review is to help the project **and** to help you grow as an
engineer, so please read each comment, make sure you understand it, and reply
in your own words.

[arrow-rs-review-example]: https://github.com/apache/arrow-rs/pull/11209#discussion_r4145512124

### AI-assisted reviews

The same standard applies to AI-generated reviews as to AI-generated code: you
should have read and understand anything you post. Raw AI review output is often
verbose with unnecessary details, and it can take substantial effort to figure
out what is actually being asked.

If you review PRs with the help of an AI tool, read all comments first, remove
detail that is unnecessary or you don't understand, and explain the rest in your
own words so that each comment makes a clear, specific request.

### Better ways to contribute than an “AI dump”

It's recommended to write a high-quality issue with a clear problem statement
and a minimal, reproducible example. The reproducer should focus on the end-user-visible
behavior rather than explaining the details of some code defect.

This will make it easier for others to contribute and for reviewers to
understand the problem being addressed.

## Conventional Commits & Labeling PRs

We generate change logs for each release using an automated process that will categorize PRs based on the title
Expand Down Expand Up @@ -192,33 +278,9 @@ The good thing about open code and open development is that any issues in one ch
Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
Commenting on the PR will remove the `stale` label.

## AI-Assisted contributions

DataFusion has the following policy for AI-assisted PRs:

- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and be able to justify the design and code during review.
- **Calls out unknowns and assumptions**. It's okay to not fully understand some bits of AI generated code. You should comment on these cases and point them out to reviewers so that they can use their knowledge of the codebase to clear up any concerns. For example, you might comment "calling this function here seems to work but I'm not familiar with how it works internally, I wonder if there's a race condition if it is called concurrently".

### Why fully AI-generated PRs without understanding are not helpful

Today, AI tools cannot reliably make complex changes to DataFusion on their own, which is why we rely on pull requests and code review.

The purposes of code review are:

1. Finish the intended task.
2. Share knowledge between authors and reviewers, as a long-term investment in the project. For this reason, even if someone familiar with the codebase can finish a task quickly, we're still happy to help a new contributor work on it even if it takes longer.

An AI dump for an issue doesn’t meet these purposes. Maintainers could finish the task faster by using AI directly, and the submitters gain little knowledge if they act only as a pass through AI proxy without understanding.

Please understand the reviewing capacity is **very limited** for the project, so large PRs which appear to not have the requisite understanding might not get reviewed, and eventually closed or redirected.

### Better ways to contribute than an “AI dump”

It's recommended to write a high-quality issue with a clear problem statement and a minimal, reproducible example. This can make it easier for others to contribute.

### CI Runners
## CI Runners

#### Runs-On
### Runs-On

We use [Runs-On](https://runs-on.com/) for some actions in the main repository, which run in the ASF AWS account to speed up CI. In forks, these actions run on the default GitHub runners since forks do not have access to ASF infrastructure.

Expand All @@ -234,7 +296,7 @@ For those actions we also use the [Runs-On action](https://runs-on.com/caching/m

For the standard GitHub runners, this action will do nothing.

##### Spot Instances
#### Spot Instances

By default, Runs-On actions run as [spot instances](https://runs-on.com/configuration/spot-instances/), which means they might occasionally be interrupted. In the CI you would see:

Expand All @@ -244,6 +306,6 @@ Error: The operation was canceled.

According to Runs-On, spot instance termination is extremely rare for instances running for less than 1h. Those actions will be restarted automatically.

#### GitHub Runners
### GitHub Runners

We also use standard GitHub runners for some actions in the main repository; these are also runnable in forks.
6 changes: 6 additions & 0 deletions docs/source/contributor-guide/pr_review.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,12 @@ Rather than a bare "LGTM", say what you actually checked ("traced the state
transitions by hand", "confirmed the hasher change cannot affect ordering")
so it is clear what was verified and what was not.

### Use AI Review Tools Thoughtfully

AI tools can help you review, but please read and edit their output before
posting it so each comment is a clear, specific request in your own words. See
[AI-assisted reviews](index.md#ai-assisted-reviews) in the contributor guide.

### Invite Additional Committers on Core Changes

For changes to core, widely shared code, leave the PR open for other
Expand Down
Loading