Skip to content

Fully clean up buffers/memory resources in tests#2243

Merged
seberg merged 1 commit into
NVIDIA:mainfrom
seberg:ft-testing-mr-cleanup
Jun 23, 2026
Merged

Fully clean up buffers/memory resources in tests#2243
seberg merged 1 commit into
NVIDIA:mainfrom
seberg:ft-testing-mr-cleanup

Conversation

@seberg

@seberg seberg commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Maybe the final (small) split-out for the free-threading test PR.

Add explicit sync points and notes around mr.close() making sure that there is no MR shutdown with pending async buffer freeing. These can currently cause occasional dead-locks when we have many MRs in parallel. I believe this is a small upstream issue (625181).

Add explicit sync points and notes around `mr.close()` making sure
that there is no MR shutdown with pending async buffer freeing.
These can currently cause occasional dead-locks when we have many
MRs in parallel.

Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>
@seberg seberg added this to the cuda.core v1.1.0 milestone Jun 23, 2026
@seberg seberg self-assigned this Jun 23, 2026
@seberg seberg added P1 Medium priority - Should do test Improvements or additions to tests cuda.core Everything related to the cuda.core module labels Jun 23, 2026
@copy-pr-bot

copy-pr-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@mdboom

mdboom commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Somewhat adjacent issue: #1719

@mdboom mdboom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no reason not to merge this.

As an aside, though... How did you find all of these and is there a way to enforce that this kind of thing doesn't creep back in in the future?

@seberg

seberg commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

is there a way to enforce that this kind of thing doesn't creep back in in the future?

That'll be the full changes at #2194 to run with pytest-run-parallel, I was just splitting things out since it seemed a bit easier to move forward.

To be fair, cleanup eventually happens so with single threaded (no pytest-run-parallel) I am not sure this can fail (at least in practice).

/ok to test b478199

@seberg

seberg commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test b478199

@github-actions

This comment has been minimized.

@seberg seberg merged commit ac3b15d into NVIDIA:main Jun 23, 2026
114 checks passed
@seberg seberg deleted the ft-testing-mr-cleanup branch June 23, 2026 16:09
@github-actions

This comment has been minimized.

1 similar comment
@github-actions

Copy link
Copy Markdown
Doc Preview CI
Preview removed because the pull request was closed or merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.core Everything related to the cuda.core module P1 Medium priority - Should do test Improvements or additions to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants