Skip to content

fix: stop get_submission from changing process-wide warning filters - #378

Open
Ardiea wants to merge 1 commit into
openedx:masterfrom
mitodl:md/202610_thread_safe_get_submission
Open

Ardiea wants to merge 1 commit into
openedx:masterfrom
mitodl:md/202610_thread_safe_get_submission

Conversation

@Ardiea

@Ardiea Ardiea commented Oct 9, 2026

Copy link
Copy Markdown

Description

get_submission() calls warnings.filterwarnings('error') before loading the submission and warnings.filterwarnings('default') in finally. It does this to turn jsonfield's invalid-JSON RuntimeWarning back into an exception, which jsonfield raised itself before 3.1.0.

warnings.filters is a single list shared by the whole process. Under a threaded WSGI server (we run Granian with 8 blocking threads per worker), this causes two problems:

  1. Unrelated requests fail. While one thread is inside get_submission, any warning raised on another thread is raised as an exception. We see this in production as unhandled DeprecationWarnings (500s) from code that has nothing to do with submissions. For example, datetime.utcnow() in SessionInactivityTimeout.process_request fails on /event, /courseware-search/enabled/ and xblock handler routes.
  2. The host's warning config is overridden. filterwarnings('default') puts a catch-all default filter at the front of the list, and it stays there after the first call. That cancels edx-platform's ignore rules in openedx/core/lib/logsettings.py (for example Setting _field_data is deprecated). The ('error', None, Warning, ...) entry also stays in the list, with only that default in front of it.

warnings.catch_warnings() would not fix this. Before Python 3.14 it also swaps the process-wide filter list.

Change

  • Submission.answer now uses AnswerJSONField, a JSONField subclass. Its from_db_value returns invalid JSON as an InvalidJSONString (a JSONString subclass) and still emits jsonfield's RuntimeWarning. jsonfield wraps both valid JSON strings and invalid raw values in JSONString, so a separate marker type is needed to tell them apart.
  • get_submission() checks for InvalidJSONString and raises SubmissionInternalError as before. It no longer touches warnings.filters.
  • AnswerJSONField.deconstruct() returns jsonfield.fields.JSONField as the field path. No migration is generated (makemigrations --check is clean), so nothing alters the submissions table. On SQLite an AlterField here rebuilds the table.
  • Version bumped to 4.0.3.

Answers are read and written the same way as before, and get_submission still uses one query.

Testing

  • test_load_non_json_answer (existing) still passes: a corrupt raw_answer raises SubmissionInternalError from get_submission and get_submission_and_student.
  • New test_get_submission_leaves_warning_filters_alone (string and dict answers) asserts that warnings.filters is the same after the call. It fails on master.
  • New test_load_json_string_answer_that_looks_like_invalid_json: a valid JSON string answer '}' comes back as the answer and does not raise.
  • Full suite: 221 passed. make test_quality (pylint, isort, pycodestyle) is clean.

🤖 Generated with Claude Code

get_submission called warnings.filterwarnings('error') to turn jsonfield's
invalid-JSON RuntimeWarning back into an exception. warnings.filters is
shared by every thread, so under a threaded server any warning raised on
another request while one thread was inside get_submission became an
unhandled exception. The 'default' filter it added in `finally` also stayed
at the front of the list for good, overriding the host application's
ignore rules.

Detect corrupt answers in the field instead: Submission.answer now returns
invalid JSON as an InvalidJSONString marker, and get_submission raises
SubmissionInternalError on that, as before. deconstruct() keeps the parent
path, so no migration touches the submissions table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Oct 9, 2026
@openedx-webhooks

openedx-webhooks commented Oct 9, 2026 •

Copy link
Copy Markdown

Thanks for the pull request, @Ardiea!

This repository is currently maintained by @openedx/committers-edx-submissions.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@github-project-automation github-project-automation Bot moved this to Needs Triage in Contributions Oct 9, 2026
@Ardiea
Ardiea marked this pull request as ready for review October 9, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

open-source-contribution PR author is not from Axim or 2U

Projects

Status: Needs Triage

Development

Successfully merging this pull request may close these issues.

2 participants