Repository navigation
fix: skip sources emptied by the eligibility filters in create_question_set - #355
Open
Sahilkumar-r wants to merge 1 commit into
Open
Sahilkumar-r wants to merge 1 commit into
Sahilkumar-r wants to merge 1 commit into
Conversation
…ion_set` `drop_questions_that_resolve_too_soon` and `drop_questions_that_resolve_too_late` build their mask with `dfq.apply(..., axis=1)`, which on an empty frame returns a Series that drops every column, so a market source whose questions were all filtered out crashed with a `KeyError`. A source left empty by `prepare_source_questions` is now skipped with a warning, and the empty-frame tests assert that the columns survive. closes forecastingresearch#335 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When the eligibility filters remove every question of a sampled source, question set creation crashes instead of skipping that source (#335).
drop_questions_that_resolve_too_soon/drop_questions_that_resolve_too_latebuild their mask withdfq.apply(..., axis=1, result_type="reduce"); on an empty frame that returns a Series that drops every column, so a market source whose questions are all resolved ended up with a column-less frame and aKeyError. Theif dfq.empty: continueguard increate_question_setonly covered an empty input frame.Fix
create_question_setlogs a warning and skips a source thatprepare_source_questionsleaves empty.test_empty_market_frame_stays_emptytests now also assert the columns are kept. New tests: an empty data-source frame keeps its columns, and a market source or a dataset source whose questions are all resolved is skipped while the others are still sampled.Not changed
An empty
question_metadata.jsonlstill raisesKeyError: 'english'(the third case in #335).test_empty_metadata_passes_throughdocuments that empty metadata passes every question through, so fixing it means deciding what empty metadata should mean (drop everything vs. skip the English filter). I left that for a maintainer decision.I saw the maintainer note on #335 that this is low priority and best done after the question retirement work (#317).
drop_retired_questionsis already onmain, so I assumed that has landed; happy to close this if you would rather hold it.Verification
Ran in a fresh venv outside the repo (Windows, Python 3.13, pandas 2.3.3):
pytest src/tests/unit/question_sampling src/tests/unit/sources/test_kalshi.py: 269 passed.ruff format --checkandruff check(0.17.1) clean on the touched directories.src/tests,make lint/make testas such, anything needing GCP.AI disclosure
Written with Claude Code (Claude Sonnet 5.5) for @Sahilkumar-r. Please review it as AI-assisted work.
🤖 Generated with Claude Code