Skip to content

Jmafoster1/329 estimator endpoint#382

Open
jmafoster1 wants to merge 15 commits intomainfrom
jmafoster1/329-estimator-endpoint
Open

Jmafoster1/329 estimator endpoint#382
jmafoster1 wants to merge 15 commits intomainfrom
jmafoster1/329-estimator-endpoint

Conversation

@jmafoster1
Copy link
Copy Markdown
Collaborator

Closes #329. This PR makes estimators and expected causal effects endpoints instead of having to manually convert via a dict.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

🦙 MegaLinter status: ✅ SUCCESS

Descriptor Linter Files Fixed Errors Elapsed time
✅ PYTHON black 32 0 1.09s
✅ PYTHON pylint 32 0 6.63s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by OX Security

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.89%. Comparing base (f4361b5) to head (6dae63f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #382      +/-   ##
==========================================
- Coverage   97.90%   97.89%   -0.01%     
==========================================
  Files          27       27              
  Lines        1572     1568       -4     
==========================================
- Hits         1539     1535       -4     
  Misses         33       33              
Files with missing lines Coverage Δ
causal_testing/main.py 100.00% <100.00%> (ø)
causal_testing/testing/metamorphic_relation.py 100.00% <100.00%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2bd52f1...6dae63f. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jmafoster1 jmafoster1 force-pushed the jmafoster1/329-estimator-endpoint branch from ea362a8 to 1458192 Compare April 29, 2026 14:04
@jmafoster1 jmafoster1 requested a review from f-allian April 29, 2026 14:14
@jmafoster1 jmafoster1 force-pushed the jmafoster1/329-estimator-endpoint branch from b8f0007 to 8c99927 Compare April 30, 2026 09:58
Comment thread causal_testing/main.py
Comment on lines -284 to 300
# Handle combined queries (global and test-specific)
test_query = test.get("query")
combined_query = None

if self.query and test_query:
combined_query = f"({self.query}) and ({test_query})"
logger.info(
f"Combining global query '{self.query}' with test-specific query "
f"'{test_query}' for test '{test['name']}'"
if test["estimator"] not in estimator_map:
raise ValueError(
f"Unsupported estimator {test['estimator']}. Supported: {sorted(estimator_map)}. "
"If you have implemented a custom estimator, you will need to add this to your entrypoints via your "
"pyproject.toml file."
)
elif test_query:
combined_query = test_query
logger.info(f"Using test-specific query for '{test['name']}': {test_query}")
elif self.query:
combined_query = self.query
logger.info(f"Using global query for '{test['name']}': {self.query}")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmafoster1 Are we dropping functionality for test-specific queries now?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the estimators all handle it anyway, so there's no need to do it here as well. The alternative would be to move the "query" argument from the estimators, which I could also do. That's possibly a bit DRY-er, but I'm not sure whether it might be nice to keep it as an option if people want to use the estimators without driving it from the main frontend?

Comment thread tests/resources/data/tests.json
Comment thread pyproject.toml
Copy link
Copy Markdown
Collaborator

@f-allian f-allian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmafoster1 Generally good to go. I'll do some stress testing later but I just had a few questions for my own benefit!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable Custom Estimators in Entrypoint

2 participants