A leakage-aware research platform for cross-sectional stock selection, downside-risk modelling, financial-news NLP, realistic execution, and paper-live monitoring.
The project asks whether machine learning can identify future relative winners while controlling downside risk more reliably than direct return prediction. It also studies whether timestamp-valid financial sentiment contributes useful incremental information.
The current evidence supports a small but measurable ranking contribution. It does not establish production alpha or a strategy that consistently beats SPY.
Status date: August 2, 2026
Current research direction: multi-task opportunity and risk estimation, followed by a constrained reward-aware portfolio policy
Historical universe: point-in-time S&P 500 membership, lagged liquid top 250, sector-capped portfolios
Paper-live monitor: independently scheduled Ridge and XGBoost arms at 5, 10, and 20 sessions
The system became narrower as its validation standard became stronger.
- The original project combined direct return prediction, optimization, pair trading, technical indicators, and sentiment. Its complete design and early experiments are preserved on the archive-experiments branch.
- Execution timing, transaction costs, label maturity, point-in-time membership, and price-adjustment audits invalidated several attractive early results. The corrected workflow now fails closed when those contracts are violated.
- The central ML question changed from predicting an exact return to ranking future relative winners. Ridge remained the hardest baseline to beat, while more complex ranking models did not consistently improve portfolio results.
- A ten-year replay showed that the ranking signal can beat matched random selection at some horizons, but none of the tested active portfolios beat schedule-matched SPY. High-recall severe-loss filters also rejected too many eventual winners.
- The next design therefore estimates both opportunity and risk: upside rank, expected residual return, downside quantiles, severe-loss probability, and uncertainty. A transparent constrained policy must pass strict controls before contextual or offline reinforcement learning is considered.
The dated evidence behind each change is linked in Current evidence. Superseded tables and long experiment narratives remain available through those reports rather than being pasted into the active README.
The latest common-date 20-session development experiment covers 520 weekly OOS decisions from December 2013 through November 2023. Daily portfolio returns are reconstructed from exact overlapping sleeves.
| Portfolio | Mean rank IC | Gross CAGR | Gross Sharpe | Gross maximum drawdown |
|---|---|---|---|---|
| Ridge return | 0.0018 | 12.09% | 0.619 | -46.44% |
| Ridge rank | 0.0123 | 11.42% | 0.590 | -49.58% |
| XGBoost rank | 0.0128 | 11.28% | 0.596 | -44.28% |
| LambdaMART, five grades | -0.0041 | 10.02% | 0.481 | -50.18% |
| SPY index | N/A | 12.29% | 0.774 | -32.05% |
The table is a zero-cost sensitivity intended to isolate model behaviour. SPY still has the highest CAGR and Sharpe and the smallest drawdown. Ranking labels improve Ridge rank IC but do not improve its portfolio return or tail risk.
Read the ten-year ranking and tail-control experiment
Frozen-score attribution found the clearest selection contribution at 40 and 50 sessions. Their median gross CAGR advantages over sector-matched random portfolios were 2.84 and 1.87 percentage points. Every tested horizon still trailed schedule-matched SPY before costs, so this is evidence of a weak selection signal rather than a market-beating strategy.
Read the horizon performance attribution
The ten-year Ridge-rank tail treatment reduced severe-loss frequency from 8.03% to 6.90% and improved CVaR. Its veto precision was only 13.84%, 53.37% of rejected observations were eventual winners, and return preservation did not generalize across models. The result motivates joint upside and downside modelling instead of another hard high-recall veto.
Read the severe-loss A/B evidence
Sentiment remains promising but unproven. Historical development comparisons showed favourable point estimates, but source coverage, timestamp quality, entity linking, and the absence of a prospective technical-only control prevent a causal performance claim. Sentiment stays in the research system while the NLP and evaluation design are strengthened.
Read the current project evidence map
The approved redesign separates prediction from portfolio policy.
Prediction heads
upside ranking
expected residual return
downside quantiles
severe-loss probability
Uncertainty estimates
fold and seed disagreement
predicted quantile range
data-quality risk
Portfolio policy
expected opportunity
less downside-risk penalty
less uncertainty penalty
subject to exposure, liquidity, turnover, drawdown, and CVaR controls
The four outcomes remain separate training targets so their contribution can be measured. Reward is introduced at the portfolio-policy layer rather than being used as one opaque stock label. The first experiment uses a deterministic constrained policy. A contextual controller and conservative offline RL are eligible only if sequential portfolio state adds value beyond that baseline.
Promotion requires every candidate to:
- beat sector-matched random portfolios and the eligible-universe control;
- produce positive gross active return against schedule-matched SPY;
- improve Sharpe without worsening maximum drawdown or CVaR;
- remain stable across folds, years, regimes, seeds, and nearby horizons;
- reduce severe-loss exposure without rejecting too many winners; and
- pass a genuinely untouched chronological confirmation period.
Read the multi-task signal, risk, and policy-learning proposal
flowchart TD
INGEST["Market, membership, fundamentals, and news ingestion"]
AUDIT["Point-in-time availability, corporate-action, and coverage audits"]
PANEL["Leakage-safe universe, features, and horizon-matched labels"]
HEADS["Opportunity and risk models: rank, return, quantile, and severe loss"]
POLICY["Uncertainty-aware scoring and constrained portfolio policy"]
REPLAY["Exact next-open portfolio replay with costs and SPY"]
LIVE["Immutable paper-live cohorts and matured outcomes"]
EVIDENCE["IC, calibration, return, drawdown, CVaR, SHAP, and controls"]
DASH["Live Monitor, Horizon Comparison, and Research Evidence dashboards"]
INGEST --> AUDIT --> PANEL --> HEADS --> POLICY --> REPLAY
POLICY --> LIVE
REPLAY --> EVIDENCE
LIVE --> EVIDENCE --> DASH
| Module | Responsibility |
|---|---|
| Market, membership, and news ingestion | Refresh adjusted market history and timestamped financial news; preserve effective-dated membership, sectors, sources, and freshness metadata |
| Leakage-safe research panel | Form the lagged liquid universe; build technical, sector-relative, risk, fundamental, sentiment, and recency features; create exact next-open horizon labels |
| Model development | Use purged temporal folds, fold-local preprocessing and feature selection, training-only early stopping, Optuna TPE, common rows, and saved OOS predictions |
| Portfolio policy and replay | Convert frozen scores into constrained holdings; execute at audited prices; track exposure, turnover, costs, cash, drawdown, CVaR, and SPY-relative results |
| Evidence and monitoring | Preserve immutable forecasts; mature outcomes without look-ahead; publish signal, portfolio, model, feature, and live-forward diagnostics |
The much larger historical architecture is available on the archive-experiments branch.
The 40- and 50-session portfolios beat sector-matched random selection most clearly, but all tested horizons trail schedule-matched SPY.
The chart keeps SPY visible beside the corrected active portfolios and reports growth, drawdown, daily return distributions, and exact sleeve turnover.
The signal report separates rank IC and score-quantile behaviour from portfolio accounting. Overlapping horizon returns are not compounded as independent weekly outcomes.
The treatment improves several tail statistics in the Ridge-rank arm but does not preserve return reliably across models, so it remains inactive.
TreeSHAP explains the frozen July 30 XGBoost 20-session cross-section. It describes model behaviour, not causal importance.
Regenerate the tracked README figures with:
python -m src.experiments.readme_figuresThe public site uses one visual system across three linked views:
- Live Forward Monitor, the default stock and portfolio monitoring view
- Horizon Comparison, the 5, 10, and 20-session prospective comparison
- Research Evidence, the historical experiment, model, risk, and feature-evidence view
The daily publisher rebuilds and validates all three pages from sanitized artifacts. A dashboard can display partial observations, but it does not label an outcome complete until the exact entry and exit prices exist.
The monitor issues daily forecasts while matching each arm's model-refit and portfolio-rebalance clock to its horizon.
| Arm | Refit | Rebalance | Label purge | Role |
|---|---|---|---|---|
| Ridge and XGBoost, 5 sessions | Every 5 sessions | Every 5 sessions | 5 sessions | Short-horizon diagnostic |
| Ridge and XGBoost, 10 sessions | Every 10 sessions | Every 10 sessions | 10 sessions | Intermediate challenger |
| Ridge and XGBoost, 20 sessions | Every 20 sessions | Every 20 sessions | 20 sessions | Provisional longer-horizon arm |
Forecasts are saved every trading day without forcing daily portfolio replacement. Model outputs, selected features, top stocks, execution dates, and maturity status are immutable once issued.
Run a completed-market-day cycle with:
python -m src.cli run-live-forward-20d --as-of YYYY-MM-DD- Membership, sectors, liquidity, and features are reconstructed as of the simulated decision time.
- Labels must mature before training and are purged from temporal boundaries.
- Imputation, scaling, feature selection, calibration, early stopping, and hyperparameter selection stay inside the training boundary.
- Candidate models use common dates, rows, folds, execution, and compute budgets.
- SPY, eligible-universe, and matched-random controls remain visible.
- Gross, zero-cost, declared-cost, and tactical results are reported separately.
- Missing prices, stale inputs, failed NLP, invalid scores, or incomplete artifacts stop the run.
- A viewed test period is development evidence and cannot be renamed an untouched confirmation period.
The reusable modelling standard is documented in
ml-framework.md.
python -m venv .venv
# Windows
.venv\Scripts\activate
pip install -r requirements.txt
pip install -e ".[tuning]"
pytest -qUseful commands:
# Run one paper-live cycle after a completed US market close
python -m src.cli run-live-forward-20d --as-of YYYY-MM-DD
# Refresh the frozen-cohort monitor and public bundle
python -m src.cli run-live-forward-daily --as-of YYYY-MM-DDRun python -m src.cli --help for the complete command surface.
The README contains only the active system, corrected evidence, and approved next direction. Detailed experiments remain linked for reproducibility.
- Documentation index
- Project status and evidence map
- Multi-task ML and policy redesign
- Horizon-selection design
- Latest horizon attribution
- Latest corrected ten-year model and tail study
- Framework-clean model comparison
- False-positive and tail-risk research record
- Superseded designs and historical experiments
Failed experiments remain evidence. They are preserved in their dated reports instead of being presented as simultaneously active strategies.
- The strongest results remain development evidence. The repeatedly inspected 2013 through 2023 period is not an untouched confirmation period.
- Public data does not completely solve delisted-security coverage, permanent-identifier history, corporate actions, or historical sectors.
- The average ranking edge is small, unstable by year and regime, and has not produced a standalone portfolio that beats schedule-matched SPY.
- The active tail treatment has insufficient precision and remains disabled.
- Sentiment lacks sufficient high-quality coverage and a prospective technical-only control for causal attribution.
- The paper-live evidence is young; partial marks and immature horizons cannot establish long-run performance.
- Static execution costs do not establish capacity, taxes, borrow availability, market impact, or partial fills.
- Reinforcement learning remains a proposal. It receives no promotion unless the simpler multi-task constrained policy passes every required gate first.
This repository is an educational and research system. Backtests, model scores, paper portfolios, and live-forward monitors are not investment advice and do not guarantee future performance.