Skip to content

Add lexical richness indices (Brunet's W, Honoré's R) to competence timeline - #52

Merged
willwade merged 1 commit into
mainfrom
feat/lexical-richness
Aug 19, 2026
Merged

Add lexical richness indices (Brunet's W, Honoré's R) to competence timeline#52
willwade merged 1 commit into
mainfrom
feat/lexical-richness

Conversation

@willwade

Copy link
Copy Markdown
Collaborator

Summary

Adds total-sample lexical-richness indices to the linguistic-competence
timeline: Brunet's W and Honoré's R, plus the hapax-legomena count.

Why

MATTR-30 (the headline lexical-diversity measure) depends on a moving window
and behaves awkwardly on the small, highly variable samples typical of AAC
output. Brunet's W (N·V^-0.165, lower = richer) and Honoré's R
(100·ln N / (1 − V₁/V), higher = richer) are total-sample type-token
measures that complement it without window-size sensitivity. Both are used in
DEPAC's lexical-complexity feature set (Tasnim et al., DEPAC 2022) and the
wider clinical-linguistics literature.

What

  • lexicalRichness(words) — pure function of the token-frequency
    distribution; language-agnostic, no word lists or resources needed (fits
    the data-free competence core). Returns { brunetsW, honoresR, hapax, types, tokens } with null guards (empty stream; R undefined when every
    type is a hapax or N ≤ 1).
  • MonthBin.lexicalRichness — computed per calendar month in
    analyzeTimeline (nulled on suppressed months alongside the other measures).
  • Only aggregate counts/ratios emitted — no word lists (privacy unchanged).

Testing

  • 5 new unit tests: closed-form values for a known distribution, null guards,
    monotonicity (W decreases as vocabulary diversifies at fixed N), and
    MonthBin wiring in analyzeTimeline.
  • Full suite: 91 suites / 841 tests pass; lint (0 errors) and type-check clean.

Consumers

Grid-Metrics-Exporter dashboard already renders lexicalRichness charts from
the timeline (omitted gracefully on ≤ 0.3.3 reports); it will pick this up on
the next versioned release.

@willwade
willwade merged commit c3e21ae into main Aug 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant