Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,33 @@ researchers who pin a tag and want to know what they are pinning to.

## [Unreleased]

### Fixed

- **This moves a published number, in one direction. Re-run
`analyse-discourse` on any corpus, then re-read every evasion rate computed
from it.** `FEDERAL_DEFLECTION`'s state-subject pattern required a copula:
`\b(?:is|are)\s+(?:a|an)\s+[Ss]tate\s+[Ss]ubject\b`. Indian legislative
English drops it. "Agriculture **being** a State subject, there is no
investment..." matched nothing. The pattern now accepts `being`.

**Measured by zero-hour over 489,469 answered records (REQ-0072):** 34,632
answers mention "State subject", 5,542 match the shipped copula pattern, and
3,080 use the participle. Of those, 2,982 match no copula, and **518 carry no
label from any tier at all**.

**Two kinds of record move, and only the first is pure recall.** The 518
unlabelled records gain `FEDERAL_DEFLECTION`, which raises both the evasive
count and the classified denominator. The other 2,464 already carry a label
from some other pattern, and `FEDERAL_DEFLECTION`'s 0.92 confidence outranks
it. Where that old label was substantive, the evasion rate rises a second
time. Where it was evasive, only the label distribution changes.

**That precedence is not new and is not introduced here.** The copula form
already outranks `ACCEPTED` and `REJECTED` today at the same confidence. The
alternation makes the participle behave exactly as the copula does. Whether
a state-subject clause should outrank a substantive disclosure is a separate
question, and it applies to the shipped copula pattern first.

## [2.7.0] — 2026-08-20

### Changed
Expand Down
9 changes: 7 additions & 2 deletions commoner_analyse/discourse.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,13 @@ def _compile(patterns: Iterable[str]) -> tuple[re.Pattern, ...]:
"Uses federalism/State Subjects to dodge Central responsibility for national standards.",
_compile([
r"library\s+is\s+a\s+state\s+subject",
# Broader state-subject dodge pattern (mined from Azad corpus UNCLASSIFIED)
r"\b(?:is|are)\s+(?:a|an)\s+[Ss]tate\s+[Ss]ubject\b",
# Broader state-subject dodge pattern (mined from Azad corpus UNCLASSIFIED).
# `being` is not optional decoration. Indian legislative English drops the
# copula in this exact construction, and the participial form carries the
# same federal dodge: "Agriculture being a State subject, ...". Measured
# over 489,469 answered records, 2,982 use the participle and match no
# copula, and 518 of those carry no label from any tier (REQ-0072).
r"\b(?:is|are|being)\s+(?:a|an)\s+[Ss]tate\s+[Ss]ubject\b",
r"\b(?:Concurrent|[Cc]oncurrent)\s+[Ll]ist\b",
r"\bunder\s+entry\s+\d+\s+of\s+the\s+[Ss]tate\s+List\b",
r"\bSeventh\s+Schedule\b",
Expand Down
48 changes: 48 additions & 0 deletions tests/test_discourse.py
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,54 @@ def test_state_subject_fires_federal_deflection(self):
)
self.assertEqual(result.label, "FEDERAL_DEFLECTION")

def test_participial_state_subject_fires_federal_deflection(self):
"""REQ-0072. Indian legislative English drops the copula in this exact
construction, and the pattern required it. Verbatim from the request,
source key ``LS|S|734|2000-05-17``, Minister of Agriculture. zero-hour
measured 2,982 answers using this form and invisible to the pattern,
of which 518 carried no label from any tier at all.
"""
from commoner_analyse.discourse import CHANNEL_QA
result = classify_response(
"Agriculture being a State subject, there is no investment in "
"agriculture directly by the Central Government in the States.",
channel=CHANNEL_QA,
)
self.assertEqual(result.label, "FEDERAL_DEFLECTION")

def test_the_participle_and_the_copula_agree(self):
"""The alternation adds recall. It does not create a second behaviour.

The two forms make the same claim, so they must reach the same label at
the same confidence. A drift between them is the defect returning.
"""
from commoner_analyse.discourse import CHANNEL_QA
pairs = [
("Agriculture is a State subject, so the Centre does not intervene.",
"Agriculture being a State subject, the Centre does not intervene."),
("Health is a State subject and States decide.",
"Health being a State subject, States decide."),
]
for copula, participle in pairs:
with self.subTest(participle=participle):
first = classify_response(copula, channel=CHANNEL_QA)
second = classify_response(participle, channel=CHANNEL_QA)
self.assertEqual(first.label, second.label)
self.assertEqual(first.confidence, second.confidence)

def test_the_copula_form_still_matches(self):
"""Acceptance 3. The alternation must not move a copula match."""
from commoner_analyse.discourse import CHANNEL_QA
for text in (
"Land is a state subject under the Seventh Schedule.",
"These are a State subject and rest with the States.",
):
with self.subTest(text=text):
self.assertEqual(
classify_response(text, channel=CHANNEL_QA).label,
"FEDERAL_DEFLECTION",
)

def test_concurrent_list_fires_federal_deflection(self):
from commoner_analyse.discourse import CHANNEL_QA
result = classify_response(
Expand Down