Skip to content

Commit 683028b

Browse files
docs: fix kl_divergence and chi_square audit arguments and semantics
The documented examples used argument names that the built-in audits do not accept, so copying them fails with "Macro variable 'source_column' is undefined": - kl_divergence takes source_column / target_column (not column / target_column) - chi_square takes column_a / column_b (not column / target_column) The chi_square description also said the audit ensures the statistic does not exceed the critical value. The implementation does the opposite by default: with dependent := true (the default) it asserts that the columns are dependent and fails when the statistic does not exceed the critical value. Document the dependent argument and both behaviours, and fix the "statistic0" typo. Also align the kl_divergence usage comment in builtin.py with its arguments. Signed-off-by: shinichitazawa <shinichitazawa.aws@gmail.com>
1 parent d69c262 commit 683028b

2 files changed

Lines changed: 7 additions & 5 deletions

File tree

‎docs/concepts/audits.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -664,13 +664,15 @@ This example asserts that the symmetrised KL Divergence between columns `age` an
664664
```sql linenums="1"
665665
MODEL (
666666
audits (
667-
kl_divergence(column := age, target_column := reference_age, threshold := 0.1)
667+
kl_divergence(source_column := age, target_column := reference_age, threshold := 0.1)
668668
)
669669
);
670670
```
671671

672672
#### chi_square, chi_square_non_blocking
673-
Ensures that the [chi-square](https://en.wikipedia.org/wiki/Chi-squared_test) statistic for two categorical columns does not exceed a critical value.
673+
Performs a [chi-square](https://en.wikipedia.org/wiki/Chi-squared_test) test of independence between two categorical columns, `column_a` and `column_b`, and compares the statistic with `critical_value`.
674+
675+
By default (`dependent := true`), the audit asserts that the columns are dependent: it fails when the chi-square statistic does not exceed the critical value. Set `dependent := false` to assert that the columns are independent: the audit then fails when the statistic exceeds the critical value.
674676

675677
You can look up the critical value corresponding to a p-value with a table (such as [this one](https://www.medcalc.org/manual/chi-square-table.php)) or by using the Python [scipy library](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.chi2.html):
676678

@@ -680,12 +682,12 @@ from scipy.stats import chi2
680682
# critical value for p-value := 0.95 and degrees of freedom := 1
681683
chi2.ppf(0.95, 1)
682684
```
683-
This example asserts that the chi-square statistic0 for columns `user_state` and `user_type` does not exceed 6.635:
685+
This example asserts that columns `user_state` and `user_type` are dependent, i.e. that their chi-square statistic exceeds 6.635:
684686

685687
```sql linenums="1"
686688
MODEL (
687689
audits (
688-
chi_square(column := user_state, target_column := user_type, critical_value := 6.635)
690+
chi_square(column_a := user_state, column_b := user_type, critical_value := 6.635)
689691
)
690692
);
691693
```

‎sqlmesh/core/audit/builtin.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -524,7 +524,7 @@ def create_non_blocking_copy(audit: Audit) -> Audit:
524524
""",
525525
)
526526

527-
# kl_divergence(column=age, target_column=normalized_age, threshold=0.1)
527+
# kl_divergence(source_column=age, target_column=normalized_age, threshold=0.1)
528528
kl_divergence_audit = ModelAudit(
529529
name="kl_divergence",
530530
defaults={"condition": exp.true()},

0 commit comments

Comments
 (0)