[jmgirard]

With p_adjust = "tukey" or "sidak", estimate_contrasts() and estimate_slopes() gave p-values that differ from emmeans in several cases, and the emmeans backend of estimate_slopes() ignored p_adjust. The Tukey adjustment was applied to comparisons that are not pairwise, Tukey and Sidak counted the wrong number of groups or tests, and slopes tested against a value other than 0 kept Tukey.

Examples. m1 <- lm(y ~ g * h) and m2 <- lm(y ~ x * g) with a 4-level g and a 2-level h (simulated data, as in test-p_adjust.R). m5 <- lm(mpg ~ gear) and m4 <- lm(mpg ~ wt * cyl) on mtcars with factored gear and cyl. First p-value of each call:

Call	Before	Now	emmeans
estimate_contrasts(m5, "gear", comparison = "reference", p_adjust = "tukey")	0.0937 (Tukey)	0.0754 (Sidak, with a message)	0.0754
estimate_contrasts(m1, "g", p_adjust = "sidak")	0.266	0.371	0.371
estimate_slopes(m2, "x", by = "g", comparison = ~pairwise, p_adjust = "tukey")	0.921	0.756	0.756
estimate_slopes(m2, "x", by = "g", hypothesis = 1, p_adjust = "tukey")	0.00753 (Tukey)	0.00554 (Sidak, with a message)	0.00554
estimate_slopes(m4, "wt", by = "cyl", backend = "emmeans", p_adjust = "holm")	0.00031 (unadjusted)	0.00094	0.00094

What changed:

• .p_adjust() keeps Tukey only for pairwise comparisons: a comparison (contrasts), or comparison, hypothesis or post_process (slopes), that is a formula with pairwise or revpairwise on the right side, with or without a | group term. Other comparisons, and slopes tested against a value, get Sidak with a message, as emmeans does. No message is printed when each family has one test, because Sidak then changes nothing.
• The family of each row is the rows within each group of the by variables that are output columns, or of the | group term. Sidak counts these rows. For pairwise comparisons, Tukey takes the number of levels or slopes k from the k * (k - 1) / 2 rows of each family. Missing by values form their own family. Sidak is computed as -expm1(n * log1p(-p)), which keeps small p-values precise.
• A partly matched hypothesis argument (for example hypo = ~pairwise), which avg_slopes() accepts, is read as hypothesis. When post_process is given, it decides the rule, because it defines the output rows.
• estimate_contrasts() on estimate_relation() output sets the by attribute again before the adjustment, because formatting drops it.
• estimate_slopes(backend = "emmeans") sets adjust on the emtrends object for the stats::p.adjust() methods, "tukey" and "sidak" (any case), and parameters then adjusts the p-values. "tukey" becomes "sidak" with a message, as in emmeans::test(). Bayesian and ordinal models are not changed. Other methods are still ignored there.
• The p_adjust docs of estimate_contrasts() and estimate_slopes(), and NEWS, describe the rules. The sentence that multiple comparisons only occur for Johnson-Neyman intervals is gone.

Tests in the new test-p_adjust.R (no skip_on_os()) compare the p-values with emmeans at a tolerance of 1e-5: Tukey and Sidak for pairwise contrasts with and without by, the reference, sequential and meandev comparisons, compared slopes with estimate_slopes() and estimate_contrasts(), grouped slope comparisons, slopes tested against 1, hypo, contrasts of estimate_relation() output, and the emmeans slope backend. They also check the messages, the p_adjust attribute and missing by values.

Not changed here, the same as on main:

• The stats::p.adjust() methods (holm, bonferroni and others) adjust over all rows with the marginaleffects backend, where emmeans adjusts within each by group.
• A post_process function drops the by columns, so Sidak then counts all rows as one family.
• post_process = ~pairwise on slopes gives unadjusted p-values from z, where comparison = ~pairwise uses t.

devtools::test() gives 0 failures locally. devtools::check() gives 0 errors and 0 warnings. The version is bumped to 0.17.0.21.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

2 open issues

• .p_adjust_family_size() is an internal (non-exported) function, so it typically won’t be… · New
• This startsWith() call is intentionally reversed to detect partial argument names (e.g., hypo… · New

3 files not reviewed

• man/estimate_contrasts.Rd: Generated file
• man/estimate_slopes.Rd: Generated file
• man/get_emmeans.Rd: Generated file

What changed in this PR

Aligns "tukey" and "sidak" p-value adjustments in estimate_contrasts() / estimate_slopes() with emmeans, including fixing the emmeans backend in estimate_slopes() to actually apply p_adjust.

Changes:

• Updates .p_adjust() to apply Tukey only to pairwise comparisons and otherwise fall back to Sidak (matching emmeans), with correct family sizing logic.
• Applies p_adjust for the emmeans backend of estimate_slopes() (for stats::p.adjust() methods + Tukey/Sidak).
• Adds extensive regression tests and updates documentation + NEWS, with a version bump.

File	Description
tests/​testthat/​test-p_adjust.R	Adds regression tests comparing adjusted p-values against emmeans across multiple scenarios/backends.
R/​p_adjust.R	Refines Tukey/Sidak rules, corrects family sizing, adds helpers for pairwise/group-term detection, and improves Sidak numeric stability.
R/​estimate_slopes.R	Ensures backend="emmeans" respects p_adjust (incl. Tukey→Sidak behavior).
R/​estimate_contrasts.R	Documents Tukey→Sidak replacement rules and Sidak family sizing behavior.
R/​estimate_contrast_methods.R	Restores by attribute before adjustment so Sidak can compute families correctly.
NEWS.md	Documents behavior changes and rationale.
DESCRIPTION	Bumps package version.

_{Review effort: Lite}