Skip to content

Appropriate to use t_to_d() on estimated marginal mean contrasts from ANOVA? #616

Closed

Description

@DallasNovakowski

Question and context

I'm running some factorial anova analyses, and would like to use t_to_d() with emmeans to retrieve cohen's d effect sizes for my pairwise contrasts. I'm just wondering whether this is statistically sound? I see the degrees of freedom are the same all across the contrasts, and am not sure whether these large df values might lead to some inflated and inaccurate estimates of cohen's d.

Reprex:

library(palmerpenguins) # dataset
#> Warning: package 'palmerpenguins' was built under R version 4.1.3
library(car) # anova functions
#> Warning: package 'car' was built under R version 4.1.3
#> Loading required package: carData
#> Warning: package 'carData' was built under R version 4.1.3
library(effectsize) # effect sizes
library(dplyr)
#> Warning: package 'dplyr' was built under R version 4.1.3
#> 
#> Attaching package: 'dplyr'
#> The following object is masked from 'package:car':
#> 
#>     recode
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

# assigns data to a dataframe we call "df"
df <- palmerpenguins::penguins

# drop rows with missing values
df <- df[complete.cases(df)==TRUE, ]


df %>%
    group_by(species, sex) %>% # Group by the specified variables
    dplyr::summarise(n()) %>%
  knitr::kable()
#> `summarise()` has grouped output by 'species'. You can override using the
#> `.groups` argument.
species sex n()
Adelie female 73
Adelie male 73
Chinstrap female 34
Chinstrap male 34
Gentoo female 58
Gentoo male 61
# Fit data
flipper_fit <- stats::aov(flipper_length_mm ~ species, data = df)
# Run anova
flipper_anova <- car::Anova(flipper_fit)


# Extract estimated marginal means
flipper_emmeans <- emmeans::emmeans(flipper_fit, specs = pairwise ~ species)

# convert estimated marginal mean contrasts to dataframe
flipper_emmeans_contrasts <- data.frame(flipper_emmeans$contrasts)

knitr::kable(flipper_emmeans_contrasts)
contrast estimate SE df t.ratio p.value
Adelie - Chinstrap -5.72079 0.9796493 330 -5.83963 0
Adelie - Gentoo -27.13255 0.8240767 330 -32.92479 0
Chinstrap - Gentoo -21.41176 1.0143492 330 -21.10887 0
effectsize::t_to_d(t = flipper_emmeans_contrasts[flipper_emmeans_contrasts$contrast == "Adelie - Chinstrap", "t.ratio"],
                   df_error = flipper_emmeans_contrasts[flipper_emmeans_contrasts$contrast == "Adelie - Chinstrap", "df"]
                   )
#> d     |         95% CI
#> ----------------------
#> -0.64 | [-0.86, -0.42]

Created on 2023-11-05 with reprex v2.0.2

Activity

  1. locked and limited conversation to collaborators on Nov 6, 2023
  2. converted this issue into a discussion #617 on Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions