Search Conferences

Type in any word, words or author name. This searchs through the abstract title, keywords and abstract text and authors. You may search all conferences or just select one conference.

 All Conferences
 EMAC 2019 Annual Conference
 EMAC 2020 Annual Conference
 EMAC 2020 Regional Conference
 EMAC 2021 Annual Conference
 EMAC 2021 Regional Conference
 EMAC 2022 Annual
 EMAC 2022 Regional Conference

EMAC 2019 Annual Conference

In p we trust (too much). Simulations of typical p-value and effect size behavior in advertising research

Published: May 28, 2019


Tim Smits, Institute for Media Studies, KU Leuven; Ben Van Calster, KU Leuven


null-hypothesis significance testing; replicability; advertising effects


With this simple illustration we aim to demonstrate the most important issues concerning the continued use of p-values and null hypothesis significance testing. We do this based on a set of simulations that pertain to the setting of an advertising researcher investigating a basic effect in a control-versus-experimental between-subjects design. These simulations were performed for different hypothetical true effect sizes and sample sizes and, in a second set of simulations, for some reference values coming from advertising meta-analyses. Important issues demonstrated in these simulations are the high variability of p-values, the variability of effect sizes, and the over-confidence when significant findings occur. These effects are worse for smaller sample sizes. We also focus on the effect of shifting the significance criterion from .05 to .005 as has been suggested recently. We also reflect on how these issues partially contribute to the replicability crisis.