Search Conferences

Type in any word, words or author name. This searchs through the abstract title, keywords and abstract text and authors. You may search all conferences or just select one conference.


 All Conferences
 EMAC 2019 Annual Conference
 EMAC 2020 Annual Conference
 EMAC 2020 Regional Conference
 EMAC 2021 Annual Conference

EMAC 2019 Annual Conference


In p we trust (too much). Simulations of typical p-value and effect size behavior in advertising research
(A2019-9521)

Published: May 28, 2019

AUTHORS

Tim Smits, Institute for Media Studies, KU Leuven; Ben Van Calster, KU Leuven

KEYWORDS

null-hypothesis significance testing; replicability; advertising effects

ABSTRACT

With this simple illustration we aim to demonstrate the most important issues concerning the continued use of p-values and null hypothesis significance testing. We do this based on a set of simulations that pertain to the setting of an advertising researcher investigating a basic effect in a control-versus-experimental between-subjects design. These simulations were performed for different hypothetical true effect sizes and sample sizes and, in a second set of simulations, for some reference values coming from advertising meta-analyses. Important issues demonstrated in these simulations are the high variability of p-values, the variability of effect sizes, and the over-confidence when significant findings occur. These effects are worse for smaller sample sizes. We also focus on the effect of shifting the significance criterion from .05 to .005 as has been suggested recently. We also reflect on how these issues partially contribute to the replicability crisis.