TY - JOUR

T1 - Out with .05, in with Replication and Measurement

T2 - Isolating and Working with the Particular Effect Sizes that are Troublesome for Inferential Statistics

AU - Bradley, Michael T

AU - Brand, Andrew

PY - 2017/11

Y1 - 2017/11

N2 - It is difficult to obtain adequate power to test a small effect size with a set criterion alpha of 0.05. Probably an inferential test will indicate non-statistical significance and not be published. Rarely, statistical significance will be obtained, and an exaggerated effect size calculated and reported. Accepting all inferential probabilities and associated effect sizes could solve exaggeration problems. Graphs, generated through Monte Carlo methods, are presented to illustrate this. The first graph presents effect sizes (Cohen's d) as lines from 1 to 0 with probabilities on the Y axis and the number of measures on the X axis. This graph shows effect sizes of .5 or less should yield non-significance with sample sizes below 120 measures. The other graphs show results with as many as 10 small sample size replications. There is a convergence of means with the effect size as sample size increases and measurement accuracy emerges.

AB - It is difficult to obtain adequate power to test a small effect size with a set criterion alpha of 0.05. Probably an inferential test will indicate non-statistical significance and not be published. Rarely, statistical significance will be obtained, and an exaggerated effect size calculated and reported. Accepting all inferential probabilities and associated effect sizes could solve exaggeration problems. Graphs, generated through Monte Carlo methods, are presented to illustrate this. The first graph presents effect sizes (Cohen's d) as lines from 1 to 0 with probabilities on the Y axis and the number of measures on the X axis. This graph shows effect sizes of .5 or less should yield non-significance with sample sizes below 120 measures. The other graphs show results with as many as 10 small sample size replications. There is a convergence of means with the effect size as sample size increases and measurement accuracy emerges.

KW - effext sizes

KW - Replication

KW - Maeasurement

KW - Inferential probabilites

U2 - 10.1080/00221309.2017.1381496

DO - 10.1080/00221309.2017.1381496

M3 - Article

C2 - 29023206

VL - 144

SP - 309

EP - 316

JO - The Journal of General Psychology

JF - The Journal of General Psychology

SN - 0022-1309

IS - 4

ER -