Hypothesis Testing Reconsidered (Elements in Perception)

Hypothesis testing is a common statistical analysis for empirical data generated by studies of perception, but its properties and limitations are widely misunderstood. This Element describes several properties of hypothesis testing, with special emphasis on analyses common to studies of perception. The author also describes the challenges and difficulties with using hypothesis testing to interpret empirical data. Many common applications of hypothesis testing inflate the intended Type I error rate. Other aspects of hypothesis tests have important implications for experimental design. Solutions are available for some of these difficulties, but many issues are difficult to deal with.