Testing Scientific Claim At Own Risk: Reproducibility Against Novelty

Artem Akopyan, The University of Western Ontario

Full paper: www.kon.org/urc/v12/akopyan.html

Abstract The article discusses the problem of validation by means of independent replication. Bem’s studies of precognition are discussed in that context, as well as the recognized measure of implicit attitudes, the Affect Misattribution Procedure (AMP). Subsequently, a review of LeBel’s replication of Payne, Hall, Cameron, and Bishara (2010) is presented. Finally, important replication-oriented initiatives are outlined.

Introduction Theorizing and empirical research are often challenging, especially in the domain of social sciences where the omnipresent lack of truly objective perceptual evaluations is all the more apparent. Hypothesis testing is generally reduced to the aggregation of data scores and derivation of average descriptive values across the sample; this in turn makes research in psychology hypersensitive to blemishes in study design. For instance, Simmons, Nelson and Simonsohn (2011) demonstrated how selective reporting and optional stoppage could drastically inflate the probability of Type I error. Thus, the excessive use of researcher degrees of freedom might jeopardize the integrity and potential benefit of psychological science. Due to the reliance of inferential statistics on the likelihood of physical states, one of the verification (or falsification) procedures may involve the minimization of combined Type I error; in fact, the notion of reliability is well-known to psychologists, and every conclusion drawn from a set of data hinge upon the assumptions of validity and reliability of measurement instruments, in other words that the test measured precisely the desired construct and that the ensuing conclusion would be found by any other researcher using the same or different sample from population.

Read the full paper: www.kon.org/urc/v12/akopyan.html