The focus of this paper is to analyze whether the unreliability of results related to certain controversial psychological phenomena may be a consequence of their low statistical power. Applying the Null Hypothesis Statistical Testing (NHST), still the widest used statistical approach, unreliability derives from the failure to refute the null hypothesis, in particular when exact or quasi-exact replications of experiments are carried out. Taking as example the results of meta-analyses related to four different controversial phenomena, subliminal semantic priming, incubation effect for problem solving, unconscious thought theory, and non-local perception, it was found that, except for semantic priming on categorization, the statistical power to detect the expected effect size (ES) of the typical study, is low or very low. The low power in most studies undermines the use of NHST to study phenomena with moderate or low ESs. We conclude by providing some suggestions on how to increase the statistical power or use different statistical approaches to help discriminate whether the results obtained may or may not be used to support or to refute the reality of a phenomenon with small ES.
Keywords: incubation effect, non-local perception, power, subliminal priming, unconscious thought theory
Citation: Tressoldi PE (2012) Replication unreliability in psychology: elusive phenomena or “elusive” statistical power? Front. Psychology 3:218. doi: 10.3389/fpsyg.2012.00218
Received: 13 April 2012; Accepted: 12 June 2012;
Published online: 04 July 2012.
Edited by:Jason W. Osborne, Old Dominion University, USA
Reviewed by:Fiona Fidler, University of Melbourne, Australia
Copyright: © 2012 Tressoldi. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.
*Correspondence: Patrizio E. Tressoldi, Dipartimento di Psicologia Generale, Università di Padova, Padova, Italy. e-mail: firstname.lastname@example.org