Abstract: In “Estimating the reproducibility of psychological science” published in 《Science》, the authors report the results of their replication of 100 experimental and correlational studies published in three prestigious psychology journals in 2008 (Open Science Collaboration, 2015). Whereas 97% of original studies had significant results (P<.05), only 36% of replications were so. The authors offer some explanations for these results and emphasise that this is not a defeat for psychology. They conclude that accumulating evidence is the scientific community’s method of self-correction and is the best available option for achieving the ultimate goal: truth. In this process, statistically significant innovations from the original studies would point out paths that are possible, whilst those that subsequently pass the test of replication would point out paths that are likely. Scientific progress relies on both innovations and replications — hence journals should not dismiss replication studies lightly. Whilst we are in broad agreement with the authors’ explanations and conclusions, we note, first, that the problem of irreproducibility is not unique to psychology. It has happened to other sciences such as oncology, haematology and genetics, sometimes to an even greater extent, and has led to better scientific tools such as double-blind procedures to reduce the risk of unconscious biases (Nuzzo, 2015). Second, many of the replications that fail to reach statistically significant P values are in the same trend as the original studies rather than contradictory. This raises the question of the real scientific significance of P values in the research process and its implications. In the second paper reviewed herein (“P values are just the tip of the iceberg” published in《Nature》), Leek and Peng (2015) criticise the decision made by the journal Basic and Applied Social Psychology in 2015 to ban the use of P values and other methods of NHST (null hypothesis significance testing). The authors argue, first, that although P values are widely abused, banning it would open more doors to dubious game statistics for getting a paper accepted for publication. Second, and more importantly, they point out that a successful study has many stages in the making before reaching the P values stage. The latter stage is only the tip of the iceberg, and its impact on the eventual success of a study as a whole is much less critical than careful research design and analysis that lie below it. Hence the more fruitful debate should be on all aspects of the research process rather than on P values primarily. This conclusion has implications not only for the reproducibility study’s over-reliance on P values, but also for the training of psychology research students in contemporary China. Too often, the training has focused on the “tip of the iceberg” of the research process to the neglect of those parts of the process below. In this training culture, students learn the technicalities competently. They also learn to reduce their cognitive uncertainties about doing research by placing their trust in authorities, and can complete super-efficiently the “required” multiple studies in a bundle rather than in a programatic manner to build one study on the results of the one preceding. Then they blame the authorities for failures instead of taking personal responsibility for their own research. The combined take-home message of the two 《Science》 and《Nature》psychology papers is perhaps best summed up by Francis Bacon (1561–1626): “If a man will begin with certainties, he shall end in doubts; but if he will be content to begin with doubts, he shall end in certainties.” (Quoted in Sun, 2004, p. 581)

Key words: Reproducibility of psychological science, Scientific innovation, Spirit of scientific research, P value, Postgraduate education

摘要： 本文评述了《科学》和《自然》杂志2015年的两篇文章。《科学》上的文章“心理科学的重现性”对100项心理学实验进行了重复验证。结果表明，只有大约多于1/3的原始研究能够重复。该文章对出现这样结果的原因作了分析，并指出这不是心理学的挫折。我们强调科学精神在于它的不确定性，因而科学才能处在不停的探索中。《自然》上的文章“P值仅仅是冰山的一角”，批评了个别心理学杂志禁用P值的做法，指出研究者应该在实验设计、数据处理等实验的前期阶段下功夫，才能防止心理科学的坍塌。我们结合国内研究生培养的现状，就改进心理学研究生的实验训练提出了建议。

关键词: 心理学, 科学的创新, 科学的重现性, P值, 研究生培养