Abstract
It is key for the clinician reading research to recognize the difference between clinical and statistical significance. In this review we will first look into the meaning of statistical significance followed by a review of the concepts that give rise to clinical significance. We will review the statistical options available to improve the interpretation of research results so that we can avoid the misinterpretation of the “dreaded p-value”. Lastly, we will review two real examples from the literature of a common situation in research studies: the finding of statistical significance without clinical significance.
References
Cohen J. The earth is round (p < .05). American Psychologist. 1994;49(12):997–1003.
Cohen J. Things I have learned (so far). American Psychologist. 1990;45(12):1304–12.
Gelman A, Stern H. The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant. null. 2006 Nov;60(4):328–31.
Cox DR. Statistical significance tests. Br J Clin Pharmacol. 1982 Sep;14(3):325–31.
Goodman S. A dirty dozen: twelve p-value misconceptions. Semin Hematol. 2008 Jul;45(3):135–40.
Wasserstein RL, Lazar NA. The ASA Statement on p-Values: Context, Process, and Purpose. The American Statistician. 2016 Apr 2;70(2):129–33.
Curran-Everett D. Multiple comparisons: philosophies and illustrations. Am J Physiol Regul Integr Comp Physiol. 2000 Jul;279(1):R1-8.
Sethuraman A, Gonzalez NM, Grenier CE, Kansagra KS, Mey KK, Nunez-Zavala SB, et al. Continued misuse of multiple testing correction methods in population genetics-A wake-up call? Mol Ecol Resour. 2019 Jan;19(1):23–6.
Sullivan GM, Feinn R. Using Effect Size—or Why the P Value Is Not Enough. J Grad Med Educ. 2012 Sep;4(3):279–82.
Kazdin AE. The meanings and measurement of clinical significance. J Consult Clin Psychol. 1999 Jun;67(3):332–9.
Benson T. Measure what we want: a taxonomy of short generic person-reported outcome and experience measures (PROMs and PREMs). BMJ Open Qual. 2020 Mar;9(1):e000789.
Glass TA, Goodman SN, Hernán MA, Samet JM. Causal inference in public health. Annu Rev Public Health. 2013;34:61–75.
Vancak V, Goldberg Y, Levine SZ. Systematic analysis of the number needed to treat. Stat Methods Med Res. 2020 Sep;29(9):2393–410.
Quinn TJ, Dawson J, Walters M. Dr John Rankin; his life, legacy and the 50th anniversary of the Rankin Stroke Scale. Scott Med J. 2008 Feb;53(1):44–7.
Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. 1989 Dec;10(4):407–15.
Westphal LP, Widmer R, Held U, Steigmiller K, Hametner C, Ringleb P, et al. Association of prestroke metformin use, stroke severity, and thrombolysis outcome. Neurology. 2020 Jul 28;95(4):e362–73.
Moore MJ, Goldstein D, Hamm J, Figer A, Hecht JR, Gallinger S, et al. Erlotinib Plus Gemcitabine Compared With Gemcitabine Alone in Patients With Advanced Pancreatic Cancer: A Phase III Trial of the National Ca