Official reprint from UpToDate®
www.uptodate.com ©2017 UpToDate, Inc. and/or its affiliates. All Rights Reserved.

Systematic review and meta-analysis

Ethan Balk, MD, MPH
Peter A L Bonis, MD
Section Editor
Joann G Elmore, MD, MPH
Deputy Editor
Carrie Armsby, MD, MPH


Clinical decisions in medicine ideally should be based upon guidance from a comprehensive assessment of the body of available knowledge. A single clinical trial, even a large one, is seldom sufficient to provide a confident answer to a clinical question. Indeed, one analysis suggested that most research claims are ultimately proven to be incorrect or inaccurate when additional studies have been performed [1]. At the same time, it is well-established that large randomized controlled trials do not always confirm the results of prior meta-analyses [2-4]. The "truth" needs to be understood by examining all sources of data as critically and objectively as possible.

There are several potential benefits to performing systematic analysis, which may also include meta-analysis:

Unique aspects to a single randomized trial, involving the participating patient population, the protocol, the setting in which the trial is performed, or the expertise of the involved clinicians, may limit its generalizable to other settings or individual patients. The conclusions of systematic reviews may be more generalizable than single studies.

Combining studies in meta-analyses increases the sample size and generally produces more precise estimates of the effect size (ie, estimates that have smaller confidence intervals) than a single randomized trial.

Clinicians rarely have the time or resources to critically evaluate the body of evidence relevant to a particular clinical question, and a systematic review can facilitate this investigation.

To continue reading this article, you must log in with your personal, hospital, or group practice subscription. For more information on subscription options, click below on the option that best describes you:

Subscribers log in here

Literature review current through: Sep 2017. | This topic last updated: Jul 12, 2016.
The content on the UpToDate website is not intended nor recommended as a substitute for medical advice, diagnosis, or treatment. Always seek the advice of your own physician or other qualified health care professional regarding any medical questions or conditions. The use of this website is governed by the UpToDate Terms of Use ©2017 UpToDate, Inc.
  1. Ioannidis JP. Why most published research findings are false: author's reply to Goodman and Greenland. PLoS Med 2007; 4:e215.
  2. LeLorier J, Grégoire G, Benhaddad A, et al. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 1997; 337:536.
  3. Cappelleri JC, Ioannidis JP, Schmid CH, et al. Large trials vs meta-analysis of smaller trials: how do their results compare? JAMA 1996; 276:1332.
  4. Villar J, Carroli G, Belizán JM. Predictive ability of meta-analyses of randomised controlled trials. Lancet 1995; 345:772.
  5. Systematic reviews, Chalmers I, Altman DG (Eds), BMJ Publishing Group, London 1995.
  6. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997; 126:376.
  7. Oxman AD, Cook DJ, Guyatt GH. Users' guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA 1994; 272:1367.
  8. Mulrow CD. The medical review article: state of the science. Ann Intern Med 1987; 106:485.
  9. Lau J, Ioannidis JP, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997; 127:820.
  10. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med 2009; 151:W65.
  11. www.prisma-statement.org/Extensions/Default.aspx (Accessed on June 26, 2016).
  12. Institute of Medicine. Finding what workds in health care: Standards for systematic reviews. The National Academies Press, Washington, DC, 2011. Available at: http://www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx (Accessed on October 10, 2011).
  13. Clarke MJ, Stewart LA. Principles of and procedures for systematic reviews. In: Systematic reviews in health care: meta-analysis in context, Egger M, Smith G, Altman D (Eds), BMJ Publishing Group, London 2001. p.23.
  14. Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997; 127:380.
  15. http://www.ncbi.nlm.nih.gov/pubmed (Accessed on July 14, 2011).
  16. http://www.embase.com/search (Accessed on July 14, 2011).
  17. http://onlinelibrary.wiley.com/o/cochrane/cochrane_clcentral_articles_fs.html.
  18. Dickersin K, Chalmers I. Recognising, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the World Health Organisation. James Lind Library 2010. Available at: www.jameslindlibrary.org (Accessed on October 10, 2011).
  19. Mathieu S, Boutron I, Moher D, et al. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 2009; 302:977.
  20. Kirkham JJ, Altman DG, Williamson PR. Bias due to changes in specified outcomes during the systematic review process. PLoS One 2010; 5:e9810.
  21. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998; 279:281.
  22. Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 2000; 53:207.
  23. Goodman S, Dickersin K. Metabias: a challenge for comparative effectiveness research. Ann Intern Med 2011; 155:61.
  24. Vevea JL, Woods CM. Publication bias in research synthesis: sensitivity analysis using a priori weight functions. Psychol Methods 2005; 10:428.
  25. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315:629.
  26. Tang JL, Liu JL. Misleading funnel plot for detection of bias in meta-analysis. J Clin Epidemiol 2000; 53:477.
  27. Terrin N, Schmid CH, Lau J, Olkin I. Adjusting for publication bias in the presence of heterogeneity. Stat Med 2003; 22:2113.
  28. Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol 2005; 58:894.
  29. Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000; 56:455.
  30. Copas J. What works?: Selectivity models and meta-analysis. Journal of the Royal Statistical Society Series A 1999; 162:95.
  31. Rosenthal R. The 'file drawer problem' and tolerance for null results. Psychol Bull 1979; 86:638.
  32. Ioannidis JP, Trikalinos TA. An exploratory test for an excess of significant findings. Clin Trials 2007; 4:245.
  33. Moher D, Jadad AR, Nichol G, et al. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995; 16:62.
  34. Verhagen AP, de Vet HC, de Bie RA, et al. The art of quality assessment of RCTs included in systematic reviews. J Clin Epidemiol 2001; 54:651.
  35. Jüni P, Altman DG, Egger M. Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 2001; 323:42.
  36. Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999; 282:1054.
  37. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273:408.
  38. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 2001; 135:982.
  39. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352:609.
  40. Linde K, Scholz M, Ramirez G, et al. Impact of study quality on outcome in placebo-controlled trials of homeopathy. J Clin Epidemiol 1999; 52:631.
  41. Balk EM, Bonis PA, Moskowitz H, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 2002; 287:2973.
  42. Savović J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med 2012; 157:429.
  43. Balk EM, Tatsioni A, Lichtenstein AH, et al. Effect of chromium supplementation on glucose metabolism and lipids: a systematic review of randomized controlled trials. Diabetes Care 2007; 30:2154.
  44. Deeks JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med 2002; 21:1575.
  45. Walter SD. Choice of effect measure for epidemiological data. J Clin Epidemiol 2000; 53:931.
  46. Wu LA, Kottke TE. Number needed to treat: caveat emptor. J Clin Epidemiol 2001; 54:111.
  47. Engels EA, Schmid CH, Terrin N, et al. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. Stat Med 2000; 19:1707.
  48. Smeeth L, Haines A, Ebrahim S. Numbers needed to treat derived from meta-analyses--sometimes informative, usually misleading. BMJ 1999; 318:1548.
  49. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7:177.
  50. Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet 1998; 351:123.
  51. Cornell JE, Mulrow CD, Localio R, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med 2014; 160:267.
  52. Bradburn MJ, Deeks JJ, Berlin JA, Russell Localio A. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med 2007; 26:53.
  53. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003; 327:557.
  54. Huedo-Medina TB, Sánchez-Meca J, Marín-Martínez F, Botella J. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods 2006; 11:193.
  55. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines 6. Rating the quality of evidence--imprecision. J Clin Epidemiol 2011; 64:1283.
  56. Dechartres A, Altman DG, Trinquart L, et al. Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA 2014; 312:623.
  57. Berlin JA, Golub RM. Meta-analysis as evidence: building a better pyramid. JAMA 2014; 312:603.
  58. Berkey CS, Hoaglin DC, Mosteller F, Colditz GA. A random-effects regression model for meta-analysis. Stat Med 1995; 14:395.
  59. Schmid CH. Exploring heterogeneity in randomized trials via metaanalysis. Drug Inf J 1999; 33:211.
  60. van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med 2002; 21:589.
  61. Ioannidis JP, Cappelleri JC, Sacks HS, Lau J. The relationship between study design, results, and reporting of randomized clinical trials of HIV infection. Control Clin Trials 1997; 18:431.
  62. Schulz KF, Grimes DA. Multiplicity in randomised trials II: subgroup and interim analyses. Lancet 2005; 365:1657.
  63. Rothman KJ, Greenland S. Modern epidemiology, 2nd ed, Lippincott-Raven, Philadelphia 1998.
  64. Povitz M, Bolo CE, Heitman SJ, et al. Effect of treatment of obstructive sleep apnea on depressive symptoms: systematic review and meta-analysis. PLoS Med 2014; 11:e1001762.
  65. Jansen JP, Fleurence R, Devine B, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health 2011; 14:417.
  66. Mills EJ, Ioannidis JP, Thorlund K, et al. How to use an article reporting a multiple treatment comparison meta-analysis. JAMA 2012; 308:1246.
  67. Bangalore S, Kumar S, Kjeldsen SE, et al. Antihypertensive drugs and risk of cancer: network meta-analyses and trial sequential analyses of 324,168 participants from randomised trials. Lancet Oncol 2011; 12:65.
  68. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods 2012; 3:80.
  69. Trinquart L, Attiche N, Bafeta A, et al. Uncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials. Ann Intern Med 2016; 164:666.