Figures
Abstract
Background
There is no consensus as to what extent of “wrap” is required in a fundoplication for correction of gastroesophageal reflux disease (GERD).
Objective
To evaluate if a complete (360 degree) or partial fundoplication gives better control of GERD.
Methods
A systematic search of MEDLINE and Scopus identified interventional and observational studies of fundoplication in children. Screening identified those comparing techniques. The primary outcome was recurrence of GERD following surgery. Dysphagia and complications were secondary outcomes of interest. Meta-analysis was performed when appropriate. Study quality was assessed using the Cochrane Risk of Bias Tool.
Results
2289 abstracts were screened, yielding 2 randomized controlled trials (RCTs) and 12 retrospective cohort studies. The RCTs were pooled. There was no difference in surgical success between partial and complete fundoplication, OR 1.33 [0.67,2.66]. In the 12 cohort studies, 3 (25%) used an objective assessment of the surgery, one of which showed improved outcomes with complete fundoplication. Twenty-five different complications were reported; common were dysphagia and gas-bloat syndrome. Overall study quality was poor.
Citation: Glen P, Chassé M, Doyle M-A, Nasr A, Fergusson DA (2014) Partial versus Complete Fundoplication for the Correction of Pediatric GERD: A Systematic Review and Meta-Analysis. PLoS ONE 9(11): e112417. https://doi.org/10.1371/journal.pone.0112417
Editor: Wan Yee J. Lau, The Chinese University of Hong Kong, Hong Kong
Received: June 22, 2014; Accepted: October 16, 2014; Published: November 11, 2014
Copyright: © 2014 Glen et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. All relevant data are within the paper and its Supporting Information files.
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Gastroesophageal reflux disease (GERD) results from pathologic reflux of stomach content into the esophagus, causing troublesome symptoms [1]. In children the severity of symptoms is highly variable, ranging from epigastric pain to life threatening aspiration [2]. Management options reflect the variation in disease severity with mild cases responding well to medical therapy; however in severe or refractory cases surgical intervention is warranted [3].
Surgical fundoplication was first described in the adult literature in 1956 by Nissen [4]. This eponymous intervention creates an anatomic barrier to reflux by wrapping the gastric fundus around the esophagus in a complete 360 degree fashion [5], [6]. Contemporaries of Nissen described techniques involving a partial wrap. Thal and Toupet described wrapping the fundus partially (180 to 270 degrees) around the esophagus [7], [8]. All procedures have been incorporated into practices worldwide along with other derivations [9]–[11]. In the era of minimally invasive surgery, laparoscopic fundoplication in children has become accepted standard of practice [12]–[14].
While the benefit of laparoscopy is well accepted, the choice of a partial or complete wrap remains controversial. Advocates of the complete wrap claim better correction of GERD; conversely proponents of partial wrap point to lower rates of dysphagia and gas-bloating. A recent meta-analysis in adults found evidence to suggest correction of GERD similar for both techniques and higher rates of dysphagia and gas-bloat resulted from complete fundoplication [15]. The conclusion in this study was that partial fundoplication was the superior technique; however recent guidelines do not explicitly endorse this conclusion [16].
The adult literature cannot be expressly translated to management of children, as paediatric GERD is associated with different pathophysiology and is often associated with neurologic disorders. In children there is little evidence to guide the choice of wrap and current guidelines suggest both are acceptable [2], [17]. Although a systematic review conducted in 2011 concluded that there was equivalent control of reflux symptoms, it employed a strategy of pooling observational data with RCT data which can introduce bias [18]. Additionally, the study used did not use a comprehensive search strategy, reported a limited quality assessment, and may have missed important evidence. These factors support the need for an updated systematic review.
Our objective was to conduct a systematic review of interventional and observational studies comparing partial and complete fundoplication for pediatric GERD. This review was designed to compare reflux control, dysphagia, and complication rates between the two techniques. Additionally, we sought to ascertain the quality of the studies addressing the question.
Materials and Methods
Study Selection
A systematic literature search strategy using text and MeSH terms was used to conduct a search of the Medline (from 1946) and Scopus (from 1960) databases on February 15, 2013. The search was performed with no language or date restrictions (File S1 and File S2).
Inclusion Criteria
Interventional and observational studies were eligible if they were designed to directly compare the outcomes of partial and complete fundoplication for the correction of gastroesophageal reflux disease (GERD). Partial fundoplication was defined as any surgical technique employing a wrap of less than 360 degrees to correct reflux disease. Complete fundoplication, commonly known as a Nissen fundoplication, was defined by a wrap of at least 360 degrees. Primary research studies involving children aged less than 18 years were included. Only studies that reported the success of anti-reflux surgery as measured by recurrence of the signs and symptoms of GERD were included. In studies for which there were multiple publications, the most recent publication was used. Only studies published in English or French were eligible.
Exclusion Criteria
Studies where the pediatric population was indistinguishable from adults were eliminated. Animal studies, case reports, review articles, and editorials were ineligible. Studies in which the surgical indication was not GERD were also excluded (e.g. achalasia and congenital diaphragmatic hernia).
Assessment of Study Eligibility
A two-stage screening of abstracts was performed to identify eligible studies. In Phase I, a standardized abstract screening tool was validated by three reviewers (PG, MC, and MD). This tool was used to eliminate ineligible studies by their titles and abstracts by the primary reviewer (PG). If there was uncertainty from the title or abstract then the article was kept for full text review. The abstracts kept after Phase I were reviewed by two reviewers in Phase II. Full text review and data abstraction were performed by PG, and MC or MD for all eligible studies.
Outcomes
The primary outcome was the success rate of fundoplication to correct symptoms of GERD. A successful surgery was defined as one which corrected the signs and symptoms of GERD (i.e. an absence of pre-operative signs and symptoms). Determination of surgical success was grouped according to whether there was an objective method (eg. radiographic contrast studies, mannometry, 24 hour-pH monitoring, or endoscopic) or subjective method (patient reported, or investigator reported outcomes) of assessment.
Rates of dysphagia following fundoplication and other reported complications of the surgery were evaluated as secondary outcomes.
Data Abstraction
The full text articles for studies with eligible abstracts were independently reviewed by the primary author (PG) and one other reviewer (MC or MD). Data from studies meeting inclusion criteria were captured using a standardized form which was tested using a pilot process. Study design elements, patient demographics, surgical techniques and definitions of surgical success were abstracted. Conflicts were resolved by consensus between the reviewers. In circumstances where the data was incomplete, an attempt was made to contact the primary author.
Data Analysis
For the purposes of this review, all surgical techniques that employed a wrap of less than 360 degrees were grouped as a partial fundoplication. These were compared to subjects who had undergone a complete fundoplication. A pre-specified sub-group analysis was planned to assess the impact of neurological impairment on outcome. All data analysis was performed using the RevMan 5 software.
The results were analyzed separately for interventional and observational studies. Results were reported as Odds Ratios with 95% confidence intervals. We performed a meta-analysis of the interventional studies using a random effects model to calculate pooled odds ratio and 95% confidence interval. Statistical heterogeneity was measured using the I2 statistic.
Risk of Bias
Each study was independently assessed for risk of bias by two reviewers using the Cochrane Risk of Bias tool [19]. Conflicts were resolved by consensus. The tool was modified to assess for the risk of bias from confounding factors which were not accounted for in the design or statistical analysis, this was evaluated within the “other” category of the tool.
Results
Search Results
Our systematic search strategy yielded 2289 abstracts after automated removal of duplicates. Phase I of screening identified 83 potentially eligible articles; Phase II identified 14 studies to be included in the final analysis: 2 randomized controlled trials (RCT) [20], [21] and 12 retrospective cohort studies [22]–[33] (Figure 1).
Study Characteristics
The included studies were published between 1994 and 2011. A total of 1927 subjects were identified, 967 (50%) were given a partial fundoplication. Patient age ranged from 2 months to 21 years (Table 1). Patient selection was based primarily on symptoms of GERD which were refractory to medical therapy (Table 1). Both RCTs required documented GERD on a pH study or upper gastrointestinal series (UGIS) before patient enrollment. Ten cohort studies required an objective measure of GERD, with one requiring only severe symptoms refractory to medical therapy [25]. In both RCTs and in 10 of the cohort studies a gastrostomy tube was a co-intervention for select patients.
The longest recorded follow up was 18 years in one study [24].
In all 14 studies the Nissen fundoplication was used as the complete wrap (Table 1). 12 of 14 studies used a single standard technique of partial fundoplication; 2 of 14 studies used at least 2 different partial wrap techniques. The most common partial fundoplications were the Thal (7/14 studies) [21], [23], [26]–[29], [31] and Toupet (5/14) [22], [25], [26], [28], [33]. The other partial techniques used included the Watson (1/14) [32], Boix-Ochoa (2/14) [24], [30], and Vertical Gastric Plication (1/14) [20].
Primary Outcome: Correction of Reflux
Two randomized trials compared the performance of partial versus complete wraps. Kubiak et al. (2011) compared the Thal and Nissen wraps [21]. Follow-up for these patients ranged from 1–109 months. In this study the intervention was deemed successful if there was no need for anti-reflux medications or surgical revision. The RCT published by Durante et al. (2007) compared Vertical Gastric Plication with a Nissen wrap [20]. In this study success was determined using a clinical scoring system. Neither of these trials demonstrated a statistically significant difference between surgical techniques independently or in the pooled analysis, OR 1.33 [0.67, 2.66] (Figure 2).
Three of the twelve cohort studies objective measures to determine recurrence of symptoms. Symptoms of recurrence were confirmed using one or more of a pH study, upper gastrointestinal series (UGIS), or esophagogastroduodenoscopy (EGD). In the remaining 9 studies only patient or surgeon reported outcomes were used to measure recurrence.
Two studies showed greater benefit from the complete fundoplication, while 2 other studies demonstrated greater benefit from the partial fundoplication when odds ratios were calculated from crude data (Figure 3). The remaining 8 studies did not demonstrate a significant difference between the two techniques. All studies compared the two techniques, but none reported adjusted odds ratios.
Subgroup Analysis of Neurologically Impaired Patients.
Both RCTs enrolled a proportion of neurologically impaired patients [20], [21]. The pooled OR of these trials favoured success with complete fundoplication but did not demonstrate a statistically significant difference with OR 1.68 [0.41, 6.81] (Figure 4).
Nine observational studies enrolled some patients with neurological impairment and four reported specific outcomes for the neurologically impaired patients. There was insufficient evidence in each individual study to conclude a difference in symptomatic relief with either wrap.
Secondary Outcome: Complications
A total of 25 different types of complications were reported in the 14 studies with the most commonly reported being dysphagia, gas-bloat syndrome, wrap disruption, and paraesophageal hernia (Table 2). Death was reported as a complication in 11 studies. All of these studies stated that death resulted from complications of an underlying chronic disease or were due to complications relating to gastrostomy tube displacement. There were no deaths attributed to the wrap technique.
Postoperative Dysphagia.
Dysphagia was reported in one interventional study. This study did not find a difference in incidence of dysphagia between the two techniques [21] (Figure 5).
Seven observational studies reported post-operative dysphagia. The definition and severity of dysphagia varied between studies. One found a significant benefit of the partial fundoplication on rates of dysphagia, OR 0.20 [0.08,0.48] [33] (Figure 5). In this study, patients were selected to receive partial fundoplication if it was felt there was underlying esophageal or gastric dysmotility. This confounder was not accounted for in the original publication. Additionally, there was no distinction made between dysphagia and gas-bloat syndrome in this publication. Attempts to reach the author for further clarification were unsuccessful.
Trial Quality Assessment
Results of the Cochrane Risk of Bias demonstrate that overall trial quality was poor (Figure 6). The RCT published by Durante scored a low risk of bias on two of the domains, unclear on one, and high in the remaining four domains. The study published by Kubiak et al. was at higher risk of bias with three unclear and four high risk domains.
Shaded: high risk of bias; checkered: unclear or indeterminate risk of bias; not shaded: low risk of bias.
The observational studies scored similarly. With respect to the control of confounding factors, all studies scored a high risk of bias as it was unclear if any control for confounding factors had been conducted.
Discussion
Surgical fundoplication for correction of GERD refractive to medical therapy is a standard intervention. The results of this review find no evidence to indicate that the extent of wrap, partial or complete, results in a superior correction of GERD symptoms. The paucity of well conducted studies in this area justifies for further study. The lack of well conducted RCTs is a problem not only for the study of this question, but many pediatric surgical questions [34].
In a previous review addressing the question of complete versus partial fundoplication, trials were pooled irrespective of their trial design [18]. This may lead to biased results as consideration for study design is vital when pooling non-randomized trial data [35]. We chose to distinguish between these two designs, and pool only RCTs. There was great variation between studies with respect to their definition of GERD, definition of treatment failure, and length of follow up (Table 1). These factors further affect the ability to draw meaningful conclusions from pooled data.
High quality RCTs are vital to providing evidence-based estimates of effectiveness and safety to clinicians [36]. The quality of trials designed to study pediatric fundoplication are on the whole poor (Figure 6). Only in the domain of completeness of outcome reporting did at least half of the trials demonstrate low risk of bias (7/14). Understanding these high risks of bias are important for understanding the limitations of this review and state of the evidence.
Our primary outcome was correction of GERD following partial and complete fundoplication. There were two RCTs designed to answer this question. The study by Durante used a symptom score to grade the recurrence of reflux [20], however there was no description given for this score. The study by Kubiak et al. was based on symptom recurrence requiring medical therapy or surgical revision [21]. With these differences in mind, we conducted a pooled analysis for the primary outcome. There was no statistical difference in correction of GERD following partial or complete fundoplication (Figure 2). We also did not find a difference in outcome in the neurologically impaired subgroup (Figure 3).
In the observational trials, heterogeneity was even more apparent (Table 1). Indications for surgery were based on a variety of tests and symptom scores; likewise there was variation in the definitions of failed surgery. Outcome reporting in these studies was generally poorly done, with none of these trials performing any statistical control for confounders. The trial heterogeneity and statistical analysis precludes any meaningful meta-analysis, as the reported information is insufficient to satisfy the Cochrane criteria for inclusion of non-randomized studies [37]. Instead, we have chosen to present the data as a Forest plot based on an intention to treat analysis conducted using the data presented in this group of trials. Figure 4 shows that there is great variation in the conclusions drawn from each of these trials, confirming the notion that there are uncontrolled confounders driving this data.
Heterogeneous trials presented similar issues for the analysis of our secondary outcome measure: post-operative dysphagia. The review published by Mauritz et al. concluded that there was a greater incidence of post-operative dysphagia following complete fundoplication [18]. We did not reach the same conclusion as we could not find a reliable means of pooling the outcomes addressing this question (Figure 5).
The variety of post-operative complications found in these studies is indicative both of the morbidity associated with the surgery and with underlying medical conditions often necessitating fundoplication. We have employed a systematic strategy to identify complications arising from surgery and shown that there are a large number of potential complications which both the surgeon and patient should be aware of. Table 2 outlines 25 post-operative complications identified across the 14 studies. Some of these certainly overlap and are indicative of synonymous description of symptoms (e.g. retching and regurgitation). It should also be noted that for many studies there was concomitant placement of gastrostomy tubes. This intervention was typically added to fundoplication in cases of severe neurologic impairment. This understanding is important when interpreting the complications. For instance, 11 trials reported mortality data, however in all of these trials death was attributed to the underlying neurologic impairment or complications attributed to the gastrostomy tube [20]–[27], [30], [32]. This spectrum of possible complications should be understood by clinicians proposing to manage GERD with anti-reflux surgery.
For surgeons who are planning fundoplication for a patient the evidence presented here suggests that an individualized treatment plan be developed. Both techniques are technically similar, with the main variation being the placement of intra-corporeal sutures. Patient factors are thus the variables which should drive the decision on the extent of wrap required. In children these factors will vary with age and severity of symptoms, as well as the tolerance for post-operative issues with dysphagia. Although the evidence is poor, we would suggest that partial fundoplication should be considered for patients in whom tolerance for post-operative dysphagia is low. If the primary concern is correction of reflux, and there is little concern for the effects of possible dysphagia, then complete Nissen fundoplication should be the intervention of choice.
Further study on fundoplication technique in children is warranted. Currently there is one study registered with clinicaltrials.gov which have not yet been reported in the literature (NCT 00480285). This study was due to have been completed in 2011 yet no publication was found. Further interventional trial research is indicated to define which technique is better able to control reflux symptoms. Research to identify pre-operative factors associated with treatment complications, such as dysphagia, would also be beneficial.
Conclusion
The comparison of partial fundoplication with complete fundoplication warrants further study. Choosing between partial and complete fundoplication will rely mainly on the preferences of the surgeon and patient as presently there is no conclusive evidence to support one technique over another. The lack of high quality RCTs in the area points to the need for a well conducted and reported study, as there is insufficient evidence to establish a conclusive answer using meta-analysis. The variation in the definition of GERD failure also suggests that a universal definition be defined.
Supporting Information
File S1.
Search string for Medline database.
https://doi.org/10.1371/journal.pone.0112417.s002
(DOCX)
Acknowledgments
Risa Shore, Librarian, The Ottawa Hospital (conducted systematic database search and article retrieval).
Author Contributions
Conceived and designed the experiments: PG MC MAD AN DAF. Performed the experiments: PG MC MAD. Analyzed the data: PG MC MAD DAF. Contributed reagents/materials/analysis tools: PG DAF. Contributed to the writing of the manuscript: PG MC MAD AN DAF.
References
- 1. Vakil N, van Zanten S V, Kahrilas P, Dent J, Jones R (2006) The Montreal definition and classification of gastroesophageal reflux disease: a global evidence-based consensus. Am J Gastroenterol 101: 1900–20.
- 2.
GERD (2009) IPEG guidelines for the surgical treatment of pediatric gastroesophageal reflux disease (GERD) (2009). J Laparoendosc Adv Surg Tech 19: x–xiii.
- 3. Vandenplas Y, Rudolph C (2009) Pediatric gastroesophageal reflux clinical practice guidelines: joint recommendations of the North American Society for Pediatric Gastroenterology, Hepatology, and. J Pediatr Gastroenterol Nutr 49: 498–547.
- 4. Nissen R (1956) A simple operation for control of reflux esophagitis. Schweiz Med Wochenschr 86: 590–592.
- 5. Nissen R (1961) Gastropexy and “Fundoplication” in surgical treatment of hiatal hernia. Am J Dig Dis 6: 954–961.
- 6. Krupp S, Rossetti M (1966) Surgical treatment of hiatal hernias by fundoplication and gastropexy (Nissen repair). Ann Surg 164: 927–934.
- 7. Toupet A (1963) Technic of esophago-gastroplasty with phrenogastropexy used in radical treatment of hiatal hernias as a supplement to Heller’s operation in cardiospasms. Mem Acad Chir (Paris) 89: 384–389.
- 8. Thal AP, Kurtzman R (1965) A new method for reconstruction of the esophagogastric junction. Surg Gynecol Obstet 120: 1225–1231.
- 9. Boix-Ochoa J (1986) The Physiologic Approach to the Management of Gastric Esophageal Reflux. J Pediatr Surg 2: 1032–1039.
- 10. Taylor T V, Knox RA, Pullan BR (1989) Vertical gastric plication: an operation for gastro-oesophageal reflux. Ann R Coll Surg Engl 71: 31–36.
- 11. Watson a, Jenkinson LR, Ball CS, Barlow a P, Norris TL (1991) A more physiological alternative to total fundoplication for the surgical correction of resistant gastro-oesophageal reflux. Br J Surg 78: 1088–1094.
- 12. Kane TD, Brown MF, Chen MK (2009) Committee M of the ANT (2009) Position paper on laparoscopic antireflux operations in infants and children for gastroesophageal reflux disease. American Pediatric Surgery Association. J Pediatr Surg 44: 1034–1040.
- 13. Siddiqui MRS, Abdulaal Y, Nisar A, Ali H, Hasan F (2011) A meta-analysis of outcomes after open and laparoscopic Nissen’s fundoplication for gastro-oesophageal reflux disease in children. Pediatr Surg Int 27: 359–366.
- 14. Ma S, Qian B, Shang L, Shi R, Zhang G (2012) A meta-analysis comparing laparoscopic partial versus Nissen fundoplication. ANZ J Surg 82: 17–22.
- 15. Broeders J a JL, Mauritz F a, Ahmed Ali U, Draaisma W a, Ruurda JP, et al. (2010) Systematic review and meta-analysis of laparoscopic Nissen (posterior total) versus Toupet (posterior partial) fundoplication for gastro-oesophageal reflux disease. Br J Surg 97: 1318–1330.
- 16. Stefanidis D, Hope WW, Kohn GP, Reardon PR, Richardson WS, et al. (2010) Guidelines for surgical treatment of gastroesophageal reflux disease. Surg Endosc 24: 1872–1878.
- 17. Mauritz F a, van Herwaarden-Lindeboom MY a, Stomp W, Zwaveling S, Fischer K, et al. (2011) The effects and efficacy of antireflux surgery in children with gastroesophageal reflux disease: a systematic review. J Gastrointest Surg 15: 1872–1878.
- 18. Mauritz F a, Blomberg B a, Stellato RK, van der Zee DC, Siersema PD, et al. (2013) Complete versus partial fundoplication in children with gastroesophageal reflux disease: results of a systematic review and meta-analysis. J Gastrointest Surg 17: 1883–1892.
- 19. Higgins J, Altman D (2011) The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Br Med J 343: 1–9.
- 20. Durante AP, Schettini ST, Fagundes DJ (2007) Vertical gastric plication versus Nissen fundoplication in the treatment of gastroesophageal reflux in children with cerebral palsy. Sao Paulo Med J 125: 15–21.
- 21. Kubiak R, Andrews J, Grant HW (2011) Long-term outcome of laparoscopic nissen fundoplication compared with laparoscopic thal fundoplication in children: a prospective, randomized study. Ann Surg 253: 44–49.
- 22. Allal H, Captier G, Lopez M, Forgues D, Galifer RB (2001) Evaluation of 142 consecutive laparoscopic fundoplications in children: effects of the learning curve and technical choice. J Pediatr Surg 36: 921–926.
- 23. Ceriati E, Guarino N, Zaccara A, Marchetti P, La Sala E, et al. (1998) Gastroesophageal reflux in neurologically impaired children: Partial or total fundoplication? Langenbeck’s Arch Surg 383: 317–319.
- 24. Cohen Z, Fishman S, Yulevich A, Kurtzbart E, Mares AJ (1999) Nissen fundoplication and Boix-Ochoa antireflux procedure: comparison between two surgical techniques in the treatment of gastroesophageal reflux in children. Eur J Pediatr Surg 9: 289–293.
- 25. Georgeson KE (1998) Laparoscopic fundoplication and gastrostomy. Semin Laparosc Surg 5: 25–30.
- 26. Goessler A, Huber-Zeyringer A, Hoellwarth ME (2007) Recurrent gastroesophageal reflux in neurologically impaired patients after fundoplication. Acta Paediatr 96: 87–93.
- 27. Kazerooni NL, VanCamp J, Hirschl RB, Drongowski RA, Coran AG (1994) Fundoplication in 160 children under 2 years of age. J Pediatr Surg 29: 677–681.
- 28. Levin DN, Diamond IR, Langer JC (2011) Complete vs partial fundoplication in children with esophageal atresia. J Pediatr Surg 46: 854–858.
- 29. Strecker-McGraw MK, Lorenz ML, Hendrickson M, Jolley SG, Tunell WP (1998) Persistent gastroesophageal reflux disease after antireflux surgery in children: I. immediate postoperative evaluation using extended esophageal pH monitoring. J Pediatr Surg 33: 1623–1627.
- 30. Subramaniam R, Dickson AP (2000) Long-term outcome of Boix-Ochoa and Nissen fundoplication in normal and neurologically impaired children. J Pediatr Surg 35: 1214–1216.
- 31. Van Der Zee DC, Rovekamp MH, Pull Ter Gunne AJ, Bax NMA (1994) Surgical treatment of reflux esophagitis: Nissen versus Thal procedure. Pediatr Surg Int 9: 334–337.
- 32. Wagener S, Sudhakaran N, Cusick E (2007) Watson fundoplication in children: a comparative study with Nissen fundoplication. J Pediatr Surg 42: 1098–1102.
- 33. Weber TR (1999) Toupet fundoplication for gastroesophageal reflux in childhood. Arch Surg 134: 711–717.
- 34. Curry BJI, Reeves B, Stringer D (2003) Randomized controlled trials in pediatric surgery: could we do better? J Pediatr Surg 38: 556–559.
- 35. Valentine JC, Thompson SG (2013) Issues relating to confounding and meta-analysis when including non-randomized studies in systematic reviews on the effects of interventions. Res Synth Methods 4: 26–35.
- 36. Jüni P, Altman DG, Egger M (2001) Assessing the quality of controlled clinical trials. BMJ 323: 42–46.
- 37.
Reeves BC, Deeks JJ, Higgins JPT, Wells GA (2008) Chapter 13: Including non-randomized studies. In: Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions.