Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Research article

Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool

Authors: Heather Menzies Munthe-Kaas, Claire Glenton, Andrew Booth, Jane Noyes, Simon Lewin

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

Qualitative evidence synthesis is increasingly used alongside reviews of effectiveness to inform guidelines and other decisions. To support this use, the GRADE-CERQual approach was developed to assess and communicate the confidence we have in findings from reviews of qualitative research. One component of this approach requires an appraisal of the methodological limitations of studies contributing data to a review finding. Diverse critical appraisal tools for qualitative research are currently being used. However, it is unclear which tool is most appropriate for informing a GRADE-CERQual assessment of confidence.

Methodology

We searched for tools that were explicitly intended for critically appraising the methodological quality of qualitative research. We searched the reference lists of existing methodological reviews for critical appraisal tools, and also conducted a systematic search in June 2016 for tools published in health science and social science databases. Two reviewers screened identified titles and abstracts, and then screened the full text of potentially relevant articles. One reviewer extracted data from each article and a second reviewer checked the extraction. We used a best-fit framework synthesis approach to code checklist criteria from each identified tool and to organise these into themes.

Results

We identified 102 critical appraisal tools: 71 tools had previously been included in methodological reviews, and 31 tools were identified from our systematic search. Almost half of the tools were published after 2010. Few authors described how their tool was developed, or why a new tool was needed. After coding all criteria, we developed a framework that included 22 themes. None of the tools included all 22 themes. Some themes were included in up to 95 of the tools.

Conclusion

It is problematic that researchers continue to develop new tools without adequately examining the many tools that already exist. Furthermore, the plethora of tools, old and new, indicates a lack of consensus regarding the best tool to use, and an absence of empirical evidence about the most important criteria for assessing the methodological limitations of qualitative research, including in the context of use with GRADE-CERQual.
Appendix
Available only for authorised users
Literature
1.
Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Systematic Reviews. 2012:1(28).
2.
Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, Noyes J, Rashidian A. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev. 2013;2013.
3.
Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin C, Gülmezoglu M, Noyes J, Booth A, Garside R, Rashidian A. Using qualitative evidence in decision making for heatlh and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med. 2015;12(10):e1001895.PubMedPubMedCentralCrossRef
4.
Guyatt G, Oxman A, Kunz R, Vist G, Falck-Ytter Y, Schunemann H. For the GRADE working group: what is "quality of evidence" and why is it important to clinicians? BMJ. 2008;336:995–8.PubMedPubMedCentralCrossRef
5.
Lewin S, Booth A, Bohren M, Glenton C, Munthe-Kaas HM, Carlsen B, Colvin CJ, Tuncalp Ö, Noyes J, Garside R, et al. Applying the GRADE-CERQual approach (1): introduction to the series. Implement Sci. 2018.
6.
Katrak P, Bialocerkowski A, Massy-Westropp N, Kumar V, Grimmer K. A systematic review of the content of critical appraisal tools. BMC Med Res Methodol. 2004:4(22).
7.
Denzin N. Qualitative inquiry under fire: toward a new paradigm dialogue. USA: Left Coast Press; 2009.
8.
Hammersley M. The issue of quality in qualitative research. International Journal of Research & Method in Education. 2007;30(3):287–305.CrossRef
9.
Smith J. The problem of criteria for judging interpretive inquiry. Educ Eval Policy Anal. 1984;6(4):379–91.CrossRef
10.
Smith J, Deemer D. The problem of criteria in the age of relativism. In: Densin N, Lincoln Y, editors. Handbook of Qualitative Research. London: Sage Publication; 2000.
11.
Noyes J, Booth A, Flemming K, Garside R, Harden A, Lewin S, Pantoja T, Hannes K, Cargo M, Thomas J. Cochrane qualitative and implementation methods group guidance series—paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018;1(97):49–58.CrossRef
12.
Soilemezi D, Linceviciute S. Synthesizing qualitative research: reflections and lessons learnt by two new reviewers. Int J Qual Methods. 2018;17(1):160940691876801.CrossRef
13.
Carroll C, Booth A. Quality assessment of qualitative evidence for systematic review and synthesis: is it meaningful, and if so, how should it be performed? Res Synth Methods. 2015;6(2):149–54.PubMedCrossRef
14.
Sandelowski M. A matter of taste: evaluating the quality of qualitative research. Nurs Inq. 2015;22(2):86–94.PubMedCrossRef
15.
Garside R. Should we appraise the quality of qualitative research reports for systematic reviews, and if so, how? Innovation: The European Journal of Social Science Research. 2013;27(1):67–79.
16.
Barusch A, Gringeri C, George M: Rigor in Qualitative Social Work Research: A Review of Strategies Used in Published Articles. Social Work Research 2011, 35(1):11–19 19p.CrossRef
17.
Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005;10:45–53.PubMedCrossRef
18.
Green J, Thorogood N: Qualitative methodology in health research. In: Qualitative methods for health research, 4th Edition. Edn. Edited by Seaman J. London, UK: Sage Publications; 2018.
19.
Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009:9(59).
20.
Gough D, Oliver S, Thomas J. An introduction to systematic reviews. London, UK: Sage; 2017.
21.
Hannes K, Macaitis K. A move to more transparent and systematic approaches of qualitative evidence synthesis: update of a review on published papers. Qual Res. 2012;12:402–42.CrossRef
22.
Santiago-Delefosse M, Gavin A, Bruchez C, Roux P, Stephen SL. Quality of qualitative research in the health sciences: Analysis of the common criteria present in 58 assessment guidelines by expert users. Social Science & Medicine. 2016;148:142–151 110p.CrossRef
23.
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.PubMedCrossRef
24.
25.
Dixon-Woods M, Sutton M, Shaw RL, Miller T, Smith J, Young B, Bonas S, Booth A, Jones D. Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. Journal of Health Services Research & Policy. 2007;12(1):42–7.CrossRef
26.
Long AF, Godfrey M. An evaluation tool to assess the quality of qualitative research studies. Int J Soc Res Methodol. 2004;7(2):181–96.CrossRef
27.
Popay J, Rogers A, Williams G. Rationale and Standards for the Systematic Review of Qualitative Literature in Health Services Research. Qual Health Res. 1998:8(3).PubMedCrossRef
28.
Dalton J, Booth A, Noyes J, Sowden A. Potential value of systematic reviews of qualitative evidence in informing user-centered health and social care: findings from a descriptive overview. J Clin Epidemiol. 2017;88:37–46.PubMedCrossRef
29.
Lundh A, Gøtzsche P. Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies. BMC Med Res Methodol. 2008;8(22).
30.
Higgins J, Sterne J, Savović J, Page M, Hróbjartsson A, Boutron I, Reeves B, Eldridge S: A revised tool for assessing risk of bias in randomized trials In: Cochrane Methods. Edited by J. C, McKenzie J, Boutron I, Welch V, vol. 2016: Cochrane Database Syst Rev 2016.
31.
Higgins J, Altman D, Gøtzsche P, Jüni P, Moher D, Oxman A, Savović J, Schulz K, Weeks L, Sterne J. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;18(343):d5928.CrossRef
32.
Crowe M, Sheppard L. A review of critical appraisal tools show they lack rigor: alternative tool structure is proposed. J Clin Epidemiol. 2011;64(1):79–89.PubMedCrossRef
33.
Majid U, Vanstone M. Appraising qualitative research for evidence syntheses: a compendium of quality appraisal tools Qualitative Health Research; 2018.
34.
Santiago-Delefosse M, Bruchez C, Gavin A, Stephen SL. Quality criteria for qualitative research in health sciences. A comparative analysis of eight grids of quality criteria in psychiatry/psychology and medicine. Evolution Psychiatrique. 2015;80(2):375–99.CrossRef
35.
36.
37.
Methods for the development of NICE public health guidance (third edition): Process and methods. In. UK: National Institute for Health and Care Excellence; 2012.
38.
39.
Baillie L: Promoting and evaluating scientific rigour in qualitative research. Nursing standard (Royal College of Nursing (Great Britain) : 1987) 2015, 29(46):36–42.PubMedCrossRef
40.
Ballinger C. Demonstrating rigour and quality? In: LFCB, editor. Qualitative research for allied health professionals: Challenging choices. Chichester, England: J. Wiley & Sons; 2006. p. 235–46.
41.
Bleijenbergh I, Korzilius H, Verschuren P. Methodological criteria for the internal validity and utility of practice oriented research. Qual Quant. 2011;45(1):145–56.CrossRef
42.
Boeije HR, van Wesel F, Alisic E. Making a difference: towards a method for weighing the evidence in a qualitative synthesis. J Eval Clin Pract. 2011;17(4):657–63.PubMedCrossRef
43.
Boulton M, Fitzpatrick R, Swinburn C. Qualitative research in health care: II. A structured review and evaluation of studies. J Eval Clin Pract. 1996;2(3):171–9.PubMedCrossRef
44.
Britton N, Jones R, Murphy E, Stacy R. Qualitative research methods in general practice and primary care. Fam Pract. 1995;12(1):104–14.CrossRef
45.
46.
Caldwell K, Henshaw L, Taylor G. Developing a framework for critiquing health research: an early evaluation. Nurse Educ Today. 2011;31(8):e1–7.PubMedCrossRef
47.
Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003;56(4):671–84.PubMedCrossRef
48.
Carter S, Little M. Justifying knowledge, justifying method, taking action: epistemologies, methodologies, and methods in qualitative research. Qual Health Res. 2007;17(10):1316–28.PubMedCrossRef
49.
Cesario S, Morin K, Santa-Donato A. Evaluating the level of evidence of qualitative research. J Obstet Gynecol Neonatal Nurs. 2002;31(6):708–14.PubMedCrossRef
50.
Cobb AN, Hagemaster JN. Ten criteria for evaluating qualitative research proposals. J Nurs Educ. 1987;26(4):138–43.PubMed
51.
Cohen D, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. The Annals of Family Medicine. 2008;6(4):331–9.PubMedCrossRef
52.
53.
Côté L, Turgeon J. Appraising qualitative research articles in medicine and medical education. Medical Teacher. 2005;27(1):71–5.PubMedCrossRef
54.
Creswell JW. Qualitative procedures. Research design: qualitative, quantitative, and mixed method approaches (2nd ed.). Thousand Oaks, CA: Sage Publications; 2003.
55.
10 questions to help you make sense of qualitative research.
56.
Crowe M, Sheppard L. A general critical appraisal tool: an evaluation of construct validity. Int J Nurs Stud. 2011;48(12):1505–16.PubMedCrossRef
57.
Currie G, McCuaig C, Di Prospero L. Systematically Reviewing a Journal Manuscript: A Guideline for Health Reviewers. Journal of Medical Imaging and Radiation Sciences. 2016;47(2):129–138.e123.PubMedCrossRef
58.
Curtin M, Fossey E. Appraising the trustworthiness of qualitative studies: guidelines for occupational therapists. Aust Occup Ther J. 2007;54:88–94.CrossRef
59.
Cyr J. The pitfalls and promise of focus groups as a data collection method. Sociol Methods Res. 2016;45(2):231–59.CrossRef
60.
61.
El Hussein M, Jakubec SL, Osuji J. Assessing the FACTS: a mnemonic for teaching and learning the rapid assessment of rigor in qualitative research studies. Qual Rep. 2015;20(8):1182–4.
62.
Elder NC, Miller WL. Reading and evaluating qualitative research studies. J Fam Pract. 1995;41(3):279–85.PubMed
63.
Elliott R, Fischer CT, Rennie DL. Evolving guidelines for publication of qualitative research studies in psychology and related fields. Br J Clin Psychol. 1999;38(3):215–29.PubMedCrossRef
64.
Farrell SE, Kuhn GJ, Coates WC, Shayne PH, Fisher J, Maggio LA, Lin M. Critical appraisal of emergency medicine education research: the best publications of 2013. Acad Emerg Med Off J Soc Acad Emerg Med. 2014;21(11):1274–83.CrossRef
65.
Fawkes C, Ward E, Carnes D. What evidence is good evidence? A masterclass in critical appraisal. International Journal of Osteopathic Medicine. 2015;18(2):116–29.CrossRef
66.
Forchuk C, Roberts J. How to critique qualitative research articles. Can J Nurs Res. 1993;25(4):47–56.PubMed
67.
Forman J, Crewsell J, Damschroder L, Kowalski C, Krein S. Quailtative research methods: key features and insights gained from use in infection prevention research. Am J Infect Control. 2008;36(10):764–71.PubMedCrossRef
68.
Fossey E, Harvey C, McDermott F, Davidson L. Understanding and evaluating qualitative research. Aust N Z J Psychiatry. 2002;36(6):717–32.PubMedCrossRef
69.
Fujiura GT. Perspectives on the publication of qualitative research. Intellectual and Developmental Disabilities. 2015;53(5):323–8.PubMedCrossRef
70.
71.
Greenhalgh T, Wengraf T. Collecting stories: is it research? Is it good research? Preliminary guidance based on a Delphi study. Med Educ. 2008;42(3):242–7.PubMedCrossRef
72.
Gringeri C, Barusch A, Cambron C. Examining foundations of qualitative research: a review of social work dissertations, 2008-2010. J Soc Work Educ. 2013;49(4):760–73.CrossRef
73.
Hoddinott P, Pill R. A review of recently published qualitative research in general practice. More methodological questions than answers? Fam Pract. 1997;14(4):313–9.PubMedCrossRef
74.
Inui T, Frankel R. Evaluating the quality of qualitative research: a proposal pro tem. J Gen Intern Med. 1991;6(5):485–6.PubMedCrossRef
75.
Jeanfreau SG, Jack L Jr. Appraising qualitative research in health education: guidelines for public health educators. Health Promot Pract. 2010;11(5):612–7.PubMedPubMedCentralCrossRef
76.
Kitto SC, Chesters J, Grbich C. Quality in qualitative research: criteria for authors and assessors in the submission and assessment of qualitative research articles for the medical journal of Australia. Med. J. Aust. 2008;188(4):243–6.PubMed
77.
Kneale J, Santry J. Critiquing qualitative research. J Orthop Nurs. 1999;3(1):24–32.CrossRef
78.
Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ. 2008;337:687–92.CrossRef
79.
Lane S, Arnold E. Qualitative research: a valuable tool for transfusion medicine. Transfusion. 2011;51(6):1150–3.PubMedCrossRef
80.
Lee E, Mishna F, Brennenstuhl S. How to critically evaluate case studies in social work. Res Soc Work Pract. 2010;20(6):682–9.CrossRef
81.
Leininger M: Evaluation criteria and critique of qualitative research studies. In: Critical issues in qualitative research methods. edn. Edited by (Ed.) JM. Thousand Oaks, CA.: Sage Publications; 1993: 95–115.
82.
Leonidaki V. Critical appraisal in the context of integrations of qualitative evidence in applied psychology: the introduction of a new appraisal tool for interview studies. Qual Res Psychol. 2015;12(4):435–52.CrossRef
83.
Critical review form - Qualitative studies (Version 2.0).
84.
Lincoln Y, Guba E. Establishing trustworthiness. In: YLEG, editor. Naturalistic inquiry. Newbury Park, CA: Sage Publications; 1985. p. 289–331.
85.
Long A, Godfrey M, Randall T, Brettle A, Grant M. Developing evidence based social care policy and practic. Part 3: Feasibility of undertaking systematic reviews in social care. In: University of Leeds (Nuffield Institute for Health) and University of Salford (Health Care Practice R&D Unit); 2002.
86.
Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet. 2001;358(9280):483–8.PubMedCrossRef
87.
Manuj I, Pohlen TL. A reviewer's guide to the grounded theory methodology in logistics and supply chain management research. International Journal of Physical Distribution & Logistics Management. 2012;42(8–9):784–803.CrossRef
88.
Marshall C, Rossman GB. Defending the value and logic of qualitative research. In: Designing qualitative research. Newbury Park, CA: Sage Publications; 1989.
89.
90.
91.
Meyrick J. What is good qualitative research? A first step towards a comprehensive approach to judging rigour/quality. J Health Psychol. 2006;11(5):799–808.PubMedCrossRef
92.
Miles MB, Huberman AM: Drawing and verifying conclusions. In: Qualitative data analysis: An expanded sourcebook (2nd ed). edn. Thousand Oaks, CA: Sage Publications; 1997: 277–280.
93.
Morse JM. A review committee's guide for evaluating qualitative proposals. Qual Health Res. 2003;13(6):833–51.PubMedCrossRef
94.
Nelson A. Addressing the threat of evidence-based practice to qualitative inquiry through increasing attention to quality: a discussion paper. Int J Nurs Stud. 2008;45:316–22.PubMedCrossRef
95.
Norena ALP, Alcaraz-Moreno N, Guillermo Rojas J, Rebolledo Malpica D. Applicability of the criteria of rigor and ethics in qualitative research. Aquichan. 2012;12(3):263–74.CrossRef
96.
O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Academic medicine : journal of the Association of American Medical Colleges. 2014;89(9):1245–51.CrossRef
97.
O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. Journal of Health Services Research & Policy. 2008;13(2):92–8.CrossRef
98.
O'HEocha C, Wang X, Conboy K. The use of focus groups in complex and pressurised IS studies and evaluation using Klein & Myers principles for interpretive research. Inf Syst J. 2012;22(3):235–56.CrossRef
99.
Oliver DP. Rigor in Qualitative Research. Research on Aging, 2011;33(4):359–360 352p.CrossRef
100.
Pearson A, Jordan Z, Lockwood C, Aromataris E. Notions of quality and standards for qualitative research reporting. Int J Nurs Pract. 2015;21(5):670–6.PubMedCrossRef
101.
Peters S. Qualitative Research Methods in Mental Health. Evidence Based Mental Health. 2010;13(2):35–40 36p.PubMedCrossRef
102.
Guidelines for Articles. Canadian Family Physician.
103.
Plochg T. Van Zwieten M (eds.): guidelines for quality assurance in health and health care research: qualitative research. Qualitative Research Network AMCUvA: Amsterdam, NL; 2002.
104.
Proposal: A mixed methods appraisal tool for systematic mixed studies reviews.
105.
Poortman CL, Schildkamp K. Alternative quality standards in qualitative research? Quality & Quantity: International Journal of Methodology. 2012;46(6):1727–51.CrossRef
106.
107.
Ravenek MJ, Rudman DL. Bridging conceptions of quality in moments of qualitative research. Int J Qual Methods. 2013;12:436–56.CrossRef
108.
109.
Rocco T. Criteria for evaluating qualitative studies. Human Research Development International. 2010;13(4):375–8.CrossRef
110.
Rogers A, Popay J, Williams G, Latham M: Part II: setting standards for qualitative research: the development of markers. In: Inequalities in health and health promotion: insights from the qualitative research literature edn. London: Health Education Authority; 1997: 35–52.
111.
Rowan M, Huston P. Qualitative research articles: information for authors and peer reviewers. Canadian Meidcal Association Journal. 1997;157(10):1442–6.
112.
Russell CK, Gregory DM. Evaluation of qualitative research studies. Evid Based Nurs. 2003;6(2):36–40.PubMedCrossRef
113.
Ryan F, Coughlan M, Cronin P. Step-by-step guide to critiquing research. Part 2: qualitative research. Br J Nurs. 2007;16(12):738–44.PubMedCrossRef
114.
115.
Sandelowski M, Barroso J. Appraising reports of qualitative studies. In: Handbook for synthesizing qualitative research. New York: Springer; 2007. p. 75–101.
116.
Savall H, Zardet V, Bonnet M, Péron M. The emergence of implicit criteria actualy used by reviewers of qualitative research articles. Organ Res Methods. 2008;11(3):510–40.CrossRef
117.
Schou L, Hostrup H, Lyngso EE, Larsen S, Poulsen I. Validation of a new assessment tool for qualitative research articles. J Adv Nurs. 2012;68(9):2086–94.PubMedCrossRef
118.
Shortell S. The emergence of qualitative methods in health services research. Health Serv Res. 1999;34(5 Pt 2):1083–90.PubMedPubMedCentral
119.
Silverman D, Marvasti A. Quality in Qualitative Research. In: Doing Qualitative Research: A Comprehensive Guide. Thousand Oaks, CA: Sage Publications; 2008. p. 257–76.
120.
Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract. 2012;18(4):746–52.PubMedCrossRef
121.
Spencer L, Ritchie J, Lewis JR, Dillon L. Quality in qualitative evaluation: a framework for assessing research evidence. In. London: Government Chief Social Researcher's Office; 2003.
122.
Stige B, Malterud K, Midtgarden T. Toward an agenda for evaluation of qualitative research. Qual Health Res. 2009;19(10):1504–16.PubMedCrossRef
123.
Stiles W. Evaluating qualitative research. Evidence-Based Mental Health. 1999;4(2):99–101.CrossRef
124.
Storberg-Walker J. Instructor's corner: tips for publishing and reviewing qualitative studies in applied disciplines. Hum Resour Dev Rev. 2012;11(2):254–61.CrossRef
125.
Tracy SJ. Qualitative quality: eight "big-tent" criteria for excellent qualitative research. Qual Inq. 2010;16(10):837–51.CrossRef
126.
Treloar C, Champness S, Simpson PL, Higginbotham N. Critical appraisal checklist for qualitative research studies. Indian J Pediatr. 2000;67(5):347–51.PubMedCrossRef
127.
Waterman H, Tillen D, Dickson R, De Konig K. Action research: a systematic review and guidance for assessment. Health Technol Assess. 2001;5(23):43–50.CrossRef
128.
Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qual Health Res. 2001;11(4):522–37.PubMedCrossRef
129.
Yardley L. Dilemmas in qualitative health research. Psychol Health. 2000;15(2):215–28.CrossRef
130.
Yarris LM, Juve AM, Coates WC, Fisher J, Heitz C, Shayne P, Farrell SE. Critical appraisal of emergency medicine education research: the best publications of 2014. Acad Emerg Med Off J Soc Acad Emerg Med. 2015;22(11):1327–36.CrossRef
131.
Zingg W, Castro-Sanchez E, Secci FV, Edwards R, Drumright LN, Sevdalis N, Holmes AH. Innovative tools for quality assessment: integrated quality criteria for review of multiple study designs (ICROMS). Public Health. 2016;133:19–37.PubMedCrossRef
132.
Zitomer MR, Goodwin D. Gauging the quality of qualitative research in adapted physical activity. Adapt Phys Act Q. 2014;31(3):193–218.
133.
Whiting P, Wolff R, Mallett S, Simera I, Savović J. A proposed framework for developing quality assessment tools. Systematic Reviews. 2017:6(204).
134.
Munthe-Kaas H, Bohren M, Glenton C, Lewin S, Noyes J, Tuncalp Ö, Booth A, Garside R, Colvin C, Wainwright M, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings - paper 3: how to assess methodological limitations. Implementation Science In press.
135.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Steward L, Group. TP-P. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews. 2014:4(1).
136.
Hannes K, Heyvært M, Slegers K, Vandenbrande S, Van Nuland M. Exploring the Potential for a Consolidated Standard for Reporting Guidelines for Qualitative Research: An Argument Delphi Approach. International Journal of Qualitative Methods. 2015;14(4):1–16.CrossRef
137.
Bosch-Caplanch X, Lavis J, Lewin S, Atun R, Røttingen J-A, al. e: Guidance for evidence-informed policies about health systems: Rationale for and challenges of guidance development. PloS Medicine 2012, 9(3):e1001185.PubMedPubMedCentralCrossRef
Metadata
Title
Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool
Authors
Heather Menzies Munthe-Kaas
Claire Glenton
Andrew Booth
Jane Noyes
Simon Lewin
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0728-6

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue

Research article

Understanding health management and safety decisions using signal processing and machine learning

Review

A review of spline function procedures in R

Research article

Sample size calculation for estimating key epidemiological parameters using serological data and mathematical modelling

Research article

Strategies for recruitment in general practice settings: the iSOLVE fall prevention pragmatic cluster randomised controlled trial

Research article

Bias in pharmacoepidemiologic studies using secondary health care databases: a scoping review

Research article

The Chinese version of attitudes towards guidelines scale: validity and reliability assessment