Skip to main content
Top
Published in: Clinical Oral Investigations 3/2012

01-06-2012 | Original Article

A new instrument for assessing the quality of studies on prevalence

Authors: Nikolaos Nikitas Giannakopoulos, Peter Rammelsberg, Lydia Eberhard, Marc Schmitter

Published in: Clinical Oral Investigations | Issue 3/2012

Login to get access

Abstract

There are numerous scientific articles of studies on the prevalence of disorders with non-standardised examination and diagnostic protocols. Because their quality is heterogeneous, a new instrument has been developed for the assessment of such studies. The new instrument is based mainly on statistical criteria. The points assigned for each of the main criteria according to the information gained from each paper are summed up to form a Total Quality Score (TQS). The interrater reliability of the instrument was tested by employing Kappa and Interrater Correlation Coefficient (ICC) statistics. The latter was assessed on the results of three independent investigators. The new quality instrument appeared to be easy to use, and the instructions were comprehensible. The ICC(2,1) for the TQS ranged between 0.94 and 1.00 indicating almost perfect agreement between the investigators. The reliability of the new instrument enables its use for scientific review purposes. In this way, its validity will also be tested. The instrument could be adopted for assessment of scientific articles of studies on the prevalence of disorders in many, similar, scientific areas.
Literature
1.
go back to reference Gross AJ, Rivera-Morales WC, Gale EN (1988) A prevalence study of symptoms associated with TM disorders. J Craniomandib Disord 2:191–195PubMed Gross AJ, Rivera-Morales WC, Gale EN (1988) A prevalence study of symptoms associated with TM disorders. J Craniomandib Disord 2:191–195PubMed
2.
go back to reference Cherkin DC, Deyo RA, Wheeler K, Ciol MA (1994) Physician variation in diagnostic testing for low back pain. Who you see is what you get. Arthritis Rheum 37:15–22PubMedCrossRef Cherkin DC, Deyo RA, Wheeler K, Ciol MA (1994) Physician variation in diagnostic testing for low back pain. Who you see is what you get. Arthritis Rheum 37:15–22PubMedCrossRef
3.
go back to reference Koes BW, van Tulder MW, Ostelo R, Kim Burton A, Waddell G (2001) Clinical guidelines for the management of low back pain in primary care: an international comparison. Spine (Phila Pa 1976) 26:2504–2513, Discussion, 2513–2504CrossRef Koes BW, van Tulder MW, Ostelo R, Kim Burton A, Waddell G (2001) Clinical guidelines for the management of low back pain in primary care: an international comparison. Spine (Phila Pa 1976) 26:2504–2513, Discussion, 2513–2504CrossRef
4.
go back to reference Bamji AN, Erhardt CC, Price TR, Williams PL (1996) The painful shoulder: can consultants agree? Br J Rheumatol 35:1172–1174PubMedCrossRef Bamji AN, Erhardt CC, Price TR, Williams PL (1996) The painful shoulder: can consultants agree? Br J Rheumatol 35:1172–1174PubMedCrossRef
5.
go back to reference Luime JJ, Koes BW, Hendriksen IJ, Burdorf A, Verhagen AP, Miedema HS, Verhaar JA (2004) Prevalence and incidence of shoulder pain in the general population; a systematic review. Scand J Rheumatol 33:73–81PubMedCrossRef Luime JJ, Koes BW, Hendriksen IJ, Burdorf A, Verhagen AP, Miedema HS, Verhaar JA (2004) Prevalence and incidence of shoulder pain in the general population; a systematic review. Scand J Rheumatol 33:73–81PubMedCrossRef
6.
go back to reference Egger M, Smith G, Altman D (2009) Systematic reviews in health care: meta-analysis in context, 2nd edn. BMJ Books, London Egger M, Smith G, Altman D (2009) Systematic reviews in health care: meta-analysis in context, 2nd edn. BMJ Books, London
7.
go back to reference von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP (2007) The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370:1453–1457CrossRef von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP (2007) The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370:1453–1457CrossRef
8.
go back to reference Shamliyan T, Kane RL, Dickinson S (2010) A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol 63:1061–1070PubMedCrossRef Shamliyan T, Kane RL, Dickinson S (2010) A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol 63:1061–1070PubMedCrossRef
9.
go back to reference Boyle MH (1998) Guidelines for evaluating prevalence studies. Evid Based Mental Health 1:37–39CrossRef Boyle MH (1998) Guidelines for evaluating prevalence studies. Evid Based Mental Health 1:37–39CrossRef
10.
go back to reference Beller EM, Gebski V, Keech AC (2002) Randomisation in clinical trials. Med J Aust 177:565–567PubMed Beller EM, Gebski V, Keech AC (2002) Randomisation in clinical trials. Med J Aust 177:565–567PubMed
11.
go back to reference Krogh-Poulsen WG (1969) Management of the occlusion of the teeth. Examination, diagnosis, treatment. In: Chayes LSCM (ed) Facial pain and mandibular dysfunction. Saunders, Philadelphia, pp 251–258 Krogh-Poulsen WG (1969) Management of the occlusion of the teeth. Examination, diagnosis, treatment. In: Chayes LSCM (ed) Facial pain and mandibular dysfunction. Saunders, Philadelphia, pp 251–258
12.
go back to reference Helkimo M (1974) Studies on function and dysfunction of the masticatory system. Ii. Index for anamnestic and clinical dysfunction and occlusal state. Sven Tandläk Tidskr 67:101–121PubMed Helkimo M (1974) Studies on function and dysfunction of the masticatory system. Ii. Index for anamnestic and clinical dysfunction and occlusal state. Sven Tandläk Tidskr 67:101–121PubMed
13.
go back to reference Levitt SR, Lundeen TF, McKinney MW (1988) Initial studies of a new assessment method for temporomandibular joint disorders. J Prosthet Dent 59:490–495PubMedCrossRef Levitt SR, Lundeen TF, McKinney MW (1988) Initial studies of a new assessment method for temporomandibular joint disorders. J Prosthet Dent 59:490–495PubMedCrossRef
14.
go back to reference Levitt SR, McKinney MW, Lundeen TF (1988) The TMJ scale: cross-validation and reliability studies. Cranio 6:17–25PubMed Levitt SR, McKinney MW, Lundeen TF (1988) The TMJ scale: cross-validation and reliability studies. Cranio 6:17–25PubMed
15.
go back to reference Lundeen TF, Levitt SR, McKinney MW (1986) Discriminative ability of the TMJ scale: age and gender differences. J Prosthet Dent 56:84–92PubMedCrossRef Lundeen TF, Levitt SR, McKinney MW (1986) Discriminative ability of the TMJ scale: age and gender differences. J Prosthet Dent 56:84–92PubMedCrossRef
16.
17.
18.
go back to reference McNeill C (1993) Temporomandibular disorders-guidelines for classification, assessment and management, 2nd edn. Quintessence Publishing, Chicago McNeill C (1993) Temporomandibular disorders-guidelines for classification, assessment and management, 2nd edn. Quintessence Publishing, Chicago
19.
go back to reference Okeson J (1996) Orofacial pain: guidelines for assessment, diagnosis, and management. Quintessence Publishing, Chicago Okeson J (1996) Orofacial pain: guidelines for assessment, diagnosis, and management. Quintessence Publishing, Chicago
20.
go back to reference Dworkin SF, LeResche L (1992) Research diagnostic criteria for temporomandibular disorders: review, criteria, examinations and specifications, critique. J Craniomandib Disord 6:301–355PubMed Dworkin SF, LeResche L (1992) Research diagnostic criteria for temporomandibular disorders: review, criteria, examinations and specifications, critique. J Craniomandib Disord 6:301–355PubMed
21.
go back to reference Wahlund K, List T, Dworkin SF (1998) Temporomandibular disorders in children and adolescents: reliability of a questionnaire, clinical examination, and diagnosis. J Orofac Pain 12:42–51PubMed Wahlund K, List T, Dworkin SF (1998) Temporomandibular disorders in children and adolescents: reliability of a questionnaire, clinical examination, and diagnosis. J Orofac Pain 12:42–51PubMed
22.
go back to reference Schmitter M, Kress B, Leckel M, Henschel V, Ohlmann B, Rammelsberg P (2008) Validity of temporomandibular disorder examination procedures for assessment of temporomandibular joint status. Am J Orthod Dentofacial Orthop 133:796–803PubMedCrossRef Schmitter M, Kress B, Leckel M, Henschel V, Ohlmann B, Rammelsberg P (2008) Validity of temporomandibular disorder examination procedures for assessment of temporomandibular joint status. Am J Orthod Dentofacial Orthop 133:796–803PubMedCrossRef
23.
go back to reference Dworkin SF, Sherman J, Mancl L, Ohrbach R, LeResche L, Truelove E (2002) Reliability, validity, and clinical utility of the research diagnostic criteria for Temporomandibular Disorders Axis II Scales: depression, non-specific physical symptoms, and graded chronic pain. J Orofac Pain 16:207–220PubMed Dworkin SF, Sherman J, Mancl L, Ohrbach R, LeResche L, Truelove E (2002) Reliability, validity, and clinical utility of the research diagnostic criteria for Temporomandibular Disorders Axis II Scales: depression, non-specific physical symptoms, and graded chronic pain. J Orofac Pain 16:207–220PubMed
24.
go back to reference John MT, Dworkin SF, Mancl LA (2005) Reliability of clinical temporomandibular disorder diagnoses. Pain 118:61–69PubMedCrossRef John MT, Dworkin SF, Mancl LA (2005) Reliability of clinical temporomandibular disorder diagnoses. Pain 118:61–69PubMedCrossRef
25.
go back to reference Schmitter M, Ohlmann B, John MT, Hirsch C, Rammelsberg P (2005) Research diagnostic criteria for temporomandibular disorders: a calibration and reliability study. Cranio 23:212–218PubMed Schmitter M, Ohlmann B, John MT, Hirsch C, Rammelsberg P (2005) Research diagnostic criteria for temporomandibular disorders: a calibration and reliability study. Cranio 23:212–218PubMed
26.
go back to reference List T, John MT, Dworkin SF, Svensson P (2006) Recalibration improves inter-examiner reliability of tmd examination. Acta Odontol Scand 64:146–152PubMedCrossRef List T, John MT, Dworkin SF, Svensson P (2006) Recalibration improves inter-examiner reliability of tmd examination. Acta Odontol Scand 64:146–152PubMedCrossRef
27.
go back to reference Cohen J (1960) A coeffiecient of agreement for nominal scales. Educ Psychol Meas 20:37–46CrossRef Cohen J (1960) A coeffiecient of agreement for nominal scales. Educ Psychol Meas 20:37–46CrossRef
28.
go back to reference Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428PubMedCrossRef Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428PubMedCrossRef
29.
go back to reference Aggarwal VR, Macfarlane TV, Macfarlane GJ (2003) Why is pain more common amongst people living in areas of low socio-economic status? A population-based cross-sectional study. Br Dent J 194:383–387, Discussion, 380PubMedCrossRef Aggarwal VR, Macfarlane TV, Macfarlane GJ (2003) Why is pain more common amongst people living in areas of low socio-economic status? A population-based cross-sectional study. Br Dent J 194:383–387, Discussion, 380PubMedCrossRef
30.
go back to reference Dworkin SF, Huggins KH, LeResche L, Von Korff M, Howard J, Truelove E, Sommers E (1990) Epidemiology of signs and symptoms in temporomandibular disorders: clinical signs in cases and controls. J Am Dent Assoc 120:273–281PubMed Dworkin SF, Huggins KH, LeResche L, Von Korff M, Howard J, Truelove E, Sommers E (1990) Epidemiology of signs and symptoms in temporomandibular disorders: clinical signs in cases and controls. J Am Dent Assoc 120:273–281PubMed
31.
go back to reference Heikinheimo K, Salmi K, Myllarniemi S, Kirveskari P (1989) Symptoms of craniomandibular disorder in a sample of Finnish adolescents at the ages of 12 and 15 years. Eur J Orthod 11:325–331PubMed Heikinheimo K, Salmi K, Myllarniemi S, Kirveskari P (1989) Symptoms of craniomandibular disorder in a sample of Finnish adolescents at the ages of 12 and 15 years. Eur J Orthod 11:325–331PubMed
32.
33.
go back to reference Nilsson IM, List T, Drangsholt M (2005) Prevalence of temporomandibular pain and subsequent dental treatment in Swedish adolescents. J Orofac Pain 19:144–150PubMed Nilsson IM, List T, Drangsholt M (2005) Prevalence of temporomandibular pain and subsequent dental treatment in Swedish adolescents. J Orofac Pain 19:144–150PubMed
34.
go back to reference Pedroni CR, De Oliveira AS, Guaratini MI (2003) Prevalence study of signs and symptoms of temporomandibular disorders in university students. J Oral Rehabil 30:283–289PubMedCrossRef Pedroni CR, De Oliveira AS, Guaratini MI (2003) Prevalence study of signs and symptoms of temporomandibular disorders in university students. J Oral Rehabil 30:283–289PubMedCrossRef
35.
go back to reference Rantala MA, Ahlberg J, Suvinen TI, Savolainen A, Kononen M (2004) Chronic myofascial pain, disk displacement with reduction and psychosocial factors in finnish non-patients. Acta Odontol Scand 62:293–297PubMedCrossRef Rantala MA, Ahlberg J, Suvinen TI, Savolainen A, Kononen M (2004) Chronic myofascial pain, disk displacement with reduction and psychosocial factors in finnish non-patients. Acta Odontol Scand 62:293–297PubMedCrossRef
36.
go back to reference Raustia AM, Peltola M, Salonen MA (1997) Influence of complete denture renewal on craniomandibular disorders: a 1-year follow-up study. J Oral Rehabil 24:30–36PubMedCrossRef Raustia AM, Peltola M, Salonen MA (1997) Influence of complete denture renewal on craniomandibular disorders: a 1-year follow-up study. J Oral Rehabil 24:30–36PubMedCrossRef
37.
go back to reference Sadowsky C, Muhl ZF, Sakols EI, Sommerville JM (1985) Temporomandibular joint sounds related to orthodontic therapy. J Dent Res 64:1392–1395PubMedCrossRef Sadowsky C, Muhl ZF, Sakols EI, Sommerville JM (1985) Temporomandibular joint sounds related to orthodontic therapy. J Dent Res 64:1392–1395PubMedCrossRef
38.
go back to reference Tallents RH, Hatala M, Katzberg RW, Westesson PL (1993) Temporomandibular joint sounds in asymptomatic volunteers. J Prosthet Dent 69:298–304PubMedCrossRef Tallents RH, Hatala M, Katzberg RW, Westesson PL (1993) Temporomandibular joint sounds in asymptomatic volunteers. J Prosthet Dent 69:298–304PubMedCrossRef
39.
go back to reference Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174PubMedCrossRef Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174PubMedCrossRef
40.
go back to reference Schouten H (1986) Nominal scale agreement among observers. Psychometrika 51:453–466CrossRef Schouten H (1986) Nominal scale agreement among observers. Psychometrika 51:453–466CrossRef
41.
42.
go back to reference Downs SH, Black N (1998) The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 52:377–384PubMedCrossRef Downs SH, Black N (1998) The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 52:377–384PubMedCrossRef
43.
go back to reference Macfarlane TV, Glenny AM, Worthington HV (2001) Systematic review of population-based epidemiological studies of oro-facial pain. J Dent 29:451–467PubMedCrossRef Macfarlane TV, Glenny AM, Worthington HV (2001) Systematic review of population-based epidemiological studies of oro-facial pain. J Dent 29:451–467PubMedCrossRef
44.
go back to reference Genaidy A, Lemasters GK, Lockey J, Succop P, Deddens J, Sobeih T, Dunning K (2007) An epidemiological appraisal instrument - a tool for evaluation of epidemiological studies. Ergonomics 50:920–960PubMedCrossRef Genaidy A, Lemasters GK, Lockey J, Succop P, Deddens J, Sobeih T, Dunning K (2007) An epidemiological appraisal instrument - a tool for evaluation of epidemiological studies. Ergonomics 50:920–960PubMedCrossRef
45.
46.
go back to reference Berlin J, Rennie D (1999) Measuring the quality of trials: the quality of quality scales. JAMA 282:1083–1085PubMedCrossRef Berlin J, Rennie D (1999) Measuring the quality of trials: the quality of quality scales. JAMA 282:1083–1085PubMedCrossRef
47.
go back to reference Greenland S (1994) Quality scores are useless and potentially misleading. Am J Epidemiol 140:300–302 Greenland S (1994) Quality scores are useless and potentially misleading. Am J Epidemiol 140:300–302
48.
go back to reference Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613PubMedCrossRef Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613PubMedCrossRef
49.
go back to reference Whiting P, Harbord R, Kleijnen J (2005) No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Med Res Methodol 5:19–27PubMedCrossRef Whiting P, Harbord R, Kleijnen J (2005) No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Med Res Methodol 5:19–27PubMedCrossRef
50.
go back to reference Tooth L, Ware R, Bain C, Purdie DM, Dobson A (2005) Quality of reporting of observational longitudinal research. Am J Epidemiol 161:280–288PubMedCrossRef Tooth L, Ware R, Bain C, Purdie DM, Dobson A (2005) Quality of reporting of observational longitudinal research. Am J Epidemiol 161:280–288PubMedCrossRef
51.
go back to reference Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic B (2004) Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the radiation therapy oncology group. BMJ 328:22–24PubMedCrossRef Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic B (2004) Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the radiation therapy oncology group. BMJ 328:22–24PubMedCrossRef
52.
go back to reference Nguyen Q, Bezemer P, Habets L, Prahl-Andersen B (1999) A systematic review of the relationship between overjet size and traumatic dental injuries. Eur J Orthod 21:503–515PubMedCrossRef Nguyen Q, Bezemer P, Habets L, Prahl-Andersen B (1999) A systematic review of the relationship between overjet size and traumatic dental injuries. Eur J Orthod 21:503–515PubMedCrossRef
53.
go back to reference Ariëns G, van Mechelen W, Bongers P, Bouter L, van der Wal G (2000) Physical risk factors for neck pain. Scand J Work Environ Health 26:7–19PubMedCrossRef Ariëns G, van Mechelen W, Bongers P, Bouter L, van der Wal G (2000) Physical risk factors for neck pain. Scand J Work Environ Health 26:7–19PubMedCrossRef
54.
go back to reference Loney P, Chambers LW, Benett KJ, Roberts JG, Stratford PW (1998) Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chronic Dis Can 19:170–176PubMed Loney P, Chambers LW, Benett KJ, Roberts JG, Stratford PW (1998) Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chronic Dis Can 19:170–176PubMed
55.
go back to reference Woodbury M, Houghton PE (2004) Prevalence of pressure ulcers in Canadian healthcare settings. Ostomy/Wound Manage 50:22–38 Woodbury M, Houghton PE (2004) Prevalence of pressure ulcers in Canadian healthcare settings. Ostomy/Wound Manage 50:22–38
56.
go back to reference Silva LC, Ordunez P, Paz Rodriguez M, Robles S (2001) A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension. Rev Panam Salud Pública 10:152–160PubMedCrossRef Silva LC, Ordunez P, Paz Rodriguez M, Robles S (2001) A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension. Rev Panam Salud Pública 10:152–160PubMedCrossRef
57.
go back to reference Barclay P, Hollender LG, Maravilla KR, Truelove EL (1999) Comparison of clinical and magnetic resonance imaging diagnosis in patients with disk displacement in the temporomandibular joint. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 88:37–43PubMedCrossRef Barclay P, Hollender LG, Maravilla KR, Truelove EL (1999) Comparison of clinical and magnetic resonance imaging diagnosis in patients with disk displacement in the temporomandibular joint. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 88:37–43PubMedCrossRef
58.
go back to reference Schmitter M, Kress B, Rammelsberg P (2004) Temporomandibular joint pathosis in patients with myofascial pain: a comparative analysis of magnetic resonance imaging and a clinical examination based on a specific set of criteria. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 97:318–324PubMedCrossRef Schmitter M, Kress B, Rammelsberg P (2004) Temporomandibular joint pathosis in patients with myofascial pain: a comparative analysis of magnetic resonance imaging and a clinical examination based on a specific set of criteria. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 97:318–324PubMedCrossRef
59.
go back to reference Emshoff R, Rudisch A (2001) Validity of clinical diagnostic criteria for temporomandibular disorders: clinical versus magnetic resonance imaging diagnosis of temporomandibular joint internal derangement and osteoarthrosis. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 91:50–55PubMedCrossRef Emshoff R, Rudisch A (2001) Validity of clinical diagnostic criteria for temporomandibular disorders: clinical versus magnetic resonance imaging diagnosis of temporomandibular joint internal derangement and osteoarthrosis. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 91:50–55PubMedCrossRef
Metadata
Title
A new instrument for assessing the quality of studies on prevalence
Authors
Nikolaos Nikitas Giannakopoulos
Peter Rammelsberg
Lydia Eberhard
Marc Schmitter
Publication date
01-06-2012
Publisher
Springer-Verlag
Published in
Clinical Oral Investigations / Issue 3/2012
Print ISSN: 1432-6981
Electronic ISSN: 1436-3771
DOI
https://doi.org/10.1007/s00784-011-0557-4

Other articles of this Issue 3/2012

Clinical Oral Investigations 3/2012 Go to the issue