Top

Social Psychiatry and Psychiatric Epidemiology

Published in:

01-04-2009 | ORIGINAL PAPER

Assessing the quality of diagnostic studies using psychometric instruments: applying QUADAS

Authors: Rachel Mann, MSc, Catherine E. Hewitt, PhD, Prof. Simon M. Gilbody, Dphil, MRCPsych

Published in: Social Psychiatry and Psychiatric Epidemiology | Issue 4/2009

Abstract

Background

There has been an increase in the number of systematic reviews of diagnostic tests, which has resulted in the introduction of two checklists: statement for reporting of diagnostic accuracy (STARD) and quality assessment of diagnostic accuracy studies (QUADAS).

Objective

To examine the validity and usefulness of QUADAS when applied to diagnostic accuracy studies using psychometric instruments and to examine the quality in reporting of these studies during practical application of the checklist.

Method

Two reviewers independently rated the quality of 54 studies using QUADAS. The proportion of agreement was used to assess overall agreement and individual agreement of QUADAS items between reviewers.

Results

The overall agreement between the two reviewers for all QUADAS items combined was 85.7%. The proportion of agreement between reviewers for each item ranged from just over 57–100% and was over 80% for 8 of the items. The poorest agreement was associated with the items for selection criteria, indeterminate results and withdrawals. None of the studies adequately reported all relevant information to enable all QUADAS item to be scored as ‘yes’.

Conclusion

Overall QUADAS was relatively easy to use and appears to be an acceptable tool for appraising the quality of diagnostic accuracy studies using psychometric instruments. The application of QUADAS was hampered by the poor quality of reporting encountered.

Abiodun OA (2006) Postnatal depression in primary care populations in Nigeria. Gen Hosp Psychiatry 28:133–136PubMedCrossRef

Adewuya AO, Eegunranti AB, Lawal AM (2005) Prevalence of postnatal depression in Western Nigerian women: a controlled study. Int J Psychiatry Clin Pract 9:60–64CrossRef

Agoub M, Moussaoui D, Battas O (2005) Prevalence of post partum depression in a Moroccan sample. Arch Womens Ment Health 8:37–43PubMedCrossRef

Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang T (2001) The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694PubMed

Areias M, Kumar R, Barros H, Figueiredo E (1996) Comparative incidence of depression in women and men, during pregnancy and after childbirth: Validation of the Edinburgh postnatal depression scale in Portugese mothers. Br J Psychiatry 169:30–35PubMedCrossRef

Aslan E, Kuey L, Kultur S (1997) The validity and reliability study of the Turkish version of the Edinburgh postnatal depression scale. In: Spring Symposium–1 Book. Psychiatric Association of Turkey, Ankara

Austin MP, Hadzi-Pavlovic D, Saint K, Parker G (2005) Antenatal screening for the prediction of postnatal depression: validation of a psychosocial Pregnancy Risk Questionnaire. Acta Psychiatr Scand 112:310–317PubMedCrossRef

Aydin N, Inandi T, Yigit A, Nalan Sahin Hodoglugil N (2004) Validation of the Turkish version of the Edinburgh postnatal depression scale among women within their first postpartum year. Soc Psychiatry Psychiatr Epidemiol 39:483–486PubMedCrossRef

Bagedahl-Strindlund M, Borjesson KM (1998) Postnatal depression: a hidden illness. Acta Psychiatr Scand 98:272–275PubMedCrossRef

10.

Barnett R, Matthey S, Gyaneshwar R (1999) Screening for postnatal depression in women of non-English speaking background. Arch Womens Ment Health 2:67–74CrossRef

11.

Beattie-Clarke P (2003) Validation of two postpartum screening scales in a sample of Saskatchewan First Nations and Metis women. In: Psychology. University of Regina

12.

Beck CT, Gable RK (2001) Comparative analysis of the performance of the postpartum depression screening scale with two other depression instruments. Nurs Res 50:242–249PubMedCrossRef

13.

Beck CT, Gable RK (2001) Further validation of the postpartum depression screening scale. Nurs Res 50:155–164PubMedCrossRef

14.

Beck CT, Gable RK (2005) Screening performance of the postpartum depression screening scale—Spanish version. J Transcult Nurs 16:331–338PubMedCrossRef

15.

Benvenuti P, Ferrara M, Niccolai C, Valoriani V, Cox JL (1999) The Edinburgh postnatal depression scale: validation for an Italian sample. J Affect Disord 53:137–141PubMedCrossRef

16.

Berle JO, Aarre TF, Mykletun A, Dahl AA, Holsten F (2003) Screening for postnatal depression. Validation of the Norwegian version of the Edinburgh postnatal depression scale, and assessment of risk factors for postnatal depression. J Affect Disord 76:151–156PubMedCrossRef

17.

Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Moher D, Rennie D, de Vet HCW, Lijmer JG (2003) The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 138:W1PubMed

18.

Boyce P, Stubbs J, Todd A (1993) The Edinburgh postnatal depression scale: validation for an Australian sample. Aust NZ J Psychiatry 27:472–476CrossRef

19.

Carpiniello B, Pariante CM, Serri F, Costa G, Carta MG (1997) Validation of the Edinburgh postnatal depression scale in Italy. J Psychosom Obstet Gynaecol 18:280–285PubMedCrossRef

20.

Cochrane Diagnostic Test Accuracy Working Group (2008) http://srdta.cochrane.org/en/index.html. Accessed 19 February 2008

21.

Cooper PJ, Murray L, Hooper R, West A (1996) The development and validation of a predictive index for postpartum depression. Psychol Med 26:627–634PubMed

22.

Cox JL, Chapman G, Murray D, Jones P (1996) Validation of the Edinburgh postnatal depression scale (EPDS) in non-postnatal women. J Affect Disord 39:185–189PubMedCrossRef

23.

Cox JL, Holden JM, Sagovsky R (1987) Detection of postnatal depression. Development of the 10-item Edinburgh postnatal depression scale. Br J Psychiatry 150:782–786PubMedCrossRef

24.

Deeks JJ (2001) Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. BMJ 323:157–162PubMedCrossRef

25.

Eberhard-Gran M, Eskild A, Tambs K, Schei B, Opjordsmoen S (2001) The Edinburgh postnatal depression scale: validation in a Norwegian community sample. Nord J Psychiatry 55:113–117PubMedCrossRef

26.

Felice E, Saliba J, Grech V, Cox JL (2006) Validation of the Maltese version of the Edinburgh postnatal depression scale. Arch Womens Ment Health 9:75–80PubMedCrossRef

27.

Garcia-Esteve L, Ascaso C, Ojuel J, Navarro P (2003) Validation of the Edinburgh postnatal depression scale (EPDS) in Spanish mothers. J Affect Disord 75:71–76PubMed

28.

Ghubash R, Abou-Saleh MT, Daradkeh TK (1997) The validity of the Arabic Edinburgh postnatal depression scale. Soc Psychiatry Psychiatr Epidemiol 32:474–476PubMed

29.

Gilbody SM, House AO, Sheldon TA (2001) Routinely administered questionnaires for depression and anxiety: systematic review. BMJ 322:406–409PubMedCrossRef

30.

Guedeney N, Fermanian J (1998) Validation study of the French version of the Edinburgh postnatal depression scale: new results about use and psychometric properties. Eur Psychiatry 13:83–89CrossRefPubMed

31.

Harris B, Huckle P, Thomas R, Johnd S, Fung H (1989) The use of rating scales to identify postnatal depression. Br J Psychiatry 154:813–817PubMedCrossRef

32.

Health Technology Assessment Review The clinical validity, utility and cost effectiveness of methods for the identification of postnatal depression (PND) in primary care: an integrated evidence synthesis and value of information analysis. http://www.ncchta.org/project/1521.asp. (Accessed 19 February 2008)

33.

Holt WJ (1995) The detection of postnatal depression in general practice using the Edinburgh postnatal depression scale. N Z Med J 108:57–59PubMed

34.

Jadresic E, Araya R, Jara C (1995) Validation of the Edinburgh postnatal depression scale (EPDS) in Chilean postpartum women. J Psychosom Obstet Gynaecol 16:187–191PubMedCrossRef

35.

Jardri R, Pelta J, Maron M, Thomas P, Delion P, Codaccioni X, Goudemand M (2006) Predictive validation study of the Edinburgh postnatal depression scale in the first week after delivery and risk analysis for postnatal depression. J Affect Disord 93:169–176PubMedCrossRef

36.

Kitamura T, Shima S, Sugawara M, Toda MA (1994) Temporal variation of validity of self-rating questionnaires: repeated use of the general health questionnaire and Zung’s self-rating depression scale among women during antenatal and postnatal periods. Acta Psychiatr Scand 90:446–450PubMedCrossRef

37.

Lantz CA, Nebenzahl E (1996) Behavior and interpretation of the [kappa] statistic: Resolution of the two paradoxes. J Clin Epidemiol 49:431–434PubMedCrossRef

38.

Lawrie TA, Hofmeyr GJ, de Jager M, Berk M (1998) Validation of the Edinburgh postnatal depression scale on a cohort of South African women. S Afr Med J 88:1340–1344PubMed

39.

Lee DTS, Yip ASK, Chan SSM, Tsui MHY, Wong WS, Chung TKH (2003) Postdelivery screening for postpartum depression. Psychosom Med 65:357–361PubMedCrossRef

40.

Lee DTS, Yip ASK, Chiu HFK, Chung TKH (2000) Screening for postnatal depression using the double-test strategy. Psychosom Med 62:258–263PubMed

41.

Lee DTS, Yip ASK, Chiu HFK, Leung TYS, Chung TKH (2001) Screening for postnatal depression: are specific instruments mandatory? J Affect Disord 63:233–238PubMedCrossRef

42.

Leverton TJ, Elliott SA (2000) Is the EPDS a magic wand?: 1. A comparison of the Edinburgh postnatal depression scale and health visitor report as predictors of diagnosis on the Present State Examination. J Reprod Infant Psychol 18:279–296CrossRef

43.

Matthey S, Barnett B, Kavanagh DJ, Howie P (2001) Validation of the Edinburgh postnatal depression scale for men, and comparison of item endorsement with their partners. J Affect Disord 64:175–184PubMedCrossRef

44.

Milgrom J, Ericksen J, Negri L, Gemmill AW (2005) Screening for postnatal depression in routine primary care: properties of the Edinburgh postnatal depression scale in an Australian sample. Aust NZ J Psychiatry 39:833–839CrossRef

45.

Moher D, Schulz KF, Altman DG (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357(9263):1191–1194PubMedCrossRef

46.

Murray L, Carothers AD (1990) The validation of the Edinburgh Post-natal Depression Scale on a community sample. Br J Psychiatry 157:288–290PubMedCrossRef

47.

Muzik M, Klier CM, Rosenblum KL, Holzinger A, Umek W, Katschnig H (2000) Are the commonly used self-report inventories suitable for screening postpartum depression and anxiety disorders? Acta Psychiatr Scand 102:71–73PubMedCrossRef

48.

Navarro P, Ascaso C, Garcia-Esteve L, Aguado J, Torres A, Martin-Santos R (2007) Postnatal psychiatric morbidity: a validation study of the GHQ–12 and the EPDS as screening tools. Gen Hosp Psychiatry 29:1–7PubMedCrossRef

49.

Ostrowski JA (2006) Accuracy of 3 diagnostic tests for anterior cruciate ligament tears. J Athl Train 41:120–121PubMed

50.

Pitanupong J, Liabsuetrakul T, Vittayanont A (2007) Validation of the Thai Edinburgh postnatal depression scale for screening postpartum depression. Psychiatry Res 149:253–259PubMedCrossRef

51.

Regmi S, Sligl W, Carter D, Grut W, Seear M (2002) A controlled study of postpartum depression among Nepalese women: validation of the Edinburgh Postpartum Depression Scale in Kathmandu. Tropical Med Int Health 7:378–382CrossRef

52.

Rushidi WMWM, Amir A, Iran H, Mahmood NM (2004) Analysis of the psychometric properties of the Malay version of the Beck depression inventory II (BDI-II) among postpartum women in Kedah, North West of Peninsular Malaysia. Malays J Med Sci 11:19–25

53.

Rushidi WMWM, Amir A, Mahmood NM (2003) Revalidation of the Malay version of the Edinburgh Postnatal Depresion Scale (EPDS) among Malay postpartum women attending the Bakar Bata Health Center in Alor Setar, Kedah, North West of Peninsular Malaysia. Malays J Med Sci 10:71–75

54.

Sackett DL, Haynes RB (2002) Evidence base of clinical diagnosis: The architecture of diagnostic research. BMJ 324:539–541PubMedCrossRef

55.

Smidt N, Rutjes AWS, van der Windt D, Ostelo R, Bossuyt PM, Reitsma JB, Bouter LM, de Vet HCW (2006) The quality of diagnostic accuracy studies since the STARD statement: Has it improved? Neurology 67:792PubMedCrossRef

56.

Teng H-W, Hsu C-S, Shih S-M, Lu M-L, Pan J-J, Shen WW (2005) Screening postpartum depression with the Taiwanese version of the Edinburgh postnatal depression scale. Compr Psychiatry 46:261–265PubMedCrossRef

57.

Thompson WM, Harris B, Lazarus J, Richards C (1998) A comparison of the performance of rating scales used in the diagnosis of postnatal depression. Acta Psychiatr Scand 98:224–227PubMedCrossRef

58.

Uwakwe R (2003) Affective (depressive) morbidity in puerperal Nigerian women: validation of the Edinburgh postnatal depression scale. Acta Psychiatr Scand 107:251–259PubMedCrossRef

59.

Vittayanont A, Liabsuetrakul T, Pitanupong J (2006) Development of Postpartum Depression Screening Scale (PDSS): A Thai version for screening postpartum depression. J Med Assoc Thai 89(1):1–7PubMed

60.

Werrett J, Clifford C (2006) Validation of the Punjabi version of the Edinburgh postnatal depression scale (EPDS). Int J Nurs Stud 43:227–236PubMedCrossRef

61.

Whiffen VE (1988) Screening for postpartum depression: a methodological note. J Clin Psychol 44:367–371PubMedCrossRef

62.

Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J (2003) The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 3:25PubMedCrossRef

63.

Whiting P, Rutjes AWS, Dinnes J, Reitsma JB, Bossuyt PMM, Kleijnen J (2005) A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools. J Clin Epidemiol 58:1–12PubMedCrossRef

64.

Whiting PF, Weswood ME, Rutjes AW, Reitsma JB, Bossuyt PN, Kleijnen J (2006) Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies. BMC Med Res Methodol 6

65.

Wickberg B, Hwang CP (1996) The Edinburgh postnatal depression scale: validation on a Swedish community sample. Acta Psychiatr Scand 94:181–184PubMedCrossRef

66.

Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weert HC (2007) Diagnostic accuracy of the mood module of the Patient Health Questionnaire: a systematic review. Gen Hosp Psychiatry 29:388–395PubMedCrossRef

67.

Yamashita H, Yoshida K, Nakano H, Tashiro N (2000) Postnatal depression in Japanese women: detecting the early onset of postnatal depression by closely monitoring the postpartum mood. J Affect Disord 58:145–154PubMedCrossRef

68.

Yoshida K, Marks MN, Kibe N, Kumar R, Nakano H, Tashiro N (1997) Postnatal depression in Japanese women who have given birth in England. J Affect Disord 43:69–77PubMedCrossRef

69.

Zelkowitz P, Milet TH (1995) Screening for post-partum depression in a community sample. Can J Psychiatry 40:80–87PubMed

Title: Assessing the quality of diagnostic studies using psychometric instruments: applying QUADAS
Authors: Rachel Mann, MSc
Catherine E. Hewitt, PhD
Prof. Simon M. Gilbody, Dphil, MRCPsych
Publication date: 01-04-2009
Publisher: Steinkopff-Verlag
Published in: Social Psychiatry and Psychiatric Epidemiology / Issue 4/2009
Print ISSN: 0933-7954
Electronic ISSN: 1433-9285
DOI: https://doi.org/10.1007/s00127-008-0440-z

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Assessing the quality of diagnostic studies using psychometric instruments: applying QUADAS

Abstract

Background

Objective

Method

Results

Conclusion

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Objective

Method

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 4/2009

Effect of socioeconomic status and parents’ education at birth on risk of schizophrenia in offspring

Suicidality in Chinese adolescents in Hong Kong: the role of family and cultural influences

The Tachikawa cohort of motor vehicle accident study investigating psychological distress: design, methods and cohort profiles

The Tachikawa cohort of motor vehicle accident study investigating psychological distress: design, methods and cohort profiles

Perceived helpfulness of websites for mental health information

To seek advice or not to seek advice about the problem: the help-seeking dilemma for obsessive-compulsive disorder