Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2014 | Research article

Influence of data quality on computed Dutch hospital quality indicators: a case study in colorectal cancer surgery

Authors: Kathrin Dentler, Ronald Cornet, Annette ten Teije, Pieter Tanis, Jean Klinkenbijl, Kristien Tytgat, Nicolette de Keizer

Published in: BMC Medical Informatics and Decision Making | Issue 1/2014

Abstract

Background

Our study aims to assess the influence of data quality on computed Dutch hospital quality indicators, and whether colorectal cancer surgery indicators can be computed reliably based on routinely recorded data from an electronic medical record (EMR).

Methods

Cross-sectional study in a department of gastrointestinal oncology in a university hospital, in which a set of 10 indicators is computed (1) based on data abstracted manually for the national quality register Dutch Surgical Colorectal Audit (DSCA) as reference standard and (2) based on routinely collected data from an EMR. All 75 patients for whom data has been submitted to the DSCA for the reporting year 2011 and all 79 patients who underwent a resection of a primary colorectal carcinoma in 2011 according to structured data in the EMR were included. Comparison of results, investigating the causes for any differences based on data quality analysis. Main outcome measures are the computability of quality indicators, absolute percentages of indicator results, data quality in terms of availability in a structured format, completeness and correctness.

Results

All indicators were fully computable based on the DSCA dataset, but only three based on EMR data, two of which were percentages. For both percentages, the difference in proportions computed based on the two datasets was significant.

All required data items were available in a structured format in the DSCA dataset. Their average completeness was 86%, while the average completeness of these items in the EMR was 50%. Their average correctness was 87%.

Conclusions

Our study showed that data quality can significantly influence indicator results, and that our EMR data was not suitable to reliably compute quality indicators. EMRs should be designed in a way so that the data required for audits can be entered directly in a structured and coded format.

Available only for authorised users

Brook RH, McGlynn EA, Cleary PD:Measuring quality of care. N Engl J Med. 1996, 335 (13): 966-970. 10.1056/NEJM199609263351311.CrossRefPubMed

Campbell SM, Braspenning J, Hutchinson A, Marshall M:Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002, 11 (4): 358-364. 10.1136/qhc.11.4.358. [http://qshc.bmj.com/content/11/4/358.full],CrossRefPubMedPubMedCentral

Blumenthal D, Tavenner M:The “Meaningful Use” regulation for electronic health records. N Engl J Med. 2010, 363: 501-504. 10.1056/NEJMp1006114. [http://www.nejm.org/doi/full/10.1056/nejmp1006114],CrossRefPubMed

Kern LM, Malhotra S, Barron Y, Quaresimo J, Dhopeshwarkar R, Pichardo M, Edwards AM, Kaushal R:Accuracy of electronically reported “Meaningful Use” clinical quality measures. Ann Intern Med. 2013, 158: 77-83. 10.7326/0003-4819-158-2-201301150-00001.CrossRefPubMed

Drolet BC, Johnson KB:Categorizing the world of registries. J Biomed Inform. 2008, 41 (6): 1009-1020. 10.1016/j.jbi.2008.01.009. [http://www.ncbi.nlm.nih.gov/pubmed/18329960],CrossRefPubMed

Dentler K, ten Teije A, de Keizer NF, Cornet R:Barriers to the reuse of routinely recorded clinical data: a field report. Studies in Health Technology and Informatics. 2013, [http://www.ncbi.nlm.nih.gov/pubmed/23920567],

Arts DGT, de Keizer NF, Scheffer GJ:Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc. 2002, 9: 600-611. 10.1197/jamia.M1087. [http://jamia.bmj.com/content/9/6/600.short],CrossRefPubMedPubMedCentral

Zichtbare Zorg:Kwaliteitsindicatoren. 2012,

van Gijn W, van de Velde CJH:Improving quality of cancer care through surgical audit. Eur J Surg Oncol (EJSO). 2010, 36 Suppl 1: [http://www.ncbi.nlm.nih.gov/pubmed/20615648],

10.

Dutch Institute for Clinical Auditing: DICA-Rapportages 2011: transparantie, keuzes en verbetering van zorg. 2011, Leiden: DICA, http://www.clinicalaudit.nl/jaarrapportage/archief/DICA\%20Jaarrapportage\%202011.pdf

11.

Dentler K, ten Teije A, Cornet R, de Keizer NF:Towards the automated calculation of clinical quality indicators. Knowledge Representation Health-Care. 2012, LNCS 6924: 51-64.CrossRef

12.

Dentler K:Formalised colorectal cancer surgery quality indicators. 2014, [http://dx.doi.org/10.6084/m9.figshare.913576],

13.

Romano PS, Mull HJ, Rivard PE, Zhao S, Henderson WG, Loveland S, Tsilimingras D, Christiansen CL, Rosen AK:Validity of selected AHRQ patient safety indicators based on VA national surgical quality improvement program data. Health Serv Res. 2009, 44: 182-204. 10.1111/j.1475-6773.2008.00905.x. [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2669628%26tool=pmcentrez%26rendertype=abstract],CrossRefPubMedPubMedCentral

14.

Bland JM, Butland BK:Comparing proportions in overlapping samples. Tech. rep. 2011, [http://www-users.york.ac.uk/~mb55/overlap.pdf],

15.

Weiskopf NG, Weng C:Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2012, 20 (144–151): [http://www.ncbi.nlm.nih.gov/pubmed/22733976],

16.

Persell SD, Wright JM, Thompson JA, Kmetik KS, Baker DW:166. Arch Intern Med. 2006, 20: 2272-[http://archinte.ama-assn.org/cgi/content/abstract/166/20/2272],CrossRef

17.

Baker DW, Persell SD, Thompson JA, Soman N, Burgner KM, Liss D, Kmetik KS:Automated review of electronic health records to assess quality of care for outpatients with heart failure. Ann Intern Med. 2006, 146 (4): 270-277. [http://ukpmc.ac.uk/abstract/MED/17310051],CrossRef

18.

Geissbuhler A, Safran C, Buchan I, Bellazzi R, Labkoff S, Eilenberg K, Leese A, Richardson C, Mantas J, Murray P, De Moor G:Trustworthy reuse of health data: a transnational perspective. Int J Med Inform. 2013, 82 (1): 1-9. 10.1016/j.ijmedinf.2012.11.003. doi:10.1016/j.ijmedinf.2012.11.003,CrossRefPubMed

19.

Weiner JP, Fowles JB, Chan KS:New paradigms for measuring clinical performance using electronic health records. Int J Qual Health Care. 2012, 24 (3): 200-205. 10.1093/intqhc/mzs011.CrossRefPubMed

20.

Fu PC, Rosenthal D, Pevnick JM, Eisenberg F:The impact of emerging standards adoption on automated quality reporting. J Biomed Inform. 2012, 45 (4): 772-781. 10.1016/j.jbi.2012.06.002. [http://www.ncbi.nlm.nih.gov/pubmed/22820003],CrossRefPubMed

21.

Kerr EA, Smith DM, Hogan MM, Krein SL, Pogach L, Hofer TP, Hayward RA:Comparing clinical automated, medical record, and hybrid data sources for diabetes quality measures. J Qual Improv. 2002, 28 (10): 555-565. [http://www.ncbi.nlm.nih.gov/pubmed/12369158] [http://www.ingentaconnect.com/content/jcaho/jcjqs/2002/00000028/00000010/art00003],

22.

Parsons A, McCullough C, Wang J, Shih S:Validity of electronic health record-derived quality measurement for performance monitoring. J Am Med Inform Assoc. 2012, 19 (4): 604-609. 10.1136/amiajnl-2011-000557. [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3384112%26tool=pmcentrez%26rendertype=abstract],CrossRefPubMedPubMedCentral

23.

MacLean CH, Louie R, Shekelle PG, Roth CP, Saliba D, Higashi T, Adams J, Chang JT, Kamberg CJ, Solomon DH, Young RT, Wenger NS:Comparison of administrative data and medical records to measure the quality of medical care provided to vulnerable older patients. Med Care. 2006, 44 (2): 141-148. 10.1097/01.mlr.0000196960.12860.de. [http: //journals.lww.com/lww-medicalcare/Abstract/2006/02000/Comparison_of_Administrative_Data_and_Medical.7.aspx],CrossRefPubMed

24.

Tang PC, Ralston M, Arrigotti MF, Qureshi L, Graham J:Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures. J Am Med Inform Assoc. 2007, 14: 10-15. [[http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2215069%26tool=pmcentrez%26rendertype=abstract]],CrossRefPubMedPubMedCentral

25.

Ancker JS, Shih S, Singh MP, Snyder A:Root causes underlying challenges to secondary use of data. AMIA Symposium Proceedings. 2011, 57-62. [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3243274/],

26.

Brook RH, McGlynn EA, Shekelle PG:Defining and measuring quality of care: a perspective from US researchers. Int J Qual Health Care. 2000, 12 (4): 281-295. 10.1093/intqhc/12.4.281. [http://www.ncbi.nlm.nih.gov/pubmed/10985266],CrossRefPubMed

27.

Powell AE, Davies HTO, Thomson RG:Using routine comparative data to assess the quality of health care: understanding and avoiding common pitfalls. Qual Saf Health Care. 2003, 12 (122–128): [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1743685%26tool=pmcentrez%26rendertype=abstract],

28.

Fowles J, Kind E, Awwad S, Weiner J, Chan K:Performance measures using electronic health records: five case studies. 2008, [http://www.commonwealthfund.org/usr_doc/Briggs_Fowles_perfmeasEHRs5casestudies_1132.pdf],

29.

Roth CP, Lim YW, Pevnick JM, Asch SM, McGlynn EA:The challenge of measuring quality of care from the electronic health record. Am J Med Qual. 2009, 24 (5): 385-394. 10.1177/1062860609336627. [http://www.ncbi.nlm.nih.gov/pubmed/19482968],CrossRefPubMed

30.

Abernethy AP, Herndon JEII, Wheeler JL, Rowe K, Marcello J, Meenal P:Poor documentation prevents adequate assessment of quality metrics in colorectal cancer. J Oncol. 2009, 5 (4): 167-174. [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2795422%26tool=pmcentrez%26rendertype=abstract] [http://jop.ascopubs.org/content/5/4/167.short],

31.

Chan KS, Fowles JB, Weiner JP:Electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev. 2010, 67 (5): 503-527. 10.1177/1077558709359007. [http://www.ncbi.nlm.nih.gov/pubmed/20150441],CrossRefPubMed

32.

Burns EM, Bottle A, Aylin P, Darzi A, Nicholls RJ, Faiz O:Variation in reoperation after colorectal surgery in England as an indicator of surgical performance: retrospective analysis of Hospital Episode Statistics. BMJ. 2011, 343: [http://www.bmj.com/cgi/doi/10.1136/bmj.d4836],

33.

Anema HA, van der Veer SN, Kievit J, Krol-Warmerdam E, Fischer C, Steyerberg E, Dongelmans DA, Reidinga AC, Klazinga NS, de Keizer NF:Influences of definition ambiguity on hospital performance indicator scores: examples from The Netherlands. Eur J Public Health. 2013, Dmv: 1-6. [http://www.ncbi.nlm.nih.gov/pubmed/23543677],

34.

Warsi AA, White S, McCulloch P:Completeness of data entry in three cancer surgery databases. Eur J Surg Oncol (EJSO). 2002, 28 (8): 850-856. 10.1053/ejso.2002.1283. [http://linkinghub.elsevier.com/retrieve/pii/S0748798302912837],CrossRef

35.

Botsis T, Hartvigsen G, Chen F, Weng C:Secondary use of EHR: data quality issues and informatics opportunities. AMIA Summits Transl Sci. 2010, 2010: 1-5. [http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3041534%26tool=pmcentrez%26rendertype=abstract],

36.

Hripcsak G, Albers DJ:Next-generation phenotyping of electronic health records. J Am Med Inform Assoc. 2013, 20 (1): 117-121. 10.1136/amiajnl-2012-001145. doi:10.1136/amiajnl-2012-001145,CrossRefPubMed

37.

Edhemovic I, Temple WJ, de Gara CJ, Stuart GCE:The computer synoptic operative report—leap forward in the science of surgery. Ann Surg Oncol. 2004, 11 (10): 941-947. 10.1245/ASO.2004.12.045. [http://www.ncbi.nlm.nih.gov/pubmed/15466354] [http://www.springerlink.com/index/CG6A4TQ38T54LP7L.pdf],CrossRefPubMed

38.

Mack LA, Bathe OF, Hebert MA, Tamano E, Buie WD, Fields T, Temple WJ:Opening the black box of cancer surgery quality: WebSMR and the Alberta experience. J Surg Oncol. 2009, 99 (9): 525-530. [http://www.ncbi.nlm.nih.gov/pubmed/19338026],CrossRefPubMed

39.

Park J, Pillarisetty VG, Brennan MF, Jarnagin WR, D’Angelica MI, Dematteo RP, Coit DG, Janakos M, Allen PJ:Electronic synoptic operative reporting: assessing the reliability and completeness of synoptic reports for pancreatic resection. J Am Coll Surg. 2010, 211 (3): 308-315. 10.1016/j.jamcollsurg.2010.05.008. [http://www.ncbi.nlm.nih.gov/pubmed/20800186],CrossRefPubMed

40.

Cornet R, Van Eldik A, De Keizer N:Inventory of tools for dutch clinical language processing. Stud Health Technol Inform. 2012, 180: 245-PubMed

41.

Wyatt J:Acquisition and use of clinical data for audit and research. J Eval Clin Pract. 1995, 1: 15-27. 10.1111/j.1365-2753.1995.tb00004.x. [http://www.ncbi.nlm.nih.gov/pubmed/9238554],CrossRefPubMed

42.

Cimino JJ:Collect once, use many: enabling the reuse of clinical data through controlled terminologies. J AHIMA. 2007, 78 (2): 24-29.PubMed

The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/14/32/prepub

Title: Influence of data quality on computed Dutch hospital quality indicators: a case study in colorectal cancer surgery
Authors: Kathrin Dentler
Ronald Cornet
Annette ten Teije
Pieter Tanis
Jean Klinkenbijl
Kristien Tytgat
Nicolette de Keizer
Publication date: 01-12-2014
Publisher: BioMed Central
Published in: BMC Medical Informatics and Decision Making / Issue 1/2014
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/1472-6947-14-32

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Influence of data quality on computed Dutch hospital quality indicators: a case study in colorectal cancer surgery

Abstract

Background

Methods

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2014

Designing privacy-friendly digital whiteboards for mediation of clinical progress

Variability of the impact of adverse events on physicians’ decision making

The challenges of real-world implementation of web-based shared care software: the HopSCOTCH Shared-Care Obesity Trial in Children

Optimal strategy for linkage of datasets containing a statistical linkage key and datasets with full personal identifiers

Manual and automated methods for identifying potentially preventable readmissions: a comparison in a large healthcare system

Lead users’ ideas on core features to support physical activity in rheumatoid arthritis: a first step in the development of an internet service using participatory design