Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Health Services Research
Published in: BMC Health Services Research 1/2012

Open Access 01-12-2012 | Research article

Measuring data reliability for preventive services in electronic medical records

Authors: Michelle Greiver, Jan Barnsley, Richard H Glazier, Bart J Harvey, Rahim Moineddin

Published in: BMC Health Services Research | Issue 1/2012

Login to get access

Abstract

Background

Improvements in the quality of health care services are often measured using data present in medical records. Electronic Medical Records (EMRs) contain potentially valuable new sources of health data. However, data quality in EMRs may not be optimal and should be assessed. Data reliability (are the same data elements being measured over time?) is a prerequisite for data validity (are the data accurate?). Our objective was to measure the reliability of data for preventive services in primary care EMRs during the transition to EMR.

Methods

Our data sources were randomly selected eligible patients’ medical records and data obtained from provincial administrative datasets. Eighteen community-based family physicians in Toronto, Ontario that implemented EMRs starting in 2006 participated in this study. We measured the proportion of patients eligible for a service (Pap smear, screening mammogram or influenza vaccination) that received the service. We compared the change in rates of selected preventive services calculated from the medical record audits with the change in administrative datasets.

Results

In the first year of EMR use (2006) services decreased by 8.7% more (95% CI −11.0%– − 6.4%, p < 0.0001) when measured through medical record audits as compared with administrative datasets. Services increased by 2.4% more (95% CI 0%–4.9%, p = 0.05) in the medical record audits during the second year of EMR use (2007).

Conclusion

There were differences between the change measured through medical record audits and administrative datasets. Problems could include difficulties with organizing new data entry processes as well as continued use of both paper and EMRs. Data extracted from EMRs had limited reliability during the initial phase of EMR implementation. Unreliable data interferes with the ability to measure and improve health care quality
Appendix
Available only for authorised users
Literature
1.
Newcomer LN: Perspective: physician, measure thyself. Health Aff. 1998, 17 (4): 32-35. 10.1377/hlthaff.17.4.32.CrossRef
2.
Romanow R: Building on values: the future of health care in Canada. 2002, National Library of Canada, Ottawa
3.
Have paper records passed their expiry date?. CMAJ. 2005, 173 (7): 725.
4.
CIHI CPCSSN data analysis report: cycle 2A. 2010, Canadian Institutes for Health Information, Toronto
5.
6.
Birtwhistle R, Keshavjee K, Lambert-Lanning A, Godwin M, Greiver M, Manca D, Lagace C: Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med. 2009, 22 (4): 412-422. 10.3122/jabfm.2009.04.090081.CrossRefPubMed
7.
Brouwer H, Bindels P, Weert H: Data quality improvement in general practice. Fam Pract. 2006, 23 (5): 529-536. 10.1093/fampra/cml040.CrossRefPubMed
8.
Whitelaw FG, Nevin SL, Milne RM, Taylor RJ, Taylor MW, Watt AH: Completeness and accuracy of morbidity and repeat prescribing records held on general practice computers in Scotland. Br J Gen Pract. 1996, 46 (404): 181-186.PubMedPubMedCentral
9.
Selak V, Wells S, Whittaker R, Stewart A: Smoking status recording in GP electronic records: the unrealised potential. Inform Prim Care. 2006, 14 (4): 235-241. discussion 242–235PubMed
10.
Mant J, Murphy M, Rose P, Vessey M: The accuracy of general practitioner records of smoking and alcohol use: comparison with patient questionnaires. J Pub Health Med. 2000, 22 (2): 198-201. 10.1093/pubmed/22.2.198.CrossRef
11.
Hollowell J: The General Practice Research Database: quality of morbidity data. Popul Trends. 1997, 87: 36-40.PubMed
12.
Mukhi S, Keshavjee K: Developing a primary care electronic medical record chronic disease surveillance system in Canada: data quality and lessons learned. 2009, In Canadian Association of Health Services and Policy Research, Calgary
13.
Birtwhistle R, Keshavjee K, Martin K: Improving data quality in EMRs for chronic disease management. 2009, Lambert-Lanning A, In Family Medicine Forum. Calgary
14.
Chan KS, Fowles JB, Weiner JP: Review: electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev. 2010, 67 (5): 503-527. 10.1177/1077558709359007.CrossRefPubMed
15.
Thiru K, Hassey A, Sullivan F: Systematic review of scope and quality of electronic patient record data in primary care. BMJ. 2003, 326 (7398): 1070-10.1136/bmj.326.7398.1070.CrossRefPubMedPubMedCentral
16.
Dean BB, Lam J, Natoli JL, Butler Q, Aguilar D, Nordyke RJ: Review: use of electronic medical records for health outcomes research: a literature review. Med Care Res Rev. 2009, 66 (6): 611-638. 10.1177/1077558709332440.CrossRefPubMed
17.
Benin AL, Vitkauskas G, Thornquist E, Shapiro ED, Concato J, Aslan M, Krumholz HM: Validity of using an electronic medical record for assessing quality of care in an outpatient setting. Med Care. 2005, 43 (7): 691-698. 10.1097/01.mlr.0000167185.26058.8e.CrossRefPubMed
18.
Tang PC, LaRosa MP, Gorden SM: Use of computer-based records, completeness of documentation, and appropriateness of documented clinical decisions. J Am Med Inform Assoc. 1999, 6 (3): 245-251. 10.1136/jamia.1999.0060245.CrossRefPubMedPubMedCentral
19.
Jordan K, Porcheret M, Croft P: Quality of morbidity coding in general practice computerized medical records: a systematic review. Fam Pract. 2004, 21 (4): 396-412. 10.1093/fampra/cmh409.CrossRefPubMed
20.
Majeed A, Car J, Sheikh A: Accuracy and completeness of electronic patient records in primary care. Fam Pract. 2008, 25 (4): 213-214. 10.1093/fampra/cmn047.CrossRefPubMed
21.
Jick H, Jick SS, Derby LE: Validation of information recorded on general practitioner based computerised data resource in the United Kingdom. BMJ. 1991, 302 (6779): 766-768. 10.1136/bmj.302.6779.766.CrossRefPubMedPubMedCentral
22.
Porcheret M, Hughes R, Evans D, Jordan K, Whitehurst T, Ogden H, Croft P, North Staffordshire General Practice Research N: Data quality of general practice electronic health records: the impact of a program of assessments, feedback, and training. J Am Med Inform Assoc. 2004, 11 (1): 78-86.CrossRefPubMedPubMedCentral
23.
Greiver M, Barnsley J, Glazier R, Harvey B, Moineddin R: Effect of electronic medical record implementation on the provision of preventive services in a pay for performance environment. Canadian Family Physician. 2011, 57: e390-e397.PubMedPubMedCentral
24.
General blended payment template. Ministry of Health and Long Term Care of Ontario and the Ontario Family Health Network Corporation. 2002, .
25.
Memorandum of agreement between the Ontario Medical Association and the Ontario Ministry of Health and Long Term Care. 2004, Government of Ontario, Toronto
26.
Memorandum of agreement between the Ontario Medical Association and the Ontario Ministry of Health and Long Term Care. 2003, Government of Ontario, Toronto
27.
Tu K, Campbell NR, Chen ZL, Cauch-Dudek KJ, McAlister FA: Accuracy of administrative databases in identifying patients with hypertension. Open Med. 2007, 1 (1): e18-e26.PubMedPubMedCentral
28.
Hux JE, Ivis F, Flintoft V, Bica A: Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002, 25 (3): 512-516. 10.2337/diacare.25.3.512.CrossRefPubMed
29.
Greiver M, Kasperski J, Barnsley J, Rachlis V: Measuring quality improvements in preventive care services in the first two family health networks in the greater Toronto area. 2006, Ontario College of Family Physicians, Toronto
30.
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW: Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007, 356 (5): 486-496. 10.1056/NEJMsa064964.CrossRefPubMed
31.
Epi Info. 2005, Centers for Disease Control and Prevention, Atlanta, 332
32.
Glazier RH, Klein-Geltink J, Kopp A, Sibley LM: Capitation and enhanced fee-for-service models for primary care reform: a population-based evaluation. CMAJ. 2009, 180 (11): E72-E81. 10.1503/cmaj.081316.CrossRefPubMedPubMedCentral
33.
Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T: Identifying patients with physician-diagnosed asthma in health administrative databases. Can Respir J. 2009, 16 (6): 183-188.CrossRefPubMedPubMedCentral
34.
Ko DT, Alter DA, Austin PC, You JJ, Lee DS, Qiu F, Stukel TA, Tu JV: Life expectancy after an index hospitalization for patients with heart failure: a population-based study. Am Heart J. 2008, 155 (2): 324-331. 10.1016/j.ahj.2007.08.036.CrossRefPubMed
35.
Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T: Identifying individuals with physcian diagnosed COPD in health administrative databases. COPD. 2009, 6 (5): 388-394. 10.1080/15412550903140865.CrossRefPubMed
36.
Jackevicius CA, Li P, Tu JV: Prevalence, predictors, and outcomes of primary nonadherence after acute myocardial infarction. Circulation. 2008, 117 (8): 1028-1036. 10.1161/CIRCULATIONAHA.107.706820.CrossRefPubMed
37.
Asch SM, Kerr EA, Keesey J, Adams JL, Setodji CM, Malik S, McGlynn EA: Who is at greatest risk for receiving poor-quality health care?. N Engl J Med. 2006, 354 (11): 1147-1156. 10.1056/NEJMsa044464.CrossRefPubMed
38.
Shortell SM: Continuity of medical care: conceptualization and measurement. Med Care. 1976, 14 (5): 377-391. 10.1097/00005650-197605000-00001.CrossRefPubMed
39.
Baltimore. 1997, Johns Hopkins University School of Hygiene and Public Health, Md
40.
41.
Steele LS, Glazier RH, Lin E, Evans M: Using administrative data to measure ambulatory mental health service provision in primary care. Med Care. 2004, 42 (10): 960-965. 10.1097/00005650-200410000-00004.CrossRefPubMed
42.
Roth CP, Lim YW, Pevnick JM, Asch SM, McGlynn EA: The challenge of measuring quality of care from the electronic health record. Am J Med Qual. 2009, 24 (5): 385-394. 10.1177/1062860609336627.CrossRefPubMed
43.
Baron RJ: Quality improvement with an electronic health record: achievable, but not automatic. Ann Intern Med. 2007, 147 (8): 549-552.CrossRefPubMed
44.
DeLone WH, McLean ER: Information systems success: the quest for the dependent variable. Inform Syst Res. 1992, 3 (1): 60-95. 10.1287/isre.3.1.60.CrossRef
45.
Delone WH, McLean ER: The DeLone and McLean model of information systems success: a ten-year update. J Manage Inf Syst. 2003, 19 (4): 9-30.
46.
Chassin MR, Galvin RW, The National Roundtable on Health Care Quality: The urgent need to improve health care quality: Institute of Medicine National Roundtable on Health Care Quality. JAMA. 1998, 280 (11): 1000-1005. 10.1001/jama.280.11.1000.CrossRefPubMed
47.
Casalino L, Gillies RR, Shortell SM, Schmittdiel JA, Bodenheimer T, Robinson JC, Rundall T, Oswald N, Schauffler H, Wang MC: External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases. Jama. 2003, 289 (4): 434-441. 10.1001/jama.289.4.434.CrossRefPubMed
48.
Perlin JB, Kolodner RM, Roswell RH: The Veterans Health Administration: quality, value, accountability, and information as transforming strategies for patient-centered care. Am J Manag Care. 2004, 10 (11 Pt 2): 828-836.PubMed
49.
Craig TJ, Perlin JB, Fleming BB: Self-reported performance improvement strategies of highly successful veterans health administration facilities. Am J Med Qual. 2007, 22 (6): 438-444. 10.1177/1062860607304928.CrossRefPubMed
50.
Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD: Improving preventive care by prompting physicians. Arch Intern Med. 2000, 160 (3): 301-308. 10.1001/archinte.160.3.301.CrossRefPubMed
51.
Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005, 293 (10): 1223-1238. 10.1001/jama.293.10.1223.CrossRefPubMed
52.
Szilagyi PG, Bordley C, Vann JC, Chelminski A, Kraus RM, Margolis PA, Rodewald LE: Effect of patient reminder/recall interventions on immunization rates: a review. Jama. 2000, 284 (14): 1820-1827. 10.1001/jama.284.14.1820.CrossRefPubMed
53.
Stone EG, Morton SC, Hulscher ME, Maglione MA, Roth EA, Grimshaw JM, Mittman BS, Rubenstein LV, Rubenstein LZ, Shekelle PG: Interventions that increase use of adult immunization and cancer screening services: a meta-analysis. Ann Intern Med. 2002, 136 (9): 641-651.CrossRefPubMed
54.
Schultz SETJ, Guttmann A, Jaakkimainen L: Characteristics of primary care practice. Edited by: Jaakkimainen LUR, Klein-Geltink JE, Leong A, Maaten S, Schultz SE, Wang L. 2006, Institute for Clinical Evaluative Sciences, Toronto
Metadata
Title
Measuring data reliability for preventive services in electronic medical records
Authors
Michelle Greiver
Jan Barnsley
Richard H Glazier
Bart J Harvey
Rahim Moineddin
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2012
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/1472-6963-12-116

Other articles of this Issue 1/2012

BMC Health Services Research 1/2012 Go to the issue

Research article

Psychiatrists′ work with sickness certification: frequency, experiences and severity of the certification tasks in a national survey in Sweden

Research article

The health services burden of heart failure: an analysis using linked population health data-sets

Research article

Mapping of multiple criteria for priority setting of health interventions: an aid for decision makers

Study protocol

Involving patients in HTA activities at local level: a study protocol based on the collaboration between researchers and knowledge users

Research article

The knowledge, attitude and behavior about public health emergencies and the response capacity of primary care medical staffs of Guangdong Province, China

Study protocol

Evaluation of primary care midwifery in the Netherlands: design and rationale of a dynamic cohort study (DELIVER)