Skip to main content
Top
Published in: BMC Primary Care 1/2019

Open Access 01-12-2019 | Care | Research article

Concordance of care processes between medical records and patient self-administered questionnaires

Authors: Cynthia Khanji, Mireille E. Schnitzer, Céline Bareil, Sylvie Perreault, Lyne Lalonde

Published in: BMC Primary Care | Issue 1/2019

Login to get access

Abstract

Background

Despite the increasing use of medical records to measure quality of care, studies have shown that their validity is suboptimal. The objective of this study is to assess the concordance of cardiovascular care processes evaluated through medical record review and patient self-administered questionnaires (SAQs) using ten quality indicators (TRANSIT indicators). These indicators were developed as part of a participatory research program (TRANSIT study) dedicated to TRANSforming InTerprofessional clinical practices to improve cardiovascular disease (CVD) prevention in primary care.

Methods

For every patient participating in the TRANSIT study, the compliance to each indicator (individual scores) as well as the mean compliance to all indicators of a category (subscale scores) and to the complete set of ten indicators (overall scale score) were established. Concordance between results obtained using medical records and patient SAQs was assessed by prevalence-adjusted bias-adjusted kappa (PABAK) coefficients as well as intraclass correlation coefficients (ICCs) and 95% confidence intervals (95% CI). Generalized linear mixed models (GLMM) were used to identify patients’ sociodemographic and clinical characteristics associated with agreement between the two data sources.

Results

The TRANSIT study was conducted in a primary care setting among patients (n = 759) with multimorbidity, at moderate (16%) and high risk (83%) of cardiovascular diseases. Quality of care, as measured by the TRANSIT indicators, varied substantially between medical records and patient SAQ. Concordance between the two data sources, as measured by ICCs (95% CI), was poor for the subscale (0.18 [0.08–0.27] to 0.46 [0.40–0.52]) and overall (0.46 [0.40–0.53]) compliance scale scores. GLMM showed that agreement was not affected by patients’ characteristics.

Conclusions

In quality improvement strategies, researchers must acknowledge that care processes may not be consistently recorded in medical records. They must also be aware that the evaluation of the quality of care may vary depending on the source of information, the clinician responsible of documenting the interventions, and the domain of care.
Appendix
Available only for authorised users
Literature
1.
go back to reference Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med. 2000;108(8):642–9.CrossRef Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med. 2000;108(8):642–9.CrossRef
2.
go back to reference Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, et al. How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Med Care. 1998;36(6):851–67.CrossRef Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, et al. How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Med Care. 1998;36(6):851–67.CrossRef
3.
go back to reference Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283(13):1715–22.CrossRef Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283(13):1715–22.CrossRef
4.
go back to reference Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743–8.CrossRef Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743–8.CrossRef
5.
go back to reference Corriol C, Daucourt V, Grenier C, Minvielle E. How to limit the burden of data collection for quality indicators based on medical records? The COMPAQH experience. BMC Health Serv Res. 2008;8:215.CrossRef Corriol C, Daucourt V, Grenier C, Minvielle E. How to limit the burden of data collection for quality indicators based on medical records? The COMPAQH experience. BMC Health Serv Res. 2008;8:215.CrossRef
6.
go back to reference Katz A, Soodeen RA, Bogdanovic B, De Coster C, Chateau D. Can the quality of care in family practice be measured using administrative data? Health Serv Res. 2006;41(6):2238–54.CrossRef Katz A, Soodeen RA, Bogdanovic B, De Coster C, Chateau D. Can the quality of care in family practice be measured using administrative data? Health Serv Res. 2006;41(6):2238–54.CrossRef
7.
go back to reference Corser W, Sikorskii A, Olomu A, Stommel M, Proden C, Holmes-Rovner M. Concordance between comorbidity data from patient self-report interviews and medical record documentation. BMC Health Serv Res. 2008;8:85.CrossRef Corser W, Sikorskii A, Olomu A, Stommel M, Proden C, Holmes-Rovner M. Concordance between comorbidity data from patient self-report interviews and medical record documentation. BMC Health Serv Res. 2008;8:85.CrossRef
8.
go back to reference Sangha O, Stucki G, Liang MH, Fossel AH, Katz JN. The self-administered comorbidity questionnaire: a new method to assess comorbidity for clinical and health services research. Arthritis Rheum. 2003;49(2):156–63.CrossRef Sangha O, Stucki G, Liang MH, Fossel AH, Katz JN. The self-administered comorbidity questionnaire: a new method to assess comorbidity for clinical and health services research. Arthritis Rheum. 2003;49(2):156–63.CrossRef
9.
go back to reference Tisnado DM, Adams JL, Liu H, Damberg CL, Hu FA, Chen WP, et al. Does the concordance between medical records and patient self-report vary with patient characteristics. Health Serv Outcome Res Methodol. 2006;6:157–75.CrossRef Tisnado DM, Adams JL, Liu H, Damberg CL, Hu FA, Chen WP, et al. Does the concordance between medical records and patient self-report vary with patient characteristics. Health Serv Outcome Res Methodol. 2006;6:157–75.CrossRef
10.
go back to reference Sudman S, Bradburn NM. Response effects in surveys. Chicago: Adeline; 1974. Sudman S, Bradburn NM. Response effects in surveys. Chicago: Adeline; 1974.
11.
go back to reference Goldman N, Lin IF, Weinstein M, Lin YH. Evaluating the quality of self-reports of hypertension and diabetes. J Clin Epidemiol. 2003;56(2):148–54.CrossRef Goldman N, Lin IF, Weinstein M, Lin YH. Evaluating the quality of self-reports of hypertension and diabetes. J Clin Epidemiol. 2003;56(2):148–54.CrossRef
12.
go back to reference Okura Y, Urban LH, Mahoney DW, Jacobsen SJ, Rodeheffer RJ. Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol. 2004;57(10):1096–103.CrossRef Okura Y, Urban LH, Mahoney DW, Jacobsen SJ, Rodeheffer RJ. Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol. 2004;57(10):1096–103.CrossRef
13.
go back to reference Rozario PA, Morrow-Howell N, Proctor E. Comparing the congruency of self-report and provider records of depressed elders' service use by provider type. Med Care. 2004;42(10):952–9.CrossRef Rozario PA, Morrow-Howell N, Proctor E. Comparing the congruency of self-report and provider records of depressed elders' service use by provider type. Med Care. 2004;42(10):952–9.CrossRef
14.
go back to reference Tisnado DM, Adams JL, Liu H, Damberg CL, Chen WP, Hu FA, et al. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006;44(2):132–40.CrossRef Tisnado DM, Adams JL, Liu H, Damberg CL, Chen WP, Hu FA, et al. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006;44(2):132–40.CrossRef
15.
go back to reference Valikodath NG, Newman-Casey PA, Lee PP, Musch DC, Niziol LM, Woodward MA. Agreement of ocular symptom reporting between patient-reported outcomes and medical records. JAMA Ophthalmol. 2017;135(3):225–31.CrossRef Valikodath NG, Newman-Casey PA, Lee PP, Musch DC, Niziol LM, Woodward MA. Agreement of ocular symptom reporting between patient-reported outcomes and medical records. JAMA Ophthalmol. 2017;135(3):225–31.CrossRef
16.
go back to reference Lalonde L, Goudreau J, Hudon E, Lussier MT, Duhamel F, Belanger D, et al. Priorities for action to improve cardiovascular preventive care of patients with multimorbid conditions in primary care - a participatory action research project. Fam Pract. 2012;29(6):733–41.CrossRef Lalonde L, Goudreau J, Hudon E, Lussier MT, Duhamel F, Belanger D, et al. Priorities for action to improve cardiovascular preventive care of patients with multimorbid conditions in primary care - a participatory action research project. Fam Pract. 2012;29(6):733–41.CrossRef
17.
go back to reference Lalonde L, Goudreau J, Hudon E, Lussier M-T, Bareil C, Duhamel F, et al. Development of an interprofessional program for cardiovascular prevention in primary care: a participatory research approach. Sage Open Med. 2014:1–11. Lalonde L, Goudreau J, Hudon E, Lussier M-T, Bareil C, Duhamel F, et al. Development of an interprofessional program for cardiovascular prevention in primary care: a participatory research approach. Sage Open Med. 2014:1–11.
18.
go back to reference Bareil C, Duhamel F, Lalonde L, Goudreau J, Hudon E, Lussier MT, et al. Facilitating implementation of Interprofessional collaborative practices into primary care: a trilogy of driving forces. J Healthc Manag. 2015;60(4):287–300.CrossRef Bareil C, Duhamel F, Lalonde L, Goudreau J, Hudon E, Lussier MT, et al. Facilitating implementation of Interprofessional collaborative practices into primary care: a trilogy of driving forces. J Healthc Manag. 2015;60(4):287–300.CrossRef
19.
go back to reference Genest J, McPherson R, Frohlich J, Anderson T, Campbell N, Carpentier A, et al. 2009 Canadian cardiovascular society/Canadian guidelines for the diagnosis and treatment of dyslipidemia and prevention of cardiovascular disease in the adult - 2009 recommendations. Can J Cardiol. 2009;25(10):567–79.CrossRef Genest J, McPherson R, Frohlich J, Anderson T, Campbell N, Carpentier A, et al. 2009 Canadian cardiovascular society/Canadian guidelines for the diagnosis and treatment of dyslipidemia and prevention of cardiovascular disease in the adult - 2009 recommendations. Can J Cardiol. 2009;25(10):567–79.CrossRef
20.
go back to reference Khanji C, Bareil C, Hudon E, Goudreau J, Duhamel F, Lussier MT, et al. Psychometric analysis of the TRANSIT quality indicators for cardiovascular disease prevention in primary care. Int J Qual Health Care. 2017;29(8):999–1005.CrossRef Khanji C, Bareil C, Hudon E, Goudreau J, Duhamel F, Lussier MT, et al. Psychometric analysis of the TRANSIT quality indicators for cardiovascular disease prevention in primary care. Int J Qual Health Care. 2017;29(8):999–1005.CrossRef
21.
go back to reference Khanji C, Lalonde L, Bareil C, Lussier M-T, Perreault S, Schnitzer ME. LASSO regression for the prediction of intermediate outcomes related to cardiovascular disease prevention using the TRANSIT quality indicators. Med Care. 2018;57(1):63–72.CrossRef Khanji C, Lalonde L, Bareil C, Lussier M-T, Perreault S, Schnitzer ME. LASSO regression for the prediction of intermediate outcomes related to cardiovascular disease prevention using the TRANSIT quality indicators. Med Care. 2018;57(1):63–72.CrossRef
22.
go back to reference Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330–3 ii.CrossRef Glanz K, Sallis JF, Saelens BE, Frank LD. Healthy nutrition environments: concepts and measures. Am J Health Promot. 2005;19(5):330–3 ii.CrossRef
23.
go back to reference Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–9.CrossRef Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–9.CrossRef
24.
go back to reference Koo TK, Li MY. A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.CrossRef Koo TK, Li MY. A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.CrossRef
25.
go back to reference Demidenko E. Mixed models: theory and applications with R, second edition: John Wiley & Sons, Inc.; 2013. Demidenko E. Mixed models: theory and applications with R, second edition: John Wiley & Sons, Inc.; 2013.
26.
go back to reference Rohrbaugh M, Rogers JC. What did the doctor do? When physicians and patients disagree. Arch Fam Med. 1994;3(2):125–8.CrossRef Rohrbaugh M, Rogers JC. What did the doctor do? When physicians and patients disagree. Arch Fam Med. 1994;3(2):125–8.CrossRef
27.
go back to reference Eder SC, Sloan EP, Todd K. Documentation of ED patient pain by nurses and physicians. Am J Emerg Med. 2003;21(4):253–7.CrossRef Eder SC, Sloan EP, Todd K. Documentation of ED patient pain by nurses and physicians. Am J Emerg Med. 2003;21(4):253–7.CrossRef
28.
go back to reference Jensdottir AB, Jonsson P, Noro A, Jonsen E, Ljunggren G, Finne-Soveri H, et al. Comparison of nurses' and physicians' documentation of functional abilities of older patients in acute care - patient records compared with standardized assessment. Scand J Caring Sci. 2008;22(3):341–7.CrossRef Jensdottir AB, Jonsson P, Noro A, Jonsen E, Ljunggren G, Finne-Soveri H, et al. Comparison of nurses' and physicians' documentation of functional abilities of older patients in acute care - patient records compared with standardized assessment. Scand J Caring Sci. 2008;22(3):341–7.CrossRef
29.
go back to reference Cricelli C, Mazzaglia G, Samani F, Marchi M, Sabatini A, Nardi R, et al. Prevalence estimates for chronic diseases in Italy: exploring the differences between self-report and primary care databases. J Public Health Med. 2003;25(3):254–7.CrossRef Cricelli C, Mazzaglia G, Samani F, Marchi M, Sabatini A, Nardi R, et al. Prevalence estimates for chronic diseases in Italy: exploring the differences between self-report and primary care databases. J Public Health Med. 2003;25(3):254–7.CrossRef
30.
go back to reference Metzger MH, Goldberg M, Chastang JF, Leclerc A, Zins M. Factors associated with self-reporting of chronic health problems in the French GAZEL cohort. J Clin Epidemiol. 2002;55(1):48–59.CrossRef Metzger MH, Goldberg M, Chastang JF, Leclerc A, Zins M. Factors associated with self-reporting of chronic health problems in the French GAZEL cohort. J Clin Epidemiol. 2002;55(1):48–59.CrossRef
31.
go back to reference Kriegsman DM, Penninx BW, van Eijk JT, Boeke AJ, Deeg DJ. Self-reports and general practitioner information on the presence of chronic diseases in community dwelling elderly. A study on the accuracy of patients' self-reports and on determinants of inaccuracy. J Clin Epidemiol. 1996;49(12):1407–17.CrossRef Kriegsman DM, Penninx BW, van Eijk JT, Boeke AJ, Deeg DJ. Self-reports and general practitioner information on the presence of chronic diseases in community dwelling elderly. A study on the accuracy of patients' self-reports and on determinants of inaccuracy. J Clin Epidemiol. 1996;49(12):1407–17.CrossRef
32.
go back to reference Simpson CF, Boyd CM, Carlson MC, Griswold ME, Guralnik JM, Fried LP. Agreement between self-report of disease diagnoses and medical record validation in disabled older women: factors that modify agreement. J Am Geriatr Soc. 2004;52(1):123–7.CrossRef Simpson CF, Boyd CM, Carlson MC, Griswold ME, Guralnik JM, Fried LP. Agreement between self-report of disease diagnoses and medical record validation in disabled older women: factors that modify agreement. J Am Geriatr Soc. 2004;52(1):123–7.CrossRef
33.
go back to reference Del Boca FK, Noll JA. Truth or consequences: the validity of self-report data in health services research on addictions. Addiction. 2000;95(Suppl 3):S347–60.CrossRef Del Boca FK, Noll JA. Truth or consequences: the validity of self-report data in health services research on addictions. Addiction. 2000;95(Suppl 3):S347–60.CrossRef
34.
go back to reference Andersen RM, Kasper J, Frankel MR. Total survey error: applications to improve health surveys. San Francisco: Jossey Bass; 1979. Andersen RM, Kasper J, Frankel MR. Total survey error: applications to improve health surveys. San Francisco: Jossey Bass; 1979.
Metadata
Title
Concordance of care processes between medical records and patient self-administered questionnaires
Authors
Cynthia Khanji
Mireille E. Schnitzer
Céline Bareil
Sylvie Perreault
Lyne Lalonde
Publication date
01-12-2019
Publisher
BioMed Central
Keyword
Care
Published in
BMC Primary Care / Issue 1/2019
Electronic ISSN: 2731-4553
DOI
https://doi.org/10.1186/s12875-019-0979-7

Other articles of this Issue 1/2019

BMC Primary Care 1/2019 Go to the issue