Skip to main content
Top
Published in: BMC Psychiatry 1/2016

Open Access 01-12-2016 | Research article

Accuracy of reported service use in a cohort of people who are chronically homeless and seriously mentally ill

Authors: Julian M. Somers, Akm Moniruzzaman, Lauren Currie, Stefanie N. Rezansoff, Angela Russolillo, Milad Parpouchi

Published in: BMC Psychiatry | Issue 1/2016

Login to get access

Abstract

Background

Self-reported service use is an integral feature of interventional research with people who are homeless and mentally ill. The objective of this study was to investigate the accuracy of self-reported involvement with major categories of publicly funded services (health, justice, social welfare) within this sub-population.

Methods

Measures were administered pre-randomization in two randomized controlled trials, using timeline follow back with calendar aids for Health, Social, and Justice Service Use, compared to linked administrative data. Variables examined were: psychiatric admissions (both extended stays of more than 6 months and two or more stays within 5 years); emergency department visits, general hospitalization and jail in the past 6 months; and income assistance in the past 1 month. Participants (n = 433) met criteria for homelessness and a least one mental illness.

Results

Prevalence adjusted and bias adjusted kappa (PABAK) values ranged between moderate and almost perfect for extended psychiatric hospital separations (PABAK: 0.77; 95 % confidence interval (CI) = 0.71, 0.83), multiple psychiatric hospitalizations (PABAK = 0.50, 95 % CI = 0.41, 0.59), emergency department visits (PABAK: 0.77; 95 % CI = 0.71, 0.83), jail (PABAK: 0.74; 95 % CI = 0.68, 0.81), and income assistance (PABAK: 0.82; 95 % CI = 0.76, 0.87). Significant differences in under versus over reporting were also found.

Conclusions

People who are homeless and mentally ill reliably reported their overall use of health, justice, and income assistance services. Evidence of under-reporting and over-reporting of certain variables has implications for specific research questions.
ISRCTN registry: 57595077 (Vancouver at Home Study: Housing First plus Assertive Community Treatment versus congregate housing plus supports versus treatment as usual); and 66721740 (Vancouver at Home study: Housing First plus Intensive Case management versus treatment as usual).
Literature
1.
go back to reference Mechanic D. Seizing the opportunities under the Affordable Care Act for transforming the mental and behavioral health system. Health Aff. 2012;31(2):376–82.CrossRef Mechanic D. Seizing the opportunities under the Affordable Care Act for transforming the mental and behavioral health system. Health Aff. 2012;31(2):376–82.CrossRef
2.
go back to reference Gaebel W, Zielasek J. Homeless and mentally ill – a mental healthcare challenge for Europe. Acta Psychiatr Scan. 2015;131(4):236–8.CrossRef Gaebel W, Zielasek J. Homeless and mentally ill – a mental healthcare challenge for Europe. Acta Psychiatr Scan. 2015;131(4):236–8.CrossRef
3.
go back to reference Goering PN, Streiner DL, Adair C, et al. The At Home/Chez Soi trial protocol: a pragmatic, multi-site, randomized controlled trial of a housing first intervention for homeless individuals with mental illness in five Canadian cities. BMJ Open. 2011;1:e000323.PubMedCentralCrossRefPubMed Goering PN, Streiner DL, Adair C, et al. The At Home/Chez Soi trial protocol: a pragmatic, multi-site, randomized controlled trial of a housing first intervention for homeless individuals with mental illness in five Canadian cities. BMJ Open. 2011;1:e000323.PubMedCentralCrossRefPubMed
4.
go back to reference Tinland A, Fortanier C, Girard V, et al. Evaluation of the housing first program in patients with severe mental disorders in France: study protocol for a randomized controlled trial. Trials. 2013;14:309.PubMedCentralCrossRefPubMed Tinland A, Fortanier C, Girard V, et al. Evaluation of the housing first program in patients with severe mental disorders in France: study protocol for a randomized controlled trial. Trials. 2013;14:309.PubMedCentralCrossRefPubMed
5.
go back to reference Johnson G, Parkinson S, Parsell C. Policy shift or program drift? Implementing Housing First in Australia, AHURI Final Report No. 184 Australian Housing and Urban Research Institute, Melbourne. 2012. Johnson G, Parkinson S, Parsell C. Policy shift or program drift? Implementing Housing First in Australia, AHURI Final Report No. 184 Australian Housing and Urban Research Institute, Melbourne. 2012.
6.
go back to reference Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: results from the national comorbidity survey replication. Arch Gen Psychiatry. 2005;62(6):629–40.CrossRefPubMed Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: results from the national comorbidity survey replication. Arch Gen Psychiatry. 2005;62(6):629–40.CrossRefPubMed
7.
go back to reference Druss BG, Rosenheck RA. Patterns of health care costs associated with depression and substance abuse in a national sample. Psychiatr Serv. 1999;50(2):214–8.CrossRefPubMed Druss BG, Rosenheck RA. Patterns of health care costs associated with depression and substance abuse in a national sample. Psychiatr Serv. 1999;50(2):214–8.CrossRefPubMed
8.
go back to reference Goldberg RW, Seybolt DC, Lehman A. Reliable self-report of health services use by individuals with serious mental illness. Psychiatr Serv. 2002;53(7):879–81.CrossRefPubMed Goldberg RW, Seybolt DC, Lehman A. Reliable self-report of health services use by individuals with serious mental illness. Psychiatr Serv. 2002;53(7):879–81.CrossRefPubMed
9.
10.
go back to reference Caslyn RJ, Morse GA, Kinkenberg WD, Trusty ML. Reliability and validity of self-report data of homeless mentally ill individuals. Eval Program Plann. 1997;20(1):47–54.CrossRef Caslyn RJ, Morse GA, Kinkenberg WD, Trusty ML. Reliability and validity of self-report data of homeless mentally ill individuals. Eval Program Plann. 1997;20(1):47–54.CrossRef
11.
go back to reference Metraux S, Stino M, Culhane DP. Validation of self-reported veteran status among two sheltered homeless populations. Public Health Rep. 2014;129(1):73–7.PubMedCentralPubMed Metraux S, Stino M, Culhane DP. Validation of self-reported veteran status among two sheltered homeless populations. Public Health Rep. 2014;129(1):73–7.PubMedCentralPubMed
12.
13.
go back to reference Clifasefi SL, Collins SE, Tanzer K, Burlington B, Hoang SE, Larimer ME. Agreement between se;f-report and archival public service utilization data among chronically homeless individuals with severe alcohol problems. J Community Psychol. 2011;39(6):631–44.CrossRef Clifasefi SL, Collins SE, Tanzer K, Burlington B, Hoang SE, Larimer ME. Agreement between se;f-report and archival public service utilization data among chronically homeless individuals with severe alcohol problems. J Community Psychol. 2011;39(6):631–44.CrossRef
14.
go back to reference Somers JM, Patterson ML, Moniruzzaman A, et al. Vancouver at home: pragmatic randomized trials investigating housing first for homeless and mentally ill adults. Trials. 2013;14:365.PubMedCentralCrossRefPubMed Somers JM, Patterson ML, Moniruzzaman A, et al. Vancouver at home: pragmatic randomized trials investigating housing first for homeless and mentally ill adults. Trials. 2013;14:365.PubMedCentralCrossRefPubMed
15.
go back to reference Rosen MI, McMahon TJ, Rosenheck RA. Homeless people whose self-reported SSI/DI status is inconsistent with social security administration records. Soc Secur Bull. 2007;67(1):53–62.PubMed Rosen MI, McMahon TJ, Rosenheck RA. Homeless people whose self-reported SSI/DI status is inconsistent with social security administration records. Soc Secur Bull. 2007;67(1):53–62.PubMed
16.
go back to reference Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev. 2006;63:217–35.CrossRefPubMed Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev. 2006;63:217–35.CrossRefPubMed
17.
go back to reference Kertesz SG, Crouch K, Milby JB, Cusimano RE, Schumacher JE. Housing first for homeless persons with active addiction: are we overreaching? Milbank Q. 2009;87(2):495–534.PubMedCentralCrossRefPubMed Kertesz SG, Crouch K, Milby JB, Cusimano RE, Schumacher JE. Housing first for homeless persons with active addiction: are we overreaching? Milbank Q. 2009;87(2):495–534.PubMedCentralCrossRefPubMed
18.
go back to reference Monahan J, Bonnie RJ, Applebaum PS, Hyde PS, Steadman HJ, Swartz MS. Mandated community treatment: beyond outpatient commitment. Psychiatr Serv. 2001;52(9):1198–205.CrossRefPubMed Monahan J, Bonnie RJ, Applebaum PS, Hyde PS, Steadman HJ, Swartz MS. Mandated community treatment: beyond outpatient commitment. Psychiatr Serv. 2001;52(9):1198–205.CrossRefPubMed
20.
go back to reference Frank JW, Wang EA, Nunez-Smith M, Comfort M. Discrimination based on criminal record and healthcare utilization among men recently released from prison: a descriptive study. Health Justice. 2014; doi:10.1186/2194-7899-2-6. Frank JW, Wang EA, Nunez-Smith M, Comfort M. Discrimination based on criminal record and healthcare utilization among men recently released from prison: a descriptive study. Health Justice. 2014; doi:10.​1186/​2194-7899-2-6.
21.
go back to reference McKee M, Dixon J, Chenet L. Making routine data adequate to support clinical audit. BMJ. 1994;(309):1246–1247 McKee M, Dixon J, Chenet L. Making routine data adequate to support clinical audit. BMJ. 1994;(309):1246–1247
22.
go back to reference Groene O, Kristensen S, Arah OA, et al. Feasibility of using administrative data to compare hospital performance in the EU. Int J Qual Health Care. 2014;26(S1):108–15.PubMedCentralCrossRefPubMed Groene O, Kristensen S, Arah OA, et al. Feasibility of using administrative data to compare hospital performance in the EU. Int J Qual Health Care. 2014;26(S1):108–15.PubMedCentralCrossRefPubMed
23.
go back to reference Andersen JS, Olivarius N, Krasnik A. The Danish National Health Service register. Scand J Pub Health. 2011;39(7):34–7.CrossRef Andersen JS, Olivarius N, Krasnik A. The Danish National Health Service register. Scand J Pub Health. 2011;39(7):34–7.CrossRef
24.
go back to reference Sheehan DV, Lecrubier Y, Sheehan KH, et al. The mini-international neuropsychiatric interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD- 10. J Clin Psychiatr. 1998;59:22–33. Sheehan DV, Lecrubier Y, Sheehan KH, et al. The mini-international neuropsychiatric interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD- 10. J Clin Psychiatr. 1998;59:22–33.
25.
go back to reference Adair CE, Holland AC, Patterson ML, et al. Cognitive interviewing methods for questionnaire pre-testing in homeless persons with mental disorders. J Urban Health. 2012;89:36–52.PubMedCentralCrossRefPubMed Adair CE, Holland AC, Patterson ML, et al. Cognitive interviewing methods for questionnaire pre-testing in homeless persons with mental disorders. J Urban Health. 2012;89:36–52.PubMedCentralCrossRefPubMed
26.
go back to reference Rezansoff SN, Moniruzzaman A, Gress C, Somers JM. Psychiatric diagnoses and multiyear criminal recidivism in a Canadian provincial offender population. Psychol Public Policy Law. 2013;19:443–53.CrossRef Rezansoff SN, Moniruzzaman A, Gress C, Somers JM. Psychiatric diagnoses and multiyear criminal recidivism in a Canadian provincial offender population. Psychol Public Policy Law. 2013;19:443–53.CrossRef
27.
28.
go back to reference Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–68.PubMed Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–68.PubMed
29.
go back to reference Marrie RA, Fisk JD, Yu BN, et al. The CIHR team in the epidemiology and impact of comorbidity on multiple sclerosis: mental comorbidity and multiple sclerosis: validating administrative data to support population-based surveillance. BMC Neurol. 2013;13(1):16. doi:10.1186/1471-2377-13-16.PubMedCentralCrossRefPubMed Marrie RA, Fisk JD, Yu BN, et al. The CIHR team in the epidemiology and impact of comorbidity on multiple sclerosis: mental comorbidity and multiple sclerosis: validating administrative data to support population-based surveillance. BMC Neurol. 2013;13(1):16. doi:10.​1186/​1471-2377-13-16.PubMedCentralCrossRefPubMed
30.
go back to reference Cibere J, Thorne A, Bellamy N, et al. Reliability of the hip examination in osteoarthritis: effect of standardization. Arthritis Rheum. 2008;59:373–81.CrossRefPubMed Cibere J, Thorne A, Bellamy N, et al. Reliability of the hip examination in osteoarthritis: effect of standardization. Arthritis Rheum. 2008;59:373–81.CrossRefPubMed
31.
go back to reference Girianelli VR, Santos Thuler LC. Evaluation of agreement between conventional and liquid-based cytology in cervical cancer early detection based on analysis of 2,091 smears: experiences at the Brazilian National Cancer Institute. Diagn Cytopathol. 2007;35:545–9.CrossRefPubMed Girianelli VR, Santos Thuler LC. Evaluation of agreement between conventional and liquid-based cytology in cervical cancer early detection based on analysis of 2,091 smears: experiences at the Brazilian National Cancer Institute. Diagn Cytopathol. 2007;35:545–9.CrossRefPubMed
32.
go back to reference Steel Z, Silove D, Chey T, Bauman A, Phan T. Mental disorders, disability and health service use amongst Vietnamese refugees and the host Australian population. Acta Psychiatr Scand. 2005;111:300–9.CrossRefPubMed Steel Z, Silove D, Chey T, Bauman A, Phan T. Mental disorders, disability and health service use amongst Vietnamese refugees and the host Australian population. Acta Psychiatr Scand. 2005;111:300–9.CrossRefPubMed
33.
go back to reference Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRefPubMed Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRefPubMed
34.
go back to reference Corp IBM. Released. IBM SPSS statistics for windows, version 22.0. IBM Corp.: Armonk, NY; 2013. Corp IBM. Released. IBM SPSS statistics for windows, version 22.0. IBM Corp.: Armonk, NY; 2013.
35.
go back to reference StataCorp. Stata statistical software: release 13. College Station, TX: StataCorp LP; 2013. StataCorp. Stata statistical software: release 13. College Station, TX: StataCorp LP; 2013.
36.
go back to reference Stavro K, Pelletier J, Potvin S. Widespread and sustained cognitive deficits in alcoholism: a meta-analysis. Addict Biol. 2014;18(2):203–13.CrossRef Stavro K, Pelletier J, Potvin S. Widespread and sustained cognitive deficits in alcoholism: a meta-analysis. Addict Biol. 2014;18(2):203–13.CrossRef
Metadata
Title
Accuracy of reported service use in a cohort of people who are chronically homeless and seriously mentally ill
Authors
Julian M. Somers
Akm Moniruzzaman
Lauren Currie
Stefanie N. Rezansoff
Angela Russolillo
Milad Parpouchi
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Psychiatry / Issue 1/2016
Electronic ISSN: 1471-244X
DOI
https://doi.org/10.1186/s12888-016-0758-0

Other articles of this Issue 1/2016

BMC Psychiatry 1/2016 Go to the issue