Skip to main content
Top
Published in: BMC Public Health 1/2013

Open Access 01-12-2013 | Research article

External validation of two prediction models identifying employees at risk of high sickness absence: cohort study with 1-year follow-up

Authors: Corné AM Roelen, Ute Bültmann, Willem van Rhenen, Jac JL van der Klink, Jos WR Twisk, Martijn W Heymans

Published in: BMC Public Health | Issue 1/2013

Login to get access

Abstract

Background

Two models including age, self-rated health (SRH) and prior sickness absence (SA) were found to predict high SA in health care workers. The present study externally validated these prediction models in a population of office workers and investigated the effect of adding gender as a predictor.

Methods

SRH was assessed at baseline in a convenience sample of office workers. Age, gender and prior SA were retrieved from an occupational health service register. Two pre-defined prediction models were externally validated: a model identifying employees with high (i.e. ≥30) SA days and a model identifying employees with high (i.e. ≥3) SA episodes during 1-year follow-up. Calibration was investigated by plotting the predicted and observed probabilities and calculating the calibration slope. Discrimination was examined by receiver operating characteristic (ROC) analysis and the area under the ROC-curve (AUC).

Results

A total of 593 office workers had complete data and were eligible for analysis. Although the SA days model showed acceptable calibration (slope = 0.89), it poorly discriminated office workers with high SA days from those without high SA days (AUC = 0.65; 95% CI 0.58–0.71). The SA episodes model showed acceptable discrimination (AUC = 0.76, 95% CI 0.70–0.82) and calibration (slope = 0.96). The prognostic performance of the prediction models did not improve in the population of office workers after adding gender.

Conclusion

The SA episodes model accurately predicted the risk of high SA episodes in office workers, but needs further multisite validation and requires a simpler presentation format before it can be used to select high-risk employees for interventions to prevent or reduce SA.
Appendix
Available only for authorised users
Literature
1.
go back to reference Marmot M, Feeney A, Shipley M, North F, Syme SL: Sickness absence as a measure of health status and functioning: from the UK Whitehall II study. J Epidemiol Community Health. 1995, 49: 124-130. 10.1136/jech.49.2.124.CrossRefPubMedPubMedCentral Marmot M, Feeney A, Shipley M, North F, Syme SL: Sickness absence as a measure of health status and functioning: from the UK Whitehall II study. J Epidemiol Community Health. 1995, 49: 124-130. 10.1136/jech.49.2.124.CrossRefPubMedPubMedCentral
2.
go back to reference Kivimäki M, Head J, Ferrie JE, Shipley MJ, Vahtera J, Marmot MG: Sickness absence as a global measure of health: evidence from mortality in the Whitehall II prospective cohort study. BMJ. 2003, 327: 364-368. 10.1136/bmj.327.7411.364.CrossRefPubMedPubMedCentral Kivimäki M, Head J, Ferrie JE, Shipley MJ, Vahtera J, Marmot MG: Sickness absence as a global measure of health: evidence from mortality in the Whitehall II prospective cohort study. BMJ. 2003, 327: 364-368. 10.1136/bmj.327.7411.364.CrossRefPubMedPubMedCentral
3.
go back to reference Bambra C, Norman P: What is the association between sickness absence, morbidity and mortality?. Health Place. 2006, 12: 728-733. 10.1016/j.healthplace.2005.02.008.CrossRefPubMed Bambra C, Norman P: What is the association between sickness absence, morbidity and mortality?. Health Place. 2006, 12: 728-733. 10.1016/j.healthplace.2005.02.008.CrossRefPubMed
4.
go back to reference Eriksson HG, von Celsing HS, Wahlström R, Janson L, Zander V, Wallman T: Sickness absence and self-reported health a population-based study of 43,600 individuals in central Sweden. BMC Publ Health. 2008, 8: 426-10.1186/1471-2458-8-426.CrossRef Eriksson HG, von Celsing HS, Wahlström R, Janson L, Zander V, Wallman T: Sickness absence and self-reported health a population-based study of 43,600 individuals in central Sweden. BMC Publ Health. 2008, 8: 426-10.1186/1471-2458-8-426.CrossRef
5.
go back to reference Labriola M: Conceptual framework of sickness absence and return to work, focusing on both the individual and contextual level. Work. 2008, 30: 377-387.PubMed Labriola M: Conceptual framework of sickness absence and return to work, focusing on both the individual and contextual level. Work. 2008, 30: 377-387.PubMed
6.
go back to reference Lund T, Kivimäki M, Labriola M, Villadsen E, Christensen KB: Using administrative sickness absence data as a marker of future disability pension: the prospective DREAM study of Danish private sector employees. Occup Environ Med. 2008, 65: 28-31. 10.1136/oem.2006.031393.CrossRefPubMed Lund T, Kivimäki M, Labriola M, Villadsen E, Christensen KB: Using administrative sickness absence data as a marker of future disability pension: the prospective DREAM study of Danish private sector employees. Occup Environ Med. 2008, 65: 28-31. 10.1136/oem.2006.031393.CrossRefPubMed
7.
go back to reference Kant IJ, Jansen NW, van Amelsvoort LG, van Leusden R, Berkouwer A: Structured early consultation with the occupational physician reduces sickness absence among office workers at high risk for long-term sickness absence: a randomized controlled trial. J Occup Rehabil. 2008, 18: 79-86. 10.1007/s10926-007-9114-z.CrossRefPubMedPubMedCentral Kant IJ, Jansen NW, van Amelsvoort LG, van Leusden R, Berkouwer A: Structured early consultation with the occupational physician reduces sickness absence among office workers at high risk for long-term sickness absence: a randomized controlled trial. J Occup Rehabil. 2008, 18: 79-86. 10.1007/s10926-007-9114-z.CrossRefPubMedPubMedCentral
8.
go back to reference Taimela S, Malmivaara A, Justén S, Läärä E, Sintonen H, Tiekso J, Aro T: The effectiveness of two occupational health intervention programmes in reducing sickness absence among employees at risk. Two randomised controlled trials. Occup Environ Med. 2008, 65: 236-241.CrossRefPubMed Taimela S, Malmivaara A, Justén S, Läärä E, Sintonen H, Tiekso J, Aro T: The effectiveness of two occupational health intervention programmes in reducing sickness absence among employees at risk. Two randomised controlled trials. Occup Environ Med. 2008, 65: 236-241.CrossRefPubMed
9.
go back to reference Taimela S, Justén S, Aronen P, Sintonen H, Läärä A, Malmivaara A, Tiekso J, Aro T: An occupational health intervention programme for workers at high risk for sickness absence. Cost effectiveness analysis based on a randomised controlled trial. Occup Environ Med. 2008, 65: 242-248. 10.1136/oem.2007.033167.CrossRefPubMed Taimela S, Justén S, Aronen P, Sintonen H, Läärä A, Malmivaara A, Tiekso J, Aro T: An occupational health intervention programme for workers at high risk for sickness absence. Cost effectiveness analysis based on a randomised controlled trial. Occup Environ Med. 2008, 65: 242-248. 10.1136/oem.2007.033167.CrossRefPubMed
10.
go back to reference Duijts SF, Kant IJ, Landeweerd JA, Swaen GMH: Prediction of sickness absence: development of a screening instrument. Occup Environ Med. 2006, 63: 564-569. 10.1136/oem.2005.024521.CrossRefPubMedPubMedCentral Duijts SF, Kant IJ, Landeweerd JA, Swaen GMH: Prediction of sickness absence: development of a screening instrument. Occup Environ Med. 2006, 63: 564-569. 10.1136/oem.2005.024521.CrossRefPubMedPubMedCentral
11.
go back to reference Roelen CA, van der Pol TR, Koopmans PC, Groothoff JW: Identifying workers at risk of sickness absence by questionnaire. Occup Med. 2006, 56: 442-446. 10.1093/occmed/kql087.CrossRef Roelen CA, van der Pol TR, Koopmans PC, Groothoff JW: Identifying workers at risk of sickness absence by questionnaire. Occup Med. 2006, 56: 442-446. 10.1093/occmed/kql087.CrossRef
12.
go back to reference Duijts SF, Kant I, van den Brandt PA, Swaen GM: Psychometrics and validation of a screening instrument for sickness absence. Occup Med. 2008, 58: 413-418. 10.1093/occmed/kqn070.CrossRef Duijts SF, Kant I, van den Brandt PA, Swaen GM: Psychometrics and validation of a screening instrument for sickness absence. Occup Med. 2008, 58: 413-418. 10.1093/occmed/kqn070.CrossRef
13.
go back to reference Kant IJ, Jansen NW, van Amelsvoort LG, Swaen GM, van Leusden R, Berkouwer A: Screening questionnaire Balansmeter proved successful in predicting future long-term sickness absence in office workers. J Clin Epidemiol. 2009, 62: 408-414. 10.1016/j.jclinepi.2008.07.003.CrossRefPubMed Kant IJ, Jansen NW, van Amelsvoort LG, Swaen GM, van Leusden R, Berkouwer A: Screening questionnaire Balansmeter proved successful in predicting future long-term sickness absence in office workers. J Clin Epidemiol. 2009, 62: 408-414. 10.1016/j.jclinepi.2008.07.003.CrossRefPubMed
14.
go back to reference Etter JF, Perneger TV: Analysis of non-response bias in a mailed health survey. J Clin Epidemiol. 1997, 50: 1123-1128. 10.1016/S0895-4356(97)00166-2.CrossRefPubMed Etter JF, Perneger TV: Analysis of non-response bias in a mailed health survey. J Clin Epidemiol. 1997, 50: 1123-1128. 10.1016/S0895-4356(97)00166-2.CrossRefPubMed
15.
go back to reference Froom P, Melamed S, Kristahl-Boneh E, Benbassat J, Ribak J: Healthy volunteer effect in industrial workers. J Clin Epidemiol. 1999, 52: 731-735. 10.1016/S0895-4356(99)00070-0.CrossRefPubMed Froom P, Melamed S, Kristahl-Boneh E, Benbassat J, Ribak J: Healthy volunteer effect in industrial workers. J Clin Epidemiol. 1999, 52: 731-735. 10.1016/S0895-4356(99)00070-0.CrossRefPubMed
16.
go back to reference Duijts SF, Kant IJ, Swaen GM: Advantages and disadvantages of an objective selection process for early intervention in employees at risk for sickness absence. BMC Publ Health. 2007, 7: 67-10.1186/1471-2458-7-67.CrossRef Duijts SF, Kant IJ, Swaen GM: Advantages and disadvantages of an objective selection process for early intervention in employees at risk for sickness absence. BMC Publ Health. 2007, 7: 67-10.1186/1471-2458-7-67.CrossRef
17.
go back to reference Roelen CA, van Rhenen W, Bültmann U, Groothoff JW, van der Klink JJ, Heymans MW: The development and validation of two prediction models to identify employees with high sickness absence. Eur J Public Health. 2013, 23: 128-133. 10.1093/eurpub/cks036.CrossRefPubMed Roelen CA, van Rhenen W, Bültmann U, Groothoff JW, van der Klink JJ, Heymans MW: The development and validation of two prediction models to identify employees with high sickness absence. Eur J Public Health. 2013, 23: 128-133. 10.1093/eurpub/cks036.CrossRefPubMed
18.
go back to reference Justice AC, Covinsky KE, Berlin JA: Assessing the generalizability of prognostic information. Ann Intern Med. 1999, 30: 515-524.CrossRef Justice AC, Covinsky KE, Berlin JA: Assessing the generalizability of prognostic information. Ann Intern Med. 1999, 30: 515-524.CrossRef
19.
go back to reference Babyak MA: What you see may not be what you get: a brief nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004, 66: 411-421. 10.1097/01.psy.0000127692.23278.a9.PubMed Babyak MA: What you see may not be what you get: a brief nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004, 66: 411-421. 10.1097/01.psy.0000127692.23278.a9.PubMed
20.
go back to reference McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS: Users’ guide to the medical literature XXII: How to use articles about clinical decision rules. J Am Med Assoc. 2000, 284: 79-84. 10.1001/jama.284.1.79.CrossRef McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS: Users’ guide to the medical literature XXII: How to use articles about clinical decision rules. J Am Med Assoc. 2000, 284: 79-84. 10.1001/jama.284.1.79.CrossRef
21.
go back to reference Allebeck P, Mastekaasa A: Risk factors for sick leave – general studies. Scand J Public Health. 2004, 32 (suppl63): 49-108.CrossRef Allebeck P, Mastekaasa A: Risk factors for sick leave – general studies. Scand J Public Health. 2004, 32 (suppl63): 49-108.CrossRef
22.
go back to reference Bekker MH, Rutte CG, van Rijswijk K: Sickness absence: a gender focused review. Psychol Health Med. 2009, 14: 405-418. 10.1080/13548500903012830.CrossRefPubMed Bekker MH, Rutte CG, van Rijswijk K: Sickness absence: a gender focused review. Psychol Health Med. 2009, 14: 405-418. 10.1080/13548500903012830.CrossRefPubMed
23.
24.
go back to reference Harrell FE: Regression modelling strategies: applications to linear models, logistic regression, and survival analysis. 2001, New York: SpringerCrossRef Harrell FE: Regression modelling strategies: applications to linear models, logistic regression, and survival analysis. 2001, New York: SpringerCrossRef
25.
26.
go back to reference Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina NJ, Kattan MW: Assessing the performance of prediction models. Epidemiology. 2010, 21: 128-138. 10.1097/EDE.0b013e3181c30fb2.CrossRefPubMedPubMedCentral Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina NJ, Kattan MW: Assessing the performance of prediction models. Epidemiology. 2010, 21: 128-138. 10.1097/EDE.0b013e3181c30fb2.CrossRefPubMedPubMedCentral
27.
go back to reference Fad J, Upadhye S, Worster A: Understanding receiver operating characteristic (ROC) curves. Can J Emerg Med. 2006, 8: 19-20. Fad J, Upadhye S, Worster A: Understanding receiver operating characteristic (ROC) curves. Can J Emerg Med. 2006, 8: 19-20.
28.
go back to reference Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD: Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004, 23: 2567-2586. 10.1002/sim.1844.CrossRefPubMed Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD: Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004, 23: 2567-2586. 10.1002/sim.1844.CrossRefPubMed
30.
go back to reference Terrin N, Schmid CH, Griffith JL, D’Agostino RB, Selker HP: External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks. J Clin Epidemiol. 2003, 56: 721-729. 10.1016/S0895-4356(03)00120-3.CrossRefPubMed Terrin N, Schmid CH, Griffith JL, D’Agostino RB, Selker HP: External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks. J Clin Epidemiol. 2003, 56: 721-729. 10.1016/S0895-4356(03)00120-3.CrossRefPubMed
31.
go back to reference Bleeker SE, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, Moons KE: External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003, 56: 826-832. 10.1016/S0895-4356(03)00207-5.CrossRefPubMed Bleeker SE, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, Moons KE: External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003, 56: 826-832. 10.1016/S0895-4356(03)00207-5.CrossRefPubMed
32.
go back to reference Heinzl H, Waldhör T, Mittlböck M: Careful use of pseudo R-squared measures in epidemiological studies. Stat Med. 2005, 24: 2867-2872. 10.1002/sim.2168.CrossRefPubMed Heinzl H, Waldhör T, Mittlböck M: Careful use of pseudo R-squared measures in epidemiological studies. Stat Med. 2005, 24: 2867-2872. 10.1002/sim.2168.CrossRefPubMed
33.
go back to reference Mavaddat N, Kinmonth AL, Sanderson S, Surtees P, Bingham S, Khaw KT: What determines self-rated health (SRH)? A cross-sectional study of SF-36 health domains in the EPIC-Norfolk Cohort. J Epidemiol Community Health. 2010, 65: 800-806.CrossRefPubMed Mavaddat N, Kinmonth AL, Sanderson S, Surtees P, Bingham S, Khaw KT: What determines self-rated health (SRH)? A cross-sectional study of SF-36 health domains in the EPIC-Norfolk Cohort. J Epidemiol Community Health. 2010, 65: 800-806.CrossRefPubMed
34.
go back to reference Duijts SF, Kant I, van den Brandt PA, Swaen GM: Effectiveness of a preventive coaching intervention for employees at risk for sickness absence due to psychosocial health complaints: results of a randomized controlled trial. J Occup Environ Med. 2008, 50: 765-776. 10.1097/JOM.0b013e3181651584.CrossRefPubMed Duijts SF, Kant I, van den Brandt PA, Swaen GM: Effectiveness of a preventive coaching intervention for employees at risk for sickness absence due to psychosocial health complaints: results of a randomized controlled trial. J Occup Environ Med. 2008, 50: 765-776. 10.1097/JOM.0b013e3181651584.CrossRefPubMed
35.
go back to reference Virtanen M, Kivimäki M, Vahtera J, Elovainio M, Sund R, Virtanen P, Ferrie JE: Sickness absence as a risk factor for job termination, unemployment, and disability pension among temporary and permanent employees. Occup Environ Med. 2006, 63: 212-217. 10.1136/oem.2005.020297.CrossRefPubMedPubMedCentral Virtanen M, Kivimäki M, Vahtera J, Elovainio M, Sund R, Virtanen P, Ferrie JE: Sickness absence as a risk factor for job termination, unemployment, and disability pension among temporary and permanent employees. Occup Environ Med. 2006, 63: 212-217. 10.1136/oem.2005.020297.CrossRefPubMedPubMedCentral
36.
go back to reference Koopmans PC, Roelen CA, Groothoff JW: Frequent and long-term sickness absence as a risk factor for work disability and job termination among employees in the private sector. Occup Environ Med. 2008, 65: 494-499. 10.1136/oem.2007.034322.CrossRefPubMed Koopmans PC, Roelen CA, Groothoff JW: Frequent and long-term sickness absence as a risk factor for work disability and job termination among employees in the private sector. Occup Environ Med. 2008, 65: 494-499. 10.1136/oem.2007.034322.CrossRefPubMed
37.
go back to reference Koopmans PC, Roelen CAM, Groothoff JW: Risk of future sickness absence in frequent and long-term absentees. Occup Med. 2008, 58: 268-274. 10.1093/occmed/kqn040.CrossRef Koopmans PC, Roelen CAM, Groothoff JW: Risk of future sickness absence in frequent and long-term absentees. Occup Med. 2008, 58: 268-274. 10.1093/occmed/kqn040.CrossRef
Metadata
Title
External validation of two prediction models identifying employees at risk of high sickness absence: cohort study with 1-year follow-up
Authors
Corné AM Roelen
Ute Bültmann
Willem van Rhenen
Jac JL van der Klink
Jos WR Twisk
Martijn W Heymans
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Public Health / Issue 1/2013
Electronic ISSN: 1471-2458
DOI
https://doi.org/10.1186/1471-2458-13-105

Other articles of this Issue 1/2013

BMC Public Health 1/2013 Go to the issue