Top

International Archives of Occupational and Environmental Health

Published in:

Open Access 01-04-2021 | Affective Disorder | Original Article

Description and application of a method to quantify criterion-related cut-off values for questionnaire-based psychosocial risk assessment

Authors: Mathias Diebig, Peter Angerer

Published in: International Archives of Occupational and Environmental Health | Issue 3/2021

Abstract

Purpose

The psychosocial risk assessment is a systematic intervention process for organizations that aims at improving psychosocial working conditions as well as employee health. Based on a screening of working conditions, interventions to reduce risk factors are implemented and evaluated. What is missing for most screening instruments however are cut-off values to categorize working conditions into uncritical vs. critical, whereas the latter indicates an elevated risk for illness. To estimate and evaluate cut-off values, two studies were conducted using the receiver operating characteristic (ROC) analysis.

Methods

In Study 1, a sample of 229 participants answered an online survey on depression (PHQ) and psychosocial working conditions using a questionnaire (DYNAMIK) that covers five factors important to workers' health: workload, boundary permeability, participation, leader support, and usability. Using the ROC analysis, criterion-related cut-off values were generated to predict depressive symptoms. In Study 2, these cut-off values were used to classify working conditions in the two categories of ‘critical’ and ‘uncritical’ in an independent sample (N = 295). It was tested for differences in the results of the two groups concerning the direct criterion of depressive symptoms and the indirect criterion of effort-reward imbalance.

Results

In Study 1, cut-off values differed between the five scales and showed different values for sensitivity and specificity. In Study 2, participants exposed to critical working conditions reported more depressive symptoms as well as an effort-reward imbalance.

Conclusions

Cut-off values are useful to identify working conditions as either critical or uncritical. This knowledge is important when deciding which working conditions should be optimized within the context of psychosocial risk assessment.

Available only for authorised users

Altman DG, Bland JM (1994) Diagnostic tests 3: receiver operating characteristic plots. Br Med J 309:188CrossRef

Amstad FT, Meier LL, Fasel U, Elfering A, Semmer NK (2011) A meta-analysis of work-family conflict and various outcomes with a special emphasis on cross-domain versus matching-domain relations. J Occup Health Psychol 16:151–169. https://doi.org/10.1037/a0022170CrossRef

Barth J, Schumacher M, Herrmann-Lingen C (2004) Depression as a risk factor for mortality in patients with coronary heart disease: a meta-analysis. Psychosom Med 66:802–813. https://doi.org/10.1097/01.psy.0000146332.53619.b2CrossRef

Beck D, Lenhardt U (2019) Consideration of psychosocial factors in workplace risk assessments: findings from a company survey in Germany. Int Arch Occup Environ Health 92:435–451. https://doi.org/10.1007/s00420-019-01416-5CrossRef

Behar E, Alcaine O, Zuellig AR, Borkovec TD (2003) Screening for generalized anxiety disorder using the Penn State Worry Questionnaire: a receiver operating characteristic analysis. J Behav Ther Exp Psychol 34:25–43. https://doi.org/10.1016/S0005-7916(03)00004-1CrossRef

Bonde JPE (2008) Psychosocial factors at work and risk of depression: a systematic review of the epidemiological evidence. Occup Environ Med 65:438–445. https://doi.org/10.1136/oem.2007.038430CrossRef

Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Erlbaum, Hillsdale

Diebig M, Jungmann F, Müller A, Wulf IC (2018) Inhalts- und prozessbezogene Anforderungen an die Gefährdungsbeurteilung psychischer Belastung im Kontext Industrie 4.0. Z Arb Organ 62:53–67. https://doi.org/10.1026/0932-4089/a000265CrossRef

Diebig M, Dragano N, Körner U, Lunau T, Wulf IC, Angerer P (2020) Development and validation of a questionnaire to measure psychosocial work stressors in modern working environments. J Occup Environ Med 62:185–193. https://doi.org/10.1097/JOM.0000000000001779CrossRef

Dragano N, Siegrist J, Nyberg ST, Lunau T, Fransson EI, Alfredsson L, Bjorner JB, Borritz M, Burr H, Erbel R, Fahlén G, Goldberg M, Hamer M, Heikkilä K, Jöckel K-H, Knutsson A, Madsen IEH, Nielsen ML, Nordin M, Oksanen T, Pejtersen JH, Pentti J, Rugulies R, Salo P, Schupp J, Singh-Manoux A, Steptoe A, Theorell T, Vahtera J, Westerholm PJM, Westerlund H, Virtanen M, Zins M, Batty GD, Kivimäki M (2017) Effort-reward imbalance at work and incident coronary heart disease: a multicohort study of 90,164 individuals. Epidemiology 28:619–626. https://doi.org/10.1097/EDE.0000000000000666CrossRef

Eddy P, Heckenberg R, Wertheim EH, Kent S, Wright BJ (2016) A systematic review and meta-analysis of the effort-reward imbalance model of workplace stress with indicators of immune function. J Psychosom Res 91:1–8. https://doi.org/10.1016/j.jpsychores.2016.10.003CrossRef

Eddy P, Wertheim EH, Kingsley M, Wright BJ (2017) Associations between the effort-reward imbalance model of workplace stress and indices of cardiovascular health: a systematic review and meta-analysis. Neurosci Biobehav Res 83:252–266. https://doi.org/10.1016/j.neubiorev.2017.10.025CrossRef

Eng J (2005) Receiver operating characteristic analysis: a primer. Acad Radiol 12:909–916. https://doi.org/10.1016/j.acra.2005.04.005CrossRef

European Agency for Safety and Health at Work (2014) E-guide to managing psychosocial risks. European Communities, Luxembourg

Follmer KB, Jones KS (2018) Mental illness in the workplace: an interdisciplinary review and organizational research agenda. J Manag 44:325–351. https://doi.org/10.1177/0149206317741194CrossRef

Ganster DC, Rosen CC (2013) Work stress and employee health: a multidisciplinary review. J Manag 39:1085–1122. https://doi.org/10.1177/0149206313475815CrossRef

Hapke U, Cohrdes C, Nübel J (2019) Depressive symptoms in a European comparison—results from the European Health Interview Survey (EHIS) 2. J Health Monit 4:57–64. https://doi.org/10.25646/6223CrossRef

Juvani A, La Oksanen T, Virtanen M, Salo P, Pentti J, Kivimäki M, Vahtera J (2018) Clustering of job strain, effort-reward imbalance, and organizational injustice and the risk of work disability: a cohort study. Scand J Work Environ Health 44:485–495. https://doi.org/10.5271/sjweh.3736CrossRef

Karasek RA (1979) Job demands, job decision latitude, and mental strain: Implications for job redesign. Admin Sci Q 24:285–308. https://doi.org/10.2307/2392498CrossRef

Kivimäki M, Virtanen M, Elovainio M, Kouvonen A, Väänänen A, Vahtera J (2006) Work stress in the etiology of coronary heart disease—a meta-analysis. Scand J Work Environ Health 32:431–442CrossRef

Kivimäki M, Pentti J, Ferrie JE, Batty GD, Nyberg ST, Jokela M, Virtanen M, Alfredsson L, Dragano N, Fransson EI, Goldberg M, Knutsson A, Koskenvuo M, Koskinen A, Kouvonen A, Luukkonen R, Oksanen T, Rugulies R, Siegrist J, Singh-Manoux A, Suominen S, Theorell T, Väänänen A, Vahtera J, Westerholm PJM, Westerlund H, Zins M, Strandberg T, Steptoe A, Deanfield J (2018) Work stress and risk of death in men and women with and without cardiometabolic disease: a multicohort study. Lancet Diabetes Endocrinol 6:705–713. https://doi.org/10.1016/S2213-8587(18)30140-2CrossRef

Kocalevent R-D, Hinz A, Brähler E (2013) Standardization of the depression screener patient health questionnaire (PHQ-9) in the general population. Gen Hosp Psychiatry 35:551–555. https://doi.org/10.1016/j.genhosppsych.2013.04.006CrossRef

Kroenke K, Spitzer RL, Williams JB (2001) The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med 16:606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.xCrossRef

Kroenke K, Spitzer RL, Williams JB (2003) The Patient Health Questionnaire-2: validity of a two-item depression screener. Med Care 41:1284–1292. https://doi.org/10.1097/01.MLR.0000093487.78664.3CCrossRef

Kroenke K, Spitzer RL, Williams JB, Löwe B (2010) The patient health questionnaire somatic, anxiety, and depressive symptom scales: a systematic review. Gen Hosp Psychiatry 32:345–359. https://doi.org/10.1016/j.genhosppsych.2010.03.006CrossRef

Lehr D, Koch S, Hillert A (2010) Where is (im)balance? Necessity and construction of evaluated cut-off points for effort-reward imbalance and overcommitment. J Occup Organ Psychol 83:251–261. https://doi.org/10.1348/096317909X406772CrossRef

Leka S, Jain A, Iavicoli S, Di Tecco C (2015) An evaluation of the policy context on psychosocial risks and mental health in the workplace in the European Union: achievements, challenges, and the future. Biomed Res Int 2015:1–18. https://doi.org/10.1155/2015/213089CrossRef

Lopez AD, Mathers CD, Ezzati M, Jamison DT, Murray CJL (2006) Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data. Lancet 367:1747–1757. https://doi.org/10.1016/S0140-6736(06)68770-9CrossRef

Löwe B, Kroenke K, Herzog W, Gräfe K (2004) Measuring depression outcome with a brief self-report instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9). J Affect Disord 81:61–66. https://doi.org/10.1016/S0165-0327(03)00198-8CrossRef

Manea L, Gilbody S, Hewitt C, North A, Plummer F, Richardson R, Thombs BD, Williams B, McMillan D (2016) Identifying depression with the PHQ-2: a diagnostic meta-analysis. J Affect Disord 203:382–395. https://doi.org/10.1016/j.jad.2016.06.003CrossRef

Nielsen K, Randall R, Holten A-L, González ER (2010) Conducting organizational-level occupational health interventions: what works? Work Stress 24:234–259. https://doi.org/10.1080/02678373.2010.515393CrossRef

Podsakoff PM, MacKenzie SB, Lee J-Y, Podsakoff NP (2003) Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol 88:879–903. https://doi.org/10.1037/0021-9010.88.5.879CrossRef

R Core Team (2018) R: a language and environment for statistical computing. R foundation for statistical computing, Vienna, Austria. https://www.R-project.org/

Richiardi L, Pizzi C, Pearce N (2013) Commentary: representativeness is usually not necessary and often should be avoided. Int J Epidemiol 42:1018–1022. https://doi.org/10.1093/ije/dyt103CrossRef

Rick J, Briner RB (2000) Psychosocial risk assessment: problems and prospects. Occup Med-C 50:310–314. https://doi.org/10.1093/occmed/50.5.310CrossRef

Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Müller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform 12:77. https://doi.org/10.1186/1471-2105-12-77CrossRef

Rothman KJ, Gallacher JEJ, Hatch EE (2013) Why representativeness should be avoided. Int J Epidemiol 42:1012–1014. https://doi.org/10.1093/ije/dys223CrossRef

Rugulies R, Aust B, Madsen IEH (2017) Effort–reward imbalance at work and risk of depressive disorders. A systematic review and meta-analysis of prospective cohort studies. Scand J Work Environ Health 43:294–306CrossRef

Schmidt S, Roesler U, Kusserow T, Rau R (2014) Uncertainty in the workplace: examining role ambiguity and role conflict, and their link to depression—a meta-analysis. Eur J Work Organ Psychol 23:91–106. https://doi.org/10.1080/1359432X.2012.711523CrossRef

Siegrist J (1996) Adverse health effects of high-effort/low-reward conditions. J Occup Health Psychol 1:27–41. https://doi.org/10.1037/1076-8998.1.1.27CrossRef

Siegrist J, Starke D, Chandola T, Godin I, Marmot M, Niedhammer I, Peter R (2004) The measurement of effort–reward imbalance at work: European comparisons. Soc Sci Med 58:1483–1499. https://doi.org/10.1016/S0277-9536(03)00351-4CrossRef

Siegrist J, Wege N, Pühlhofer F, Wahrendorf M (2009) A short generic measure of work stress in the era of globalization: effort-reward imbalance. Int Arch Occup Environ Health 82:1005–1013. https://doi.org/10.1007/s00420-008-0384-3CrossRef

Stansfeld S, Candy B (2006) Psychosocial work environment and mental health—a meta-analytic review. Scand J Work Environ Health 32:443–462CrossRef

Swets JA (1988) Measuring the accuracy of diagnostic systems. Science 240:1285–1293. https://doi.org/10.1126/science.3287615CrossRef

Zweig MH, Campbell G (1993) Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 39:561–577. https://doi.org/10.1093/clinchem/39.4.561CrossRef

Title: Description and application of a method to quantify criterion-related cut-off values for questionnaire-based psychosocial risk assessment
Authors: Mathias Diebig
Peter Angerer
Publication date: 01-04-2021
Publisher: Springer Berlin Heidelberg
Keywords: Affective Disorder
Affective Disorder
Published in: International Archives of Occupational and Environmental Health / Issue 3/2021
Print ISSN: 0340-0131
Electronic ISSN: 1432-1246
DOI: https://doi.org/10.1007/s00420-020-01597-4

ACC 2024 Congress

Springer Medicine

Description and application of a method to quantify criterion-related cut-off values for questionnaire-based psychosocial risk assessment

Abstract

Purpose

Methods

Results

Conclusions

ACC 2024 Congress

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 3/2021

SARS-CoV-2: an occupational hazard presenting challenges for low- and middle-income countries

Physical activity and ability to meet different work requirements among adult working men with or without current depressive symptoms

Correction to: The association between occupational categories and grip strength in Korean male workers

Associations of sustained smoking and smoking cessation with work-related outcomes: a longitudinal analysis

Whole body vibration influence on bus fare collectors evaluated by using a brain training mobile app

The association between occupational categories and grip strength in Korean male workers