Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Health and Quality of Life Outcomes
Published in: Health and Quality of Life Outcomes 1/2020

Open Access 01-12-2020 | Research

Measuring stress in Australia: validation of the perceived stress scale (PSS-14) in a national sample

Authors: Pedro H. Ribeiro Santiago, Tine Nielsen, Lisa Gaye Smithers, Rachel Roberts, Lisa Jamieson

Published in: Health and Quality of Life Outcomes | Issue 1/2020

Login to get access

Abstract

Background

In Australia, the stress levels have increased over the years, impacting on the physical and mental health of the general population. The aim of the present study was to evaluate the validity and reliability of the PSS-14 in an Australian population.

Methods

The PSS-14 was applied to a large national sample comprising 3857 Australians in the population-based cross-sectional study Australia’s National Survey of Adult Oral Health 2004–2006. The psychometric properties analyzed with the Rasch model and Graphical Log-linear Rasch models were: model fit, item fit, local dependence, differential item functioning, unidimensionality, reliability, targeting and criterion validity.

Results

The PSS-14 did not fit the pure RM (χ2 (55) = 3828.3, p = < 0.001) and the unidimensionality of the whole scale was rejected (p = < 0.001). The Perceived Stress (χ2 (27) = 1409.7, p = < 0.001) and Perceived Control (χ2 (27) = 713.4, p = < 0.001) subscales did not fit the pure RM. After the deletion of two items, the Perceived Stress subscale (χ2 (96) = 94.4, p = 0.440) fitted a GLLRM, while the Perceived Control scale (χ2 (55) = 62.50, p = 0.224) fitted a GLLRM after the exclusion of four misfitting items.

Conclusions

The Perceived Stress subscale displayed adequate psychometric properties after the deletion of two items; however, the majority of problems centered around the Perceived Control subscale. The presence of differential item functioning among four items indicates that adjustment of total scores is required to avoid measurement bias. Recommendations for future applications in Australia are provided.
Appendix
Available only for authorised users
Footnotes
1
Technical and Further Education (or TAFE) is the biggest provider of post-secondary education in Australia. TAFE offers a broad range of courses, at the operative, trade and paraprofessional level, that can last from a few hours (refreshment courses) to three years (diploma courses). Unlike universities, which are composed mostly of full-time students, TAFE institutions allow students to combine study and work, and encourage programs of apprenticeships and traineeships [73].
 
2
For simplicity, the term uniform is omitted when referred to uniform LD or uniform DIF from now on.
 
Literature
1.
Casey L, Mathews R. Stress and wellbeing in Australia in 2011: a state-of-the-nation survey. InPsych Bull Aust Psychol Soc. 2011;33(6):32.
2.
Australian Psychological Society. Stress & wellbeing: how Australians are coping with life: the findings of the Australian Psychological Society stress and wellbeing in Australia survey 2015. Melbourne: Australian Psychological Society; 2015.
3.
Casey L, Liang RP-T. Stress and wellbeing in Australia survey 2014: Australian Psychological Society; 2014.
4.
Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24(4):385.PubMedCrossRef
5.
Taylor JM. Psychometric analysis of the ten-item perceived stress scale. Psychol Assess. 2015;27(1):90.PubMedCrossRef
6.
Manzar MD, Salahuddin M, Peter S, Alghadir A, Anwer S, Bahammam AS, et al. Psychometric properties of the perceived stress scale in Ethiopian university students. BMC Public Health. 2019;19(1):41.PubMedPubMedCentralCrossRef
7.
Kechter A, Black DS, Riggs NR, Warren CM, Ritt-Olson A, Chou C-P, et al. Factors in the perceived stress scale differentially associate with mindfulness disposition and executive function among early adolescents. J Child Fam Stud. 2019;28(3):1–8.
8.
Lazarus RS. Psychological stress and the coping process. New York: McGraw-Hill; 1966.
9.
Cohen S, Janicki-Deverts D, Miller GE. Psychological stress and disease. JAMA. 2007;298(14):1685–7.PubMedCrossRef
10.
Harkness KL, Monroe SM. The assessment and measurement of adult life stress: basic premises, operational principles, and design requirements. J Abnorm Psychol. 2016;125(5):727.PubMedCrossRef
11.
Holmes TH, Rahe RH. The social readjustment rating scale. J Psychosom Res. 1967;11:213–8.
12.
Cohen S, Williamson G. Psychological stress in a probability sample of the United States. In: Spacapan S, Oskamp S, editors. The social psychology of health: Claremont symposium on applied social psychology. Newbury Park: Sage Publications; 1988. p. 31–7.
13.
14.
Lee E-H. Review of the psychometric evidence of the perceived stress scale. Asian Nurs Res (Korean Soc Nurs Sci). 2012;6(4):121–7.
15.
Leung DY, T-h L, Chan SS. Three versions of perceived stress scale: validation in a sample of Chinese cardiac patients who smoke. BMC Public Health. 2010;10(1):513.PubMedPubMedCentralCrossRef
16.
Wang Z, Chen J, Boyd JE, Zhang H, Jia X, Qiu J, et al. Psychometric properties of the Chinese version of the perceived stress scale in policewomen. PLoS One. 2011;6(12):e28610.PubMedPubMedCentralCrossRef
17.
18.
Patterson PG, Smith T. Modeling relationship strength across service types in an eastern culture. Int J Serv Ind Manag. 2001;12(2):90–113.CrossRef
19.
Andreou E, Alexopoulos EC, Lionis C, Varvogli L, Gnardellis C, Chrousos GP, et al. Perceived stress scale: reliability and validity study in Greece. Int J Environ Res Public Health. 2011;8(8):3287–98.PubMedPubMedCentralCrossRef
20.
Dudin MN, Gayduk VI, Sekerin VD, Bank SV, Gorohova AE. Financial crisis in Greece: challenges and threats for the global economy. Int J Econ Financ Issues. 2016;6(5S):1–6.
21.
Pong RW, DesMeules M, Lagacé C. Rural–urban disparities in health: how does Canada fare and how does Canada compare with Australia? Aust J Rural Health. 2009;17(1):58–64.PubMedCrossRef
22.
Hewitt PL, Flett GL, Mosher SW. The perceived stress scale: factor structure and relation to depression symptoms in a psychiatric sample. J Psychopathol Behav Assess. 1992;14(3):247–57.CrossRef
23.
Karam F, Bérard A, Sheehy O, Huneau MC, Briggs G, Chambers C, et al. Reliability and validity of the 4-item perceived stress scale among pregnant women: results from the OTIS antidepressants study. Res Nurs Health. 2012;35(4):363–75.PubMedCrossRef
24.
World Bank. World Bank country and lending groups. Washington (DC): World Bank Data Help Desk; 2017.
25.
Archibald L. Decolonization and healing: indigenous experiences in the United States, New Zealand, Australia and Greenland. Ottawa: Aboriginal Healing Foundation; 2006.
26.
Papanicolas I, Woskie LR, Jha AK. Health care spending in the United States and other high-income countries. JAMA. 2018;319(10):1024–39.PubMedCrossRef
27.
WHO. United States of America: WHO statistical profile 2015.
28.
WHO. Australia: WHO statistical profile 2015.
29.
American Psychological Association. Stress in America: Paying with our health. 2015.
30.
Garvey D. Review of the social and emotional wellbeing of Indigenous Australian peoples; 2008.
31.
Kowal E, Gunthorpe W, Bailie RS. Measuring emotional and social wellbeing in aboriginal and Torres Strait islander populations: an analysis of a negative life events scale. Int J Equity Health. 2007;6(1):18.PubMedPubMedCentralCrossRef
32.
Santiago PHR, Roberts R, Smithers LG, Jamieson L. Stress beyond coping? A Rasch analysis of the perceived stress scale (PSS-14) in an aboriginal population. PLoS One. 2019;14(5):e0216333.PubMedPubMedCentralCrossRef
33.
Lavoie JA, Douglas KS. The perceived stress scale: evaluating configural, metric and scalar invariance across mental health status and gender. J Psychopathol Behav Assess. 2012;34(1):48–57.CrossRef
34.
35.
Lakey B, Cohen S. Social support and theory. Social support measurement and intervention: A guide for health and social scientists, vol. 29; 2000.
36.
Thoits PA. Stress, coping, and social support processes: where are we? What next? J Health Soc Behav. 1995:53–79.
37.
Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of perceived social support. J Pers Assess. 1988;52(1):30–41.CrossRef
38.
Cohen, Wills TA. Stress, social support, and the buffering hypothesis. Psychol Bull. 1985;98(2):310.PubMedCrossRef
39.
Siegrist J. Adverse health effects of high-effort/low-reward conditions. J Occup Health Psychol. 1996;1(1):27.PubMedCrossRef
40.
Siegrist J, Wege N, Pühlhofer F, Wahrendorf M. A short generic measure of work stress in the era of globalization: effort–reward imbalance. Int Arch Occup Environ Health. 2009;82(8):1005.PubMedCrossRef
41.
Webster C. Effects of researcher presence and appeal on response quality in hand-delivered, self-administered surveys. J Bus Res. 1997;38(2):105–14.CrossRef
42.
Sanders AE, Slade GD. Gender modifies effect of perceived stress on orofacial pain symptoms: National Survey of Adult Oral Health. J Orofac Pain. 2011;25(4):317–26.
43.
Baruch Y, Holtom BC. Survey response rate levels and trends in organizational research. Hum Relat. 2008;61(8):1139–60.CrossRef
44.
Slade GD, Spencer AJ, Roberts-Thomson KF. Australia’s dental generations. Natl Surv Adult Oral Health. 2004;6(2007):274.
45.
46.
47.
Gitchel WD, Roessler RT, Turner RC. Gender effect according to item directionality on the perceived stress scale for adults with multiple sclerosis. Rehabil Couns Bull. 2011;55(1):20–8.CrossRef
48.
Nielsen T, Dammeyer J. Measuring higher education students’ perceived stress: an IRT-based construct validity study of the PSS-10. Stud Educ Eval. 2019;63:17–25.CrossRef
49.
Sharp LK, Kimmel LG, Kee R, Saltoun C, Chang C-H. Assessing the perceived stress scale for African American adults with asthma and low literacy. J Asthma. 2007;44(4):311–6.PubMedCrossRef
50.
Medvedev ON, Krägeloh CU, Hill EM, Billington R, Siegert RJ, Webster CS, et al. Rasch analysis of the perceived stress scale: transformation from an ordinal to a linear measure. J Health Psychol. 2017;24(8):1359105316689603.
51.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.CrossRef
52.
Novick MR, Lewis C. Coefficient alpha and the reliability of composite measurements. Psychometrika. 1967;32(1):1–13.PubMedCrossRef
53.
Zimet GD, Powell SS, Farley GK, Werkman S, Berkoff KA. Psychometric characteristics of the multidimensional scale of perceived social support. J Pers Assess. 1990;55(3–4):610–7.PubMedCrossRef
54.
Christensen KB, Kreiner S, Mesbah M. Rasch models in health. London: Wiley; 2013.
55.
Andersen EB. Asymptotic properties of conditional maximum-likelihood estimators. J R Stat Soc Ser B (Methodological). 1970;32(2):283–301.
56.
Fisher WP. Rasch, Frisch, and two fishers: a social history of the econometric origins of some widely used psychometric models; 2010.
57.
Rasch G. On specific objectivity: an attempt of formalizing the generality and validity of scientific statements; 1997.
58.
59.
Kreiner S, Christensen KB. Validity and objectivity in health-related scales: Analysis by graphical loglinear Rasch models. Multivariate and mixture distribution Rasch models. New York: Springer; 2007. p. 329–46.
60.
Kreiner S, Christensen KB. Item screening in graphical loglinear Rasch models. Psychometrika. 2011;76(2):228–56.CrossRef
61.
Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–5.
62.
Messick S. Validity. ETS Res Rep Ser. 1987;1987(2):i–208.
63.
Kelderman H, Rijkes CP. Loglinear multidimensional IRT models for polytomously scored items. Psychometrika. 1994;59(2):149–76.CrossRef
64.
Kelderman H. Loglinear Rasch model tests. Psychometrika. 1984;49(2):223–45.CrossRef
65.
Whittaker J. Graphical models in applied multivariate statistics. Chichester: Wiley publishing; 2009.
66.
Masters GN. A Rasch model for partial credit scoring. Psychometrika. 1982;47(2):149–74.CrossRef
67.
Warm TA. Weighted likelihood estimation of ability in item response theory. Psychometrika. 1989;54(3):427–50.CrossRef
68.
Graham JW. Missing data analysis: making it work in the real world. Annu Rev Psychol. 2009;60:549–76.PubMedCrossRef
69.
Kreiner S, Nielsen T. Item Analysis in Digram-Notes on the use of DIGRAM for item analysis by graphical loglinear Rasch models. Copenhagen: Department of Biostatistics-University of Copenhagen; 2013.
70.
Kreiner S. Introduction to DIGRAM. Denmark: Department of Biostatistics, University of Copenhagen Copenhagen; 2003.
71.
R Core Team. R: A language and environment for statistical computing. 2013.
72.
Andersen EB. A goodness of fit test for the Rasch model. Psychometrika. 1973;38(1):123–40.CrossRef
73.
Goozee G. The development of TAFE in Australia. Adelaide: National Centre for Vocational Education Research; 2001.
74.
75.
Goodman LA, Kruskal WH. Measures of association for cross classifications. J Am Stat Assoc. 1954;49(268):732–64.
76.
Kreiner S, Christensen KB. Analysis of local dependence and multidimensionality in graphical loglinear Rasch models. Commun Stat Theory Methods. 2004;33(6):1239–76.CrossRef
77.
Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B (Methodological). 1995;57(1):289–300.
78.
79.
Hamon A, Mesbah M. Questionnaire reliability under the Rasch model. Statistical methods for quality of life studies. Boston: Springer; 2002. p. 155–68.
80.
Nielsen JB, Kyvsgaard JN, Sildorf SM, Kreiner S, Svensson J. Item analysis using Rasch models confirms that the Danish versions of the DISABKIDS® chronic-generic and diabetes-specific modules are valid and reliable. Health Qual Life Outcomes. 2017;15(1):44.PubMedPubMedCentralCrossRef
81.
Kendall SM. Rank correlation. Van Nostrand's Scientific Encyclopedia; 1948.
82.
Davidson M. Known-groups validity. Encyclopedia of quality of life and well-being research. Dordrecht: Springer; 2014. p. 3481–2.
83.
Faro A. Confirmatory factor analysis of three versions of the perceived stress scale (PSS): a population-based study. Psicologia Reflexão e Crítica. 2015;28(1):21–30.CrossRef
84.
Wu SM, Amtmann D. Psychometric evaluation of the perceived stress scale in multiple sclerosis. ISRN Rehabil. 2013;2013:1–9.
85.
Ezzati A, Jiang J, Katz MJ, Sliwinski MJ, Zimmerman ME, Lipton RB. Validation of the perceived stress scale in a community sample of older adults. Int J Geriatr Psychiatry. 2014;29(6):645–52.PubMedCrossRef
86.
Pedrero Pérez EJ, Olivar AÁ. Estrés percibido en adictos a sustancias en tratamiento mediante la escala de Cohen: propiedades psicométricas y resultados de su aplicación. Anales de Psicología. 2010;26(2):302–9.
87.
de la Rubia JM, de León FC. VALIDACIÓN DE LA ESCALA DE ESTRÉS PERCIBIDO (PSS-14) EN LA POBLACIÓN DE DENTISTAS COLEGIADOS DE MONTERREY. Ansiedad Y Estrés. 2014;20:193–209.
88.
Ramírez MTG, Hernández RL. Factor structure of the perceived stress scale (PSS) in a sample from Mexico. Span J Psychol. 2007;10(1):199–206.PubMedCrossRef
89.
Yokokura AVCP, Silva AAM, Fernandes JKB, Del-Ben CM, Figueiredo FP, Barbieri MA, et al. Perceived Stress Scale: confirmatory factor analysis of the PSS14 and PSS10 versions in two samples of pregnant women from the BRISA cohort. Cad Saude Publica. 2017;33:e00184615.PubMedCrossRef
90.
Nielsen T, Skogstad S, Dammeyer J, editors. The Perceived Stress Scale ia higher education context: a Rasch-based construct validity study: Book of Abstracts page 95. Lisbon: 14th European Conference on Psychological Assessment; 2017.
91.
Fleishman JA, Spector WD, Altman BM. Impact of differential item functioning on age and gender differences in functional disability. J Gerontol Ser B Psychol Sci Soc Sci. 2002;57(5):S275–S84.
92.
Shevlin M, Bailey F, Adamson G. Examining the factor structure and sources of differential functioning of the Eysenck personality questionnaire revised—abbreviated. Pers Individ Dif. 2002;32(3):479–87.CrossRef
93.
Lange R, Thalbourne MA, Houran J, Lester D. Depressive response sets due to gender and culture-based differential item functioning. Pers Individ Dif. 2002;33(6):937–54.CrossRef
94.
Matud MP. Gender differences in stress and coping styles. Pers Individ Dif. 2004;37(7):1401–15.CrossRef
95.
Rosario M, Shinn M, Mørch H, Huckabee CB. Gender differences in coping and social supports: testing socialization and role constraint theories. J Community Psychol. 1988;16(1):55–69.CrossRef
96.
Smith LL, Reise SP. Gender differences on negative affectivity: an IRT study of differential item functioning on the multidimensional personality questionnaire stress reaction scale. J Pers Soc Psychol. 1998;75(5):1350.PubMedCrossRef
97.
Van Dam NT, Earleywine M, Forsyth JP. Gender bias in the sixteen-item anxiety sensitivity index: an application of polytomous differential item functioning. J Anxiety Disord. 2009;23(2):256–9.PubMedCrossRef
98.
Real T. I don't want to talk about it: overcoming the secret legacy of male depression: Simon and Schuster; 1998.
99.
Strother E, Lemberg R, Stanford SC, Turberville D. Eating disorders in men: underdiagnosed, undertreated, and misunderstood. Eat Disord. 2012;20(5):346–55.PubMedPubMedCentralCrossRef
100.
Marais I, Andrich D. Formalizing dimension and response violations of local independence in the unidimensional Rasch model. J Appl Meas. 2008;9(3):200–15.PubMed
101.
Andrich D, Kreiner S. Quantifying response dependence between two dichotomous items using the Rasch model. Appl Psychol Meas. 2010;34(3):181–92.CrossRef
102.
Perera MJ, Brintz CE, Birnbaum-Weitzman O, Penedo FJ, Gallo LC, Gonzalez P, et al. Factor structure of the perceived stress Scale-10 (PSS) across English and Spanish language responders in the HCHS/SOL sociocultural ancillary study. Psychol Assess. 2017;29(3):320.PubMedCrossRef
103.
Benoy C, Knitter B, Knellwolf L, Doering S, Klotsche J, Gloster AT. Assessing psychological flexibility: validation of the open and engaged state questionnaire. J Contextual Behav Sci. 2018;12:253–60.
104.
Mondo M, Sechi C, Cabras C. Psychometric evaluation of three versions of the Italian perceived stress scale. Curr Psychol. 2019:1–9.
105.
Örücü MÇ, Demir A. Psychometric evaluation of perceived stress scale for Turkish university students. Stress Health. 2009;25(1):103–9.CrossRef
106.
Nunnally JC, Bernstein IH. Psychometric theory: McGraw-Hill New York; 1967.
107.
Bond LA. Norm-and criterion-referenced testing. Pract Assess Res Eval. 1996;5(2):120–5.
108.
Engelhard G, Engelhard E, Rabbitt M. Measurement of household food insecurity: two decades of invariant measurement. Rasch Meas Trans. 2016;30(3):1598–9.
109.
Olsbjerg M, Christensen KB. Marginal and conditional approaches to longitudinal Rasch models. Pub Inst Stat Univ Paris. 2013;57:1–2.
110.
Richiardi L, Pizzi C, Pearce N. Commentary: representativeness is usually not necessary and often should be avoided. Int J Epidemiol. 2013;42(4):1018–22.PubMedCrossRef
111.
Metadata
Title
Measuring stress in Australia: validation of the perceived stress scale (PSS-14) in a national sample
Authors
Pedro H. Ribeiro Santiago
Tine Nielsen
Lisa Gaye Smithers
Rachel Roberts
Lisa Jamieson
Publication date
01-12-2020
Publisher
BioMed Central
Published in
Health and Quality of Life Outcomes / Issue 1/2020
Electronic ISSN: 1477-7525
DOI
https://doi.org/10.1186/s12955-020-01343-x

Other articles of this Issue 1/2020

Health and Quality of Life Outcomes 1/2020 Go to the issue

Letter to the Editor

Methodological and interpretive concerns about Beemster et al.’s article ‘The interpretation of change score of the pain disability index after vocational rehabilitation is baseline dependent’: a letter to the editor

Research

Assessing the effect of child’s gender on their father–mother perception of the PedsQL™ 4.0 questionnaire: an iterative hybrid ordinal logistic regression/item response theory approach with Monte Carlo simulation

Research

Development and use of health outcome descriptors: a guideline development case study

Research

Quality of life impact of refractive correction (QIRC) results three years after SMILE and FS-LASIK

Research

Evaluating prevalence of depression, anxiety and hopelessness in patients with Vitiligo on an Iranian population

Research

Mediating role of occupational stress and job satisfaction on the relationship between neuroticism and quality of life among Chinese civil servants: a structural equation model