Measuring the well-being of health care professionals in the Punjab: a psychometric evaluation of the Warwick–Edinburgh Mental Well-being Scale in a Pakistani population

Questionnaire and instrument

The questionnaire consisted of two sections. The first section recorded information related to respondents’ demographics and profession, and the second section assessed mental well-being levels with the WEMWBS.

The WEMWBS was developed and extensively validated in the UK general population (Tennant et al., 2007). This self-administered questionnaire comprises 14 positively-worded items that assess eudaimonic (focused on psychological functioning and a sense of meaning and purpose in life) and hedonic (focused on happiness, contentment and life satisfaction) constructs of mental well-being. Responses are recorded on a 5-point Likert scale ranging from “none of the time” (score 1) to “all the time” (score 5). For analysis, the total score is obtained by summing the scores for all of the items. There is no reverse coding. Higher scores indicate greater well-being. The lowest possible score is 14 (low well-being) and the highest possible score is 70 (high well-being).

Use in the UK and in other countries (Castellví et al., 2014) showed that the scale has good psychometric properties, and is feasible, reliable, and sensitive to change. Cronbach’s alpha for the scale lies between 0.87 and 0.91, and 1-week test-retest reliability studies reported very good correlation with r = 0.83. Validation studies have been done in different population samples in the UK, such as Scottish undergraduate and postgraduate students, the general population (Tennant et al., 2007), English and Scottish teenagers (Clarke et al., 2011), and Pakistani and Chinese ethnic minorities (Taggart et al., 2013). The WEMWBS has also demonstrated high internal consistency, good content and face validity, normal response distribution and no floor or ceiling effects in previous validation studies (Tennant et al., 2007; Clarke et al., 2011; Taggart et al., 2013)

Pilot survey

A pilot study was conducted in which a convenience sample of 50 health care professionals including doctors, nurses and dentists in the Combined Military Hospitals in Lahore completed and commented on the WEMWBS to tell us whether the original English-language version could be easily understood and completed. Positive feedback from these participants indicated that the WEMWBS was very easy to understand. We therefore felt it was not necessary to translate this instrument into Urdu (the participants’ first language).

Sample size calculation

Sample size is generally calculated on the basis of variability in the sample and the expected effect size. Usually both of these are unknown before starting the study, and numbers are estimated from previous studies, or sometimes with “rules of thumb” based on previous experience. Most studies of questionnaires validation in social sciences use 5 to 10 respondents per questionnaire item for factor analysis. Factor analyses are subject to the strength of communalities and factor loading, so these statistical tests should instead be based on large samples for reliable results (MacCallum et al., 1999). Comrey and Lee recommend at least 500 cases, and according to their rating scale a sample size of 1,000 or more is considered excellent for factor analysis studies (Lee, 1992). We therefore aimed for a sample size of at least 1,000 HCPs.

Procedure

This cross-sectional survey was carried out from June 2013 to December 2014. The study sample consisted of doctors, nurses, physiotherapists, pharmacists and dentists practicing either in hospital-based or private settings in various cities of the province of Punjab, Pakistan. Seven districts—Lahore, Faisalabad, Gujrat, Multan, Rawalpindi, Islamabad and Sheikhupura—were included in the survey for a study sample representative of the Punjabi population. However, due to resource limitations, we could not obtain a systematic random sample. Data were therefore collected with a convenience sampling approach. Sixteen medical students (currently in their final year of the MBBS degree program) approached HCPs in all available clinical settings in the Punjab province. An independent sample of 50 HCPs from CMH Lahore participated in the test-retest assessment of WEMWBS and completed the questionnaire twice, with a test-retest interval of 7 days.

Ethical approval was obtained from the Ethical Review Committee of CMH Lahore Medical College, Lahore Cantt, Pakistan. Written informed consent was provided by all the study participants. The respondents were ensured anonymity and informed that only group-level findings would be reported.

Statistical analysis

All data were analyzed with SPSS v.21 (IBM Chicago, IL, USA). Frequencies and descriptive statistics were calculated for demographic variables and total scores on the WEMWBS. Histograms and quantile–quantile (Q–Q) plots were visualized to check the assumption of normality for WEMWBS scores, and we also looked for floor and ceiling effects in the response distribution. Factor analysis with principal component analysis (PCA) and quartimax rotation were done to evaluate the construct validity of the WEMWBS. Prior to PCA, its suitability was assessed with the following criteria: correlation coefficient >0.3 for all variables, an overall Kaiser–Meyer–Olkin (KMO) measure greater than 0.6, and a statistically significant Barlett’s test of sphericity (P < 0.05). The number of components to retain was determined with Cartell’s scree plot, eigenvalues >1, interpretability criterion, the amount of variance explained and reliability analysis.

Only those statements were included that had a factor loading >0.3. Cronbach’s alpha coefficient was calculated to estimate the internal consistency of WEMWBS in our study sample, and a value of 0.70–0.9 was considered acceptable (Streiner, 2008). Item-total correlations were calculated with Pearson’s product moment correlation coefficient, and values that were substantial but not excessive (greater than or equal to 0.2 and less than 0.8) were sought (Streiner, 2008). Test-retest reliability was evaluated as the intraclass correlation coefficient with a two-factor mixed effects model and type consistency.

Floor and ceiling effects were sought by examining response patterns for each of the factors derived from factor analysis. Scores were graphed as a histogram, and the distribution of scores was inspected; the percentage of individuals with the lowest and highest possible score in each of the factors was recorded, and values greater than 20% were considered as floor and ceiling effects.

Associations between participant characteristics and WEMWBS scores were examined with independent sample t-tests. Although focus groups have been used to evaluate participants’ comprehension and ease of completion (Taggart et al., 2013), previous studies of the WEMBWS have not examined its readability. Therefore we used the Flesch Reading Ease score and Flesch–Kincaid Grade Level function to determine the readability of the questionnaire (Flesch, 1948).

Response rate

The total response rate was 90.7% (1271/1400).

Demographics

1,271 individuals responded to the questionnaire. Respondent characteristics are summarized in Table 1. Participants’ ages ranged between 20 and 68 years, with a mean age (SD) of 31.8 (9.4) years. Mean age was greater for men (34.1 (10.3) years) than women (30.0 (8.1) years). Most of the respondents were doctors (n = 840, 66%) or nurses (n = 218, 17.2%), female (n = 720, 57%), and practicing in Lahore (n = 70%).

There were some differences in respondent age and gender distribution in relation to profession: nurses were almost all (97%) female, whereas among doctors and dentists there was a nearly equal gender representation. 42 of the pharmacists were male (79%), as were 16 (64%) of the physiotherapists. There were no significant age differences between the professional groups except for dentists in the sample, whose mean age was younger than for the other professions (28.5 (6.6) years; P < 0.001).

Descriptive statistics and floor or ceiling effects

Mean (SD) score on the WEMBWS was 48.1 (9.4), and median score was 48. Analysis of item response frequencies showed little evidence of highly skewed distributions, with a mild skewness of −0.31 (0.07) and a kurtosis of 0.42 (0.14). Visual inspection of the histograms and Q-Q plots did not reveal any significant deviation of response distribution from normality. All response categories were used by at least one person for all the scale items. Mean values for individual items ranged from 2.88 (1.1) for “I’ve had energy to spare” to 3.74 (0.99) for “I’ve been feeling confident” (Table 2). A histogram of the distribution of mean scores is shown in Fig. 1. Based on these results, we conclude that the WEMWBS is valid for the Pakistani population is free from any floor or ceiling effects.

As in the UK validation study (Tennant et al., 2007) mean WEMWBS scores were significantly higher for men (49.16) than for women (47.30) (t = 3.486, df = 1,269, P = 0.001) (Table 1).

There were differences in well-being scores between professional groups. Doctors’ and nurses’ mean WEMWBS scores were lower than those for the other professional groups. The difference between WEMWBS score for doctors and nurses combined, compared to the other professional groups combined, was statistically significant (P = 0.002, mean difference 2.22, t = − 3.14, df = 1,269). A difference in well-being score was evident across age groups, with participants aged 30 years and older scoring higher on the WEMWBS than younger participants (P = 0.013, mean difference 1.33, t = 2.48, df = 1,269).

Items 2, 10, 11 and 12 received the most positive ratings by both men and women, whereas items 3, 4 and 5 received the least positive ratings by participants of both genders.

Factor analysis with principal components analysis

The WEMWBS responses were subjected to PCA, and the correlation matrix showed that all variables had at least one correlation coefficient greater than 0.3. The overall KMO measure was 0.83, with individual KMO measures all greater than 0.7, thus indicating a “middling” to “meritorious” classification. Bartlett’s test of sphericity was statistically significant (P < 0.0005), indicating that the data were suitable for factor analysis. Our PCA identified two components with eigenvalues greater than 1, and which explained 42.1% and 8.5% of the total variance. Eigenvalues, visual inspection of Cattell’s scree plot, interpretability criteria and Cronbach’s alpha value indicated that one component should be retained. A single-component solution explained 42.1% of the total variance. Total score was obtained by summing all items. Factor loadings of the rotated component solution for all 14 items of the WEMWBS based on eigenvalues greater than 1 are shown in Table 3. The scree plot is illustrated in Fig. 2.

For interested readers, the results of exploratory factor analysis with the principal axis factoring method are provided as Table S1.

Internal consistency and content validity

The WEMWBS consists of 14 items. Cronbach’s alpha for single-factor structure of the WEMWBS was 0.89. Corrected item correlations for all items were greater than 0.3, thus denoting the same construct for all items. Item-total statistics for all items are summarized in Table 4. To further analyze internal consistency, we calculated item-total score correlations for each item, with adjustment for overlap. These correlations ranged between 0.46 and 0.68, which are substantial but not excessive values. The item-total statistics for all items in the WEMWBS are shown in Table 4.

Test-retest reliability

The test-retest reliability evaluated as the intraclass correlation coefficient was 0.921, with a 95% confidence interval of 0.861–0.955, thus indicating a very high degree of reliability.

Readability

The values for Flesch reading ease and FK grade level score were calculated in Microsoft Word 2013. Flesch reading ease score for the WEMWBS was 73.5, and FK grade level score was 4.4, which indicate that the WEMWBS was easy for our participants to read and understand.

Recommendations

The WEMWBS is particularly useful for monitoring mental well-being levels in HCPs and medical students who work in highly demanding roles and environments. Further studies are required to develop and validate an Urdu version of this scale in the general community. Along with general studies of the range of factors that are associated with mental health and well-being, the WEMWBS may also be useful to evaluate the efficacy of stress reduction programs such as mindfulness programs, relaxation exercises, cognitive therapies, and social support and group therapies which have proved useful in academic and hospital-based settings (Shapiro, Shapiro & Schwartz, 2000).