Skip to main content
Top
Published in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie 11/2018

01-11-2018 | Reports of Original Investigations

Developing and validating a tool for measuring the educational environment in clinical anesthesia

Authors: Navdeep S. Sidhu, MBChB, FANZCA, MClinEd, FAcadMEd, Eleri Clissold, MBBS

Published in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie | Issue 11/2018

Login to get access

Abstract

Purpose

We aimed to develop a contemporary measure for anesthesia teaching and learning in the operating theatre that was applicable to a variety of training jurisdictions, the Measure for the Anaesthesia Theatre Educational Environment (MATE).

Methods

A systematic review of the literature and modified Delphi approach was used to identify items for content validity. Reliability and exploratory factor analyses were conducted after a pilot survey of trainees to show construct validity, with removal of redundant items. Item domains were identified through a global assessment of factor structure accuracy and relation to real-world constructs.

Results

Literature review generated an initial 73-item list. A modified Delphi approach with 24 experts identified 44 relevant items. The pilot survey generated 390 responses. Reliability analysis, exploratory factor analysis, and global assessment refined the measure to 33 items. Four domains were identified according to factor structure: teaching preparation and practice, assessment and feedback, procedures and responsibility, and overall atmosphere. The educational environment was rated by trainees at 74.6 ± 15.6% with excellent internal consistency (Cronbach’s α = 0.975).

Conclusion

The MATE survey tool generated valid and reliable scores when measuring the educational environment in the operating theatre. Further research is required to investigate possible differences between the training countries and age of junior doctors and the associated underlying factors. Other researchers are invited to administer the survey and share results within a central database.
Appendix
Available only for authorised users
Literature
2.
go back to reference Soemantri D, Herrera C, Riquelme A. Measuring the educational environment in health professions studies: a systematic review. Med Teach 2010; 32: 947-52.CrossRef Soemantri D, Herrera C, Riquelme A. Measuring the educational environment in health professions studies: a systematic review. Med Teach 2010; 32: 947-52.CrossRef
3.
go back to reference Holt MC, Roff S. Development and validation of the Anaesthetic Theatre Educational Environment Measure (ATEEM). Med Teach 2004; 26: 553-8.CrossRef Holt MC, Roff S. Development and validation of the Anaesthetic Theatre Educational Environment Measure (ATEEM). Med Teach 2004; 26: 553-8.CrossRef
4.
go back to reference Smith NA, Castanelli DJ. Measuring the clinical learning environment in anaesthesia. Anaesth Intensive Care 2015; 43: 199-203.PubMed Smith NA, Castanelli DJ. Measuring the clinical learning environment in anaesthesia. Anaesth Intensive Care 2015; 43: 199-203.PubMed
5.
go back to reference Castanelli DJ, Smith NA. Measuring the anaesthesia clinical learning environment at the department level is feasible and reliable. Br J Anaesth 2017; 118: 733-9.CrossRef Castanelli DJ, Smith NA. Measuring the anaesthesia clinical learning environment at the department level is feasible and reliable. Br J Anaesth 2017; 118: 733-9.CrossRef
6.
go back to reference Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000; 32: 1008-15.PubMed Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000; 32: 1008-15.PubMed
7.
go back to reference Clayton MJ. Delphi: a technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol 1997; 17: 373-86.CrossRef Clayton MJ. Delphi: a technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol 1997; 17: 373-86.CrossRef
8.
go back to reference Beavers AS, Lounsbury JW, Richards JK, Huck SW, Skolits GJ, Esquivel SL. Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research & Evaluation 2013; 18: 1-13. Available from URL: http://pareonline.net/pdf/v18n6.pdf (accessed May 2018). Beavers AS, Lounsbury JW, Richards JK, Huck SW, Skolits GJ, Esquivel SL. Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research & Evaluation 2013; 18: 1-13. Available from URL: http://​pareonline.​net/​pdf/​v18n6.​pdf (accessed May 2018).
10.
go back to reference Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods 1999; 4: 272-99.CrossRef Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods 1999; 4: 272-99.CrossRef
11.
go back to reference Cleave-Hogg D, Benedict C. Characteristics of good anaesthesia teachers. Can J Anaesth 1997; 44: 587-91.CrossRef Cleave-Hogg D, Benedict C. Characteristics of good anaesthesia teachers. Can J Anaesth 1997; 44: 587-91.CrossRef
12.
go back to reference Ortwein H, Blaum WE, Spies CD. Anesthesiology residents’ perspective about good teaching - a qualitative needs assessment. Ger Med Sci 2014; 12: Doc05. Ortwein H, Blaum WE, Spies CD. Anesthesiology residents’ perspective about good teaching - a qualitative needs assessment. Ger Med Sci 2014; 12: Doc05.
13.
go back to reference Haydar B, Charnin J, Voepel-Lewis T, Baker K. Resident characterization of better-than- and worse-than-average clinical teaching. Anesthesiology 2014; 120: 120-8.CrossRef Haydar B, Charnin J, Voepel-Lewis T, Baker K. Resident characterization of better-than- and worse-than-average clinical teaching. Anesthesiology 2014; 120: 120-8.CrossRef
14.
go back to reference Lombarts KM, Bucx MJ, Arah OA. Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology 2009; 111: 709-16.CrossRef Lombarts KM, Bucx MJ, Arah OA. Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology 2009; 111: 709-16.CrossRef
15.
go back to reference de Oliveira Filho GR, Dal Mago AJ, Garcia JM, Goldschmidt R. An instrument designed for faculty supervision evaluation by anesthesia residents and its psychometric properties. Anesth Analg 2008; 107: 1316-22.CrossRef de Oliveira Filho GR, Dal Mago AJ, Garcia JM, Goldschmidt R. An instrument designed for faculty supervision evaluation by anesthesia residents and its psychometric properties. Anesth Analg 2008; 107: 1316-22.CrossRef
16.
go back to reference Moors G, Kieruj ND, Vermunt JK. The effect of labeling and numbering of response scales on the likelihood of response bias. Sociol Methodol 2014; 44: 369-99.CrossRef Moors G, Kieruj ND, Vermunt JK. The effect of labeling and numbering of response scales on the likelihood of response bias. Sociol Methodol 2014; 44: 369-99.CrossRef
17.
go back to reference Alwin DF, Krosnick JA. The reliability of survey attitude measurement: the influence of question and respondent attributes. Sociol Methods Res 1991; 20: 139-81.CrossRef Alwin DF, Krosnick JA. The reliability of survey attitude measurement: the influence of question and respondent attributes. Sociol Methods Res 1991; 20: 139-81.CrossRef
18.
go back to reference Saris WE, Gallhofer IN. Design, Evaluation, and Analysis of Questionnaires for Survey Research. Hoboken, NJ: John Wiley & Sons; 2007.CrossRef Saris WE, Gallhofer IN. Design, Evaluation, and Analysis of Questionnaires for Survey Research. Hoboken, NJ: John Wiley & Sons; 2007.CrossRef
19.
go back to reference Kanashiro J, McAleer S, Roff S. Assessing the educational environment in the operating room - a measure of resident perception at one Canadian institution. Surgery 2006; 139: 150-8.CrossRef Kanashiro J, McAleer S, Roff S. Assessing the educational environment in the operating room - a measure of resident perception at one Canadian institution. Surgery 2006; 139: 150-8.CrossRef
20.
go back to reference Mahoney A, Crowe PJ, Harris P. Exploring Australasian surgical trainees’ satisfaction with operating theatre learning using the ‘surgical theatre educational environment measure’. ANZ J Surg 2010; 80: 884-9.CrossRef Mahoney A, Crowe PJ, Harris P. Exploring Australasian surgical trainees’ satisfaction with operating theatre learning using the ‘surgical theatre educational environment measure’. ANZ J Surg 2010; 80: 884-9.CrossRef
21.
go back to reference Yin T, Child S. The Auckland Surgical Theatre Educational Environment Measure: does attending surgery benefit house officers? N Z Med J 2015; 128: 94-8.PubMed Yin T, Child S. The Auckland Surgical Theatre Educational Environment Measure: does attending surgery benefit house officers? N Z Med J 2015; 128: 94-8.PubMed
22.
go back to reference Binsaleh S, Babaeer A, Rabah D, Madbouly K. Evaluation of urology residents’ perception of surgical theater educational environment. J Surg Educ 2014; 72: 73-9.CrossRef Binsaleh S, Babaeer A, Rabah D, Madbouly K. Evaluation of urology residents’ perception of surgical theater educational environment. J Surg Educ 2014; 72: 73-9.CrossRef
23.
go back to reference Palmgren PJ, Sundberg T, Laksov KB. Reassessing the educational environment among undergraduate students in a chiropractic training institution: a study over time. J Chiropr Educ 2015; 29: 110-26.CrossRef Palmgren PJ, Sundberg T, Laksov KB. Reassessing the educational environment among undergraduate students in a chiropractic training institution: a study over time. J Chiropr Educ 2015; 29: 110-26.CrossRef
24.
go back to reference Wong PN, John DN, Deslandes RE, Hughes ML. Same syllabus, different country - using DREEM to compare the educational environments at two Pharmacy schools. Pharm Educ 2015; 15: 87-92. Wong PN, John DN, Deslandes RE, Hughes ML. Same syllabus, different country - using DREEM to compare the educational environments at two Pharmacy schools. Pharm Educ 2015; 15: 87-92.
25.
go back to reference Leung Y, Salfinger S, Mercer A. The positive impact of structured teaching in the operating room. Aust NZ J Obstet Gynaecol 2015; 55: 601-5.CrossRef Leung Y, Salfinger S, Mercer A. The positive impact of structured teaching in the operating room. Aust NZ J Obstet Gynaecol 2015; 55: 601-5.CrossRef
26.
go back to reference Finn Y, Avalos G, Dunne F. Positive changes in the medical educational environment following introduction of a new systems-based curriculum: DREEM or reality? Curricular change and the environment. Ir J Med Sci 2014; 183: 253-8.CrossRef Finn Y, Avalos G, Dunne F. Positive changes in the medical educational environment following introduction of a new systems-based curriculum: DREEM or reality? Curricular change and the environment. Ir J Med Sci 2014; 183: 253-8.CrossRef
27.
go back to reference Shankar PR, Bharti R, Ramireddy R, Balasubramanium R, Nuguri V. Students’ perception of the learning environment at Xavier University School of Medicine, Aruba: a follow-up study. J Educ Eval Health Prof 2014; 11: 9.CrossRef Shankar PR, Bharti R, Ramireddy R, Balasubramanium R, Nuguri V. Students’ perception of the learning environment at Xavier University School of Medicine, Aruba: a follow-up study. J Educ Eval Health Prof 2014; 11: 9.CrossRef
28.
go back to reference Qin Y, Wang Y, Floden RE. The effect of problem-based learning on improvement of the medical educational environment: a systematic review and meta-analysis. Med Princ Pract 2016; 25: 525-32.CrossRef Qin Y, Wang Y, Floden RE. The effect of problem-based learning on improvement of the medical educational environment: a systematic review and meta-analysis. Med Princ Pract 2016; 25: 525-32.CrossRef
29.
go back to reference Chan CY, Sum MY, Lim WS, Chew NW, Samarasekera DD, Sim K. Adoption and correlates of Postgraduate Hospital Educational Environment Measure (PHEEM) in the evaluation of learning environments - a systematic review. Med Teach 2016; 38: 1248-55.CrossRef Chan CY, Sum MY, Lim WS, Chew NW, Samarasekera DD, Sim K. Adoption and correlates of Postgraduate Hospital Educational Environment Measure (PHEEM) in the evaluation of learning environments - a systematic review. Med Teach 2016; 38: 1248-55.CrossRef
30.
go back to reference Shimizu T, Tsugawa Y, Tanoue Y, et al. The hospital educational environment and performance of residents in the General Medicine In Training Examination: a multicenter study in Japan. Int J Gen Med 2013; 6: 637-40.CrossRef Shimizu T, Tsugawa Y, Tanoue Y, et al. The hospital educational environment and performance of residents in the General Medicine In Training Examination: a multicenter study in Japan. Int J Gen Med 2013; 6: 637-40.CrossRef
31.
go back to reference Al-Ansari AA, El Tantawi MM. Predicting academic performance of dental students using perception of educational environment. J Dent Educ 2015; 79: 337-44.PubMed Al-Ansari AA, El Tantawi MM. Predicting academic performance of dental students using perception of educational environment. J Dent Educ 2015; 79: 337-44.PubMed
32.
go back to reference Sarwar S, Tarique S. Perception of educational environment: does it impact academic performance of medical students? J Pak Med Assoc 2016; 66: 1210-4.PubMed Sarwar S, Tarique S. Perception of educational environment: does it impact academic performance of medical students? J Pak Med Assoc 2016; 66: 1210-4.PubMed
33.
go back to reference Payne LK, Glaspie T. Associations between baccalaureate nursing students’ perceptions of educational environment and HESI™ scores and GPA. Nurse Educ Today 2014; 34: e64-8.CrossRef Payne LK, Glaspie T. Associations between baccalaureate nursing students’ perceptions of educational environment and HESI™ scores and GPA. Nurse Educ Today 2014; 34: e64-8.CrossRef
34.
go back to reference Tempski P, Santos IS, Mayer FB, et al. Relationship among medical student resilience, educational environment and quality of life. PLoS One 2015; 10: e0131535.CrossRef Tempski P, Santos IS, Mayer FB, et al. Relationship among medical student resilience, educational environment and quality of life. PLoS One 2015; 10: e0131535.CrossRef
Metadata
Title
Developing and validating a tool for measuring the educational environment in clinical anesthesia
Authors
Navdeep S. Sidhu, MBChB, FANZCA, MClinEd, FAcadMEd
Eleri Clissold, MBBS
Publication date
01-11-2018
Publisher
Springer US
Published in
Canadian Journal of Anesthesia/Journal canadien d'anesthésie / Issue 11/2018
Print ISSN: 0832-610X
Electronic ISSN: 1496-8975
DOI
https://doi.org/10.1007/s12630-018-1185-0

Other articles of this Issue 11/2018

Canadian Journal of Anesthesia/Journal canadien d'anesthésie 11/2018 Go to the issue