Skip to main content
Top
Published in: Surgical Endoscopy 3/2016

01-03-2016

Development and implementation of the Structured Training Trainer Assessment Report (STTAR) in the English National Training Programme for laparoscopic colorectal surgery

Authors: Susannah M. Wyles, Danilo Miskovic, Zhifang Ni, Ara W. Darzi, Roland M. Valori, Mark G. Coleman, George B. Hanna

Published in: Surgical Endoscopy | Issue 3/2016

Login to get access

Abstract

Background

There is a lack of educational tools available for surgical teaching critique, particularly for advanced laparoscopic surgery. The aim was to develop and implement a tool that assesses training quality and structures feedback for trainers in the English National Training Programme for laparoscopic colorectal surgery.

Methods

Semi-structured interviews were performed and analysed, and items were extracted. Through the Delphi process, essential items pertaining to desirable trainer characteristics, training structure and feedback were determined. An assessment tool (Structured Training Trainer Assessment Report—STTAR) was developed and tested for feasibility, acceptability and educational impact.

Results

Interview transcripts (29 surgical trainers, 10 trainees, four educationalists) were analysed, and item lists created and distributed for consensus opinion (11 trainers and seven trainees). The STTAR consisted of 64 factors, and its web-based version, the mini-STTAR, included 21 factors that were categorised into four groups (training structure, training behaviour, trainer attributes and role modelling) and structured around a training session timeline (beginning, middle and end). The STTAR (six trainers, 48 different assessments) demonstrated good internal consistency (α = 0.88) and inter-rater reliability (ICC = 0.75). The mini-STTAR demonstrated good inter-item reliability (α = 0.79) and intra-observer reliability on comparison of 85 different trainer/trainee combinations (r = 0.701, p = <0.001). Both were found to be feasible and acceptable. The educational report for trainers was found to be useful (4.4 out of 5).

Conclusions

An assessment tool that evaluates training quality was developed and shown to be reliable, acceptable and of educational value. It has been successfully implemented into the English National Training Programme for laparoscopic colorectal surgery.
Appendix
Available only for authorised users
Literature
1.
go back to reference Birch DW, Bonjer HJ, Crossley C et al (2009) Canadian consensus conference on the development of training and practice standards in advanced minimally invasive surgery: Edmonton, Alta., Jun. 1, 2007. Can J Surg 52(4):321–327 Birch DW, Bonjer HJ, Crossley C et al (2009) Canadian consensus conference on the development of training and practice standards in advanced minimally invasive surgery: Edmonton, Alta., Jun. 1, 2007. Can J Surg 52(4):321–327
2.
go back to reference Blue AV, Griffith CH 3rd, Wilson J et al (1999) Surgical teaching quality makes a difference. Am J Surg 177(1):86–89CrossRefPubMed Blue AV, Griffith CH 3rd, Wilson J et al (1999) Surgical teaching quality makes a difference. Am J Surg 177(1):86–89CrossRefPubMed
3.
go back to reference Hull L, Arora S, Aggarwal R et al (2012) The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 214(2):214–230CrossRefPubMed Hull L, Arora S, Aggarwal R et al (2012) The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 214(2):214–230CrossRefPubMed
4.
go back to reference Sevdalis N, Davis R, Koutantji M et al (2008) Reliability of a revised NOTECHS scale for use in surgical teams. Am J Surg 196(2):184–190CrossRefPubMed Sevdalis N, Davis R, Koutantji M et al (2008) Reliability of a revised NOTECHS scale for use in surgical teams. Am J Surg 196(2):184–190CrossRefPubMed
5.
go back to reference Haynes AB, Weiser TG, Berry WR et al (2009) A surgical safety checklist to reduce morbidity and mortality in a global population. New Engl J Med 360(5):491–499CrossRefPubMed Haynes AB, Weiser TG, Berry WR et al (2009) A surgical safety checklist to reduce morbidity and mortality in a global population. New Engl J Med 360(5):491–499CrossRefPubMed
6.
go back to reference Peyton J (1998) Teaching and learning in medical practice. Manticore Europe Limited, Rickmansworth Peyton J (1998) Teaching and learning in medical practice. Manticore Europe Limited, Rickmansworth
7.
go back to reference Skeff KM, Stratos GA, Berman J et al (1992) Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med 152(6):1156–1161CrossRefPubMed Skeff KM, Stratos GA, Berman J et al (1992) Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med 152(6):1156–1161CrossRefPubMed
8.
go back to reference Copertino N, Blackham R, Hamdorf JM (2010) A short course for surgical supervisors and trainers: effecting behavioural change. ANZ J Surg 80(12):896–901CrossRefPubMed Copertino N, Blackham R, Hamdorf JM (2010) A short course for surgical supervisors and trainers: effecting behavioural change. ANZ J Surg 80(12):896–901CrossRefPubMed
10.
go back to reference Copeland HL, Hewson MG (2000) Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Acad Med 75(2):161–166CrossRefPubMed Copeland HL, Hewson MG (2000) Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Acad Med 75(2):161–166CrossRefPubMed
11.
go back to reference Cohen R, MacRae H, Jamieson C (1996) Teaching effectiveness of surgeons. Am J Surg 171(6):612–614CrossRefPubMed Cohen R, MacRae H, Jamieson C (1996) Teaching effectiveness of surgeons. Am J Surg 171(6):612–614CrossRefPubMed
12.
go back to reference Beckman TJ, Ghosh AK, Cook DA et al (2004) How reliable are assessments of clinical teaching? A review of the published instruments. J Gen Intern Med 19(9):971–977PubMedCentralCrossRefPubMed Beckman TJ, Ghosh AK, Cook DA et al (2004) How reliable are assessments of clinical teaching? A review of the published instruments. J Gen Intern Med 19(9):971–977PubMedCentralCrossRefPubMed
13.
go back to reference Miskovic D, Wyles SM, Ni M et al (2010) Systematic review on mentoring and simulation in laparoscopic colorectal surgery. Ann Surg 252(2):943–951CrossRefPubMed Miskovic D, Wyles SM, Ni M et al (2010) Systematic review on mentoring and simulation in laparoscopic colorectal surgery. Ann Surg 252(2):943–951CrossRefPubMed
14.
go back to reference Coleman MG, Hanna GB, Kennedy R (2011) The National Training Programme for laparoscopic colorectal surgery in England: a new training paradigm. Colorectal Dis 13(6):614–616CrossRefPubMed Coleman MG, Hanna GB, Kennedy R (2011) The National Training Programme for laparoscopic colorectal surgery in England: a new training paradigm. Colorectal Dis 13(6):614–616CrossRefPubMed
15.
go back to reference Wyles SM, Miskovic D, Ni M et al (2012) “Trainee” evaluation of the English National Training Programme for laparoscopic colorectal surgery. Colorectal Dis 14(6):352–357CrossRef Wyles SM, Miskovic D, Ni M et al (2012) “Trainee” evaluation of the English National Training Programme for laparoscopic colorectal surgery. Colorectal Dis 14(6):352–357CrossRef
17.
go back to reference Kennedy TJ, Lingard LA (2006) Making sense of grounded theory in medical education. Med Educ 40(2):101–108CrossRefPubMed Kennedy TJ, Lingard LA (2006) Making sense of grounded theory in medical education. Med Educ 40(2):101–108CrossRefPubMed
21.
go back to reference Mackenzie H, Cuming T, Miskovic D et al (2013) Design, delivery, and validation of a trainer curriculum for the national laparoscopic colorectal training program in England. Ann Surg 261(1):149–156CrossRef Mackenzie H, Cuming T, Miskovic D et al (2013) Design, delivery, and validation of a trainer curriculum for the national laparoscopic colorectal training program in England. Ann Surg 261(1):149–156CrossRef
22.
go back to reference Streiner N, Norman G (2008) Health measurement scales: a practical guide to their development and use, 4th edn. Oxford University Press, OxfordCrossRef Streiner N, Norman G (2008) Health measurement scales: a practical guide to their development and use, 4th edn. Oxford University Press, OxfordCrossRef
23.
go back to reference Whyte WF (1982) Interviewing in field research. George Allen and Unwin, London Whyte WF (1982) Interviewing in field research. George Allen and Unwin, London
24.
go back to reference Lingard L, Regehr G, Espin S et al (2006) A theory-based instrument to evaluate team communication in the operating room: balancing measurement authenticity and reliability. Qual Saf Health Care 15(6):422–426PubMedCentralCrossRefPubMed Lingard L, Regehr G, Espin S et al (2006) A theory-based instrument to evaluate team communication in the operating room: balancing measurement authenticity and reliability. Qual Saf Health Care 15(6):422–426PubMedCentralCrossRefPubMed
25.
go back to reference Irby DM (1978) Clinical teacher effectiveness in medicine. J Med Educ 53(10):808–815PubMed Irby DM (1978) Clinical teacher effectiveness in medicine. J Med Educ 53(10):808–815PubMed
26.
go back to reference Nisar PJ, Scott HJ (2011) Key attributes of a modern surgical trainer: perspectives from consultants and trainees in the United kingdom. J Surg Educ 68(3):202–208CrossRefPubMed Nisar PJ, Scott HJ (2011) Key attributes of a modern surgical trainer: perspectives from consultants and trainees in the United kingdom. J Surg Educ 68(3):202–208CrossRefPubMed
27.
28.
go back to reference Vygotsky L (1978) Mind in society. Harvard University Press, Cambridge Vygotsky L (1978) Mind in society. Harvard University Press, Cambridge
29.
go back to reference Miskovic D, Ni M, Wyles SM et al (2012) Learning curve and case selection in laparoscopic colorectal surgery: systematic review and international multicenter analysis of 4852 cases. Dis Colon Rectum 55(12):1300–1310CrossRefPubMed Miskovic D, Ni M, Wyles SM et al (2012) Learning curve and case selection in laparoscopic colorectal surgery: systematic review and international multicenter analysis of 4852 cases. Dis Colon Rectum 55(12):1300–1310CrossRefPubMed
30.
go back to reference Mackenzie H, Miskovic D, Ni M et al (2014) Risk prediction score in laparoscopic colorectal surgery training: experience from the English National Training Program. Ann Surg 261(2):338–344CrossRef Mackenzie H, Miskovic D, Ni M et al (2014) Risk prediction score in laparoscopic colorectal surgery training: experience from the English National Training Program. Ann Surg 261(2):338–344CrossRef
31.
go back to reference Arora S, Aggarwal R, Sevdalis N et al (2010) Development and validation of mental practice as a training strategy for laparoscopic surgery. Surg Endosc 24(1):179–187CrossRefPubMed Arora S, Aggarwal R, Sevdalis N et al (2010) Development and validation of mental practice as a training strategy for laparoscopic surgery. Surg Endosc 24(1):179–187CrossRefPubMed
32.
go back to reference Arora S, Aggarwal R, Sirimanna P et al (2011) Mental practice enhances surgical technical skills: a randomized controlled study. Ann Surg 253(2):265–270CrossRefPubMed Arora S, Aggarwal R, Sirimanna P et al (2011) Mental practice enhances surgical technical skills: a randomized controlled study. Ann Surg 253(2):265–270CrossRefPubMed
Metadata
Title
Development and implementation of the Structured Training Trainer Assessment Report (STTAR) in the English National Training Programme for laparoscopic colorectal surgery
Authors
Susannah M. Wyles
Danilo Miskovic
Zhifang Ni
Ara W. Darzi
Roland M. Valori
Mark G. Coleman
George B. Hanna
Publication date
01-03-2016
Publisher
Springer US
Published in
Surgical Endoscopy / Issue 3/2016
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-015-4281-z

Other articles of this Issue 3/2016

Surgical Endoscopy 3/2016 Go to the issue