Skip to main content
Top
Published in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie 6/2009

01-06-2009 | Reports of Original Investigations

Validation of the Imperial College Surgical Assessment Device (ICSAD) for labour epidural placement

Authors: Megan A. Hayter, MD, Zeev Friedman, MD, M. Dylan Bould, MB, John G. Hanlon, MD, Rita Katznelson, MD, Bruno Borges, MD, Viren N. Naik, MD

Published in: Canadian Journal of Anesthesia/Journal canadien d'anesthésie | Issue 6/2009

Login to get access

Abstract

Purpose

Technical proficiency in anesthesia has historically been determined subjectively. The purpose of this study was to establish the construct validity for the Imperial College Surgical Assessment Device (ICSAD), a measure of hand motion efficiency, as an objective assessment tool for technical skill performance, by examining its ability to distinguish between operators of different levels of experience performing a labour epidural. Concurrent validity for the ICSAD was investigated by comparison to a validated task specific checklist (CL) and global rating scale (GRS).

Methods

A single blinded, prospective, controlled study design compared three groups of subjects: novice residents (<30 epidurals), experienced residents (>100 epidurals), and staff anesthesiologists (>500 epidurals). Performance was measured using the ICSAD (number of movements, path length, time) and scores from a CL and GRS graded by examiners blinded to the level of training. Data were analyzed by multivariate analysis of variance (MANOVA).

Results

Twenty-nine subjects were recruited. Novice residents had longer path lengths compared to experienced residents (P = 0.031) and staff anesthesiologists (P = 0.0004), made more movements (P = 0.012) and took more time than staff (P = 0.009). Novice residents scored significantly worse on the GRS compared to experienced residents (P = 0.029) and staff (P = 0.01) and had significantly lower CL scores compared to staff (P = 0.003).

Conclusions

Construct and concurrent validity for the ICSAD was established for a regional anesthesia technique by demonstrating that it can distinguish between operators of different levels of experience and by comparing it to the current standards of technical skill assessment.
Appendix
Available only for authorised users
Literature
1.
go back to reference Naik VN, Perlas A, Chandra DB, Chung DY, Chan VW. An assessment tool for brachial plexus regional anesthesia performance: establishing construct validity and reliability. Reg Anesth Pain Med 2007; 32: 41–5.PubMed Naik VN, Perlas A, Chandra DB, Chung DY, Chan VW. An assessment tool for brachial plexus regional anesthesia performance: establishing construct validity and reliability. Reg Anesth Pain Med 2007; 32: 41–5.PubMed
2.
go back to reference Friedman Z, Katznelson R, Devito I, Siddiqui M, Chan V. Objective assessment of manual skills and proficiency in performing epidural anesthesia—video-assisted validation. Reg Anesth Pain Med 2006; 31: 304–10.PubMed Friedman Z, Katznelson R, Devito I, Siddiqui M, Chan V. Objective assessment of manual skills and proficiency in performing epidural anesthesia—video-assisted validation. Reg Anesth Pain Med 2006; 31: 304–10.PubMed
3.
go back to reference Naik V, Matsumoto E, Houston P, et al. Learning fiberoptic intubation on a simple model transfers to the O.R. Can J Anesth 2001; 48: A2. abstract.CrossRef Naik V, Matsumoto E, Houston P, et al. Learning fiberoptic intubation on a simple model transfers to the O.R. Can J Anesth 2001; 48: A2. abstract.CrossRef
4.
go back to reference Neequaye SK, Aggarwal R, Van Herzeele I, Darzi A, Cheshire NJ. Endovascular skills training and assessment. J Vasc Surg 2007; 46: 1055–64.PubMedCrossRef Neequaye SK, Aggarwal R, Van Herzeele I, Darzi A, Cheshire NJ. Endovascular skills training and assessment. J Vasc Surg 2007; 46: 1055–64.PubMedCrossRef
5.
go back to reference Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative “bench station” examination. Am J Surg 1997; 173: 226–30.PubMedCrossRef Reznick R, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via an innovative “bench station” examination. Am J Surg 1997; 173: 226–30.PubMedCrossRef
6.
go back to reference Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998; 73: 993–7.PubMedCrossRef Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998; 73: 993–7.PubMedCrossRef
7.
go back to reference Ridgway PF, Ziprin P, Datta VK, et al. Laboratory-based validation of a novel suture technique for wound closure. Ann Plast Surg 2002; 49: 291–6.PubMedCrossRef Ridgway PF, Ziprin P, Datta VK, et al. Laboratory-based validation of a novel suture technique for wound closure. Ann Plast Surg 2002; 49: 291–6.PubMedCrossRef
8.
go back to reference Konrad C, Schupfer G, Wietlisbach M, Gerber H. Learning manual skills in anesthesiology:is there a recommended number of cases for anesthetic procedures? Anesth Analg 1998; 86: 635–9.PubMedCrossRef Konrad C, Schupfer G, Wietlisbach M, Gerber H. Learning manual skills in anesthesiology:is there a recommended number of cases for anesthetic procedures? Anesth Analg 1998; 86: 635–9.PubMedCrossRef
9.
go back to reference Naik VN, Devito I, Halpern SH. Cusum analysis is a useful tool to assess resident proficiency at insertion of labour epidurals. Can J Anesth 2003; 50: 694–8.PubMedCrossRef Naik VN, Devito I, Halpern SH. Cusum analysis is a useful tool to assess resident proficiency at insertion of labour epidurals. Can J Anesth 2003; 50: 694–8.PubMedCrossRef
10.
go back to reference Kopacz DJ, Neal JM, Pollock JE. The regional anesthesia “learning curve”. What is the minimum number of epidural and spinal blocks to reach consistency? Reg Anesth 1996; 21: 182–90.PubMed Kopacz DJ, Neal JM, Pollock JE. The regional anesthesia “learning curve”. What is the minimum number of epidural and spinal blocks to reach consistency? Reg Anesth 1996; 21: 182–90.PubMed
11.
go back to reference Brydges R, Classen R, Larmer J, Xeroulis G, Dubrowski A. Computer-assisted assessment of one-handed knot tying skills performed within various contexts: a construct validity study. Am J Surg 2006; 192: 109–13.PubMedCrossRef Brydges R, Classen R, Larmer J, Xeroulis G, Dubrowski A. Computer-assisted assessment of one-handed knot tying skills performed within various contexts: a construct validity study. Am J Surg 2006; 192: 109–13.PubMedCrossRef
12.
go back to reference Carifio J, Perla R. Resolving the 50-year debate around using and misusing Likert scales. Med Educ 2008; 42: 1150–2.PubMedCrossRef Carifio J, Perla R. Resolving the 50-year debate around using and misusing Likert scales. Med Educ 2008; 42: 1150–2.PubMedCrossRef
13.
go back to reference Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 1977; 33: 363–74.PubMedCrossRef Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 1977; 33: 363–74.PubMedCrossRef
14.
go back to reference Cohen J. Statistical power analysis. Curr Dir Psychol Sci 1992; 1: 98–101.CrossRef Cohen J. Statistical power analysis. Curr Dir Psychol Sci 1992; 1: 98–101.CrossRef
15.
go back to reference Mylopoulos M, Regehr G. Cognitive metaphors of expertise and knowledge: prospects and limitations for medical education. Med Educ 2007; 41: 1159–65.PubMed Mylopoulos M, Regehr G. Cognitive metaphors of expertise and knowledge: prospects and limitations for medical education. Med Educ 2007; 41: 1159–65.PubMed
16.
go back to reference Moulton CA, Regehr G, Mylopoulos M, MacRae HM. Slowing down when you should: a new model of expert judgment. Acad Med 2007; 82: S109–16.PubMedCrossRef Moulton CA, Regehr G, Mylopoulos M, MacRae HM. Slowing down when you should: a new model of expert judgment. Acad Med 2007; 82: S109–16.PubMedCrossRef
17.
go back to reference Watts J, Feldman WB. Assessment of technical skills. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer; 1985. p. 259–74. Watts J, Feldman WB. Assessment of technical skills. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer; 1985. p. 259–74.
18.
go back to reference Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 2001; 193: 479–85.PubMedCrossRef Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 2001; 193: 479–85.PubMedCrossRef
19.
go back to reference Moorthy K, Munz Y, Dosis A, Bello F, Chang A, Darzi A. Bimodal assessment of laparoscopic suturing skills: construct and concurrent validity. Surg Endosc 2004; 18: 1608–12.PubMed Moorthy K, Munz Y, Dosis A, Bello F, Chang A, Darzi A. Bimodal assessment of laparoscopic suturing skills: construct and concurrent validity. Surg Endosc 2004; 18: 1608–12.PubMed
20.
go back to reference Saleh GM, Voyatzis Y, Hance J, Ratnasothy J, Darzi A. Evaluating surgical dexterity during corneal suturing. Arch Ophthalmol 2006; 124: 1263–6.PubMedCrossRef Saleh GM, Voyatzis Y, Hance J, Ratnasothy J, Darzi A. Evaluating surgical dexterity during corneal suturing. Arch Ophthalmol 2006; 124: 1263–6.PubMedCrossRef
21.
go back to reference Smith SG, Torkington J, Brown TJ, Taffinder NJ, Darzi A. Motion analysis. Surg Endosc 2002; 16: 640–5.PubMedCrossRef Smith SG, Torkington J, Brown TJ, Taffinder NJ, Darzi A. Motion analysis. Surg Endosc 2002; 16: 640–5.PubMedCrossRef
22.
go back to reference Borgeat A, Ekatodramis G, Schenker CA. Postoperative nausea and vomiting in regional anesthesia: a review. Anesthesiology 2003; 98: 530–47.PubMedCrossRef Borgeat A, Ekatodramis G, Schenker CA. Postoperative nausea and vomiting in regional anesthesia: a review. Anesthesiology 2003; 98: 530–47.PubMedCrossRef
23.
Metadata
Title
Validation of the Imperial College Surgical Assessment Device (ICSAD) for labour epidural placement
Authors
Megan A. Hayter, MD
Zeev Friedman, MD
M. Dylan Bould, MB
John G. Hanlon, MD
Rita Katznelson, MD
Bruno Borges, MD
Viren N. Naik, MD
Publication date
01-06-2009
Publisher
Springer-Verlag
Published in
Canadian Journal of Anesthesia/Journal canadien d'anesthésie / Issue 6/2009
Print ISSN: 0832-610X
Electronic ISSN: 1496-8975
DOI
https://doi.org/10.1007/s12630-009-9090-1

Other articles of this Issue 6/2009

Canadian Journal of Anesthesia/Journal canadien d'anesthésie 6/2009 Go to the issue