Top

Published in:

01-06-2017 | Original Research

Assessment of Residents Readiness to Perform Lumbar Puncture: A Validation Study

Authors: Mikael Johannes Vuokko Henriksen, MD, Troels Wienecke, MD, PhD, Helle Thagesen, MD, Rikke Vita Borre Jacobsen, MD, PhD, Yousif Subhi, MD, Charlotte Ringsted, MD, PhD, Lars Konge, MD, PhD

Published in: Journal of General Internal Medicine | Issue 6/2017

Abstract

Background

Lumbar puncture is a common procedure in many specialties. The procedure serves to diagnose life-threatening conditions, often requiring rapid performance. However, junior doctors possess uncertainties regarding performing the procedure and frequently perform below expectations. Hence, proper training and assessment of performance is crucial before entering clinical practice.

Objective

To develop and collect validity evidence for an assessment tool for lumbar puncture performance, including a standard to determine when trainees are ready for clinical practice.

Design

Development of a new tool, based on clinician interviews and a literature review, was followed by an explorative study to gather validity evidence.

Participants and Main Measures

We interviewed 12 clinicians from different specialties. The assessment tool was used to assess 11 doctors at the advanced beginners’ level and 18 novices performing the procedure in a simulated, ward-like setting with a standardized patient. Procedural performance was assessed by three content experts. We used generalizability theory to explore reliability. The discriminative ability of the tool was explored by comparing performance scores between the two groups. The contrasting groups method was used to set a pass/fail standard and the consequences of this was explored.

Key results

The interviews identified that in addition to the technical aspects of the procedure, non-technical elements involving planning and conducting the procedure are important. Cronbach’s alpha = 0.92, Generalizability-coefficient was 0.88 and a Decision-study found one rater was sufficient for low-stakes assessments (G-coefficient 0.71). The discriminative ability was confirmed by a significant difference between the mean scores of novices, 40.9 (SD 6.1) and of advanced beginners, 47.8 (SD 4.0), p = 0.004. A standard of 44.0 was established which was consistent with the raters’ global judgments of pass/fail.

Conclusion

We developed and demonstrated strong validity evidence for the lumbar puncture assessment tool. The tool can be used to assess readiness for practice.

Fitch MT, van de Beek D. Emergency diagnosis and treatment of adult meningitis. Lancet Infect Dis. 2007;7(3):191–200.CrossRefPubMed

Martin SCG, Teo MKCH, Young AMH, et al. Defending a traditional practice in the modern era: the use of lumbar puncture in the investigation of subarachnoid haemorrhage. Br J Neurosurg. 2015;29(6):799–803.CrossRefPubMed

Nathan BR, Kincaid O. Does experience doing lumbar punctures result in expertise? A medical maxim bites the dust. Neurology. 2012;79(2):115–116.CrossRefPubMed

Lammers RL, Temple KJ, Wagner MJ, Ray D. Competence of new emergency medicine residents in the performance of lumbar punctures. Acad Emerg Med. 2005;12(7):622–628.CrossRefPubMed

Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents’ lumbar puncture skills. Neurology. 2012;79(2):132–137.CrossRefPubMedPubMedCentral

Kneen R. The role of lumbar puncture in suspected CNS infection--a disappearing skill? Arch Dis Child. 2002;87(3):181–183.CrossRefPubMedPubMedCentral

Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med. 2009;4(7):397–403.CrossRefPubMed

Lineberry M, Soo Park Y, Cook DA, Yudkowsky R. Making the case for mastery learning assessments: key issues in validation and justification. Acad Med. 2015;90(11):1445–1450.CrossRefPubMed

McGaghie WC, Siddall VJ, Mazmanian PE, Myers J. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education: american college of chest physicians evidence-based educational guidelines. Chest. 2009;135(3 Suppl):62S–68S.CrossRefPubMed

10.

Berg K, Riesenberg LA, Berg D, et al. The development of a validated checklist for adult lumbar puncture: preliminary results. Am J Med Qual. 2013;28(4):330–334.CrossRefPubMed

11.

Haji FA, Khan R, Regehr G, Ng G, de Ribaupierre S, Dubrowski A. Operationalising elaboration theory for simulation instruction design: a Delphi study. Med Educ. 2015;49(6):576–588.CrossRefPubMed

12.

Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166. e7-16.CrossRefPubMed

13.

Sullivan ME, Yates KA, Inaba K, Lam L, Clark RE. The use of cognitive task analysis to reveal the instructional limitations of experts in the teaching of procedural skills. Acad Med. 2014;89(5):811–816.CrossRefPubMed

14.

Krippendorff K. Content analysis. 3rd ed. Thousand Oaks, California: SAGE Publications, Inc; 2013.

15.

Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–1288.CrossRefPubMed

16.

Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–115.CrossRefPubMed

17.

Kopacz DJ, Neal JM, Pollock JE. The regional anesthesia “learing curve”: What is the minimum number of epidural and spinal blocks to reach consistancy? Reg Anesth Pain Med. 1996;21(3):182–190.

18.

Subhi Y, Todsen T, Konge L. An integrable, web-based solution for easy assessment of video-recorded performances. Adv Med Educ Pract. 2014;5:103–105.CrossRefPubMedPubMedCentral

19.

Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.CrossRef

20.

Bloch R, Norman G. Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. Med Teach. 2012;34(11):960–992.CrossRefPubMed

21.

Yudkowsky R, Downing SM, Tekian A. Standard setting. In: Downing S, Yudkowsky R, eds. Assesment in health professions education. Secondth ed. New York: Roudledge Taylor and francis Group; 2009:119–148.

22.

Brunckhorst O, Shahid S, Aydin A, et al. The relationship between technical and nontechnical skills within a simulation-based ureteroscopy training environment. J Surg Educ. 2015;72(5):1039–1044.CrossRefPubMed

23.

Ilgen JS, Ma IWY, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49(2):161–173.CrossRefPubMed

24.

Gerard JM, Kessler DO, Braun C, Mehta R, Scalzo AJ, Auerbach M. Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure. Simul Healthc. 2013;8(3):148–154.CrossRefPubMed

25.

Ma IWY, Zalunardo N, Pachev G, et al. Comparing the use of global rating scale with checklists for the assessment of central venous catheterization skills using simulation. Adv Heal Sci Educ. 2012;17(4):457–470.CrossRef

26.

Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37(9):830–837.CrossRefPubMed

27.

Downing SM, Haladyna TM. Validity and its threats. In: Downing S, Yudkowsky R, eds. Assesment In health professions education. firstth ed. New York: Roudledge Taylor and francis Group; 2009:21–55.

28.

Downing SM. Reliability: on the reproducibility of assessment data. Med Educ. 2004;38(9):1006–1012.CrossRefPubMed

29.

Cook DA. Much ado about differences: why expert-novice comparisons add little to the validity argument. September: Adv Health Sci Educ Theory Pract; 2014.

30.

Konge L, Vilmann P, Clementsen P, Annema JT, Ringsted C. Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer. Endoscopy. 2012;44(10):928–933.CrossRefPubMed

31.

Kneebone R, Kidd J, Nestel D, Asvall S, Paraskeva P, Darzi A. An innovative model for teaching and learning clinical procedures. Med Educ. 2002;36(7):628–634.CrossRefPubMed

32.

Hamstra SJ, Brydges R, Hatala R, Zendejas B, Cook DA. Reconsidering fidelity in simulation-based training. Acad Med. 2014;89(3):387–392.CrossRefPubMed

33.

Brydges R, Hatala R, Zendejas B, Erwin PJ, Cook DA. Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis. Acad Med. 2015;90(2):246–256.CrossRefPubMed

34.

Søreide E, Morrison L, Hillman K, et al. The formula for survival in resuscitation. Resuscitation. 2013;84(11):1487–1493.CrossRefPubMed

35.

Tung CE. Education research: changing practice: Residents’ adoption of the atraumatic lumbar puncture needle. Neurology. 2013;80(17):e180–2.CrossRefPubMed

36.

Shaikh F, Brzezinski J, Alexander S, et al. Ultrasound imaging for lumbar punctures and epidural catheterisations: systematic review and meta-analysis. BMJ. 2013;346:f1720.CrossRefPubMed

37.

Edwards C, Leira EC, Gonzalez-Alegre P. Residency training: a failed lumbar puncture is more about obesity than lack of ability. Neurology. 2015;84(10):e69–e72.CrossRefPubMed

38.

McGaghie WC. Mastery learning: It is time for medical education to join the 21st century. Acad Med. 2015;90(11):1438–1441.CrossRefPubMed

39.

Yudkowsky R, Park YS, Lineberry M, Knox A, Ritter EM. Setting mastery learning standards. Acad Med. 2015;90(11):1495–1500.CrossRefPubMed

40.

Conroy SM, Bond WF, Pheasant KS, Ceccacci N. Competence and retention in performance of the lumbar puncture procedure in a task trainer model. Simul Healthc. 2010;5(3):133–138.CrossRefPubMed

Title: Assessment of Residents Readiness to Perform Lumbar Puncture: A Validation Study
Authors: Mikael Johannes Vuokko Henriksen, MD
Troels Wienecke, MD, PhD
Helle Thagesen, MD
Rikke Vita Borre Jacobsen, MD, PhD
Yousif Subhi, MD
Charlotte Ringsted, MD, PhD
Lars Konge, MD, PhD
Publication date: 01-06-2017
Publisher: Springer US
Published in: Journal of General Internal Medicine / Issue 6/2017
Print ISSN: 0884-8734
Electronic ISSN: 1525-1497
DOI: https://doi.org/10.1007/s11606-016-3981-y

ACC 2024 Congress Coverage

Heart Failure Video

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Assessment of Residents Readiness to Perform Lumbar Puncture: A Validation Study

Abstract

Background

Objective

Design

Participants and Main Measures

Key results

Conclusion

ACC 2024 Congress Coverage

Year in Review: Heart failure and cardiomyopathies

Semaglutide benefits people with type 2 diabetes and obesity-related heart failure

Incentivised to exercise

New intervention for STEMI-related cardiogenic shock

» View more highlights on the ACC 2024 congress hub page

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Objective

Design

Participants and Main Measures

Key results

Conclusion

Please log in to get access to this content

Other articles of this Issue 6/2017

Capsule Commentary on Krause et al., Impact of Pregnancy and Gender on Internal Medicine Resident Evaluations: A Retrospective Cohort Study

Immune Reconstitution Inflammatory Syndrome Presenting as Mycobacterium Avium Complex Lymphadenitis

Effect of Combination Therapy on Adherence Among US Patients Initiating Therapy for Hypertension: a Cohort Study

Year-End Clinic Handoffs: A National Survey of Academic Internal Medicine Programs

Epiploic Appendagitis: A Benign Process at Risk of Unnecessary Hospitalization and Interventions

The Importance of Framing

ACC 2024 Congress Coverage

Year in Review: Heart failure and cardiomyopathies

Semaglutide benefits people with type 2 diabetes and obesity-related heart failure

Incentivised to exercise

New intervention for STEMI-related cardiogenic shock

» View more highlights on the ACC 2024 congress hub page