TY - JOUR

T1 - Development and validation of a new tool for assessment of trainees' interventional musculoskeletal ultrasound skills

AU - Carstensen, Stine Maya Dreier

AU - Just, Søren Andreas

AU - Pfeiffer-Jensen, Mogens

AU - Østergaard, Mikkel

AU - Konge, Lars

AU - Terslev, Lene

N1 - © The Author(s) 2024. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For permissions, please email: journals.permissions@oup.com.

PY - 2024

Y1 - 2024

N2 - OBJECTIVES: Interventional musculoskeletal ultrasound (MSUS) procedures are routinely performed in rheumatology practice. However, the efficacy and safety of the procedures rely on the competence of the physician, and assessment of skills is crucial. Thus, this study aimed to develop and establish validity evidence for a tool assessing trainees' interventional MSUS skills.METHODS: An expert panel of rheumatologists modified an existing tool for assessing competences in invasive abdominal and thoracic ultrasound procedures. The new tool (the Assessment of Interventional Musculoskeletal Ultrasound Skills (AIMUS) tool) reflects the essential steps in interventional MSUS. To establish validity evidence, physicians with different levels of interventional MSUS experience were enrolled and performed two procedures on a rubber fantom, simulating real patient cases. All performances were video recorded, anonymized, and assessed in random order by two blinded raters using the AIMUS tool.RESULTS: Sixty-five physicians from 21 different countries were included and categorized into groups based on their experience, resulting in 130 videos for analysis. The internal consistency of the tool was excellent, with a Cronbach's alpha of 0.96. The inter-case reliability was good with a Pearson's correlation coefficient (PCC) of 0.74 and the inter-rater reliability was moderate to good (PCC 0.58). The ability to discriminate between different levels of experience was highly significant (p< 0.001).CONCLUSIONS: We have developed and established validity evidence for a new interventional MSUS assessment tool. The tool can be applied in future competency-based educational programs, provide structured feedback to trainees in daily clinical practice, and ensure end-of-training competence.

AB - OBJECTIVES: Interventional musculoskeletal ultrasound (MSUS) procedures are routinely performed in rheumatology practice. However, the efficacy and safety of the procedures rely on the competence of the physician, and assessment of skills is crucial. Thus, this study aimed to develop and establish validity evidence for a tool assessing trainees' interventional MSUS skills.METHODS: An expert panel of rheumatologists modified an existing tool for assessing competences in invasive abdominal and thoracic ultrasound procedures. The new tool (the Assessment of Interventional Musculoskeletal Ultrasound Skills (AIMUS) tool) reflects the essential steps in interventional MSUS. To establish validity evidence, physicians with different levels of interventional MSUS experience were enrolled and performed two procedures on a rubber fantom, simulating real patient cases. All performances were video recorded, anonymized, and assessed in random order by two blinded raters using the AIMUS tool.RESULTS: Sixty-five physicians from 21 different countries were included and categorized into groups based on their experience, resulting in 130 videos for analysis. The internal consistency of the tool was excellent, with a Cronbach's alpha of 0.96. The inter-case reliability was good with a Pearson's correlation coefficient (PCC) of 0.74 and the inter-rater reliability was moderate to good (PCC 0.58). The ability to discriminate between different levels of experience was highly significant (p< 0.001).CONCLUSIONS: We have developed and established validity evidence for a new interventional MSUS assessment tool. The tool can be applied in future competency-based educational programs, provide structured feedback to trainees in daily clinical practice, and ensure end-of-training competence.

U2 - 10.1093/rheumatology/keae050

DO - 10.1093/rheumatology/keae050

M3 - Journal article

C2 - 38273715

JO - Rheumatology

JF - Rheumatology

SN - 1462-0324

ER -