Automating creation of high-fidelity holographic hand animations for surgical skills training using mixed reality headsets

Regina W. K. Leung; Ge Shi; Christina A. Lim; Matthew Van Oirschot

doi:10.1117/12.3008787

29 March 2024 Automating creation of high-fidelity holographic hand animations for surgical skills training using mixed reality headsets

Regina W. K. Leung, Ge Shi, Christina A. Lim, Matthew Van Oirschot

Author Affiliations +

Proceedings Volume 12928, Medical Imaging 2024: Image-Guided Procedures, Robotic Interventions, and Modeling; 129282F (2024) https://doi.org/10.1117/12.3008787
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Conference Poster

Abstract

Purpose: Virtual holographic simulation skills training has a growing role in supplementing surgical skills training given increasingly limited healthcare resources and recent advancements in mixed reality technology. However, creating highfidelity 3D holographic animations (particularly hand animations) is expensive, time-consuming, and complex. We propose a low-cost solution using mixed reality headsets for motion capture of hands to automatically create high-fidelity 3D holographic hand animations. Methods: In this study, a 3D animation of a single-handed knot tie was created using the Oculus Quest 2 and APS Mocap Fusion app for display on a Microsoft HoloLens 2. To assess the feasibility and quality of the created 3D holographic animation, a qualitative and quantitative pilot study of 20 participants was conducted comparing learning one-handed knot ties from an in-person demonstration versus the 3D holographic hand-tie animation. Results: Our pilot study demonstrated participants were able to learn one-handed knot ties from the holographic animation (70% of participants) and was comparable to in-person (80%). Promisingly, based on the Likert scale questionnaire, participants found learning from the holographic animation was more effective (4.4 vs 3.3), easier (3.6 vs 3.3), and felt more confident in learning the knot-tie (4.4 vs 3.5) in comparison to in-person demonstration. Furthermore, participants felt the holographic animation was comparable to real-life hands (4). Overall, we successfully illustrated a low-cost automated methodology of creating high-fidelity 3D holographic hand animations from mixed reality headset motion capture data with potential for use in surgical simulation skills training.

Citation Download Citation

Regina W. K. Leung, Ge Shi, Christina A. Lim, and Matthew Van Oirschot "Automating creation of high-fidelity holographic hand animations for surgical skills training using mixed reality headsets", Proc. SPIE 12928, Medical Imaging 2024: Image-Guided Procedures, Robotic Interventions, and Modeling, 129282F (29 March 2024); https://doi.org/10.1117/12.3008787

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available