In mitral valve reconstruction, an annuloplasty band is sutured the patient's valve to restore valve function. The size of the band is determined intraoperatively by inserting a measuring device. However, the insertion of the measuring device is difficult in minimally invasive surgery with small incisions, which has become popular in recent years. We previously proposed a system that superimposes a virtual image of a valve ring on an endoscopic image using AR (Augmented Reality) technology to assist in surgery. However, the system was limited to superimposing a virtual image of a ring on an endoscopic image taken from a single viewpoint. In this study, therefore, a more accurate simulation that can be checked from various directions based on 3D shape data. First, the 3D information of the marker area is estimated using stereo method. Next, the spectral reflectance is estimated from color images obtained from a stereo endoscope and marker areas are extracted. Finally, the annuloplasty band automatically selected from 3D positions of the markers is displayed on the 3D simulation. To evaluate this method, we performed the spectral reflectance estimation and 3D simulation using a pig heart marked with surgical maker and confirmed its effectiveness. This system enables size selection without inserting measuring instruments during surgery and is expected to shorten the operation time.
In cardiac surgery, a method avoids cutting the sternum and ribs using an endoscope is known as Minimally Invasive Cardiac Surgery (MICS). MICS can be used for mitral valve prolapse. In this surgery, an annuloplasty ring is inserted to form the valve and since each person has a different valve size, an appropriate size must be selected by applying a sizer. We propose a system that supports surgery by superimposing a virtual image of the valve ring on the endoscopic image using Augmented reality (AR) technology. Marker detection needs to be performed for AR display. However, since organ surfaces are similar in color, we use the spectral reflectance of the material to obtain more information than RGB image. To avoid the use of special optics to measure the spectral reflectance, we estimate spectral reflectance from RGB images using principal component analysis. The areas containing the marker are obtained by similarity index and the 3D positions of the markers are determined from center of gravity measurements. Then, we superimpose the AR image onto the endoscope image by perspective transformation to the image plane and masking using estimated 3D position of markers. To evaluate our method, we validated the spectral reflectance estimation and AR display using pig hearts marked with crystal violet. We found that the accuracy of spectral reflectance estimation is sufficient for AR display and the AR image was almost the same as the actual ring size. Therefore, our proposed method is expected to increase the efficiency of the operation.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.