João Ramalhinho, Maria Robu, Stephen Thompson, Philip Edwards, Crispin Schneider, Kurinchy Gurusamy, David Hawkes, Brian Davidson, Dean Barratt, Matthew Clarkson
Laparoscopic Ultrasound (LUS) is regularly used during laparoscopic liver resection to locate critical vascular
structures. Many tumours are iso-echoic, and registration to pre-operative CT or MR has been proposed as
a method of image guidance. However, factors such as abdominal insufflation, LUS probe compression and
breathing motion cause deformation of the liver, making this task far from trivial. Fortunately, within a smaller
local region of interest a rigid solution can suffice. Also, the respiratory cycle can be expected to be consistent.
Therefore, in this paper we propose a feature-based local rigid registration method to align tracked LUS data
with CT while compensating for breathing motion. The method employs the Levenberg-Marquardt Iterative
Closest Point (LMICP) algorithm, registers both on liver surface and vessels and requires two LUS datasets,
one for registration and another for breathing estimation. Breathing compensation is achieved by fitting a 1D
breathing model to the vessel points. We evaluate the algorithm by measuring the Target Registration Error
(TRE) of three manually selected landmarks of a single porcine subject. Breathing compensation improves
accuracy in 77% of the measurements. In the best case, TRE values below 3mm are obtained. We conclude that
our method can potentially correct for breathing motion without gated acquisition of LUS and be integrated in
the surgical workflow with an appropriate segmentation.
Camera calibration is a key requirement for augmented reality in surgery. Calibration of laparoscopes provides two challenges that are not sufficiently addressed in the literature. In the case of stereo laparoscopes the small distance (less than 5mm) between the channels means that the calibration pattern is an order of magnitude more distant than the stereo separation. For laparoscopes in general, if an external tracking system is used, hand-eye calibration is difficult due to the long length of the laparoscope. Laparoscope intrinsic, stereo and hand-eye calibration all rely on accurate feature point selection and accurate estimation of the camera pose with respect to a calibration pattern. We compare 3 calibration patterns, chessboard, rings, and AprilTags. We measure the error in estimating the camera intrinsic parameters and the camera poses. Accuracy of camera pose estimation will determine the accuracy with which subsequent stereo or hand-eye calibration can be done. We compare the results of repeated real calibrations and simulations using idealised noise, to determine the expected accuracy of different methods and the sources of error. The results do indicate that feature detection based on rings is more accurate than a chessboard, however this doesn’t necessarily lead to a better calibration. Using a grid with identifiable tags enables detection of features nearer the image boundary, which may improve calibration.
Eli Gibson, Maria Robu, Stephen Thompson, P. Eddie Edwards, Crispin Schneider, Kurinchi Gurusamy, Brian Davidson, David Hawkes, Dean Barratt, Matthew Clarkson
Motivation: For primary and metastatic liver cancer patients undergoing liver resection, a laparoscopic approach can
reduce recovery times and morbidity while offering equivalent curative results; however, only about 10% of tumours
reside in anatomical locations that are currently accessible for laparoscopic resection. Augmenting laparoscopic video
with registered vascular anatomical models from pre-procedure imaging could support using laparoscopy in a wider
population. Segmentation of liver tissue on laparoscopic video supports the robust registration of anatomical liver models
by filtering out false anatomical correspondences between pre-procedure and intra-procedure images. In this paper, we
present a convolutional neural network (CNN) approach to liver segmentation in laparoscopic liver procedure videos.
Method: We defined a CNN architecture comprising fully-convolutional deep residual networks with multi-resolution
loss functions. The CNN was trained in a leave-one-patient-out cross-validation on 2050 video frames from 6 liver
resections and 7 laparoscopic staging procedures, and evaluated using the Dice score.
Results: The CNN yielded segmentations with Dice scores ≥0.95 for the majority of images; however, the inter-patient
variability in median Dice score was substantial. Four failure modes were identified from low scoring segmentations:
minimal visible liver tissue, inter-patient variability in liver appearance, automatic exposure correction, and pathological
liver tissue that mimics non-liver tissue appearance.
Conclusion: CNNs offer a feasible approach for accurately segmenting liver from other anatomy on laparoscopic video,
but additional data or computational advances are necessary to address challenges due to the high inter-patient variability
in liver appearance.
We present a framework for multi-atlas based segmentation in situations where we have a small number of segmented atlas images, but a large database of unlabeled images is also available. The novelty lies in the application of graph-based registration on a manifold to the problem of multi-atlas registration. The approach is to place all the images in a learned manifold space and construct a graph connecting near neighbors. Atlases are selected for any new image to be segmented based on the shortest path length along the manifold graph. A multi-scale non-rigid registration takes place via each of the nodes on the graph. The expectation is that by registering via similar images, the likelihood of misregistrations is reduced. Having registered multiple atlases via the graph, patch-based voxel weighted voting takes place to provide the final segmentation. We apply this approach to a set of T2 MRI images of the prostate, which is a notoriously difficult segmentation task. On a set of 25 atlas images and 85 images overall, we see that registration via the manifold graph improves the Dice coefficient from 0:82±0:05 to 0:86±0:03 and the average symmetrical boundary distance from 2:89±0:62mm to 2:47±0:51mm. This is a modest but potentially useful improvement in a difficult set of images. It is expected that our approach will provide similar improvement to any multi-atlas segmentation task where a large number of unsegmented images are available.
We propose a learning-based approach to segment the seminal vesicles (SV) via random forest classifiers. The proposed discriminative approach relies on the decision forest using high-dimensional multi-scale context-aware spatial, textual and descriptor-based features at both pixel and super-pixel level. After affine transformation to a template space, the relevant high-dimensional multi-scale features are extracted and random forest classifiers are learned based on the masked region of the seminal vesicles from the most similar atlases. Using these classifiers, an intermediate probabilistic segmentation is obtained for the test images. Then, a graph-cut based refinement is applied to this intermediate probabilistic representation of each voxel to get the final segmentation. We apply this approach to segment the seminal vesicles from 30 MRI T2 training images of the prostate, which presents a particularly challenging segmentation task. The results show that the multi-scale approach and the augmentation of the pixel based features with the super-pixel based features enhances the discriminative power of the learnt classifier which leads to a better quality segmentation in some very difficult cases. The results are compared to the radiologist labeled ground truth using leave-one-out cross-validation. Overall, the Dice metric of 0:7249 and Hausdorff surface distance of 7:0803 mm are achieved for this difficult task.
Increasing attention has been focused on the estimation of the deformation of the endocardium to aid the diagnosis
of cardiac malfunction. Landmark tracking can provide sparse, anatomically relevant constraints to help establish
correspondences between images being tracked or registered. However, landmarks on the endocardium are often
characterized by ambiguous appearance in cardiac MR images which makes the extraction and tracking of these
landmarks problematic.
In this paper we propose an automatic framework to select and track a sparse set of distinctive landmarks in
the presence of relatively large deformations in order to capture the endocardial motion in cardiac MR sequences.
To achieve this a sparse set of the landmarks is identified using an entropy-based approach. In particular we use
singular value decomposition (SVD) to reduce the search space and localize the landmarks with relatively large
deformation across the cardiac cycle. The tracking of the sparse set of landmarks is performed simultaneously
by optimizing a two-stage Markov Random Field (MRF) model. The tracking result is further used to initialize
registration based dense motion tracking.
We have applied this framework to extract a set of landmarks at the endocardial border of the left ventricle in
MR image sequences from 51 subjects. Although the left ventricle undergoes a number of different deformations,
we show how the radial, longitudinal motion and twisting of the endocardial surface can be captured by the
proposed approach. Our experiments demonstrate that motion tracking using sparse landmarks can outperform
conventional motion tracking by a substantial amount, with improvements in terms of tracking accuracy of 20:8%
and 19:4% respectively.
Optical coherence tomography (OCT) is a light-based, high resolution imaging technique to guide stent deployment
procedure for stenosis. OCT can accurately differentiate the most superficial layers of the vessel wall as
well as stent struts and the vascular tissue surrounding them. In this paper, we automatically detect the struts
of coronary stents present in OCT sequences. We propose a novel method to detect the strut shadow zone and
accurately segment and reconstruct the strut in 3D. The estimation of the position of the strut shadow zone
is the key requirement which enables the strut segmentation. After identification of the shadow zone we use
probability map to estimate stent strut positions. This method can be applied to cross-sectional OCT images
to detect the struts. Validation is performed using simulated data as well as in four in-vivo OCT sequences and
the accuracy of strut detection is over 90%. The comparison against manual expert segmentation demonstrates
that the proposed strut identification is robust and accurate.
In this paper, we present a novel approach for coronary artery motion modeling from cardiac Computed Tomography(
CT) images. The aim of this work is to develop a 4D motion model of the coronaries for image guidance
in robotic-assisted totally endoscopic coronary artery bypass (TECAB) surgery. To utilize the pre-operative
cardiac images to guide the minimally invasive surgery, it is essential to have a 4D cardiac motion model to be
registered with the stereo endoscopic images acquired intraoperatively using the da Vinci robotic system. In
this paper, we are investigating the extraction of the coronary arteries and the modelling of their motion from
a dynamic sequence of cardiac CT. We use a multi-scale vesselness filter to enhance vessels in the cardiac CT
images. The centerlines of the arteries are extracted using a ridge traversal algorithm. Using this method the
coronaries can be extracted in near real-time as only local information is used in vessel tracking. To compute
the deformation of the coronaries due to cardiac motion, the motion is extracted from a dynamic sequence of
cardiac CT. Each timeframe in this sequence is registered to the end-diastole timeframe of the sequence using
a non-rigid registration algorithm based on free-form deformations. Once the images have been registered a
dynamic motion model of the coronaries can be obtained by applying the computed free-form deformations to
the extracted coronary arteries. To validate the accuracy of the motion model we compare the actual position of
the coronaries in each time frame with the predicted position of the coronaries as estimated from the non-rigid
registration. We expect that this motion model of coronaries can facilitate the planning of TECAB surgery, and
through the registration with real-time endoscopic video images it can reduce the conversion rate from TECAB
to conventional procedures.
The aim of the work described in this paper is registration of a 4D preoperative motion model of the heart to
the video view of the patient through the intraoperative endoscope. The heart motion is cyclical and can be
modelled using multiple reconstructions of cardiac gated coronary CT.
We propose the use of photoconsistency between the two views through the da Vinci endoscope to align to
the preoperative heart surface model from CT. The temporal alignment from the video to the CT model could
in principle be obtained from the ECG signal. We propose averaging of the photoconsistency over the cardiac
cycle to improve the registration compared to a single view. Though there is considerable motion of the heart,
after correct temporal alignment we suggest that the remaining motion should be close to rigid.
Results are presented for simulated renderings and for real video of a beating heart phantom. We found much
smoother sections at the minimum when using multiple phases for the registration, furthermore convergence was
found to be better when more phases are used.
We propose a novel system for image guidance in totally endoscopic coronary artery bypass (TECAB). A key requirement
is the availability of 2D-3D registration techniques that can deal with non-rigid motion and deformation. Image guidance
for TECAB is mainly required before the mechanical stabilization of the heart, thus the most dominant source of non-rigid
deformation is the motion of the beating heart.
To augment the images in the endoscope of the da Vinci robot, we have to find the transformation from the coordinate
system of the preoperative imaging modality to the system of the endoscopic cameras.
In a first step we build a 4D motion model of the beating heart. Intraoperatively we can use the ECG or video processing
to determine the phase of the cardiac cycle. We can then take the heart surface from the motion model and register it to
the stereo-endoscopic images of the da Vinci robot using 2D-3D registration methods. We are investigating robust feature
tracking and intensity-based methods for this purpose.
Images of the vessels available in the preoperative coordinate system can then be transformed to the camera system and
projected into the calibrated endoscope view using two video mixers with chroma keying. It is hoped that the augmented
view can improve the efficiency of TECAB surgery and reduce the conversion rate to more conventional procedures.
In this paper we present a novel approach to the problem of fitting a 4D statistical shape model of the myocardium to
cardiac MR and CT image sequences. The 4D statistical model has been constructed from 25 cardiac MR image sequences
from normal volunteers. The model is controlled by two sets of shape parameters. The first set of shape parameters
describes shape changes due to inter-subject variability while the second set of shape parameters describes shape changes
due to intra-subject variability, i.e. the cardiac contraction and relaxation. A novel fitting approach is used to estimate the
optimal parameters of the cardiac shape model. The fitting of the model is performed simultaneously for the entire image
sequences. The method has been tested on 5 cardiac MR image sequences. Furthermore, we have also tested the method
using a cardiac CT image sequence. The result demonstrate that the method is not only able to fit the 4D model to cardiac
MR image sequences, but also to cardiac image sequences from a different modality (CT).
KEYWORDS: Diffraction, Black bodies, Sensors, Radiometry, Temperature metrology, Roads, Near field diffraction, Tolerancing, Geometrical optics, Far-field diffraction
Estimating the effects of diffraction is essential in modern
radiometric experiments. The majority of the tools used for this
date back to the pioneering work by W. Blevin and W. Steel. These
were analytical in nature, obtained by aggressive use of approximate techniques applied to the Fresnel diffraction integral; further, blackbodies were treated as uniform sources that could be described by a single characteristic wavelength, enabling diffraction effects to be determined through a single monochromatic calculation. This requires diffraction effects to change linearly with wavelength. The domain over which this is satisfied to the error tolerance required by contemporary radiometry is unclear. This paper investigates the single wavelength technique and establishes criteria for its use.
We present the first use of ultrasound to instantiate and register a statistical shape model of bony structures. Our aim is to provide accurate image-guided total hip replacement without the need for a preoperative computed tomography (CT) scan. We propose novel methods to determine the location of the bone surface intraoperatively using percutaneous ultrasound and, with the aid of a statistical shape model, reconstruct a complete three-dimensional (3D) model of relevant anatomy. The centre of the femoral head is used as a further constraint to improve accuracy in regions not accessible to ultrasound. CT scans of the femur from a database were aligned to one target CT scan using a non-rigid registration algorithm. The femur surface from the target scan was then propagated to each of the subjects and used to produce a statistical shape model. A cadaveric femur not used in the shape model construction was scanned using freehand 3D ultrasound. The iterative closest point (ICP) algorithm was used to match points corresponding to the bone Surface derived from ultrasound with the statistical bone surface model. We used the mean shape and the first five modes of variation of the shape model. The resulting root mean square (RMS) point-to-surface distance from ICP was minimised to provide the best fit of the model to the ultrasound data.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.