Time of flight (ToF) range cameras illuminate the scene with an amplitude-modulated continuous wave light source and measure the returning modulation envelopes: phase and amplitude. The phase change of the modulation envelope encodes the distance travelled. This technology suffers from measurement errors caused by multiple propagation paths from the light source to the receiving pixel. The multiple paths can be represented as the summation of a direct return, which is the return from the shortest path length, and a global return, which includes all other returns. We develop the use of a sinusoidal pattern from which a closed form solution for the direct and global returns can be computed in nine frames with the constraint that the global return is a spatially lower frequency than the illuminated pattern. In a demonstration on a scene constructed to have strong multipath interference, we find the direct return is not significantly different from the ground truth in 33/136 pixels tested; where for the full-field measurement, it is significantly different for every pixel tested. The variance in the estimated direct phase and amplitude increases by a factor of eight compared with the standard time of flight range camera technique.
Time-of-flight range imaging cameras are capable of acquiring depth images of a scene. Some algorithms require these cameras to be run in `raw mode', where any calibrations from the off-the-shelf manufacturers are lost. The calibration of the MESA SR4000 is herein investigated, with an attempt to reconstruct the full calibration. Possession of the factory calibration enables calibrated data to be acquired and manipulated even in “raw mode.” This work is motivated by the problem of motion correction, in which the calibration must be separated into component parts to be applied at different stages in the algorithm. There are also other applications, in which multiple frequencies are required, such as multipath interference correction. The other frequencies can be calibrated in a similar way, using the factory calibration as a base. A novel technique for capturing the calibration data is described; a retro-reflector is used on a moving platform, which acts as a point source at a distance, resulting in planar waves on the sensor. A number of calibrations are retrieved from the camera, and are then modelled and compared to the factory calibration. When comparing the factory calibration to both the “raw mode” data, and the calibration described herein, a root mean squared error improvement of 51:3mm was seen, with a standard deviation improvement of 34:9mm.
Spherical harmonic cross-correlation is a robust registration technique that uses the normals of two overlapping meshes to bring them into coarse rotational alignment. The amount of overlap between the two meshes is the primary determinant of whether the spherical harmonic cross-correlation achieves correct registration. By weighting each normal or clusters of normals, their contribution to the registration is influenced, allowing beneficial normals to be emphasized and deemphasizing those that are not. In this paper we evaluate how different weighting schemes impact registration efficiency and accuracy. It is found that two of the proposed weighting schemes are capable of correctly registering 22% of the mesh pairs, while the baseline, which equally weighted all normals, registered 14% of the mesh pairs. Using Fibonacci binning to equally weight surfaces provided the best all-round advantage, especially if efficiency is considered, as binning allows spherical harmonics to be pre-computed. By increasing the threshold that is applied to the weighting schemes, meshes with minimal overlap can be registered, with one case only having 2% overlap. The performed analysis shows that weighting normals when applied in a conducive manner can achieve considerable improvements improvements to registration accuracy.
Time-of-flight (TOF) full-field range cameras use a correlative imaging technique to generate three-dimensional measurements of the environment. Though reliable and cheap they have the disadvantage of high measurement noise and errors that limit the practical use of these cameras in industrial applications. We show how some of these limitations can be overcome with standard image processing techniques specially adapted to TOF camera data. Additional information in the multimodal images recorded in this setting, and not available in standard image processing settings, can be used to improve reduction of measurement noise. Three extensions of standard techniques, wavelet thresholding, adaptive smoothing on a clustering based image segmentation, and an extended anisotropic diffusion filtering, make use of this information and are compared on synthetic data and on data acquired from two different off-the-shelf TOF cameras. Of these methods, the adapted anisotropic diffusion technique gives best results, and is implementable to perform in real time using current graphics processing unit (GPU) hardware. Like traditional anisotropic diffusion, it requires some parameter adaptation to the scene characteristics, but allows for low visualization delay and improved visualization of moving objects by avoiding long averaging periods when compared to traditional TOF image denoising.
KEYWORDS: Signal attenuation, Modulation, LIDAR, Reflectivity, Backscatter, Phase shift keying, Fourier transforms, Reconstruction algorithms, Cameras, Signal to noise ratio
We present two new closed-form methods for mixed pixel/multipath interference separation in AMCW lidar
systems. The mixed pixel/multipath interference problem arises from the violation of a standard range-imaging
assumption that each pixel integrates over only a single, discrete backscattering source. While a numerical
inversion method has previously been proposed, no close-form inverses have previously been posited. The first
new method models reflectivity as a Cauchy distribution over range and uses four measurements at different
modulation frequencies to determine the amplitude, phase and reflectivity distribution of up to two component
returns within each pixel. The second new method uses attenuation ratios to determine the amplitude and phase
of up to two component returns within each pixel. The methods are tested on both simulated and real data and
shown to produce a significant improvement in overall error. While this paper focusses on the AMCW mixed
pixel/multipath interference problem, the algorithms contained herein have applicability to the reconstruction of
a sparse one dimensional signal from an extremely limited number of discrete samples of its Fourier transform.
Spherical harmonic cross-correlation is a robust registration technique that uses the normals of two overlapping
point clouds to bring them into coarse rotational alignment. This registration technique however has a high
computational cost as spherical harmonics need to be calculated for every normal. By binning the normals, the
computational efficiency is improved as the spherical harmonics can be pre-computed and cached at each bin
location. In this paper we evaluate the efficiency and accuracy of the equiangle grid, icosahedron subdivision
and the Fibonacci spiral, an approach we propose. It is found that the equiangle grid has the best efficiency as
it can perform direct binning, followed by the Fibonacci spiral and then the icosahedron, all of which decrease
the computational cost compared to no binning. The Fibonacci spiral produces the highest achieved accuracy
of the three approaches while maintaining a low number of bins. The number of bins allowed by the equiangle
grid and icosahedron are much more restrictive than the Fibonacci spiral. The performed analysis shows that
the Fibonacci spiral can perform as well as the original cross-correlation algorithm without binning, while also
providing a significant improvement in computational efficiency.
Time-of-flight range imaging sensors acquire an image of a scene, where in addition to standard intensity information,
the range (or distance) is also measured concurrently by each pixel. Range is measured using a correlation technique,
where an amplitude modulated light source illuminates the scene and the reflected light is sampled by a gain modulated
image sensor. Typically the illumination source and image sensor are amplitude modulated with square waves, leading to
a range measurement linearity error caused by aliased harmonic components within the correlation waveform. A simple
method to improve measurement linearity by reducing the duty cycle of the illumination waveform to suppress
problematic aliased harmonic components is demonstrated. If the total optical power is kept constant, the measured
correlation waveform amplitude also increases at these reduced illumination duty cycles.
Measurement performance is evaluated over a range of illumination duty cycles, both for a standard range imaging
camera configuration, and also using a more complicated phase encoding method that is designed to cancel aliased
harmonics during the sampling process. The standard configuration benefits from improved measurement linearity for
illumination duty cycles around 30%, while the measured amplitude, hence range precision, is increased for both
methods as the duty cycle is reduced below 50% (while maintaining constant optical power).
Time-of-flight range imaging cameras measure distance and intensity simultaneously for every pixel in an image. With
the continued advancement of the technology, a wide variety of new depth sensing applications are emerging; however
a number of these potential applications have stringent electrical power constraints that are difficult to meet with the
current state-of-the-art systems. Sensor gain modulation contributes a significant proportion of the total image sensor
power consumption, and as higher spatial resolution range image sensors operating at higher modulation frequencies (to
achieve better measurement precision) are developed, this proportion is likely to increase. The authors have developed
a new sensor modulation technique using resonant circuit concepts that is more power efficient than the standard mode
of operation. With a proof of principle system, a 93-96% reduction in modulation drive power was demonstrated across
a range of modulation frequencies from 1-11 MHz. Finally, an evaluation of the range imaging performance revealed
an improvement in measurement linearity in the resonant configuration due primarily to the more sinusoidal shape of the
resonant electrical waveforms, while the average precision values were comparable between the standard and resonant
operating modes.
Time-of-flight range cameras acquire a three-dimensional image of a scene simultaneously for all pixels from a single
viewing location. Attempts to use range cameras for metrology applications have been hampered by the multi-path
problem, which causes range distortions when stray light interferes with the range measurement in a given pixel.
Correcting multi-path distortions by post-processing the three-dimensional measurement data has been investigated, but
enjoys limited success because the interference is highly scene dependent. An alternative approach based on separating
the strongest and weaker sources of light returned to each pixel, prior to range decoding, is more successful, but has only
been demonstrated on custom built range cameras, and has not been suitable for general metrology applications. In this
paper we demonstrate an algorithm applied to both the Mesa Imaging SR-4000 and Canesta Inc. XZ-422 Demonstrator
unmodified off-the-shelf range cameras. Additional raw images are acquired and processed using an optimization
approach, rather than relying on the processing provided by the manufacturer, to determine the individual component
returns in each pixel. Substantial improvements in accuracy are observed, especially in the darker regions of the scene.
Time-of-flight range imaging is typically performed with the amplitude modulated continuous wave method. This
involves illuminating a scene with amplitude modulated light. Reflected light from the scene is received by the sensor
with the range to the scene encoded as a phase delay of the modulation envelope. Due to the cyclic nature of phase, an
ambiguity in the measured range occurs every half wavelength in distance, thereby limiting the maximum useable range
of the camera.
This paper proposes a procedure to resolve depth ambiguity using software post processing. First, the range data is
processed to segment the scene into separate objects. The average intensity of each object can then be used to determine
which pixels are beyond the non-ambiguous range. The results demonstrate that depth ambiguity can be resolved for
various scenes using only the available depth and intensity information. This proposed method reduces the sensitivity to
objects with very high and very low reflectance, normally a key problem with basic threshold approaches.
This approach is very flexible as it can be used with any range imaging camera. Furthermore, capture time is not
extended, keeping the artifacts caused by moving objects at a minimum. This makes it suitable for applications such as
robot vision where the camera may be moving during captures.
The key limitation of the method is its inability to distinguish between two overlapping objects that are separated by a
distance of exactly one non-ambiguous range. Overall the reliability of this method is higher than the basic threshold
approach, but not as high as the multiple frequency method of resolving ambiguity.
Time of flight range imaging is an emerging technology that has numerous applications in machine vision. In this paper we cover the use of a commercial time of flight range imaging camera for calibrating a robotic arm. We do this by identifying retro-reflective targets attached to the arm, and centroiding on calibrated spatial data, which allows precise measurement of three dimensional target locations. The robotic arm is an inexpensive model that does not have positional feedback, so a series of movements are performed to calibrate the servos signals to the physical position of the arm. The calibration showed a good linear response between the control signal and servo angles. The calibration procedure also provided a transformation between the camera and arm coordinate systems. Inverse kinematic control was then used to position the arm. The range camera could also be used to identify objects in the scene. With the object location now known in the arm's coordinate system (transformed from the camera's coordinate system) the arm was able to move allowing it to grasp the object.
Time-of-flight range imaging cameras operate by illuminating a scene with amplitude modulated light and measuring
the phase shift of the modulation envelope between the emitted and reflected light. Object distance can
then be calculated from this phase measurement. This approach does not work in multiple camera environments
as the measured phase is corrupted by the illumination from other cameras. To minimize inaccuracies in multiple
camera environments, replacing the traditional cyclic modulation with pseudo-noise amplitude modulation has
been previously demonstrated. However, this technique effectively reduced the modulation frequency, therefore
decreasing the distance measurement precision (which has a proportional relationship with the modulation frequency).
A new modulation scheme using maximum length pseudo-random sequences binary phase encoded onto
the existing cyclic amplitude modulation, is presented. The effective modulation frequency therefore remains
unchanged, providing range measurements with high precision. The effectiveness of the new modulation scheme
was verified using a custom time-of-flight camera based on the PMD19-K2 range imaging sensor. The new
pseudo-noise modulation has no significant performance decrease in a single camera environment. In a two camera
environment, the precision is only reduced by the increased photon shot noise from the second illumination
source.
We present a new two-stage method for parametric spatially variant blind deconvolution of full-field Amplitude Modulated Continuous Wave lidar image pairs taken at different aperture settings subject to limited depth of field. A Maximum Likelihood based focal parameter determination algorithm uses range information to reblur the image taken with a smaller aperture size to match the large aperture image. This allows estimation of focal parameters without prior calibration of the optical setup and produces blur estimates which have better spatial resolution and less noise than previous depth from defocus (DFD) blur measurement algorithms. We compare blur estimates from the focal parameter determination method to those from Pentland's DFD method, Subbarao's S-Transform method and estimates from range data/the sampled point spread function. In a second stage the estimated focal parameters are applied to deconvolution of total integrated intensity lidar images improving depth of field. We give an example of application to complex domain lidar images and discuss the trade-off between recovered amplitude texture and sharp range estimates.
We present two novel Poisson noise Maximum Likelihood based methods for identifying the individual returns
within mixed pixels for Amplitude Modulated Continuous Wave rangers. These methods use the convolutional
relationship between signal returns and the recorded data to determine the number, range and intensity of returns
within a pixel. One method relies on a continuous piecewise truncated-triangle model for the beat waveform
and the other on linear interpolation between translated versions of a sampled waveform. In the single return
case both methods provide an improvement in ranging precision over standard Fourier transform based methods
and a decrease in overall error in almost every case. We find that it is possible to discriminate between two
light sources within a pixel, but local minima and scattered light have a significant impact on ranging precision.
Discrimination of two returns requires the ability to take samples at less than 90 phase shifts.
Range imaging cameras measure depth simultaneously for every pixel in a given field of view. In most implementations
the basic operating principles are the same. A scene is illuminated with an intensity modulated light source and the
reflected signal is sampled using a gain-modulated imager. Previously we presented a unique heterodyne range imaging
system that employed a bulky and power hungry image intensifier as the high speed gain-modulation mechanism. In this
paper we present a new range imager using an internally modulated image sensor that is designed to operate in
heterodyne mode, but can also operate in homodyne mode. We discuss homodyne and heterodyne range imaging, and
the merits of the various types of hardware used to implement these systems. Following this we describe in detail the
hardware and firmware components of our new ranger. We experimentally compare the two operating modes and
demonstrate that heterodyne operation is less sensitive to some of the limitations suffered in homodyne mode, resulting
in better linearity and ranging precision characteristics. We conclude by showing various qualitative examples that
demonstrate the system's three-dimensional measurement performance.
A number of full field image sensors have been developed that are capable of simultaneously measuring intensity and
distance (range) for every pixel in a given scene using an indirect time-of-flight measurement technique. A light source
is intensity modulated at a frequency between 10-100 MHz, and an image sensor is modulated at the same frequency,
synchronously sampling light reflected from objects in the scene (homodyne detection). The time of flight is manifested
as a phase shift in the illumination modulation envelope, which can be determined from the sampled data simultaneously
for each pixel in the scene. This paper presents a method of characterizing the high frequency modulation response of
these image sensors, using a pico-second laser pulser. The characterization results allow the optimal operating
parameters, such as the modulation frequency, to be identified in order to maximize the range measurement precision for
a given sensor. A number of potential sources of error exist when using these sensors, including deficiencies in the
modulation waveform shape, duty cycle, or phase, resulting in contamination of the resultant range data. From the
characterization data these parameters can be identified and compensated for by modifying the sensor hardware or
through post processing of the acquired range measurements.
KEYWORDS: Signal to noise ratio, Multiplexing, Light sources, Computer programming, Spectroscopy, Light, Data acquisition, Modulation, Hyperspectral imaging, Imaging systems
A hyperspectral imaging system is in development. The system uses spatially modulated Hadamard patterns to encode image information with implicit stray and ambient light correction and a reference beam to correct for source light changes over the spectral image capture period. In this study we test the efficacy of the corrections and the multiplex advantage for our system. The signal to noise ratio (SNR) was used to demonstrate the advantage of spatial multiplexing in the system and observe the effect of the reference beam correction. The statistical implications of the data acquisition technique, illumination source drift and correction of such drift, were derived. The reference beam correction was applied per spectrum before Hadamard decoding and alternately after decoding to all spectra in the image. The reference beam method made no fundamental change to SNR, therefore we conclude that light source drift is minimal and other possibly rectifiable error sources are dominant. The multiplex advantage was demonstrated ranging from a minimum SNR boost of 1.5 (600-975 nm) to a maximum of 11 (below 500 nm). Intermediate SNR boost was observed in 975-1700 nm. The large variation in SNR boost is also due to some other error source.
A range imaging camera produces an output similar to a digital photograph, but every pixel in the image contains
distance information as well as intensity. This is useful for measuring the shape, size and location of objects in a scene,
hence is well suited to certain machine vision applications.
Previously we demonstrated a heterodyne range imaging system operating in a relatively high resolution (512-by-512)
pixels and high precision (0.4 mm best case) configuration, but with a slow measurement rate (one every 10 s).
Although this high precision range imaging is useful for some applications, the low acquisition speed is limiting in many
situations. The system's frame rate and length of acquisition is fully configurable in software, which means the
measurement rate can be increased by compromising precision and image resolution.
In this paper we demonstrate the flexibility of our range imaging system by showing examples of high precision ranging
at slow acquisition speeds and video-rate ranging with reduced ranging precision and image resolution. We also show
that the heterodyne approach and the use of more than four samples per beat cycle provides better linearity than the
traditional homodyne quadrature detection approach. Finally, we comment on practical issues of frame rate and beat
signal frequency selection.
As JPEG compression at source is ubiquitous in retinal imaging, and the block artefacts introduced are known
to be of similar size to microaneurysms (an important indicator of diabetic retinopathy) it is prudent to evaluate
the effect of JPEG compression on automated detection of retinal pathology. Retinal images were acquired at
high quality and then compressed to various lower qualities. An automated microaneurysm detector was run on
the retinal images of various qualities of JPEG compression and the ability to predict the presence of diabetic
retinopathy based on the detected presence of microaneurysms was evaluated with receiver operating characteristic
(ROC) methodology. The negative effect of JPEG compression on automated detection was observed even at
levels of compression sometimes used in retinal eye-screening programmes and these may have important clinical
implications for deciding on acceptable levels of compression for a fully automated eye-screening programme.
Full field range imaging cameras are used to simultaneously measure the distance for every pixel in a given scene using
an intensity modulated illumination source and a gain modulated receiver array. The light is reflected from an object in
the scene, and the modulation envelope experiences a phase shift proportional to the target distance. Ideally the waveforms are sinusoidal, allowing the phase, and hence object range, to be determined from four measurements using an arctangent function. In practice these waveforms are often not perfectly sinusoidal, and in some cases square waveforms are instead used to simplify the electronic drive requirements. The waveforms therefore commonly contain odd harmonics which contribute a nonlinear error to the phase determination, and therefore an error in the range measurement. We have developed a unique sampling method to cancel the effect of these harmonics, with the results showing an order of magnitude improvement in the measurement linearity without the need for calibration or lookup tables, while the acquisition time remains unchanged. The technique can be applied to existing range imaging systems without having to change or modify the complex illumination or sensor systems, instead only requiring a change to the signal generation and timing electronics.
Solid-state full-field range imaging technology, capable of determining the distance to objects in a scene simultaneously
for every pixel in an image, has recently achieved sub-millimeter distance measurement precision. With this level of
precision, it is becoming practical to use this technology for high precision three-dimensional metrology applications.
Compared to photogrammetry, range imaging has the advantages of requiring only one viewing angle, a relatively short
measurement time, and simplistic fast data processing. In this paper we fist review the range imaging technology, then
describe an experiment comparing both photogrammetric and range imaging measurements of a calibration block with
attached retro-reflective targets. The results show that the range imaging approach exhibits errors of approximately
0.5 mm in-plane and almost 5 mm out-of-plane; however, these errors appear to be mostly systematic. We then proceed
to examine the physical nature and characteristics of the image ranging technology and discuss the possible causes of
these systematic errors. Also discussed is the potential for further system characterization and calibration to compensate
for the range determination and other errors, which could possibly lead to three-dimensional measurement precision
approaching that of photogrammetry.
KEYWORDS: Modulation, Cameras, Imaging systems, Image intensifiers, Phase shift keying, Ranging, Heterodyning, Camera shutters, Video, Signal to noise ratio
Previously, we demonstrated a novel heterodyne based solid-state full-field range-finding imaging system. This system is comprised of modulated LED illumination, a modulated image intensifier, and a digital video camera. A 10 MHz drive is provided with 1 Hz difference between the LEDs and image intensifier. A sequence of images of the resulting beating intensifier output are captured and processed to determine phase and hence distance to the object for each pixel. In a previous publication, we detailed results showing a one-sigma precision of 15 mm to 30 mm (depending on signal strength). Furthermore, we identified the limitations of the system and potential improvements that were expected to result in a range precision in the order of 1 mm. These primarily include increasing the operating frequency and improving optical coupling and sensitivity. In this paper, we report on the implementation of these improvements and the new system characteristics. We also comment on the factors that are important for high precision image ranging and present configuration strategies for best performance. Ranging with sub-millimeter precision is demonstrated by imaging a planar surface and calculating the deviations from a planar fit. The results are also illustrated graphically by imaging a garden gnome.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.