This PDF file contains the front matter associated with SPIE-IS&T Proceedings Volume 6805, including the Title Page, Copyright information, Table of Contents, and the Conference Committee listing.

In this paper, we propose an efficient image-based technique to reproduce transparent and refractive objects by using quotient image with much smaller number of images for a fixed viewpoint. Our method uses a CRT display for controlling 2D incident illumination. We use the coded structured light technique for searching corresponding points between the incident illumination and the image pixel by constructing space-coded image. Although we can obtain the light path from the space-coded image, the light intensity can not be determined. To measure the transparency of the object, we approximate the transmittance by calculating the quotient value of two images which are captured with and without the object in font of display with displaying all pixels white (we call this display "floodlit display"). In the reproduction process, we can reproduce the object by only multiplying the color (the intensity of the light) which is determined from the coded structured light technique and the transmittance which is obtained from quotient image.

The quality of industrial products is gradually increasing. In the same time, products and their parts most often have complex geometry which has to be described accurately by thousands or millions of measurement points. One of the most important quality control requirement is dimensional object to reference model consistency condition. Actually, large-size production process, especially in airplane, car and power industries, requires more effective measurement techniques. These new methods should introduce faster measurement speed and simultaneously high accuracy. In the paper we present a concept that combines the accuracy of Coordinate Measurement Machines (CMM) and the speed of full-field optical methods. We present a model of Opto-Mechanical Measurement Machine (OMMM) based on the integration of an optical measurement system with a CMM. To minimize measurement time, the following sequence is proposed: optical measurement of whole surface, data analysis, contact re-measurement of critical object's areas and final metrological characterization. To prove correctness of the OMMM idea a laboratory prototype had been constructed. In the paper some experimental results of car body measurement are presented with uncertainty sources discussion.

We have designed and tested a 3D vision system for measuring microchip surface heights relative to a substrate. The microchip is mounted with an adhesive to the substrate. The goal is to check the thickness of the adhesive layer between microchip and substrate before it is encapsuled by a plastic mold compound. This thickness has a significant influence on the reliability and electrical performance of the microchip. The system consists of one camera, a telecentric lens and three semi-transparent mirrors (beamsplitters). Reference patterns on the microchip and the substrate are imaged and illuminated from opposite 45° angles. This yields sets of coordinates which are used to extract the orientation of the chip relative to the substrate. We found that the vertical resolution of the system is greatly influenced by the setup of the image processing system. In principle, the reference patterns are identical for all chips and substrates of a production lot. Thus, the reference needs to be learned only once on a particular chip. With this setup we achieved a resolution of 2 micrometer. On the other hand, if the reference pattern is learned for each chip individually, we achieved a higher resolution of 1 micrometer. However, learning the pattern for each chip individually is time-consuming and may not be applicable for an on-line production inspection system with 2 - 3 chips per second.

A flexible and highly configurable 3D vision system targeted for in-line product inspection is presented. The system includes a low cost 3D camera based on structured light and a set of flexible software tools that automate the measurement process. The specification of the measurement tasks is done in a first manual step. The user selects regions of the point cloud to analyze and specifies primitives to be characterized within these regions. After all measurement tasks have been specified, measurements can be carried out on successive parts automatically and without supervision. As a test case, a measurement cell for inspection of a V-shaped car component has been developed. The car component consists of two steel tubes attached to a central hub. Each of the tubes has an additional bushing clamped to its end. A measurement is performed in a few seconds and results in an ordered point cloud with 1.2 million points. The software is configured to fit cylinders to each of the steel tubes as well as to the inside of the bushings of the car part. The size, position and orientation of the fitted cylinders allow us to measure and verify a series of dimensions specified on the CAD drawing of the component with sub-millimetre accuracy.

In this paper, we present the system architecture of a 360 degree view 3D imaging system. The system consists of multiple 3D sensors synchronized to take 3D images around the object. Each 3D camera employs a single high-resolution digital camera and a color-coded light projector. The cameras are synchronized to rapidly capture the 3D and color information of a static object or a live person. The color encoded structure lighting ensures the precise reconstruction of the depth of the object. A 3D imaging system architecture is presented. The architecture employs the displacement of the camera and the projector to triangulate the depth information. The 3D camera system has achieved high depth resolution down to 0.1mm on a human head sized object and 360 degree imaging capability.

The practical three-dimensional measurement system with a high resolution based on a space code light pattern projection and a phase-shifted light pattern projection is presented. Three-dimensional measurement device's so-called, rangefinder, are expected to be applied in the apparel, medical and various fields. The performance of a rangefinder is evaluated by the measurements of time, depth, size, etc. The system using a space code technique can stably acquire the depth of an object although the resolution of the depth is not good because an object's space is coded in the shape of a wedge. On the other hand, in a phase shift technique, the high resolution depth of an object is able to be theoretically acquired because the object's space is divided finely by phase shifted light projection. But it is difficult to stably acquire the depth of an object because of the phase connection problem. In this paper, these problems (which are high resolution and phase connection problem) are able to be solved by both the space code technique and the phase shift technique. The effectiveness of this system is also described.

Automated Explosive Detection Systems utilizing Computed Tomography perform a series X-ray scans of passenger bags being checked in at the airport, and produce various 2-D projection images and 3-D volumetric images of the bag. The determination as to whether the passenger bag contains an explosive and needs to be searched manually is performed through trained Transportation Security Administration screeners following an approved protocol. In order to keep the screeners vigilant with regards to screening quality, the Transportation Security Administration has mandated the use of Threat Image Projection on 2-D projection X-ray screening equipment used at all US airports. These algorithms insert visual artificial threats into images of the normal passenger bags in order to test the screeners with regards to their screening efficiency and their screening quality at determining threats. This technology for 2-D X-ray system is proven and is widespread amongst multiple manufacturers of X-ray projection systems. Until now, Threat Image Projection has been unsuccessful at being introduced into 3-D Automated Explosive Detection Systems for numerous reasons. The failure of these prior attempts are mainly due to imaging queues that the screeners pickup on, and therefore make it easy for the screeners to discern the presence of the threat image and thus defeating the intended purpose. This paper presents a novel approach for 3-D Threat Image Projection for 3-D Automated Explosive Detection Systems. The method presented here is a projection based approach where both the threat object and the bag remain in projection sinogram space. Novel approaches have been developed for projection based object segmentation, projection based streak reduction used for threat object isolation along with scan orientation independence and projection based streak generation for an overall realistic 3-D image. The algorithms are prototyped in MatLab and C++ and demonstrate non discernible 3-D threat image insertion into various luggage, and non discernable streak patterns for 3-D images when compared to actual scanned images.

For over thirty years researchers have been trying to solve the shape from shading problem of determining 3D shape from a single image with a single light source. The basic problem of determining shape from shading is made more difficult due to challenges of light orientation, camera type, ambiguity, multiple materials, and specular highlights. This paper shows how some of these challenges can be overcome through the use of other techniques such as image segmentation and stereopsis. We present a new hybrid method of shape from shading that can be used to autonomously capture 3D information from two 2D images of single objects with multiple peaks and multiple materials with specular components.

An automatic alignment technique for multiple rangefinders is proposed. We obtain 3D data of object using a rangefinder which is composed of a camera and a projector. Generally, we can only obtain partial figure when we measure an object from one direction. Therefore, we can acquire all around configuration by measuring an object from multiple viewpoint. The data obtained with rangefinder exists independent camera coordinate system each other. It is needed to integrate each range maps to acquire all around figure. Heretofore, we integrate multiple range maps by estimating camera parameter using calibration rig whose scale is known. Calibration rig has many feature points which have world coordinate each. Camera parameter is estimated from the relation between camera coordinate and world coordinate of feature points on the rig. However, we cannot identify the measuring position when we measure a part of the rig, because the pattern painted on the rig is monotonous pattern. In this paper, we use "De Bruijn Sequence pattern" on the calibration rig. This pattern enables us to identify the measuring spot wherever we measure the rig, and enable us to automatically integrate multiple range maps.

In this paper we present experimental evidence of the redundancy of depth maps for 3D visualization. To this end, we performed a series of statistical experiments, devised to measure the effect of depth map quantization and the resolving power of 3D perception. The results of these tests show that for good 3D perception and 3D visualization, one does not need to use depth map of the same resolution neither with the same quantization as the original images. These results indicate that depth map based visualization can be based on low resolution, coarsely quantized, depth maps without significant degradation in the perceived 3D image.

Full field range imaging cameras are used to simultaneously measure the distance for every pixel in a given scene using an intensity modulated illumination source and a gain modulated receiver array. The light is reflected from an object in the scene, and the modulation envelope experiences a phase shift proportional to the target distance. Ideally the waveforms are sinusoidal, allowing the phase, and hence object range, to be determined from four measurements using an arctangent function. In practice these waveforms are often not perfectly sinusoidal, and in some cases square waveforms are instead used to simplify the electronic drive requirements. The waveforms therefore commonly contain odd harmonics which contribute a nonlinear error to the phase determination, and therefore an error in the range measurement. We have developed a unique sampling method to cancel the effect of these harmonics, with the results showing an order of magnitude improvement in the measurement linearity without the need for calibration or lookup tables, while the acquisition time remains unchanged. The technique can be applied to existing range imaging systems without having to change or modify the complex illumination or sensor systems, instead only requiring a change to the signal generation and timing electronics.

In order to measure the 3D structure of a number of objects a comparably new technique in computer vision exists, namely time of flight (TOF) cameras. The overall principle is rather easy and has been applied using sound or light for a long time in all kind of sonar and lidar systems. However in this approach one uses modulated light waves and receives the signals by a parallel pixel array structure. Out of the travelling time at each pixel one can estimate the depth structure of a distant object. The technique requires measuring the intensity differences and ratios of several pictures with extremely high accuracy; therefore one faces in practice rather high noise levels. Object features as reflectance and roughness influence the measurement results. This leads to partly high noise levels with variances dependent on the illumination and material parameters. It can be shown that a reciprocal relation between the variance of the phase and the squared amplitude of the signals exists. On the other hand, objects can be distinguished using these dependencies on surface characteristics. It is shown that based on local variances assigned to separated objects appropriate denoising can be performed based on Wavelets and edge-preserving smoothing methods.

Although range scanning technology has offered great improvements to digital model creation in recent years, it has also introduced some new concerns. Specifically, recent work shows that topological errors such as tiny handles can significantly lower the overall quality of range-scanned models for down-stream applications (such as simplification and parameterization). In this paper we present our investigation into the source of this topological error in the range scanning process, and our methods to alleviate the error. We concentrated our investigation of the scanning process on: (1) signal noise or calibration error in the laser scanner (resulting in bad data points) and (2) error during the model reconstruction phase. We found that by modifying the surface reconstruction phase of the range scanning process, we were able to reduce the amount of topological noise in the resulting 3D model by up to 60 percent.

We have developed a series of new quality metrics that are generalizable to a variety of laser range scanning systems, including those acquiring measurements in the mid-field. Moreover, these metrics can be integrated into either an automated scanning system, or a system that guides a minimally trained operator through the scanning process. In particular, we represent the quality of measurements with regard to aliasing and sampling density for mid-field measurements, two issues that have not been well addressed in contemporary literature. We also present a quality metric that addresses the issue of laser spot motion during sample acquisition. Finally, we take into account the interaction between measurement resolution and measurement uncertainty where necessary. These metrics are presented within the context of an adaptive scanning system in which quality metrics are used to minimize the number of measurements obtained during the acquisition of a single range image.

The knowledge of the interior structure of buildings is important in many civilian and military applications. While it is possible to obtain interior maps by exhaustively inspecting every room in a building, in many situations this is infeasible due to security or safety reasons. In this paper, we develop a method to generate interior building floor plans from the exterior of the structure. We use a laser scanner to measure the range of hundreds of thousands of points on interior walls of the building, exploiting the fact that the laser can go through unobstructed windows. We develop an algorithm to fit planes to the point cloud resulting from the laser data. To accomplish this, the optimal locations of horizontal planes are found such that they model the ceiling and the floors; subsequently, vertical planes are placed perpendicular to those and aligned to fit the data. Once these planes are found and localized, floor plans for each floor are extracted. We show that our proposed method is effective in recovering partial floor plans for three separate building examples.

To construct a 3D model of an environment with minimal information loss, integrated sensor data from many viewpoints are needed. However, this increases the amount of useless data, and it also takes too much time. Based on this problem, we developed a scalable sensing scheme using a robot system to reconstruct indoor environments. We mounted a freely rotating range sensor on a mobile robot to acquire range data in a real office environment, and we constructed a 3D model of that environment. The scheme determines the frequency of the measurement in each direction according to the complexity of the shape. If the shape on the direction of the sensor's angle is simple, the frequency of the measurement becomes low. On the other hand, if the shape is compleex, the frequency is high. It also does not acquire data in areas that have alreaedy been measured. The results showed that a 3D model can be constructed with less frequent measurements with our scheme.

To overcome the viewing resolution limit defined by the Nyquist sampling theorem for a given lenslet pitch, a Moving Array-Lens Technique (MALT) was developed in 3-D integral imaging technique. Even though the MALT is an effective method for resolution improvement of Integral Imaging, this cannot be applied to a real-time 3-D integral imaging display system because of its mechanical movement. In this paper, we propose an integral imaging display using a computational pick-up method based on Intermediate-View Reconstruction Technique instead of optical moving pickup. We show that the proposed system can provide optically resolution-improved 3-D images of integral imaging by use of EIs generated by the IVRT through the optical experiments.

Advances in rapid prototyping technologies have led to the emergence of three-dimensional printers which can fabricate physical artefacts, including the application of surface colours. In light of these developments, this paper asserts that the need to print colour accurately is just as important for designers using three-dimensional colour printing as it is for two-dimensional inkjet printing. Parallels can be made with two-dimensional digital Inkjet printing and 2D common problems include: on screen previsualisation, colour management methods, colour gamut and maintaining colour accuracy. However, for three dimensional colour printed objects, there are more variables that will affect the finished colour. These are: the powder and process inks, unevenness of the surface, wax post-processing and other infiltration media and procedures. Furthermore, in some 3D printers, the K channel is replaced by the binder and so the printer is only using the cyan, magenta and yellow channels. The paper will suggest methods for improving pre-visualisation and accurate pre-viewing of the colours through the manufacture of three-dimensional colour charts as a reference guide for designers so that they can make accurate coloured artefacts. A series of case studies will be demonstrated.

In this paper we present a novel sensor system for three-dimensional face scanning applications. Its operating principle is based on active triangulation with a color coded light approach. As it is implemented in the near infrared band, the used light is invisible for human perception. Though the proposed sensor is primarily designed for face scanning and biometric applications, its performance characteristics are beneficial for technical applications as well. The acquisition of 3d data is real-time capable, provides accurate and high resolution depthmaps and shows high robustness against ambient light. Hence most of the limiting factors of other sensors for 3d and face scanning applications are eliminated, such as blinding and annoying light patterns, motion constraints and highly restricted scenarios due to ambient light constraints.

A common limitation of laser line three-Dimensional (3D) scanners is the inability to scan objects with surfaces that are either parallel to the laser line or that self-occlude. Filling in missing areas adds some unwanted inaccuracy to the 3D model. Capturing the human head with a Cyberware PS Head Scanner is an example of obtaining a model where the incomplete areas are difficult to fill accurately. The PS scanner uses a single vertical laser line to illuminate the head and is unable to capture data at top of the head, where the line of sight is tangent to the surface, and under the chin, an area occluded by the chin when the subject looks straight forward. The Cyberware PX Scanner was developed to obtain this missing 3D head data. The PX scanner uses two cameras offset at different angles to provide a more detailed head scan that captures surfaces missed by the PS scanner. The PX scanner cameras also use new technology to obtain color maps that are of higher resolution than the PS Scanner. The two scanners were compared in terms of amount of surface captured (surface area and volume) and the quality of head measurements when compared to direct measurements obtained through standard anthropometry methods. Relative to the PS scanner, the PX head scans were more complete and provided the full set of head measurements, but actual measurement values, when available from both scanners, were about the same.

The calibration method for a multiple range finders system is proposed in this paper. Generally, multiple rangefinders are required to obtain the whole surface shape of a large object such as a human body. The proposed method solves range data registration by a prior calibration using a reference plane with rectangular markers. The world coordinates system is defined on the reference plane. Because range data have about two hundred thousands of 3-D points, the normal vector of the reference plane is accurately estimated by fitting the regression plane to the 3-D points. If the Z-axis of the world coordinates system for our calibration method is defined as the axis which cross meets the reference plane, it is determined by the normal vector. On the other hand the X and Y axes are defined as the horizontal line and vertical line of rectangular markers. They are determined by detecting and extracting the rectangular markers from the intensity image. Therefore, the orientation of each rangefinder is estimated based on the world coordinates system. In the experiments, the multiple rangefinders system which consists of twelve rangefinders is used. Experimental results indicate that the RMSE is 2.3 mm in the case of measuring a cylinder object.

This paper presents the preliminary results of applying multi image photogrammetric techniques for the three-dimensional monitoring of the intraoperative brainshift. The "brainshift" is the motion of cerebral structures occurring in neurosurgery after the craniotomy (opening of the skull in order to access the brain for surgical repair). The causes of this effect are mainly the changes of pressure and loss of cerebrospinal liquid. The phenomenon of brainshift can influence negatively the planning and execution of neurosurgical intervention. A research project at the Clinic of Neuroradiology and Neurosurgery of the Medical University of Innsbruck (Austria) aims at the quantification of the intraoperative brainshift by means of photogrammetry. The goals of the project are: (i) the development of a multi-image photogrammetric system for the quantitative monitoring of intraoperative brainshift by means of 3D measurements performed on the surface of the brain during neurosurgery after craniotomy, (ii) transformation of the pre-operative performed MR and CT datasets in function of the quantified intra-operative brainshift. This paper presents the proposed multi-image photogrammetric system, as well as, the first results achieved, in collaboration with Hometrica Consulting, for the automatic 3D measurement and tracking of selected points on the surface of the brain.

In this paper, we present a stochastic framework for articulated 3D human motion tracking. Tracking full body human motion is a challenging task, because the tracking performance normally suffers from several issues such as self-occlusion, foreground segmentation noise and high computational cost. In our work, we use explicit 3D reconstructions of the human body based on a visual hull algorithm as our system input, which effectively eliminates self-occlusion. To improve tracking efficiency as well as robustness, we use a Kalman particle filter framework based on an interacting multiple model (IMM). The posterior density is approximated by a set of weighted particles, which include both sample means and covariances. Therefore, tracking is equivalent to searching the maximum a posteriori (MAP) of the probability distribution. During Kalman filtering, several dynamical models of human motion (e.g., zero order, first order) are assumed which interact with each other for more robust tracking results. Our measurement step is performed by a local optimization method using simulated physical force/moment for 3D registration. The likelihood function is designed to be the fitting score between the reconstructed human body and our 3D human model, which is composed of a set of cylinders. This proposed tracking framework is tested on a real motion sequence. Our experimental results show that the proposed method improves the sampling efficiency compared with most particle filter based methods and achieves high tracking accuracy.

This paper presents the FaceTOON system, a semi-automatic platform dedicated to the creation of verbal and emotional facial expressions, within the applicative framework of 2D cartoon production. The proposed FaceTOON platform makes it possible to rapidly create 3D facial animations with a minimum amount of user interaction. In contrast with existing commercial 3D modeling softwares, which usually require from the users advanced 3D graphics skills and competences, the FaceTOON system is based exclusively on 2D interaction mechanisms, the 3D modeling stage being completely transparent for the user. The system takes as input a neutral 3D face model, free of any facial feature, and a set of 2D drawings, representing the desired facial features. A 2D/3D virtual mapping procedure makes it possible to obtain a ready-for-animation model which can be directly manipulated and deformed for generating expressions. The platform includes a complete set of dedicated tools for 2D/3D interactive deformation, pose management, key-frame interpolation and MPEG-4 compliant animation and rendering. The proposed FaceTOON system is currently considered for industrial evaluation and commercialization by the Quadraxis company.

Physical security increasingly involves sophisticated, real-time visual tracking of a person's location inside a given environment, often in conjunction with biometrics and other security-related technologies. However, demanding real-world conditions like crowded rooms, changes in lighting and physical obstructions have proved incredibly challenging for 2D computer vision technology. In contrast, 3D imaging technology is not affected by constant changes in lighting and apparent color, and thus allows tracking accuracy to be maintained in dynamically lit environments. In addition, person tracking with a 3D stereo camera can provide the location and movement of each individual very precisely, even in a very crowded environment. 3D vision only requires that the subject be partially visible to a single stereo camera to be correctly tracked; multiple cameras are used to extend the system's operational footprint, and to contend with heavy occlusion. A successful person tracking system, must not only perform visual analysis robustly, but also be small, cheap and consume relatively little power. The TYZX Embedded 3D Vision systems are perfectly suited to provide the low power, small footprint, and low cost points required by these types of volume applications. Several security-focused organizations, including the U.S Government, have deployed TYZX 3D stereo vision systems in security applications. 3D image data is also advantageous in the related application area of gesture tracking. Visual (uninstrumented) tracking of natural hand gestures and movement provides new opportunities for interactive control including: video gaming, location based entertainment, and interactive displays. 2D images have been used to extract the location of hands within a plane, but 3D hand location enables a much broader range of interactive applications. In this paper, we provide some background on the TYZX smart stereo cameras platform, describe the person tracking and gesture tracking systems implemented on this platform, and discuss some deployed applications.

We present the development of advanced neural network and 3D image processing algorithms to identify hand gestures for a novel Surgeon-Computer-Interface (SCI) in the operating room. Feature extraction methods have been identified to reliably extract unique attributes and recognize dynamic hand gestures such as "Point and Click" and "Hand Waiving" features. We show an experimental demonstration of a non-linear neural network classifier that is capable of reliably recognizing 8 complex hand gesture patterns.

Many low or middle level 3D reconstruction algorithms involve a robust estimation and selection step by which parameters of the best model are estimated and inliers fitting this model are selected. The RANSAC algorithm is the most widely used robust algorithm for this step. However, this robust algorithm is computationally demanding. A new version of RANSAC, called distributed RANSAC (D-RANSAC), is proposed in this paper to save computation time and improve accuracy. We compare our results with those of classical RANSAC and another state of the art version of it. Experiments show that D-RANSAC is superior to RANSAC in computational complexity and accuracy, and comparable with other proposed improved versions.

This article presents an analysis of thermal influences on the image acquisition process of an electronic camera. It is shown that temperature changes lead to thermal expansion of the mechanical camera components and thus to a displacement of the camera sensor and/or to a displacement of the center of projection. This change in the imaging geometry leads to changing camera parameters which have to be adjusted if the camera is used for high accurate measurements. The slight change in the imaging geometry can be determined by an analysis of the optical flow of a static calibration pattern. The dependency between temperature change and the change of the camera parameters can be modeled using methods from linear time invariant system theory. Once the model is determined for a specific camera it can be used for adjusting the camera parameters according to the current temperature and thus increasing the accuracy of an optical measurement device.

This paper presents preliminary results on the development of a 3D audiovisual model of the Anta Pintada (painted dolmen) of Antelas, a Neolithic chamber tomb located in Oliveira de Frades and listed as Portuguese national monument. The final aim of the project is to create a highly accurate Virtual Reality (VR) model of this unique archaeological site, capable of providing not only visual but also acoustic immersion based on its actual geometry and physical properties. The project started in May 2006 with in situ data acquisition. The 3D geometry of the chamber was captured using a Laser Range Finder. In order to combine the different scans into a complete 3D visual model, reconstruction software based on the Iterative Closest Point (ICP) algorithm was developed using the Visualization Toolkit (VTK). This software computes the boundaries of the room on a 3D uniform grid and populates its interior with "free-space nodes", through an iterative algorithm operating like a torchlight illuminating a dark room. The envelope of the resulting set of "free-space nodes" is used to generate a 3D iso-surface approximating the interior shape of the chamber. Each polygon of this surface is then assigned the acoustic absorption coefficient of the corresponding boundary material. A 3D audiovisual model operating in real-time was developed for a VR Environment comprising head-mounted display (HMD) I-glasses SVGAPro, an orientation sensor (tracker) InterTrax 2 with 3 Degrees Of Freedom (3DOF) and stereo headphones. The auralisation software is based on a geometric model. This constitutes a first approach, since geometric acoustics have well-known limitations in rooms with irregular surfaces. The immediate advantage lies in their inherent computational efficiency, which allows real-time operation. The program computes the early reflections forming the initial part of the chamber's impulse response (IR), which carry the most significant cues for source localisation. These early reflections are processed through Head Related Transfer Functions (HRTF) updated in real-time according to the orientation of the user's head, so that sound waves appear to come from the correct location in space, in agreement with the visual scene. The late-reverberation tail of the IR is generated by an algorithm designed to match the reverberation time of the chamber, calculated from the actual acoustic absorption coefficients of its surfaces. The sound output to the headphones is obtained by convolving the IR with anechoic recordings of the virtual audio source.

We have been researching three-dimensional (3D) reconstruction from images captured by multiple cameras. Currently, we are investigating how to convert 3D models into stereoscopic images. We are interested in integral photography (IP), one of many stereoscopic display systems, because the IP display system reconstructs complete 3D auto-stereoscopic images in theory. This system consists of a high-resolution liquid-crystal panel and a lens array. It enables users to obtain a perspective view of 3D auto-stereoscopic images from any direction. We developed a method for converting 3D models into IP images using the OpenGL API. This method can be applied to normal CG objects because the 3D model is described in a CG format. In this paper, we outline our 3D modeling method and the performance of an IP display system. Then we discuss the method for converting 3D models into IP images and report experimental results.

We describe a method that synthesizes a high-quality arbitrary-viewpoint image from multiple weakly-calibrated cameras. We reconstructed the approximate shape of the object reconstructed by the shape from silhouette method on a projective grid space(PGS). However, the distortion is caused in the synthesized image by the difference with the actual object and the approximate shape. In particular, the distortion is caused by a concave region. Our method synthesizes the arbitrary-viewpoint image with a little distortion by revising a depthmap obtained from the approximate shape. The depthmap is revised by using the all in-focus rendering method that makes light field rendering expand. When all camera parameters were uncalibrated, we could not apply this rendering method. Our method enabled applying uncalibrated cameras using the all in-focus rendering method by calculating each camera parameter on the PGS. We demonstrate the effectiveness of the method with arbitrary-viewpoint images and real images that were taken from multiple uncalibrated cameras.

We have developed an improved version of a high-definition real-time depth-mapping Axi-Vision Camera that enhances the speed of the depth detection, reduces external light interference, and is more compact. The new depth-detection method uses a high-resolution, fast-frame-rate CMOS sensor with a rolling shutter. The depth is calculated from four near-infrared light (NIR) images in successive video frames to cancel the effects of external light interference. Therefore, the external light interference immunity is improved so that the depth information can be detected accurately under external light at an intensity up to 30% of that of NIR light illumination. Furthermore, the compact-imaging optical system is achieved by using a specially designed HDTV camera lens, which internally separates only NIR light and provides NIR images on an image intensifier coupled with a CMOS sensor. Further compact sizing of the camera system is achieved by arranging specially developed small, high-power reflection-type LED arrays for the NIR illumination around the camera lens. As a result of this compact sizing, the volume is 1/5 that of the prototype system. We also demonstrated a method to obtain 3-D information from image data captured by this Axi-Vision Camera.

We have developed the small three dimensional image capture system by Thin Observation Module by Bound Optics(TOMBO). As a micro lens array, we used GRIN lens array. It decreases a cross talk which lens-to-lens occurs on image sensor. This module uses a micro-lens array to form multiple images, which are captured on a photo-detector array. Digital processing of the captured multiple images is used to extract the surface profile. Preliminary experiments were executed on an evaluation system to verify the principles of the system. In this paper, we have proposed ultra thin three dimensional capture system. A compound-eye imaging system and post-processing are employed. Experimental results verify the principle of the proposed method and show the potential capability of the proposed system architecture.

For the sake of providing 3D contents for up-coming 3D display devices, a real-time automatic depth fusion 2D-to-3D conversion system is needed on the home multimedia platform. We proposed a priority depth fusion algorithm with a 2D-to-3D conversion system which generates the depth map from most of the commercial video sequences. The results from different kinds of depth reconstruction methods are integrated into one depth map by the proposed priority depth fusion algorithm. Then the depth map and the original 2D image are converted to stereo images for showing on the 3D display devices. In this paper, a 2D-to-3D conversion algorithm set is combined with the proposed depth fusion algorithm to show the improved results. With the converted 3D contents, the needs for 3D display devices will also increase. As long as the two technologies evolve, the 3D-TV era will come as soon as possible.