In Volume 34 (1995) we published 3427 technical pages, a downturn from Volume 33. Figure 1 shows the number of technical pages published per year since 1977. I did state in my editorial of March 1995 that ‘‘we do not project this [1993–94] rate of increase to continue for 1995, but rather expect a flattening of the curve.

This PDF file contains the editorial “Sensor Fusion 1996” for OE Vol. 35 Issue 03

Recursive strategies for enhancing the reliability of decisions in multisensor environments through temporal fusion of the individual imperfect decisions are presented in this study. A confidence measure can be associated with each imperfect decision and as such, decisions obtained with a confidence less than an acceptable threshold value are viewed as nondecisions in developing the fused decision. The relative merits of these alternative strategies are discussed and quantitatively compared to identify the most appropriate overall set of strategies for decision fusion in imperfect multisensor environments. Initial, recursive phase, and asymptotic performances of these alternative strategies are compared and discussed to delineate the most appropriate overall operational strategy.

The problem of distributed decision fusion is studied in the case when the probability distributions of the individual detectors are not available. The detector system is available so that a training sample can be generated by sensing objects with known parameters or classification. Earlier solutions to this problem required some knowledge of the error distributions of the detectors, for example, either in a parametric form or in a closed analytical form. Here we present three methods that, given a sufficiently large training sample, yield an approximation to the optimal fusion rule with an arbitrary level of confidence. These methods are based on (i) empirical estimation, (ii) approximate decision rule, and (iii) nearest-neighbor rule. We show that a nearest-neighbor rule provides a computationally viable solution, which approximates a neural network-based one while ensuring fast computation.

A decentralized detection system operating in asynchronous mode is presented. The local sensors generate their decisions asynchronously and convey them to a global decision maker (GDM). A sequential decision fusion scheme is implemented at the GDM in order to reach a global decision. It is assumed that local decisions are generated according to a Poisson process at each local sensor. A performance comparison with the equivalent fixed-sample size test is presented. Truncation of the sequential test is also addressed.

Alternative paradigms for fusion of a mix of information at the data, feature, and decision levels, acquired from multiple sources (sensors as well as feature extractors and/or decision processors) are explored in this study. Four alternative approaches—a self-partitioning neural net, an adaptive fusion process, an evidential reasoning approach, and a concurrence-seeking approach, were selected for consideration and relatively evaluated from a conceptual viewpoint followed by some limited simulation and testing. As a result of this relative assessment, an adaptive fusion processor, based on certain innovative advances of the nearest-neighbor concept which were developed previously in the context of automatic target recognition (ATR) designed for operation in partially exposed environments, was selected for detailed implementation and testing. Details of this adaptive fusion processor are presented here along with examples of tests conducted using some real-world field data which clearly bring out the feasibility and effectiveness of the new multilevel fusion concepts. The results show the benefits of fusion in terms of improved performance compared with those obtainable from the individual component information streams being input to the fusion processor.

A pixel-based color-mapping algorithm is presented that produces a fused false color rendering of two gray-level images representing different sensor modalities. The resulting images have a higher information content than each of the original images and retain sensorspecific image information. The unique component of each image modality is enhanced in the resulting fused color image representation. First, the common component of the two original input images is determined. Second, the common component is subtracted from the original images to obtain the unique component of each image. Third, the unique component of each image modality is subtracted from the image of the other modality. This step serves to enhance the representation of sensor-specific details in the final fused result. Finally, a fused color image is produced by displaying the images resulting from the last step through, respectively, the red and green channels of a color display. The method is applied to fuse thermal and visual images. The results show that the color mapping enhances the visibility of certain details and preserves the specificity of the sensor information. The fused images also have a fairly natural appearance. The fusion scheme involves only operations on corresponding pixels. The resolution of a fused image is therefore directly related to the resolution of the input images. Before fusing, the contrast of the images can be enhanced and their noise can be reduced by standard image-processing techniques. The color mapping algorithm is computationally simple. This implies that the investigated approaches can eventually be applied in real time and that the hardware needed is not too complicated or too voluminous (an important consideration when it has to fit in an airplane, for instance).

This paper presents a technique for fusing radar and imaging sensor data for target tracking. First, it is shown how angular position measurements can be computed from the two-dimensional intensity pattern provided by the imaging sensor. Then a model for the angular measurements is developed and accurate noise statistics are given. The measurement model that is derived differs from the one usually used in data fusion applications involving imaging sensors. In particular, it is shown that the noise is signal dependent and this affects the filtering equations which form the basis of the fusion operation. It is also shown that the two angular coordinates should not be decoupled because of the noise cross-correlation. The relationship between the imaging sensor’s angular measurements and the radar’s angular measurements is then explored by considering a simple example of a two-point target. It is shown that we cannot assume in general that the two sensors are measuring the same target point and that it is necessary to accommodate the difference between the two angular measurements within the data fusion process. A simple technique for accommodating this difference is developed and the equations necessary for fusing such measurements within the filtering framework are then presented. Finally, at the end of the paper it is pointed out that the same techniques can be applied to data fusion when tracking targets at very long ranges. The techniques developed in this paper are evaluated using a computer simulation with simulated data.

The integration of data obtained from several different sensors has often been proposed as a strategy by which accurate estimates of the values of physical variables being measured may be obtained when the sensor data are corrupted by noise. This paper considers a pair of detection-localization sensor systems that have capabilities complementing each other. One has higher resolution than the other but is more susceptible to non-Gaussian multiplicative and additive noise than the sensor with lower resolution. Both are subject to additive Gaussian white noise. Studies have been made in the past to characterize such systems and to make accurate estimates of signals of interest. We propose an alternative computational framework that makes fewer assumptions and thereby makes the system more realistic. The distinguishing feature of our method is that our solution involves only polynomial time and space complexity and hence is well suited for use in realtime applications. Extensive simulation results are included to prove the effectiveness of our solution under varied random noise levels in the sensor data.

Modern combat systems based on active radar sensing suffer disadvantages against low-flying targets in cluttered backgrounds. Use of passive infrared sensors with these systems, either in cooperation or as an alternative, shows potential for improving target detection and declaration range for targets crossing the horizon. Realization of this potential requires fusion of target position data from dissimilar sensors, or passive sensor measurement of target range. The availability of passive sensors that can supply both range and bearing data on such targets would significantly extend the robustness of an integrated ship selfdefense system. We consider a new method of range determination with passive sensors based on the principle of triangulation, extending the principle to two orthogonal baselines. The performance of single or double baseline triangulation depends on sensor bearing precision and direction to target. An expression for maximum triangulation range at a required accuracy is derived as a function of polar angle relative to the center of the dual-baseline system. Limitations in the dual-baseline model due to the geometrically assessed horizon are also considered.

Data association is the process of relating sensor measurements in a data fusion system. It can be structured in a basic framework very similar to that of the classic traveling salesman problem. The derivation of the energy function is presented, and the solution is based on a modified Hopfield network which uses the Runge–Kutta method and Aiyer’s network structure. The neural data association is then applied to the problem of multiple-target tracking (MTT). The proposed neural MTT system consists of a modified Hough transform track initiator, a Kalman filter state estimator and the Hopfield probabilistic data association. Real-life air surveillance data are used to evaluate the practicality of the neural MTT system, and the results show that the neural system works efficiently in real-life tracking environments.

The problem of target classification with high-resolution, fully polarimetric, synthetic aperture radar (SAR) imagery is considered. We propose a framework of using a Bayesian network for feature fusion to deal with the difficult problem of SAR target classification. One difficult problem in SAR feature identification and fusion for target classification is that the features identified may not be independent and that it is not easy to find the ‘‘right’’ fusion rule to combine them. The Bayesian network model when constructed properly can explicitly represent the conditional independence and dependence between various features and therefore provide a sound and natural framework for feature fusion. This paper summarizes our recent work in SAR target recognition using a feature-based Bayesian inference approach. The approach works on the selected features which are chosen so that the separability of the original data are well maintained for later classification. Once the original data are mapped into feature space, the probabilistic model between features and the target is estimated and represented by a Bayesian network, which is then used to calculate the probabilities that a target belongs to one of the given classes based on the observed features. A comparison between the above technique and the traditional statistical approaches such as nearest mean and Fisher pairwise is illustrated based upon performance on a fully polarimetric ISAR (inverse SAR) image data set. Note that although the feature set used in the paper is obtained from the same sensor, the concepts of feature selection and Bayesian network formulation discussed in the paper are not restricted to this case only. They can be applied for multisensor feature-level fusion as well.

The fusion of information from different sensors can provide features that cannot be obtained from either of the sensors. In this paper we examine a feature that is obtained from the fusion of information sensed from thermal (long wave infrared) and visual imagery. Robust object recognition requires object features that are invariant to scene conditions and viewpoint. Previous effort in developing such features using explicit constraints of invariance has mostly considered only visual data. The investigation of the variation of quasi-invariant features for different ranges of viewing/scene parameters has also been limited to geometric features derived from visual imagery. Such investigation has not been performed for features derived from multisensory imagery or from thermal (infrared) imagery. In this paper we conduct a detailed analysis of the variation of invariant and quasi-invariant features that have been proposed for the analysis of thermal (infrared) imagery, and also for the integrated analysis of thermal and visual imagery. The features are based on thermophysical models of energy exchange between the object and the scene. Examination of feature variation is based on simulating the various scene and object energy components for varying scene parameter values and predicting the feature value. This approach eliminates the expensive and impractical task of collecting real imagery under all possible scene conditions. We develop thermophysical object models based on equivalent thermal circuits. The models use a small number of nodes to predict the surface temperatures (thermal image gray levels) and to predict the feature values on the surfaces of complex objects. This approach reduces the computational cost of simulation and facilitates model construction. The sensitivity of the features to object and scene parameter variations is evaluated. The features are shown to be relatively robust within specific ranges of scene conditions.

A novel optical time-division multiple-access (OTDMA) technique is introduced that can offer multirate data communications. The proposed multirate time-division multiplexing and demultiplexing can be implemented in the optical domain by using simple optical delay lines. The architectures for all-optical clock-rate multipliers are also presented. It is shown that the complexity of the proposed OTDMA network is almost the same as that of an equal-rate OTDMA network, but using our scheme can significantly improve the network flexibility.

A novel design for imitating continuous phase profiles by binary subwavelength structures is presented. The working principle has recently been shown for regular gratings like beam deflectors (Haidner et al., 1993) or polarizing beamsplitters (Collischon et al., 1994). Our objective is to transfer this principle to non-strictly periodic gratings such as lenses and further to avoid the strong polarization dependence by using 2-D modulated subwavelength structures. A further objective is to examine the origin of this effect. Two possibilities exist—the resonance effect, because the structure size is nearly as large as the wavelength, and the averaging effect, which should be responsible if the light can not resolve the small structures.

The use of the ABCD matrix formalism to study fractional Fourier transforms (FRFTs) opens new perspectives on the optical implementation of the FRFTs and on the analysis and synthesis of optical systems. ABCD matrices of basic geometries implementing perfect and imperfect FRFTs are presented. The ABCD matrix method is used to demonstrate the cascading performances of FRFT elementary units and to support the established cascading rules. The matrix formalism is used to show that real and complex optical systems realize FRFTs of complex order.

A novel analytical model is provided to evaluate the performance of subcarrier modulated optical heterodyne frequency modulation (FM) using a distributed feedback (DFB) laser with nonuniform FM response taking into account the combined effect of laser phase noise and receiver noise. Two different subcarrier signals, viz. minimum shift keying (MSK) and biphase, are used for direct frequency modulation of the DFB laser diode. The modified Monte-Carlo simulation of the system using the ‘‘importance sampling’’ technique is also carried out. The MSK subcarrier modulated optical FM is found to counteract effectively the degrading effect of nonuniform FM response leaving only a small penalty of 0.2 dB and provides better performance than biphase frequency shift keying.

The influence of the image acquisition by a color CCD camera in color pattern recognition is studied. Two different types of cameras are used: a camera with a single CCD element and color filters (1CCD camera), and a camera with dichroic mirrors and three CCD elements (3CCD camera). Two problems are considered: in the first problem, both the target and the scene are captured by the camera; in the second problem, only the scene is captured by the camera and the target is computer generated. A wide range of gain/offset values are used for image acquisition and pattern recognition results are obtained by computer simulation. The results reveal that the 3CCD camera is much more suitable than the 1CCD camera for pattern recognition purposes.

According to the principle of human discrimination of small targets from a natural scene that there is a signature of discontinuity between the object and its neighboring regions, we develop an efficient method for multiscale small target detection using template matching based on a dissimilarity measure, which is called an average gray absolute difference maximum map (AGADMM), and infer the criterion of recognizing multiscale small objects from the properties of the AGADMM of the natural scene, which is a spatially independent and stable Gaussian random field. We explain how the AGADMM increases the ratio of the signal of object-to-background perturbations, improves the detectable probability, and keeps the false alarm probability very low. We analyze the complexity of computing an AGADMM and justify the validity and efficiency. Experiments with images of a natural scene such as a sky and sea surface have shown the great potential of the proposed method for distinguishing multiscale small objects from a natural scene.

A monochromatic optical field carrying an image is known to be scrambled due to propagation through a multimode fiber. Two schemes of the image acquisition through such a fiber are suggested. Both are based on the assumption that the input-to-output propagation Green function is stabilized by some technique. In both schemes the object is characterized by the transverse distribution of reflection pattern, and the reflected signal is to be collected by a photodetector for further computer processing. The applications for endoscopy are discussed.

We built a new system for object recognition by combining the joint transform structure with the wavelet transform. The system can perform preprocessing and correlation in a single step. It has many good features including high antinoise capability and sharp correlation peaks without sidelobes. To test the effectiveness of this system, experimental results on a plane model are also given.

Lossy compression drastically reduces the operating costs of digital medical imaging systems by enabling more efficient use of transmission and archival facilities in hospitals and doctor’s offices. To be able to develop standards for the application of this technology, reliable tools are needed for measuring the quality of reconstructed images. Among the most common measures presently used, the normalized mean squared error (NMSE) does not provide any information concerning the type of impairment, and receiver operating characteristic (ROC) analyses are expensive and time-consuming. This paper evaluates the performance of three quantitative multidimensional measures for image quality. Mimicking the human visual system, they compute local features, and produce a graphical output. Improved Eskicioglu charts, in particular, are shown to be appropriate tools for characterizing compression losses in reconstructed medical images.

The extraction of spatial information, specifically the length scale, from imagery of background scenes can be indirectly made using an estimation of the correlation length. However, sampling frequency and the record size of the data affect the precision of the resultant estimation. The resolving power of the correlation length as a function of the image scale is examined in detail. Several models designed to estimate image length scale s from computed correlation length xL are studied. The relative accuracy of the model, as well as the performance of different integration strategies for computing xL are examined. It is found that s -dependent integration schemes are superior, although their use necessitates a more involved ‘‘fixed point’’ procedure to estimate s.

A technique for realizing accurate phase measurement based on additive-subtractive speckle pattern interferometry is presented. Additive fringe patterns corresponding to two deformation states of an object vibrating sinusoidally are first obtained and these are then subtracted from that with zero deformation. The technique requires five frames of additive speckle patterns S0, S1, ... , S4, where S0 is taken with the laser illumination pulsed at instants when the vibration reaches its two zero amplitude positions within one harmonic vibration cycle of the object, and the remaining four (S1 through S4) are taken with the illumination pulses arranged at times when the vibration reaches its maximum and minimum amplitudes. During the acquisition of S1 through S4, the phase of the reference beam is appropriately shifted between the two pulses within each vibration cycle. The phase shifts are synchronized with the pulses. Speckle patterns S1 through S4 are then subtracted from S0 to generate four correlation fringe patterns with relative phase shifts of 0, ?/2, ? and 3?/2, respectively. These fringe patterns are then utilized to derive a phase map using a four-step phase calculation algorithm and quantitative vibration amplitudes can be readily obtained from the phase map. Results obtained from electronic speckle pattern interferometry and shearography are presented, and the ability of the technique to perform in relatively severe noisy environments is demonstrated.

A novel method is presented for precision displacement measurement based on active heterodyne interferometry. A pair of laser diodes, each of which is equipped with an external cavity, are used as light sources. A small displacement of a piezoelectric transducer, which carries the reflecting mirror of the external cavity, causes lasing frequency shift due to optical feedback on the laser diode. Detection of the optical frequency shift is accomplished by a heterodyne arrangement. Direct counting of the heterodyne beat frequency enables precise measurement of a dynamic displacement without any complex numerical calculation. Excellent performance of the presented method is illustrated by experimental results on the measurement for small displacement with an accuracy of ±5 nm and a time resolution of 33 ms.

A simple technique to measure the phase retardation property as a function of the driving voltage of a liquid crystal television (LCTV) accurately using Young’s double-slit interference principle is presented. The phase property of an Epson LCTV is characterized by measuring the irradiance change at a fixed position at the center of the Young’s interference pattern. The phase retardation is also characterized by measuring the fringe shift as previous authors have done, and the measured results obtained with the two methods are consistent.

Optical design methodology for a dual-beam combiner head-up display (HUD) is discussed. It has been shown that binocular disparity and image doubling are the deciding factors for image quality for such systems and not resolution. Consequently there should be much tighter control over aberrations in such systems than otherwise dictated by resolution requirement only. An actual dual-beam combiner HUD is designed and various plots describing its performance are shown. This system is almost diffraction limited as dictated by the requirement of strict control over binocular disparity and image doubling.

The effects of changing the center flint glass of a Cooke triplet anastigmat from the optimum glass type are investigated in an attempt to determine how many different glass types are necessary. To this end, about 90 triplets were designed, each with a flint glass displaced from its optimum index and dispersion. Sixteen different combinations of field and aperture and three different crown glass types were used. Variations of performance with flint changes are presented. An ‘‘acceptable’’ variation of the flint glass V value appears to be approximately ±2.

Optical interconnects that rely on holographic optical elements commonly suffer from strong chromatic aberrations. These aberrations limit the coupling efficiency of optical links if the laser wavelength varies with time or from sample to sample. Semiconductor lasers are the only candidates for optical links because of their efficiency and compactness. Unfortunately, these lasers show a strong tendency to wavelength drifts. Therefore, the flexibility of holographic interconnects can be ensured only if the chromatic aberrations are reduced. Four different achromatized optical interconnecting schemes are discussed that have been under investigation for the last few years in our department. The achromatization efforts for space-invariant and space-variant interconnects are summarized.

Two methods that exploit the capabilities of optical pattern recognition techniques are investigated for characterizing spherical and nonspherical particulates. Experimental data are presented demonstrating the ability of the circular harmonic filter and the computer-generated holographic wedge detector to distinguish three classes of particles. The three particle types represented include opaque spherical particles, fibrous particles and brass flakes. A simple figure-of-merit capable of discriminating between the three particle types is defined using the outputs from the computer-generated holographic wedge detector. The figure-ofmerit is independent of laser intensity and invariant with particle scale and rotation changes.

An optical method using Fourier transformation and spatial filtering is used to reveal defects in textured materials in real time. New optical structured filter types were developed including a self-adaptable mask made of photochromic polymers. The characteristics of these materials make possible very promising applications in pattern recognition such as that represented by a fabric.

An important problem in robot vision is to be able to recognize metal objects. The task is made difficult because of specularities and interreflections. Machined surfaces often have different reflectance properties in different directions due to structures created by the machining process. It is often impossible to describe the reflectance using a simple reflectance function. Therefore, it is important to learn more about the reflective properties of machined metal surfaces to foresee and deal with them. The optical behavior of machined surfaces is studied. Machining operations are considered geometrically, mechanically, optically, and at the microstructure level. The conclusions are that the reflection properties of machined surfaces must be divided into two functions, one parallel to the machining direction, which is likely to be Gaussian distributed, and one perpendicular to the structure, which should be considered as a signature of the machining process and will vary accordingly. The position of the maximum intensity perpendicular to the parallel reflection is, however, very predictable, and an experiment is discussed that illustrates how specular reflections caused by point source illuminators can be used to estimate surface shape. The experiment also verifies the suggested reflection model.

We report angle resolved scattering characteristics of individual submicrometer polystyrene spheres on silicon and on a 91.5 nm thick film of oxide on silicon. Both these surfaces are considered optically smooth because the root mean square (rms) vertical roughness ? is much less than the wavelength ? (???). Scattering was measured as a function of polarization using a He-Ne and argon laser at varying degrees of incident angle. The experimental results are compared with a semi-analytical model of sphere/surface light scattering and a finiteelement time-domain electromagnetics code used in conjunction with a far-field extrapolator called EXTRMAP. The scattering measurements showed good to excellent agreement with both models. The semianalytical model could be improved by fully accounting for standing wave effects, a phenomenon the finite-element time-domain model inherently includes. The results are qualitatively explained with reference to both standing wave phenomena and the silicon surface reflectance, which varies significantly as the beam incident angle and polarization are changed.

Using a novel perturbation analysis of 3-D optical waveguides subjected to deformation, we present simple formulas to calculate changes in propagation constants and coupling coefficients. We apply these formulas to design an integrated optical (Si3N4/SiO2/Si) Mach- Zehnder displacement and pressure sensor based on optical path deformation and elasto-optic effect.

A new application of IR sensors for the detection of game is presented. The device detects wildlife (mammals, birds) by using the difference in IR radiation of wildlife and its environment (meadow, field). The system can be used to save the animals’ lives during mowing actions as well as for wildlife management purposes such as surveying and registration. The aspects of development and system layout are explained. The design is based on a model that describes the scenario (radiation, dynamics) and the features of important components (optics, detector, electronics). With reference to the simulation results, the device is constructed and basic optoelectronic parameters are investigated. First experiences concerning field measurements are given.

A bidimensional laboratory simulator to measure and control the attitude of a tether-stabilized platform for orbiting telescopes is presented and discussed. Yaw and libration angles are measured by plane tracking of the image of a laser spot, reflected on a screen by a mirror fixed to a floating platform. A multiple-reflection optical lever enables amplification of platform motion. The libration and yaw angles are related to the coordinates of the spot projected on the screen. Disturbances due to overall tether dynamics and telescope slewing are both reproduced and accounted for. Attitude control is achieved by moving the attachment point of the tether on the platform. Displacements are computed and commands are sent to a dc servo-motor via a PC. This measurement system allows platform libration ? and yaw ? angles to be measured to an accuracy of 0.06×10-3 rad; an accuracy of 0.4×10-3 rad in 10 s is reached in attitude control.

Time resolved reflectivity measurements of thin aluminum films undergoing damage by picosecond laser pulses are described. Experimental results are compared with results from reflectance calculations using the model of hydrodynamic expansion of a metal layer heated by a short laser pulse. Both experimental and theoretical results confirm that thin metal films are useful as damageable optical elements in an optical power limiting device.

Refractive indices and thicknesses of organic-dye-doped thin films are measured by a prism-coupling method. Anomalous dispersion of refractive indices which is caused by absorption of dyes, can be observed. The measured film thicknesses are used to determine the absorption coefficients of those films together with transmission measurements. In a highly absorptive wavelength region, the accuracy of measurements is decreased as a consequence of the broadening of m line corresponding to each guided mode. From the measurements in eosin-Y-doped polymer films, it is found that this method is not available when transmission is less than about 80%.

This PDF file contains the editorial “Guest Editorial: Special Section on Sensor Fusion 1996” for OE Vol. 35 Issue 03

This PDF file contains the editorial “The Physical Principles of Magneto‐Optical Recording” for OE Vol. 35 Issue 03