New picture storage and display technologies have drawn increasing attention to a very old (and difficult) scientific problem, human perception of complex visual patterns. One strategy, 'color appearance' research, involves application of sensory concepts (e.g., contrast and adaptation) and experimental methods (e.g., complete color matching) to slightly more complex stimuli than the simple disk-of-light-in-a-dark- surround of traditional color sensation research. Recent progress in the field of image analysis indicates that the color appearance approach cannot capture the processes responsible for visual analysis of images of real scenes. Perceptual competence in image analysis requires use of spatial structure that is not exploited by the quasi-local analyses of sensory adaptation and contrast. These distinctions are easily illustrated with simple demonstrations. The concepts of image analysis have inspired a number of recent quantitative studies of human surface color perception. Work along these lines should develop a knowledge base useful in practical problems of human image manipulation

At the OSA topical meeting on Applied Vision in July, 1989, we presented evidence showing that, when focal examples of basic colors were employed, as many as nine kinds of irrelevant colors could be added to an information display with very little increase in the time required to search for a target of known color containing a critical spatial element. [1] Here we compare search performance using seven chromatic basic colors with that obtained with seven nonbasic ones. The seven nonbasic colors were selected to be separated from other nonbasic colors by approximately the same number of discriminably-different steps as were the seven basic colors separated from one another. Our prediction that critical targets of basic colors would nevertheless be spotted faster than those of nonbasic colors was not supported. Compared to performance when the critical-target color was unknown, search times were very short and similar for the two color types, including a condition involving 140 stimuli in the display (70 basic and 70 nonbasic), 130 of which were distractors of 13 irrelevant colors. We conclude that useful color coding for visual search probably depends upon the number of discriminable steps separating the colors, rather than their basic or nonbasic character.

Many electronic displays, including large-screen flight simulator displays, have maximum luminances of about 10 photopic candelas per square meter (10 photopic nits). Most references (for example, Hood & Finkelstein, 19862, p.5-3) place this luminance in the region of overlap between photopic and mesopic vision.

The present system of colorimetry, dating from 1930, was early found (by Judd, Wright, and Stiles — three of its formulators — for example) to have serious deficiencies, despite its enormous usefulness. Astonishingly little incisive examination of these deficiencies, their characteristics, and possible cures has been published. We use direct spectroradiometry of the stimuli from the traditional bipartite field of a visual colorimeter, several primary-sets, and visual matching of narrow-band, multiple-band and broadband lights, to elucidate'the differences between normal human vision and the mathe-matical construct called the Standard Observer. Results: 1. Large errors in computed chromaticity, with pairs of highly-metameric lights pronounced to match exactly by a normal human observer, when there is strong content in one or more of the following spectral regions: violet, blue-green near 500nm, yellow near 570nm, or in the deep-red. 2. Agreement is sometimes poor between visual definition of complementary lights, and definition by means of the color-diagram. 3. Color diagram chromaticity coordinates are defined: x=X/(X+Y+Z). The denominator is not representative of "total perceived brightness" and so is not a legitimate element in the construction of a proper color diagram. In addition, the traditional method of construction simply fails, with some sets of real primaries. 4. Perceived-brightness-per-watt of mixtures can be doubled, at constant color, by choice of components. 5. Spectral content in the neighborhood of 500 nm sharply reduces the perceived brightness of white-light mixtures. 6. Although stringent tests of Grassmann's "Additivity Law" show it to hold visually, still "transformation of primaries" and "normalization of color-matching functions” appear never to have been legitimate. 7. Spectral regions near 450 nm, 530 nm and 610 nm appear to excite maximum visual response per watt.

The traditional means for specifying color appearance, such as tristimulus colorimetry and matching to standard samples, have drawbacks in many applied settings: they need precise equipment, standardized viewing conditions, and may not work well with many self-luminous sources. Furthermore, the results often tell only about a match and not about appearance. We have a technique for specifying color appearance that is reliable, very rapid, and can be used in any situation, since it requires no additional apparatus: subjects look at a stimulus and then simply state the proportions of their sensations using the four unique hue names, red, yellow, green, and blue; for completeness, they also state the apparent saturation. We have shown that the results are not biased by methodological factors including context and range effects, subjects’ linguistic backgrounds, and amount of practice. The procedure can be repeated quickly whenever viewing conditions change. For analysis, the scaled sensory values elicited by a set of stimuli are used to derive the locations of the stimuli on a color diagram that is based on appearance and that has a uniform metric; we term this a Uniform Appearance Diagram (UAD). the orthogonal axes of this space are red-green and yellow-blue; the location of a stimulus specifies its hue and its distance from the origin specifies its apparent saturation. It would be very useful if the results of our methods could be related to those from other specification systems. We argue that we can do so by deriving "traditional” measures of discriminability from our UADs. For example, we find that distances among stimuli on a UAD can be used to predict wavelength discrimination under the given viewing conditions.

Algorithms for color machine vision rely upon an explicit or implicit model of how color images are formed. These models usually include an idealized description of how camera systems measure color. Unfortunately, relying on these idealizations can cause serious errors in the performance of algorithms. In this paper, we describe several problems that occur in real color images and how they can be dealt with. A common problem is that many cameras do not have a linear response. Failure to correct this problem will not only cause hue shifts, but will cause problems in any algorithms that rely on the linear assumption. We show how this effect may be measured and corrected. More insidious are the problems of color clipping and blooming, since they can cause sudden changes in the apparent hue. Although it is not possible to correct the problem, we show how it may be detected so that affected measurements may be discounted. Most models assume that integration is performed over the visible spectrum, but many cameras are sensitive to other regions as well. Failure to compensate for this can lead to unexpected results. Also the camera’s varying spectral sensitivity within the visible spectrum will cause some measurements to be more prone to noise than others. We show how the use of a computer- controlled lens can compensate for this problem. Chromatic aberration in typical video camera lenses can cause hue changes, and we show how the further use of computer-controlled lens can compensate for this problem. Using these methods to detect and/or correct problems, we can obtain the accuracy needed for applying physics- based methods to actual color images.

The transformation from the RGB primaries of a television monitor to the CIE XYZ color notation system is usually formulated as a three by three matrix applied to a one by three vector. In the general case, however, this is not true because the black point of the monitor gamut is displaced from the origin of CIE XYZ space. An affine transformation is necessary to accomodate this translation of the monitor gamut, and this affine transformation can be reformulated as a four by three matrix times a one by four vector. Balancing the monitor guns so that they maintain the same luminance ratio over their entire dynamic range can minimize the error of assuming a three by three transform when a four by three matrix is actually called for. This procedure also leads to improved calibration for applications where the front panel brightness and contrast controls will be adjusted. The technique of balancing the monitor guns has been employed in the broadcast television industry for many years.1 This paper presents the theory to support the validity of this approach. First, the affine transformation that is necessary to fully characterize the RGB to CIE XYZ transformation is developed. Next, colorimetric errors are identified for balanced and unbalanced monitors when the assumption of a simple three by three matrix transform is made. Finally, the effect on both balanced and unbalanced monitor gamuts of adjusting the front panel brightness and contrast controls is discussed.

About 10 years ago the author developed a computer program for plotting the color gamuts of sets of pigments. The application of this program was discussed in an internal DuPont publication, but, the details of its operation were never published.

As more research is conducted into the storage of colour images by digital image processing, the quality of colour has come under greater scrutiny; this has in turn led to a reassessment of the methods for coding and analyzing colour.

Lythgoe (1979) reviews evidence that photoreceptor spectral sensitivities in certain species are matched to the animal’s environment. The possibility that photoreceptors are chosen to optimize visual performances leads to the question, what choices of photoreceptor spectral sensitivities are best for various visual tasks? In this paper I review and critique previous work concerning optimal choices of receptors for certain color vision tasks

This paper describes a visual experiment in which a group of observers made forced choice judgments of the location of neutral in test images presented on a color CRT display and photographic reflection prints. The CRT and photographic images were presented both separately and side-by-side in a simulated office environment under two conditions of ambient illumination, tungsten and daylight fluorescent. The results indicate that an observer's state of chromatic adaptation during image viewing is mainly dependent on image areas with little or no dependence upon the surrounding environment. With reflection images viewed in normal conditions, observers were noted to automatically discount ambient illumination. When viewing self-luminous images however, observers formed relative judgements only under certain conditions. These results are discussed in terms of their use in choosing white points for color reproduction calculations.

The spectral power distribution of the light that reflects from a surface to the eye depends both on the reflectance function of the surface and on the spectral power distribution of the illuminant The human visual system actively adjusts to reduce the dependence of surface color appearance on the illumination. We use a matching paradigm to measure the change in the cone coordinates of a color signal necessary to maintain constant color appearance across an illuminant change. If the visual system made no adjustment, the measured cone coordinate change would be zero. If the visual system were perfectly color constant, the measured change would compensate exactly for the physical change in cone coordinates due to the change in illuminant It is not possible to measure the visual system’s adjustment to all combinations of illuminant changes and surfaces. Therefore we develop and test a model of this adjustment. In our laboratory experiments two variables govern the adjustment: the cone coordinates of the reflected color signal and the change in the spectral power distribution of the illuminant We model the visual system’s adjustment using a bilinear function. We determine the parameters of the bilinear function from a small number of measurements. We show that the bilinear model predicts the visual system’s adjustment to many other surface and illuminant combinations.

A new technique is described for making color liquid crystal displays with only two color channels. Red and white channels were tried first, expanding upon the photographic approach of Edwin Land. A test image simulating the liquid crystal display was produced on a standard color cathode ray tube. With shades of red and white arranged in alternating diagonals of a checkerboard pattern, observers saw more colors than predicted by simple color mixtures of red and white — for example green was seen by all observers. But why white and red channels? One brightness and one color opponent channel (one which switches between opposing colors such as red and cyan) better describes the major two components in the color vision system. A test image with one white channel and one opponent red/blue channel in a checkerboard pattern gave rise to an even wider range of colors than did the Land-type display. Photographic print and slide film versions of the displays also showed the effects. In the liquid crystal display, the opposing colors can be produced by using a dichroic polarizer. Such two-channel displays have the advantages of higher spatial resolution or lower cost than conventional color displays. A prototype liquid-crystal two-channel display should be developed.

Color can play a very important role in the in-process, in-situ inspection of integrated circuit structures. Optical interference effects caused by thin films give colors that are characteristic of film thicknesses. For these optically transparent thin films (such as silicon dioxide) that make up the IC structures, interference effects take place when white light impinges upon the wafer. As a result, the features on an IC wafer are inherently colorful. We take advantage of this color dimension in our automated IC wafer inspection system in three ways: 1) Since the colors of the IC patterns are characteristic of film thickness, it is possible to measure analytically the varying thin film thicknesses based on their colors. The relationship between the measured color and thin film thickness has been determined empirically and also modeled analytically. We can rapidly (< 100 msec) measure oxide and other thin film thicknesses based on color in the range of 390A - 10,030A with 30A accuracy. 2) Since the patterns on an integrated circuit have distinct colors, it is possible to segment the images based on their color attributes. During inspection, we employ a color clustering method along with a unique multiple-valued connectivity algorithm to perform rapid, robust segmentation of IC pattern images. 3) Since the majority of IC wafer inspection systems today do not use color vision, they miss an entire class of color defects that occur during various stages of the IC manufacturing process. It has been shown that a number of these defects are detectable solely in color, rather than in grayscale images. Therefore, color allows us to detect a larger number of potential defects than systems that employ grayscale imaging alone.

Pseudocolor display of high contrast, single-band images has been suggested as a method of enhancing image interpretability by increasing the number of distinguishable intensity levels. Pseudocolor scales, whose design is often based on characteristics of color perception, are most often evaluated using subjective criteria, i.e. does the color scale appear to increase sensitivity to the image. In order to determine whether pseudocolor scales quantitatively improve acquisition of image information, a two- alternative, forced-choice experiment was performed to evaluate the effect of pseudocolor display on target detection in Synthetic Aperture Radar (SAR) images. A black-white-red scale and a black-white-blue scale were chosen for testing on the basis of subjective evaluation. A black-white-red (or black-white-blue) scale begins at black for low image values, increases in intensity to white and then increases in saturation to red (or blue) and displays the targets, which have high image values, in color. The black-white-red scale, black-white-blue scale and a control gray scale were tested using a set of 53 SAR images pairs (53 images with targets and 53 images without targets). Each of five observers performed one set of trials (53 forced- choice trials with a single color scale) with each color scale. Analysis of Variance revealed no significant difference in performance as a function of color scale. These results are in agreement with a similar study using medical images and are consistent with the hypothesis that pseudocolor display of single-band images does not enhance target detectability.

The increasing research and debate during the last decade on the use of color as an information code indicates that color displays are here to stay—because they are preferred to monochrome displays and because they are generally more effective, provided the color coding is done properly. Depending on the function of the colors, the requirements for legibility, color differences, color contrasts, and color appearance may be widely different. To meet design requirements for video display units (VDUs), perceptually based color spaces are needed. During the last decade color appearance systems have come into use for color selection in color VDUs. These systems are gaining prominence over colors defined in numerical RGB values, which are not related to the perceptually relevant dimensions of hue, chroma (chromaticness, saturation), lightness, and blackness. We have developed a 'Palette' based on the Natural Color System (NCS) and on the CIE 1976 (L* u* v* ) system (CIELUV). With the 'Palette', colors can be selected interactively as well as automatically, according to a) NCS notations and/or CIELUV values, b) every day language or, c) the functional significance of the color in the image. A palette is presented in a frame around the image. The user is then free to try the different colors one at a time at his/her convenience.

There exists a considerable body of research on what is ergonomically suitable in the selection of colors, both from the point of view of discrimination and distinguishing between physical energy and perceived sensation. We find little precedent in the study of the psychological impact of the use of color in the display of information. There are no readily available mechanisms for using color to communicate an expressive mood or climate. We postulate that colors can be chosen for conveying the substance of a message effectively: its scale, duration and format of exposure. In this paper, we add these considerations as enhancements to the definition of color ergonomics, and the selection and application of color. We propose an organization of color which facilitates its application beyond the vagaries of chance, i.e. the formulation of predictable expressive signals. The methodology for the quantification of the relative expressive qualities of color is discussed. We describe an experiment to distinguish between informational and expressive aspects of color. Reading speed is measured as a function of the contrast of value and the hue-to-hue alignment of pairs of colors. These same dyads are sorted by expressive qualities.

Jay Hannah has been studying changes in color with distance for a number of years. He has studied this phenomenon by making a series of over 100 paintings. With the keen eye of an artist he has made many dramatic demonstrations of color change. An example is that a 1.2 cm white stripe will appear bright yellow at a distance of 14 meters. Hannah has shown that color is influenced by the surround, but not by the usual complementary color rules. This paper describes a series of experiments that use color matching to quantify the appearance of a number of Hannah's painting at close (1.4 m) and far (14 m) distances This study uses the changes in appearance to establish the visual equivalent at different distances. These equivalent colors can be used to study the underlying mechanism of color appearance near spatial-frequency threshold.

It is necessary to look at each of my experimental paintings from a close distance (6 feet or less) and then look at the painting from a middle (20 feet) or a far (43 feet) or a long (65 feet) distance. Some paintings will have a strong yellow at 43 feet and no observable yellow at a close distance. Still other paintings observed at 20 feet change close-distance greens to highly saturated cyan blues. My distances are for daylight with blue sky present. In sunlight the same color changes take place at a still greater set of distances. Conversley, the same color changes do take place under an incandescent light source, but the distances become shorter than the daylight distances. Further, a black line one-eighth of an inch wide in the center of a light tint of orange-red is black at a close distance and the same black line is a bright red at the long distance of 65 feet in daylight. Below each painting is a complete set of small squares identical to each color used in the painting, and each is isolated by a white surround. These sample colors darken, but do not do not change with increased distances.

Research in early (low-level) vision, tooth for machines and humans, has traditionally been based on the study of idealized images or image patches such as step edges, gratings, flat fields, and Mondrians. Real images, however, exhibit much richer and more complex structure, whose nature is determined by the physical and geometric properties of illumination, reflection, and imaging. By understanding these physical relationships, a new kind of early vision analysis is made possible. In this paper, we describe a progression of models of imaging physics that present a much more complex and realistic set of image relationships than are commonly assumed in early vision research. We begin with the Dichromatic Reflection Model, which describes how highlights and color are related in images of dielectrics such as plastic and painted surfaces. This gives rise to a mathematical relationship in color space to separate highlights from object color. Perceptions of shape, surface roughness/texture, and illumination color are readily derived from this analysis. We next show how this can be extended to images of several objects, by deriving local color variation relationships from the basic model. The resulting method for color image analysis has been successfully applied in machine vision experiments in our laboratory. Yet another extension is to account for inter-reflection among multiple objects. We have derived a simple model of color inter-reflection that accounts for the basic phenomena, and report on this model and how we are applying it. In general, the concept of illumination for vision should account for the entire "illumination environment", rather than being restricted to a single light source. This work shows that the basic physical relationships give rise to very structured image properties, which can be a more valid basis for early vision than the traditional idealized image patterns.

The image of a uniform wall illuminated by a spotlight often gives a strong impression of the illuminant color. How can it be possible to know if it is a white wall illuminated by yellow light or a yellow wall illuminated by white light? If the wall is a Lambertian reflector, it would not be possible to tell the difference. However, in the real world, some amount of specular reflection is often present. An empirical reflection model describes light reflection from an inhomogeneous surface as a mixture of a specular (interface) component and a diffuse (body) component. Since the spatial scale over which the interface reflection changes significantly is much smaller than that of the body reflection, it can be shown that one can effectively exploit the scale difference to find a unique solution, which is often quite accurate. The method can also be generalized to compute the illuminant chromaticity for a nonuniform smooth surface.

We present an approach to the construction of computational model for color image segmentation based on the physical properties of sensors, illumination lights and surface reflectances. We use the dichromatic model for dielectric materials and develop a metric space of intensity, hue and saturation in order to better interpret light- surface interactions. While the spectral distribution of surface reflectance is not changed by shading and shadows, it is affected by highlights, as well as inter-reflections. Using the established model, we perform color image segmentation based on the material change with detection of small inter-reflections between different objects and highlights. Since usual illuminations are spectrally colored, discounting illuminations from a perceived color image (color constancy) is necessary for obtaining correct surface reflectances. However applications of many color constancy algorithms are limited by severe spectral and spatial assumptions. Our approach is to use a reference plate or specularity to achieve color constancy for global illumination. The detected inter-reflections represent the local variation of illumination due to reflected light from other objects.

The evolutionary theory of color perception is described. It is based on three main propositions: 1) The physical nature of light and the environmental distributions of its information carrying parameters, 2) Constraints implied by the available biological material and physiological processes, and 3) The evolutionary tendency toward optimal usefulness for the survival of the species. The theory leads directly to the main properties of color perception: a) Newtonian color circle, metamers, additive and subtructive color mixtures, b) Adaptive (relativistic) transformations and color constancy (invariance), c) An operational procedure of color measurements. Among the major predictions are Lorentz type formulas for color transformations, and the explanation of two-color projections. Predictions based on color transformations will be demonstrated with a simple demonstration kit.

Physical models for color image formation provide constraints which are useful for interpreting 3-D scenes. I summarize the physics underlying color image formation. Models for surface and body reflection from metals and dielectrics are analyzed in detail. This analysis allows us to evaluate the benefits we stand to gain by using color information in machine vision. I show from the reflection models that color allows the computation of image statistics which are independent of scene geometry. This principle has been used to develop an efficient algorithm for segmenting images of 3-D scenes using normalized color. The algorithm applies to images of a wide range of materials and surface textures and is useful for a wide variety of machine vision tasks including 3-D recognition and 3-D inspection. Experimental results are presented to demonstrate the scope of the models and the capabilities of the segmentation algorithm.

An understanding of the natural history of color vision would provide significant insight into the question of why we have color vision. Such information is not at hand, but studies of the biological mechanisms underlying color vision, and consideration of the variations of these mechanisms across species, provide the basis for some thoughts about the evolution of the mechanisms for color vision. The focus is mammalian color vision

What ecological advantages do animals gain by being able to detect, extract and exploit wavelength information in their environment? And what are the advantages of representing that information in the form of hue qualities? The benefits of adding chromatic to achromatic vision are perhaps marginal when it comes to object detection, but become more apparent in tasks involving object recognition and in receiving biological signals. It is argued that this improved performance is a direct consequence of the fact that the visual system reduces wavelength information to combinations of four basic hues. This engenders a simple categorical scheme that permits a rich amount of sensory information to be rapidly and efficiently employed by cognitive machinery of limited capacity.

For normal human observers, color is three-dimensional. The color appearance of any surface can be matched by adjusting just three variables (such as lightness, hue, and saturation). This trichromacy is mediated, but not ultimately explained, by the existence of three classes of photoreceptors -- retinal cones having peak sensitivities to longer, medium, and shorter wavelengths of visible light. Possibly, this trichromacy is the result of a more or less arbitrary compromise between (a) the larger number of dimensions required to represent the full spectral reflectance distributions of surfaces and (b) the smaller number of classes of retinal receptors needed to maintain, through denser retinal packing, higher spatial resolution for each receptor class. Alternatively, building on a linear model for color vision, as formulated by Maloney and Wandell, this paper seeks a less arbitrary basis for trichromacy in the prevailing degrees of freedom of terrestrial illumination. Natural selection may have favored trichromacy because it permits the achievement of color constancy despite terrestrial transformations of sunlight, which, unlike the variations in surface reflectances, have been limited to essentially three degrees of freedom. Any visual system, whether biological or artificial (including a system that requires merely an achromatic, shades-of-gray representation), can achieve constancy of color (and of lightness) only by first analyzing its optical input into three chromatic channels in order to compensate for the prevailing three degrees of freedom of terrestrial illumination.

This paper explores the use of a perceptually-based color space in the context of color image enhancement. Both algorithmic processing of images as well as visualization of the effects of processing algorithms are examined using CIELUV as a uniform perceptual model. Both the methodology and observations regarding the limited nature of real world device color gamuts are not specifically tied to the LUV color space, and should apply to other uniform perceptual color models as well.

We have reconstructed the color spaces of normal observers and of protanomalous and deuteranomalous observers by measuring the time that each requires to decide whether pairs of colors are the same or different. By means of a multidimensional scaling procedure these response times were ordered into a space such that colors that were more quickly discriminated were farther from one another. When presented with a set of colors that yields an approximately rectangular color space for normal observers, anomalous observers have difficulty in discriminating colors that lie on lines parallel to the red-green cardinal axis. Their color spaces suggest that the gamut of color that anomalous observers experience is far more impoverished than is usually thought to be the case, and that their ability to interpret color-coded displays is correspondingly limited.

The paper presents a model of the influence of the color temperature of the radiation source and of the optical fiber transmission properties on the operation of an optical fiber color sensor and results of a computer analysis of the effect of these influences on color discrimination by color recognition systems. The performed analysis provided a basis for the formulation of same general conclusions and recommendations for designing different types of color recognition and color vision systems employing optical fibers.