We propose an active vision technique for recovering object shape and absolute position information. The algorithm relies on the controlled motion of a point light source and reasonable assumptions on the general shape of the objects. The sequence of shadow images created by the illuminant is used to produce the surface information. In this approach the light source is not at infinity and light rays are not parallel. For the case of objects with `sharp' edges, our approach requires only solving a linear system of equations. We provide a sensitivity analysis as a basis for robust estimation of surface shape and depth values. Simulations and experiments are performed to demonstrate the efficacy of the algorithm.

This paper deals with new and simple representations of 3-D points in a moving-observer coordinate system. Assuming rectilinear motion with no rotation of an observer where the optical axis coincides with the direction of motion, and a stationary scene, points in 3-D space that lie on a particular 3-D surface produce constant value of some nonlinear function of the measurable image optical flow. Five sets of different surfaces are introduced and there is one optical-flow based constant value for each surface. We called these values `invariants.' It is shown how to extract these invariants and how to use them for representing 3-D space.

In binocular systems, vergence is the process of directing the gaze so that the optical axes intersect at the point of interest. Region based methods of disparity analysis provide fast and reliable estimates of the vergence error. Unfortunately, it is difficult to determine on what image features these approaches are in fact verging. Previous approaches to vergence control have for the most part failed to ensure that both cameras actually verge on the object of interest, especially in complex scenes. This paper presents a system that addresses this problem. By using the cepstral filter in a multiresolution setting with a dominant camera, our system can verge accurately in complex scenes. Specifically, the system adaptively refines the vergence angle in a scale space consisting of the center patches of a Gaussian pyramid. The effects of the cepstrum in a multiresolution system are analyzed, and the precision and performance of the new system are verified on natural scenes.

It is a challenge to develop autonomous vehicles capable of operating in complicated, unpredictable, and hazardous environments. To navigate autonomous vehicles safely, obstacles such as protrusions, depressions, and steep terrains must be discriminated from terrain before any path planning and obstacle avoidance activity is undertaken. A purposive and direct solution to obstacle detection for safe navigation has been developed. The method finds obstacles in a 2-D image-based space, as opposed to a 3-D reconstructed space, using optical flow. The theory derives from new visual linear invariants based on optical flow. Employing the linear invariance property, obstacles can be directly detected by using a reference flow line obtained from measured optical flow. The main features of this approach are that (1) 2-D visual information (i.e., optical flow) is directly used to detect obstacles; no range, 3-D motion, or 3-D scene geometry is recovered; (2) the method finding protrusions and depressions is valid for the vehicle (or camera) undergoing general motion (both translation and rotation); (3) the error sources involved are reduced to a minimum, since the only information required is one component of optical flow. Experiments using both synthetic and real image data suggest that the approach is effective and robust. The method is demonstrated on both ground and air vehicles.

In this paper, we propose a method for quantitatively analyzing the performance of tracking algorithms for generic objects (objects about which the system has no prior knowledge). The sensitivity of algorithms are measured as a function of changes in scale, rotation, clipping, and brightness. We then compare our quantitative measures with a subjective evaluation of the algorithm's performance on real-world images. We use a simple color-based matched filtering algorithm to illustrate the method.

This paper concerns the use of visual feedback to verify whether an object has been properly grasped by a manipulator. The work is motivated by the fact that many general-purpose manipulators are equipped with very simple grippers which may not be well suited to grasping common objects. Furthermore, many robotic systems do not verify that a grasp operation has been successfully executed. This paper describes a system under development at Virginia Tech which utilizes visual feedback to guide relative camera-object movements for the purpose of estimating the pose of the object. The goal is to assist in computing object pose relative to a coordinate system embedded in the gripper. Object shape is assumed to be known in advance. Two methods are discussed, both of which utilize visually guided movements to search for a minimum in an objective function. The first method is to align the gripper with the image plane, facilitating the computation of object orientation about the normal to the image plane. The second involves moving the object to align its image with a desired view of the object. Extensive calibration of the camera or manipulator is not required. The methods discussed here are still at the conceptual stage, but illustrate the potential of the active approach.

This paper describes a method for obtaining a composite focused image from a monocular image sequence. The image sequence is obtained using a novel non-frontal camera that has sensor elements at different distances from the lens. This paper first describes the motivation behind the non-frontal camera, followed by the description of an algorithm to obtain a focused image of a large scene. Large scenes are scenes that are deep and wide (panoramic). Consequently, the camera has to be panned in order to image all objects/surfaces of interest. The described algorithm integrates panning and generation of focused images. Results of experiments to generate extended depth of field images of wide scenes are also shown.

We define autonomous exploration as a process by which an active observer can interact with its surroundings, e.g., by intentionally moving around and collecting information, in order to learn about its environment. Such an ability is essential for autonomous systems that must operate in unstructured environments, i.e., where it is difficult (if not impossible) to characterize the environment beforehand. This paper describes a working system that implements an autonomous explorer whose function is to describe the environment in terms of articulated volumetric models. A novel feature of the system is that it uses feedback to reduce reliance on a priori knowledge.

Due to delays in image acquisition and processing, prediction is a critical factor for successful visual tracking of moving objects (both for humans and for vision machines). This paper explores some alternative techniques for predicting object motion for the purpose of tracking with an active camera system. In particular, one of our goals is to develop a system that will track an object undergoing `random' motion quite well, but that will track much better (at higher speeds with less lag) if the object settles into a periodic motion of some kind. Rather than identify parameters for specific signal models to accomplish this, we propose to use a finite set of previous joint states for the signal model. The advantages and problems associated with this approach are discussed. Results of experiments using different prediction algorithms with TRICLOPS, a high-performance active vision system, are also presented.

In this paper an implementation of a high level symbolic scene interpreter for an active vision system is considered. The scene interpretation module uses low level image processing and feature extraction results to achieve object recognition and to build up a 3D environment map. The module is structured to exploit spatio-temporal context provided by existing partial world interpretations and has spatial reasoning to direct gaze control and thereby achieve efficient and robust processing using spatial focus of attention. The system builds and maintains an awareness of an environment which is far larger than a single camera view. Experiments on image sequences have shown that the system can: establish its position and orientation in a partially known environment, track simple moving objects such as cups and boxes, temporally integrate recognition results to establish or forget object presence, and utilize spatial focus of attention to achieve efficient and robust object recognition. The system has been extensively tested using images from a single steerable camera viewing a simple table top scene containing box and cylinder-like objects. Work is currently progressing to further develop its competences and interface it with the Surrey active stereo vision head, GETAFIX.

This paper describes a novel technique in 3D sensory fusion for autonomous mobile vehicles. The primary sensor is a monocular camera mounted on a robot manipulator which pans to up to three positions on a 0.5 m vertical circle, while mounted on the mobile vehicle. The passive scene is analyzed using a method of inverse perspective, which is described and the resulting scene analysis comprises 3D wire frames of all surfaces detected in 3D. The 3D scene analysis uses a dual T-800 transputer based multiprocessor which cycles through generating primary scene information at rates of 1 update per 10 seconds. A PC-based 3D matching algorithm is then used to match the segmented objects to a database of pre-taught 3D wire frames. The matching software is written in Prolog.

This paper deals with recognition of three-dimensional (3D) polyhedral objects from two- dimensional (2D) line-drawing images. A new scheme is presented, which is aimed at learning, representing, and recognizing complicated 3D polyhedral objects such as concave, self-occluded and articulated objects, yet with only very few learning samples and can distinguish objects with very similar patterns. Several examples are illustrated.

This paper describes a method of optimal 3D object surface identification. First, the spatial and frequency domain approaches are presented for the determination of an optimal grid spacing for 3D object surface representation. The spatial domain approach is based on the measurement of the total variations of the object surface function f(x,y) in the constant x and constant y planes and the frequency domain approach utilizes the Nyquist criterion and an ideal 2D rectangular low-pass filter in an iterative operation. Two error criteria are defined based on the magnitude and shape features of f(x,y) as the measure of effectiveness of the optimal grid spacing for 3D object surface representation. Second, a feature-based object surface identification method is described based on the boundary features of the object and model surface cross sections.

Most conventional SFS (shape from shading) algorithms have been developed under three basic assumptions about surface properties and imaging geometry to simplify the problem. They are the Lambertian surface property, the orthographic projection, and the distant single light source. However, since these assumptions are not appropriate for many real applications, apparent distortions of reconstructed surfaces occur with conventional SFS algorithms. To obtain a more practically useful and accurate SFS algorithm, it is necessary to relax these restrictive assumptions and adopt more general conditions or models about the imaging process. In this research, we propose a new direct shape recovery algorithm from one or multiple shaded images generated by a general reflectance model which includes diffuse and specular effects, perspective projection, and a nearby point light source. The basic idea of our approach is to use a finite triangular surface model and express the image irradiance in terms of surface nodal depth variables through a linearized reflectance map under the perspective projection model. The object shape is recovered by determining all nodal depth variable through a cost minimization process.

In human binocular vision, the two retinal images are unified into a single image with perception of depth through a mechanism called `binocular sensory fusion.' The term cyclopean vision refers to the unified visual scene of the world obtained from fusion of the images projected to the two eyes. This paper describes an algorithm which simulates the fusion process for depth perception. The disparity information of the entire image is obtained by a convolution operation followed by a local maxima detection. The computation burden, and therefore, the processing time, is largely reduced.

In this paper, an implemented structured lighting system is described. Experiments are performed on the system using various objects, and method of range data acquisition is verified. After the calibration is explained with a set of numerical data, the procedural steps of image analysis are illustrated using a simple object. The obtained range data are plotted for verification of the method. The performance of the system, based on error analysis, is also described.

This paper describes some of the first analytic work performed on data collected by an autonomous 3D machine vision system. The system, which is described in detail elsewhere and briefly described here, uses fast, dedicated hardware to generate a visible surface mapping directly into a memory array that can be accessed from a host computer. The techniques described here have shown some success in the separation of individual surfaces in multi- surface objects and a range of results are presented.

In order to study the motion of non-rigid objects in an image sequence, it is often necessary to compute a dense optical flow field. Gradient-based techniques have been relatively successful at computing dense flow fields for gray-scale images when additional constraints are added. However, these constraints often introduce errors at the edges of objects, where the object motion is different from the background motion. Our approach is to use color images to obtain three constraint equations corresponding to three color components. This gives us an overdetermined linear system with three equations and two unknowns (x and y direction components of optical flow) which can be solved using linear least-squares algorithm. No further constraints are necessary. This paper presents such a color-based optical flow estimation and includes some initial promising results on synthetic motion image data. Further research directions are also discussed.

The paper deals with geometrical model correction of 3D rigid objects situated in the robotic environment. This problem is extremely important both for intelligent robots and for industrial robotic workcells. The proposed approach is based on computing homogeneous transformation matrix that describes the relation between locations in real robot environment and their images in a world model maintained by a robot control system. Detail error analysis and computational experiments were carried out to investigate proposed algorithm robustness and reliability. Experimental verification of the algorithm is presented. The workcell calibration method was integrated into a robot off-line programming system for PUMA and KUKA robots.

Stereoscopic techniques for recovering depth in scenes are computationally intensive and difficult to specify sufficiently well to ensure that optimal solutions are obtained in any given situation. Apparent motion cues are a far richer and more easily exploitable source of information on depth, but computing depth from motion in spatio-temporal image sequences has many pit-falls associated with it. We show that many of these can be avoided by simultaneous capture of two or more views of the scene, projected onto a single CCD sensor, using angled mirrors. The resulting fixed-camera ranging device is immune to camera vibration and motion as well as to changes in ambient illumination that occur during image capture. A 1-D, generalized gradient scheme is used to compute the apparent image motion induced by objects in the scene and hence the range to the corresponding objects. Furthermore, the fixed camera configuration enables the shape and size of the viewing filter to be preselected to optimize performance and maximize range resolution.

In this paper, a new method to research 3D fields is provided. That is technology of image processing. The stereo images and shapes of 3D field are reconstructed. The geometric parameters and their transform of 3D field are acquired. Three-dimensional fields are analyzed and researched directly using this method. It has many advantages, especially there are several 3D fields researched at the same time. From the examples, we know that the method provided in this paper is correct, and the results are reliable.

In this paper, a new calculating method is given for determining object surface orientation based on two images by using spherical representation of a surface and the solution is gotten more easily than by using gradient space. Furthermore, the relationship of the solutions reveals that the true solution vectors and the extraneous solution vectors are symmetric with respect to the light source plane. The surface rebuilt by one set of solutions is the true object surface and the surface rebuilt by another set of solutions is a mirror image of true surface with respect to the light source plane. The methods of rebuilding surface shape from surface orientation are given respectively in an orthographic projection and a perspective projection. The methods mentioned above are proved by simulated calculation.

A linear symmetry based 3D edge orientation estimation method is presented. By introducing a `triple number' concept, which can be used to represent 3D space vector, the 3D edge orientation can be easily estimated. The basic principle and the algorithm are described.

Mobile robotics is a broad field, and many mobile systems have been developed as forms of automated guided vehicle systems (AGVSs), on/off line teleoperator systems such as remotely piloted vehicles (RPVs), personal mobile robots, and experimental mobile robots. Navigation and motion control are the key elements required for safe, reliable, and accurate operation of the mobile unit. In this paper a technique for mobile robot navigation using a wide angle imaging system to follow a line is used. The sensed position information provides a basis for controlling the motion of mobile robot by comparing the current position to the desired position to determine range and bearing correction signals for the robot motor controllers. These positions are along a predetermined path. Ultrasonic sensors are also used to avoid obstacles.

This paper presents our design approach and implementation of a grasp strategy that selects robot finger contact locations based on direct sensory information. The overall design consists of a reactive component coupled with a deliberative component. The reactive component, called the Grasp Reactor, uses direct measurements to produce appropriate actions for the environmental situation present. The deliberative component, called the Grasp Advisor, communicates global constraints to the Grasp Reactor to improve its decision making capability. This paper concentrates on the reactive component and on understanding how it can be helped by the deliberative component. Executing within the Grasp Reactor, our grasp strategy uses a hill-climbing technique to select stable contact locations on the object; 2-D visual information about the object is used to make this selection. This visual information is continually extracted from the environment as the gripper approaches the object. The novelty of this approach is that the strategy controls the robot system, not only to preposition the gripper, but also to simplify the processing of visual information received for grasp synthesis. We have implemented this strategy using a Philips Multi-Functional Gripper and performed experimental runs to observe the strategy's real-world performance.

This paper describes an integral approach to the acquisition of randomly moving objects with a robot. The acquisition problem is phrased as a feedback control problem that explicitly models the target dynamics but justifiably excludes the robot dynamics. The target's position deviation with respect to the robot is sensed by a camera mounted within the end-effector and used to servo the robot governed by controller-observer pairs. Acquisition takes place once the system is sufficiently certain about the target's motion pattern. A good balance is found for the trade- off in this phase between a decreasing field of view for the camera and the need to get close for acquisition. The approach was tested on a PUMA-560 robot with a gripper mounted camera. The experimental system can acquire objects that are moving randomly on a table; an early version was integrated in a kitting robot cell.

A typical robot vision scenario might involve a vehicle moving with an unknown 3D motion (translation and rotation) while taking intensity images of an arbitrary environment. This paper describes the theory and implementation issues of constructing a sequence of tracked images in which a desired point in the environment is tracked using an image sequence with arbitrary motion. This method is performed fully in software without any need to mechanically move the camera relative to the vehicle. This tracking technique is simple and inexpensive. Furthermore, it does not use either optical flow or feature correspondence. Instead, the spatio- temporal gradients of the input intensity images are used directly. We have also introduced the gradient maps as an intuitive way of visually representing spatio-temporal brightness gradients. We used such gradient maps to qualitatively examine the quality of our constructed image sequence. The experimental results presented support the idea of tracking in software. The final result is a sequence of tracked images where the desired point is kept stationary in the images independent of the nature of the relative motion that occurred.

This paper describes the theory behind laser tracking measurement systems (LTMS) and the development of a prototype LTMS system at Newcastle. An assessment is made of the accuracy of positioning achieved by the system in the control of the end-effector position of a Puma 560 robot manipulator using a CCD camera positioning sensor and a hollow cube retro- reflector placed on the robot wrist.

This paper addresses an approach to the problem of determining the 3D location of points of an object in the environment of a moving camera mounted on a robot arm, based on a monocular image sequence obtained by the camera. These points can be either endpoints of the line segments or other feature points. The robot arm's velocity and position are assumed to be known via the robot arm controller. The motion model of the camera incorporates the robot arm dynamics. The resulting model is a linear time-varying one. This model overcomes the common assumption of a constant velocity camera motion between consecutive image frames. The motion of the 3D points in the camera reference frame is maintained by tracking between frames. This is done recursively using the extended Kalman filter (EKF). The 3D motion stereo equations which are derived serve as the measurement model for the corresponding EKF without the need to solve them explicitly. The resulting measurement equations are linear time-varying ones with multiplicative noise. The 3D location of points of the selected object are then updated recursively using the EKF in conjunction with different views of the object. These models are particularly suitable for the EKF implementation. Correspondence between two 2D image points in consecutive frames of the same 3D scene point is constrained by statistical distance produced by the EKF. Simulation results are presented to illustrate the approach.

Robot vision is a specialty of intelligent machines which describes the interaction between robotic manipulators and machine vision. Early robot vision systems were built to demonstrate that a robot with vision could adapt to changes in its environment. More recently attention is being directed toward machines with expanded adaptation and learning capabilities. The use of robot vision for automatic inspection and recognition of objects for manipulation by an industrial robot or for guidance of a mobile robot are two primary applications. Adaptation and learning characteristics are often lacking in industrial automation and if they can be added successfully, result in a more robust system. Due to a real time requirement, the robot vision methods that have proven most successful have been ones which could be reduced to a simple, fast computation. The purpose of this paper is to discuss some of the fundamental concepts in sufficient detail to provide a starting point for the interested engineer or scientist. A detailed example of a camera system viewing an object and for a simple, two dimensional robot vision system is presented. Finally, conclusions and recommendations for further study are presented.

The paper is devoted to robot accuracy improvement via calibration and contains correspondent algorithms, hard- and software description. The experimental part of this work has been performed with PUMA robots. The hardware includes a sensor unit mounted on the robot arm consisting of a CCD camera and four optical proximity sensors. The proposed algorithm is based on an integral estimate of straight path distortions and minimization of them by altering the parameters of the robot model.

An original approach to robotic deburring is presented here. It advocates the use of visual servoing as a preliminary step to the classic deburring process in the force control framework. The contour following task using a hand-eye robotic system is described from a theoretical point of view. The motion of the contour relative to the camera sensor is estimated in real-time by processing the measured optical flow of a set of relevant feature points. The desired motion of the end-effector is computed with the objective of keeping the visual features always at the same target location in the image plane. The task-function approach to robot control is used to address control issues. The careful study of experimental results carried out on an industrial part for heavy vehicles validates our approach and suggests many directions for further investigations.

The work presented here focuses on the problem of positioning a mechanical structure with respect to a deep underwater bore-hole by using a visual servoing approach. First, we briefly recall a paradigm we have been using as a theoretical framework for implementing visual servoing tasks. Then, we apply this general formalism to our particular case where the visual features are two ellipses perceived in the image which correspond to the projection of two circles bounding the bore-hole in the scene. The last part is devoted to implementation aspects. We show some results which have been obtained both in simulation and on our testbed consisting of a 6 degrees-of-freedom arm with a camera mounted on its end effector. Among implementation aspects, the control loop sampling rate is the most crucial issue. In a vision- based control approach, we have to deal with strong real-time constraints in terms of image processing in order to ensure performances and stability of the closed loop control scheme. To solve this problem, we have designed a new parallel machine vision architecture fully adapted to vision-based control approaches and able to achieve real-time video rate performances for the servoing loop.

Intrusive examination of patients using flexible endoscopes is commonly employed by surgeons in pre- and post-operative examination of treatment related to laryngeal illness. The insertion of naso-endoscopes through the nasal cavity is a skilled and awkward task, and viewing is limited due to the small image viewed through an eyepiece. This paper described tests undertaken in the laboratory on a scale model of the human head where an anthropomorphic robot manipulator has been adapted to allow it to assist the surgeon in endoscopic examination. Visual feedback is provided to allow fine corrections to be made automatically by the robot for focusing and glottis orientation. Corrections are adjusted by a joystick control.

The application of robots to high-speed manufacturing is limited by problems such as teaching locations, accuracy and cycle time. This paper discusses how visual servoing could be used to ameliorate these problems. Preliminary experimental results are presented showing the use of visual servoing for robot end-point control. A VME-bus based real-time system performing both robot control and image processing is used. The machine vision subsystem, based on Datacube pipeline processing modules and an APA-512+ binary feature extraction module, analyzes the camera data at CCIR video field rate (50 Hz).

In this paper we present a method for 2D temporal tracking of region in a monocular sequence of images. The approach we propose starts with a temporal matching phase which consists of forecasting the future system state. Kalman filtering is used during this phase. The goal here is to estimate, with a given reliability rate, the position and the size of the region in the next image in which the potential corresponding principal axis can be found. This phase is only applied to the principal inertia axes of each region. The following phase is the spatial matching which consists of finding the most probable matching among the principal inertia axes present in the search area. A similarity function is then used. Namely, the Mahalanobis distance, which we apply to descriptors of these principal axes. We then propagate this matching to regions. The same similitude function is used, but is now applied on regions descriptors.

The work described in this paper takes place in the context of an attempt to develop a clearer understanding of what is meant by task oriented visual processing. The traditional separation of task and sensory processing within computer vision is something which has lead to an overemphasis on representation, modularity, and the single functionality of artificial vision systems. These issues are briefly discussed here in the context that the Karhunen Loeve Transform (KLT) is an algorithmic approach which allows us to begin to move away from a preconceived view of what vision is, how it works, and what it does. We summarize the mathematical background and basic operation of the KL coding procedure. We examine the application of the eigenvector decomposition to the analysis of human motion over several posture cycles of a walking sequence. The approach is used to examine the potential for recognition through motion, for posture description and for natural motion reconstruction processes. Various classification and analysis techniques, including artificial neural networks, the Fourier transform, and heuristic operations are used to extract the information available about the different cues investigated.

In this paper, a technique of image processing for air vehicle and the method property of researching air vehicle using this technique is presented for the first time. In this method, the original data are obtained using several photographs of air vehicle and the three dimensional coordinates of air vehicle configuration are solved. After interpolating and surface smoothing, the grid drawing of air vehicle is made. The 3D image of air vehicle configuration is reconstructed using image processing techniques for this grid drawing. According to this 3D image, the wing area of air vehicle and other parameters are calculated, so the air dynamic property and flying property are researched. It is expressed through actual application for an aircraft that the air vehicle configuration can be obtained precisely using the method presented in this paper, so a very difficult problem is solved in this field.

Recent works dealing with the development of automatic image processing (IP) systems not dedicated to any specific application are all based on the search of a plan of treatments adapted to the nature of the problem and the images, among a base of predefined plans. In our approach on the contrary, we are interested in solving IP problems by building plans of treatment in a dynamic way and to use explicit knowledge for reasoning. Our system hinges upon hierarchical, incremental, and opportunistic planning within the blackboard architecture. The system reasoning makes use of explicit knowledge about expertise in IP in order to find out and set the value of parameters and form a sequence of classical IP operators.

A task in a distributed computing system consists of a set of related modules. Each of the modules will execute on one of the processors of the system and communicate with some other modules. In addition, precedence relationships may exist among the modules. Task allocation is an essential activity in distributed-software design. This activity is of importance to all phases of the development of a distributed system. This paper establishes task completion-time models and task allocation models for minimizing task completion time. Current work in this area is either at the experimental level or without the consideration of precedence relationships among modules. The development of mathematical models for the computation of task completion time and task allocation will benefit many real-time computer applications such as radar systems, navigation systems, industrial process control systems, image processing systems, and artificial intelligence oriented systems.

Surface mounted technology (SMT) in automated assembly facilities requires the use of automatic surface-mount-device (SMD) placement machines. One of the problems involved in the electronic printed circuit board (PCB) assembly process is the verification of the SMD placement operation within tight tolerances. The high throughput of modern manufacturing lines along with the required accuracy demand the use of automatic inspection systems to verify SMD placement. Image complexity of the board makes the use of machine vision for the inspection process a difficult task. This is complicated by the fact that misclassification errors should be kept to a minimum. Additionally, it is desirable that the inspection results provide enough information to be used for statistical process control (SPC). The strategy adopted to solve this problem was to simplify the complexity of the image by means of special illumination devices. The simplified image was then suitable for analysis by simple processing, segmentation, and detection algorithms that, sequentially applied to the image, met the required repeatability and accuracy specifications for the inspection system. The scope of this paper is to describe the techniques explored by the authors to solve the SMD inspection problem in order to develop a working industrial SMD inspection system.

This paper builds upon work in developing a theory of planning as adaptation of a reactive system. In previous work, we introduced the planner-reactor architecture and investigated the structure of the reactor and the planner. In this paper we present our results in representing hierarchical reactive systems and in making safe structural adaptations to an ongoing reactive system. This work is done in the context of automated robotic kitting for assembly and the examples in this paper are taken from this area. Although hierarchical representation is essential for complex tasks, it poses the problem that it may destroy the reactiveness of the system. We demonstrate that it is possible to choose a `flat' representation that both preserves reactivity and allows hierarchical structuring. Previous work on modifying ongoing reactive systems has not addressed the crucial issue of making the modifications in a safe and consistent manner. We present a set of desirable properties for an adaptation mechanism: safeness (reactions aren't interrupted), consistency (use only the old reactions or only the new ones), and boundedness (the changes will eventually occur). We then introduce an adaptation mechanism that has these properties.

In this paper, a new approach is proposed to plan occlusion-free next view for a light stripe range finder. To expand its viewing scope, the range finder is mounted on the gripper of a manipulator so that it can take range images at any point in the space from any direction. In order to avoid self occlusion from occurring, we make the light plane of each striping orthogonal to the tangent plane of some representative point (RP) on the object, and maintain constant the distance between the view point and the RP. Instead of blind scanning to obtain dense range images of the object, we utilize already acquired data as knowledge to plan the next scanning view purposively. First, the range finder is guided right above the object by processing an intensity image taken from above the worktable, and an optimal initial scanning direction is determined through test stripings. Secondly, four initial stripings along the scanning direction with a default displacement are carried out, and their images are segmented at abrupt and sharp turning points. The longest corresponding segments are first fitted with a B-splines surface, and the middle point of the boundary along the scanning direction is viewed as RP for the initial patch. The next view point is determined by approximating the surface with a cylindrical surface within a small neighboring area around the RP, calculating the curvature and torsion of the spiral curve on the cylindrical surface passing through the RP. Thirdly, the initial patch is extended to a new one by merging the stripe from the determined view point. This procedure is repeated until touch of the object with the worktable is reached. And finally, we get complete description by connecting all the patches starting from each initial segment. As can be seen from the above, the proposed approach acts very close to the perception process of a human. We utilize a simulation system to show the effectiveness of our approach and its advantages over the existing ones.

Segmentation is an important process in 3D vision since it is at this stage that all higher level processing begins. Range images are useful because they can provide direct depth measurements of objects in the scene which can be used for navigation and object manipulation tasks. Range images are characterized by two principal types of discontinuities: step edges that represent discontinuities in depth, and roof (or trough) edges that represent orientation discontinuities. A Gaussian weighted least squares technique is developed for extracting these two types of edges from range images. Edge extraction is followed by a surface-based region splitting algorithm in order to generate a more complete partition of the image. The result is fed into a surface-based region growing algorithm which yields the final segmentation image. The algorithm is tested using synthetic and real range image data which illustrate the importance of each of these steps in yielding final segmentation results that are robust and consistent.