From an engineer who has been designing Automatic Target Recognition (ATR) systems for 40 years comes this step-by-step guide to producing state-of-the-art ATR systems. The full spectrum of ATR designs is covered, from systems that just suggest targets to the warfighter to ATRs that could serve as the “brains” of lethal autonomous robots.
Unfortunately, when it comes to ATR, some practitioners claim that their off-the-shelf, canned algorithms magically leap from academic research to deployment with scant domain knowledge or system engineering. Deep learning is marketed more than deep understanding, deep explainability, or deep fusion of on-platform resources. Inexperienced practitioners might twist a few algorithmic knobs and test on data of uncertain virtue until performance seems superb. Unfortunately, with the enemy and ever-changing environment conspiring to defeat detection and recognition, naively designed ATRs can fail in unexpected and spectacular ways.
Trustworthy ATRs need to fuse multiple data and metadata sources, continuously learn from and adapt to their environment, interact with humans in natural language, and deal with in-library and out-of-library targets and confuser objects. This fourth edition, with a new chapter on autonomous lethal weapons and ATR, provides a blueprint for smarter, more autonomous, more sophisticated ATR designs.
Target classification algorithms have generally kept pace with developments in the academic and commercial sectors since the 1970s. However, most recently, investment into object classification by internet companies and various Human Brain Projects have far outpaced that of the defense sector. Implications are noteworthy. There are some unique characteristics of the military classification problem. Target classification is not solely an algorithm design problem, but is part of a larger system design task. The design flows down from a concept of operations (ConOps) and key performance parameters (KPPs). Inputs are image and/or signal data and time-synchronized metadata. The operation is real-time. The implementation minimizes size, weight and power (SWaP). The output must be conveyed to a time-strapped operator who understands the rules of engagement. It is assumed that the adversary is actively trying to defeat recognition. The target list is often mission dependent, not necessarily a closed set, and may change on a daily basis. It is highly desirable to obtain sufficiently comprehensive training and testing data sets, but costs of doing so are very high and data on certain target types are scarce. The training data may not be representative of battlefield conditions suggesting the avoidance of highly tuned designs. A number of traditional and emerging target classification strategies are reviewed in the context of the military target problem.
A number of image quality metrics (IQMs) and video quality metrics (VQMs) have been proposed in the literature for evaluating techniques and systems for mitigating degraded visual environments. Some require both pristine and corrupted imagery. Others require patterned target boards in the scene. None of these metrics relates well to the task of landing a helicopter in conditions such as a brownout dust cloud.
We have developed and used a variety of IQMs and VQMs related to the pilot’s ability to detect hazards in the scene and to maintain situational awareness. Some of these metrics can be made agnostic to sensor type. Not only are the metrics suitable for evaluating algorithm and sensor variation, they are also suitable for choosing the most cost effective solution to improve operating conditions in degraded visual environments.
The subject being addressed is how an automatic target tracker (ATT) and an automatic target recognizer (ATR) can be fused together so tightly and so well that their distinctiveness becomes lost in the merger. This has historically not been the case outside of biology and a few academic papers. The biological model of ATT∪ATR arises from dynamic patterns of activity distributed across many neural circuits and structures (including retina). The information that the brain receives from the eyes is “old news” at the time that it receives it. The eyes and brain forecast a tracked object’s future position, rather than relying on received retinal position. Anticipation of the next moment – building up a consistent perception – is accomplished under difficult conditions: motion (eyes, head, body, scene background, target) and processing limitations (neural noise, delays, eye jitter, distractions). Not only does the human vision system surmount these problems, but it has innate mechanisms to exploit motion in support of target detection and classification. Biological vision doesn’t normally operate on snapshots. Feature extraction, detection and recognition are spatiotemporal. When vision is viewed as a spatiotemporal process, target detection, recognition, tracking, event detection and activity recognition, do not seem as distinct as they are in current ATT and ATR designs. They appear as similar mechanism taking place at varying time scales. A framework is provided for unifying ATT and ATR.
Hundreds of simple target-detection algorithms were tested on mid- and long-wave forward-looking infrared images. Each algorithm is briefly described. Indications are given as to which performed well. Most of these simple algorithms are loosely derived from standard tests of the difference of two populations. For target detection, these are populations of pixel grayscale values or features derived from them. The statistical tests are implemented in the form of sliding triple window filters. Several more elaborate algorithms are also described with their relative performances noted. They utilize neural networks, deformable templates, and adaptive filtering. Algorithm design issues are broadened to cover system design issues and concepts of operation. Since target detection is such a fundamental problem, it is often used as a test case for developing technology. New technology leads to innovative approaches for attacking the problem. Eight inventive paradigms, each with deep philosophical underpinnings, are described in relation to their effect on target detector design.
Forward Looking Infrared (FLIR) automatic target recognition (ATR) systems depend upon the capacity of the atmosphere
to propagate thermal radiation over long distances. To date, not much research has been conducted on analyzing
and mitigating the effects of the atmosphere on FLIR ATR performance, even though the atmosphere is often the
limiting factor in long-range target detection and recognition. The atmosphere can also cause frame-to-frame inconsistencies
in the scene, affecting the ability to detect and track moving targets. When image quality is limited by turbulence,
increasing the aperture size or improving the focal plane array cannot improve ATR performance. Traditional
single frame image enhancement does not solve the problem.
A new approach is described for reducing the effects of turbulence. It is implemented under a lucky-region-imaging
framework using short integration time and spatial domain processing. It is designed to preserve important target and
scene structure. Unlike previous Fourier-based approaches originating from the astronomical community, this new approach
is intended for real-time processing from a moving platform, with ground as the background. The system produces
a video stream with minimal delay.
A new video quality measure (VQMturb) is presented for quantifying the success of turbulence mitigation on real data
where no reference imagery is available. The VQMturb is the core of the innovation because it allows a wide range of
algorithms to be quantitatively compared. An algorithm can be chosen, and then tuned, to best-fit available processing
power, latency requirements, scenarios and sensor characteristics.
Humans are better at detecting targets in literal imagery than any known algorithm. Recent advances in modeling visual
processes have resulted from f-MRI brain imaging with humans and the use of more invasive techniques with monkeys. There are four startling new discoveries. 1) The visual cortex does not simply process an incoming image. It constructs a physics based model of the image. 2) Coarse category classification and range-to-target are estimated quickly - possibly through the dorsal pathway of
the visual cortex, combining rapid coarse processing of image data with expectations and goals. This data is then
fed back to lower levels to resize the target and enhance the recognition process feeding forward through the ventral
pathway. 3) Giant photosensitive retinal ganglion cells provide data for maintaining circadian rhythm (time-of-day) and modeling
the physics of the light source. 4) Five filter types implemented by the neurons of the primary visual cortex have been determined. A computer model for automatic target detection has been developed based upon these recent discoveries. It uses an
artificial neural network architecture with multiple feed-forward and feedback paths. Our implementation's efficiency
derives from the observation that any 2-D filter kernel can be approximated by a sum of 2-D box functions. And, a 2-D
box function easily decomposes into two 1-D box functions. Further efficiency is obtained by decomposing the largest
neural filter into a high pass filter and a more sparsely sampled low pass filter.
In November of 2000, the Deputy Under Secretary of Defense for Science and Technology Sensor Systems (DUSD (S&T/SS)) chartered the ATR Working Group (ATRWG) to develop guidelines for sanctioned Problem Sets. Such Problem Sets are intended for development and test of ATR algorithms and contain comprehensive documentation of the data in them. A problem set provides a consistent basis to examine ATR performance and growth. Problem Sets will, in general, serve multiple purposes. First, they will enable informed decisions by government agencies sponsoring ATR development and transition. Problem Sets standardize the testing and evaluation process, resulting in consistent assessment of ATR performance. Second, they will measure and guide ATR development progress within this standardized framework. Finally, they quantify the state of the art for the community. Problem Sets provide clearly defined operating condition coverage. This encourages ATR developers to consider these critical challenges and allows evaluators to assess over them. Thus the widely distributed development and self-test portions, along with a disciplined methodology documented within the Problem Set, permit ATR developers to address critical issues and describe their accomplishments, while the sequestered portion permits government assessment of state-of-the-art and of transition readiness. This paper discusses the elements of an ATR problem set as a package of data and information that presents a standardized ATR challenge relevant to one or more scenarios. The package includes training and test data containing targets and clutter, truth information, required experiments, and a standardized analytical methodology to assess performance.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.