1.

Background

The ever-increasing rate of object identification through automation outpaces the ability of human analysts to process the captured images and has become an increasingly time-critical task.^1,2 Analysts must be able to discern signals (targets) from noise that can lead to four possible decisions: hit, miss, false alarm, and correct rejection.³ Anomalies refer to finding outliers or patterns in a dataset that do not conform to expected or normal behavior.⁴ Certain factors increase the difficulty of timely anomaly detection due to their random and unexpected nature.^5,6 Occupational noise is a frequent exposure in the analysts’ work environment in military intelligence, surveillance, and reconnaissance (ISR) operations. Noise impacts cognitive performance by acting as a stressor, which could interfere with the analysts’ decision-making process.⁷ It has been well-documented that occupational noise has a negative impact on job satisfaction.^8–11 Previous studies have shown that noise stress has a negative effect on attention, working memory, and episodic recall. It has been shown that the effect of noise varies based on task complexity. For instance, studies have shown that noise has no effect on simple tasks but does play a factor in more complex tasks.¹² Theories have been proposed to explain why there is a tendency for people to perform better in silence compared to an environment that contains background noise.^13,14 One such theory is that background noise captures the individual’s attention and reallocates their attention away from the target information, causing an interruption in the task.^14,15 However, prior research between the relationship of cognitive workload and noise stress has been ambiguous.¹⁶ It is well-established that the prefrontal cortex (PFC) plays a key role in higher order cognitive function,¹⁷ thus, the use of functional near-infrared spectroscopy (fNIRS) to measure workload is a natural step.

fNIRS is an optical method that measures the changes in concentration of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbD). fNIRS provides independent measures of the chromophores present in HbO and HbD by utilizing at least two different wavelengths that are differentially absorbed.¹⁸ fNIRS utilizes wavelengths within the 600- to 950-nm near-infrared region in tissues where scattering is the main photon transport mechanism.¹⁹ Active neurons attract more blood flood flow to their region as they require additional energy.²⁰ Cortical neuronal activation increases cerebral oxygenation (neurovascular coupling) when specific stimuli are present, which make it possible to associate specific regions of the brain to cognitive functions.²¹ Previous studies have shown that by isolating the PFC, one can specifically look at mental workload.²² Another study revealed that when a person is stressed, the cortical activation in the PFC is reduced.²³ Hemodynamic responses from the PFC have been shown to be a reliable measurement in complex naturalistic tasks in terms of quantifying cognitive workload levels.²⁴ Using fNIRS to detect workload effects in the PFC, as a result of visual stimulation, has also been investigated.²⁵ Ayaz et al.²⁶ used fNIRS to detect the level of workload for participants piloting unmanned air vehicles. Solovey et al.²⁷ performed a multitasking assignment to emulate real-world environments as opposed to the standard $n$-back task studies used to investigate mental workload.

In our study, we utilize fNIRS to address anomaly detection research by highlighting activation when anomalies are correctly detected and also missed while measuring performance under a variety of noise conditions. Behavioral data that include eye tracking, reaction time, and accuracy rate of target identification were also utilized in conjunction with fNIRS to aid in verifying the hemodynamic results. The study was undertaken specifically to quantify the effect of different noise stimuli on analysts’ performance and workload in anomaly detection by simulating a noisy work environment.

2.

Materials and Methods

2.1.

Instrumentation

A continuous wave, single phase, compact NIRScout imaging system (NIRx Medical Technologies LLC)—with eight 760- and 850-nm wavelength LED sources (power: $>5\mathrm{mW}/\text{wavelength}$) and eight Si photodiode detectors (sensitivity and dynamic range: $<1\mathrm{pW}$ and 90 dB)—was used to measure changes in HbO and HbD at a sampling rate of 7.81 Hz. Changes in concentration of both HbO and HbD were defined as the differences between an average baseline and a stimulus-induced visual target. Data were recorded using NIRStar 14.2 (NIRx Medical Technologies LLC). With the aid of a retaining cap, all sources and detectors (optodes) were arranged to overlie the PFC (Fig. 1).

Fig. 1

(a) Sources (black outline gray fill) and detectors (black outline white fill) layout based on 10–20 system to give a total of eight sources, eight detectors, and 23 channels. (b) Illustration of (i) task paradigm timing. A total of seven videos were used. (ii) Videos consisted of real-world scene with instruction to identify particular targets. (c) A zoomed-in view of the schematic shown in (a). Significant channels are indicated with asterisks and channel numbers. Channels 2, 9, and 15 were found to be significant for HbO. Channels 2, 8, 9, and 10 were significant for HbD.

3.

Results

From the 10 subjects that were tested, a total of eight subjects were used in the analysis as two subjects were excluded due to fNIRS system failure and noise-filled signals. After removing outliers and noisy data, the hemodynamic responses for both HbO and HbD were calculated for all of the channels (1 to 23). A representation of the block-averaged hemodynamic responses for a target response for all the participants and type of noise levels are shown in Fig. 2.

Fig. 2

An illustration of the block-averaged hemodynamic responses for a target response across all the participants and noise types. The error bars represent SE. HbO curves are asterisk lines and HbD curves are circular lines. The solid vertical line indicates the onset of the target. The dashed vertical line indicates the exit of the target. (a) Represents the no noise, (b) is the short-12, (c) is the short-60, and (d) is the long noise stimulus.

From the 23 channels that were analyzed, there were three statistically significant channels for HbO (channels 2, 9, and 15) and four significant channels (channels 2, 8, 9, and 10) for HbD. The significant channels are illustrated in Fig. 1. For HbO in channel 2, the long-intermittent noise ($\text{mean}=-0.11$, $\mathrm{SE}=0.05\mu \mathrm{mol}/\mathrm{L}$) was significantly lower compared to short-60 ($0.089\pm 0.05\mu \mathrm{mol}/\mathrm{L}$, $p\text{-}\text{value}=0.026$), but there were no significant differences between the long, no noise ($-0.016\pm 0.07\mu \mathrm{mol}/\mathrm{L}$), and short-12 ($-0.019\pm 0.05\mu \mathrm{mol}/\mathrm{L}$) levels. The long noise level ($-0.12\pm 0.06\mu \mathrm{mol}/\mathrm{L}$) for HbO in channel 9 was significantly lower than short-60 ($0.15\pm 0.06\mu \mathrm{mol}/\mathrm{L}$, $p\text{-}\text{value}=0.022$), but the long noise was not significantly different in comparison to the no noise ($0.063\pm 0.09\mu \mathrm{mol}/\mathrm{L}$) and short-12 ($-0.018\pm 0.06\mu \mathrm{mol}/\mathrm{L}$) levels. In channel 15 for HbO, short-60 ($0.098\pm 0.08\mu \mathrm{mol}/\mathrm{L}$, $p\text{-}\text{value}=0.019$) was significantly higher in comparison to both the no noise ($-0.24\pm 0.12\mu \mathrm{mol}/\mathrm{L}$) and long ($-0.17\pm 0.08\mu \mathrm{mol}/\mathrm{L}$) levels, but there were no statistical differences between the short-60 and short-12 ($0.018\pm 0.08\mu \mathrm{mol}/\mathrm{L}$) noise stimuli. The box and whisker plots for the significant HbO channels are shown in Fig. 3.

Fig. 3

Mean HbO concentration changes ($\mu \mathrm{mol}/\mathrm{L}$) for significant channels across the different noise types. (a) The long noise level is significantly lower than the short-60 condition for HbO in channel 2. (b) In channel 9 for HbO, the long noise stimulus is significantly lower than the short-60 level. (c) The short-60 is significantly higher compared to both the long and no noise levels for HbO in channel 15.

For HbD in channel 2, the long-intermittent noise ($0.12\pm 0.05\mu \mathrm{mol}/\mathrm{L}$, $p\text{-}\text{value}=0.0060$) was significantly larger in comparison to both the short-12 ($-0.029\pm 0.05\mu \mathrm{mol}/\mathrm{L}$) and short-60 ($-0.11\pm 0.05\mu \mathrm{mol}/\mathrm{L}$) levels but not statistically different than the no noise condition ($0.020\pm 0.07\mu \mathrm{mol}/\mathrm{L}$). For HbD in channel 8, the long noise ($0.10\pm 0.04\mu \mathrm{mol}/\mathrm{L}$, $p\text{-}\text{value}=0.0082$) was significantly larger than the short-60 condition ($-0.095\pm 0.04\mu \mathrm{mol}/\mathrm{L}$). There were no statistical differences between the long, no noise ($-0.0089\pm 0.06\mu \mathrm{mol}/\mathrm{L}$), and short-12 ($-0.011\pm 0.04\mu \mathrm{mol}/\mathrm{L}$) levels. The long-intermittent noise ($0.077\pm 0.03\mu \mathrm{mol}/\mathrm{L}$, $p\text{-}\text{value}=0.0014$) for HbD in channel 9 was significantly larger in comparison to both the no noise ($-0.072\pm 0.05\mu \mathrm{mol}/\mathrm{L}$) and short-60 ($-0.080\pm 0.0\mu \mathrm{mol}/\mathrm{L}$) levels. There were no statistical differences between the long and short-12 ($-0.011\pm 0.03\mu \mathrm{mol}/\mathrm{L}$) noise stimuli. In channel 10 for HbD, the long noise stimulus ($0.10\pm 0.04\mu \mathrm{mol}/\mathrm{L}$, $p\text{-}\text{value}=0.0016$) was significantly larger than both the short-12 ($-0.016\pm 0.04\mu \mathrm{mol}/\mathrm{L}$) and short-60 ($-0.088\pm 0.04\mu \mathrm{mol}/\mathrm{L}$) noise levels. There were no statistical differences between the long and no noise ($-0.022\pm 0.05\mu \mathrm{mol}/\mathrm{L}$) conditions. The box and whisker plots for the significant HbD channels are shown in Fig. 4.

Fig. 4

Mean HbD concentration changes ($\mu \mathrm{mol}/\mathrm{L}$) for significant channels across the different noise types. (a) The long level is significantly higher in comparison to both the short-12 and short-60 levels in channel 2 for HbD. (b) In channel 8 for HbD, the long noise is significantly higher compared to the short-60 level. (c) The long noise is significantly higher compared to both the no noise and short-60 levels for HbD in channel 9. (d) In HbD for channel 10, the long level was significantly higher compared to both the short-12 and short-60 stimuli.

Target-in and target-out latency was analyzed via paired student’s $t$-test, and there was no statistical significance between the noise levels. However, short-12 had the highest mean for both target-in and target-out latency ($6.57\pm 0.90$ and $4.16\pm 0.97$, respectively) while short-60 ($5.49\pm 0.90$ and $2.52\pm 0.97$, respectively) had the lowest. There was also no statistical difference for both the false alarm rate and missed targets. However, short-12 ($2.63\pm 0.47$) and short-60 ($1.69\pm 0.47$) had higher mean false alarms compared to the long ($1.38\pm 0.47$) and no noise ($1.13\pm 0.67$) levels. For the missed targets, the short-12 ($1.63\pm 0.41$) and no noise ($1.50\pm 0.58$) levels had the highest average while the long ($1.44\pm 0.41$) and short-60 ($\text{mean}=1.06\pm 0.41$) had the lowest.

Target fixation count rate and target fixation duration rate had statistical significance. In both cases, the mean of the long noise ($1.12\pm 0.05$ and $0.11\pm 0.005$, respectively) was the greatest compared to the other levels, and short-12 ($0.92\pm 0.05$ and $0.070\pm 0.005$, respectively) was the lowest. For the fixation count rate, the long-intermittent noise ($p\text{-}\text{value}=0.0047$) was significantly different than both the short-12 ($p\text{-}\text{value}=0.045$) and no noise levels. For the target fixation duration rate, the long noise ($p\text{-}\text{value}<0.0001$) was statistically different than the short-60, no noise, and short-12 ($p\text{-}\text{value}<0.0001$) levels.

4.

Discussion and Conclusion

This study aimed to assess the fNIRS signal activation at the PFC in adults during visual search tasks in various noise-filled environments to simulate the workload of an ISR analyst. The salient findings of our study both partially support and go against our hypothesis that PFC activity would increase as a result of a noise stimulus during a visual target search task. The findings partially support our hypothesis as a result of HbO activation in channel 15 for short-60 being significantly higher in comparison to the no noise condition, hence indicating that this noise condition causes an increase in cognitive workload. The findings go against our hypothesis as a result of the no noise condition having a higher PFC activation compared to the long noise in channel 9 for HbD. Also, there were no significant differences between the no noise, long, and short-12 levels for HbO. This could be due to increased workload of the noise stimulus causing blood flow to be redirected to other portions of the brain, thus reducing the amount of blood flow to the PFC leading to no statistical difference for noise versus no noise. During the periods of multitasking, load sensitive brain regions can elicit either transient or sustained activation, which suggests that these regions could be involved in time-constrained activities such as memory updating. According to Wickens et al.,³⁴ “the resources on which this updating activity depends seem to be limited in their availability, and, when deployed in the service of one task, their availability to be of service to other tasks is reduced.” Due to the temporal aspect of these activations during multitasking, some of these neural stimulations could explain the decline in stimulation in one region (specifically, PFC) even though this region could be heavily involved. On the same note, increasing load on working memory and workload does not always result in a decrease in performance,³⁵ which could also explain why there were no significant findings with the reaction time and performance measures.

However, there were significant differences for HbO activation in channels 2, 9, and 15 due to short-60 being significantly higher in comparison to the long noise stimulus. Also, both short-12 and short-60 were found to have a significantly higher neural activation compared to the long noise stimulus in channels 2 and 10 for HbD. Therefore, the short-intermittent noises could mimic “alarms” that distract the participants attention away from their primary task (target identification) while the long noise is more continuous and could serve as “typical background noise” that the participants experience in their daily life. Furthermore, previous studies have indicated that HbO activation due to continuous noise is significantly lower than short-intermittent noise in regions of the PFC.^36–39 This implies that workload and working memory could be adversely affected by pulsated noises as these distractions can result in increased neuronal activation as it takes a larger cognitive effort to perform the task at hand. Pulsated noise also activates auditory regions of the brain to a larger degree compared to more continuous noises.^39,40 Therefore, pulsated noise habituates to a lesser degree compared to continuous noise due to it containing more information.³⁹

Stressors, such as time and noise, also play a factor in anomaly detection due to their influence on attention, executive function, and memory. Prior research has been conducted to assess the relationship between noise exposure and cognitive performance; however, the results have been ambiguous, as the effects of noise on performance have been found to be facilitative, detrimental, or even absent.¹⁶ Research has also shown that situations of high cognitive load correlate to increased error rates in anomaly detection by the human analyst.¹⁶ The target fixation duration rate, another indicator of cognitive workload, resulted in a significantly higher mean for the long-intermittent noise compared to the other levels. This suggests that the participants were more focused on the targets during the long noise as they entered and exited the screen. A previous study found that a decrease in target fixation rates could be indicative of “visual tunneling” on nontarget stimuli due to the participants having a reduced range of scanning. This reduced range of scanning could be the outcome of increased workload.⁴¹ This increased workload can account for the significant difference in HbO and HbD activation in the short-noise setting compared to the long noise.

With regards to fNIRS, the PFC serves a vital role as it is related to memory tasks.⁴² It is believed that the PFC promotes mental manipulations with regards to the central executive of working memory. Studying the PFC during periods of high working memory-intensive tasks is important because during this time, there is a change in the regional cerebral blood flow. This would indicate an increase in neuronal activation, in which fNIRS measures in the form of oxy-Hb concentration.⁴³ Herff et al.⁴² showed that fNIRS signals obtained from the PFC are indicators that can be utilized to quantify user workload. Bendall et al.⁴⁴ also demonstrated that NIRS is a well-suited technique to measure PFC activity during cognitive tasks. Specifically, previous fMRI studies have demonstrated that the posterior medial and dorsolateral PFCs are active in both working memory and workload tasks.^45–48 The posterior medial PFC has also been shown to play a key role in decision-making.^48,49 The significant channels in this study (Fig. 1) correspond to the previous findings as they are mainly congregated around the posterior medial PFC.

Overall, fNIRS is a promising domain, which can have major impacts in terms of military applications. Specifically, being able to “quantify” mental workload does not only impact anomaly detection but can play a huge role in influencing the probability of human error during remotely operated vehicle (ROV) operations, for example, where safety hinges on the operator’s ability to make instantaneous decisions.⁵⁰ A high workload can result in the operator making rash and ill-advised decisions, which can have catastrophic outcomes. According to the Office of Aerospace Medicine in the United States, human factors-related deficiencies are responsible for 21% to 67% of ROV accidents in the US Army, Navy, and Air Force.⁵¹ These are preventable, and fNIRS can play a major role in helping to bring these numbers down. By analyzing the effect of workload and environmental noise stressors via the utilization of fNIRS technology, the optimal environment for the ROV operators can be determined. This can possibly lead to an increase in work efficiency and a decrease in the number of human factor-related accidents.

Like any experiment, our study has several limitations. This study only investigated HbO and HbD activation in the PFC region; therefore, we were unable to measure other regions of the brain, such as the motor, visual or auditory cortices, which could experience significant changes in cerebral blood flow. Cap configurations for the former two cortices have been tested and optimized.^52,53 Signals from the motor cortex as a result of hand movements could be used as a regressor to clean up the data attained from the PFC. Correlation of the visual and auditory cortices with the PFC could further elucidate cognitive workload findings. Live video feed was used as the visual stimuli to simulate an ISR analyst task that could make one of the seven videos that the participant faced more challenging or easier than the others that could alter our results. fNIRS is unable to measure deep-brain regions due to experiencing limited sensitivity to hemodynamic changes in these areas.⁵⁴ Therefore, fNIRS can only measure cortical regions of the brain.⁵⁵ Additional fNIRS limitations include interferences from nontargeted chromophores (HbO and HbD), indefinite differential pathlength, and unknown scattering loss factor.⁵⁶ Future work will consist of investigating the overall trend of HbO and HbD during overlapping targets.

In conclusion, fNIRS has been shown to measure significant differences in both HbO and HbD activation in the PFC cortex between short- and long-intermittent noises during a visual search task. This suggests a difference in workload or working memory under different noise stimuli as certain regions of the brain could be heavily involved for the processing of one type of noise due to it acting like a distraction but not so much for another noise type. A more comprehensive study, including different noise amplitude and noise frequency, could lead to significant results in performance measures that could further explain the impact of noise on anomaly detection.