Utilizing functional near-infrared spectroscopy for prediction of cognitive workload in noisy work environments

Ryan D. Gabbard; Mary Fendley; Irfaan A. Dar; Rik Warren; Nasser H. Kashou

doi:10.1117/1.NPh.4.4.041406

18 August 2017 Utilizing functional near-infrared spectroscopy for prediction of cognitive workload in noisy work environments

Ryan D. Gabbard; Mary Fendley; Irfaan A. Dar; Rik Warren; Nasser H. Kashou;

Author Affiliations +

Neurophotonics, 4(4), 041406 (2017). https://doi.org/10.1117/1.NPh.4.4.041406

Abstract

Occupational noise frequently occurs in the work environment in military intelligence, surveillance, and reconnaissance operations. This impacts cognitive performance by acting as a stressor, potentially interfering with the analysts’ decision-making process. We investigated the effects of different noise stimuli on analysts’ performance and workload in anomaly detection by simulating a noisy work environment. We utilized functional near-infrared spectroscopy (fNIRS) to quantify oxy-hemoglobin (HbO) and deoxy-hemoglobin concentration changes in the prefrontal cortex (PFC), as well as behavioral measures, which include eye tracking, reaction time, and accuracy rate. We hypothesized that noisy environments would have a negative effect on the participant in terms of anomaly detection performance due to the increase in workload, which would be reflected by an increase in PFC activity. We found that HbO for some of the channels analyzed were significantly different across noise types (p<0.05). Our results also indicated that HbO activation for short-intermittent noise stimuli was greater in the PFC compared to long-intermittent noises. These approaches using fNIRS in conjunction with an understanding of the impact on human analysts in anomaly detection could potentially lead to better performance by optimizing work environments.

Gabbard, Fendley, Dar, Warren, and Kashou: Utilizing functional near-infrared spectroscopy for prediction of cognitive workload in noisy work environments

1. Background

The ever-increasing rate of object identification through automation outpaces the ability of human analysts to process the captured images and has become an increasingly time-critical task.¹^,² Analysts must be able to discern signals (targets) from noise that can lead to four possible decisions: hit, miss, false alarm, and correct rejection.³ Anomalies refer to finding outliers or patterns in a dataset that do not conform to expected or normal behavior.⁴ Certain factors increase the difficulty of timely anomaly detection due to their random and unexpected nature.⁵^,⁶ Occupational noise is a frequent exposure in the analysts’ work environment in military intelligence, surveillance, and reconnaissance (ISR) operations. Noise impacts cognitive performance by acting as a stressor, which could interfere with the analysts’ decision-making process.⁷ It has been well-documented that occupational noise has a negative impact on job satisfaction.⁸9.10.^–¹¹ Previous studies have shown that noise stress has a negative effect on attention, working memory, and episodic recall. It has been shown that the effect of noise varies based on task complexity. For instance, studies have shown that noise has no effect on simple tasks but does play a factor in more complex tasks.¹² Theories have been proposed to explain why there is a tendency for people to perform better in silence compared to an environment that contains background noise.¹³^,¹⁴ One such theory is that background noise captures the individual’s attention and reallocates their attention away from the target information, causing an interruption in the task.¹⁴^,¹⁵ However, prior research between the relationship of cognitive workload and noise stress has been ambiguous.¹⁶ It is well-established that the prefrontal cortex (PFC) plays a key role in higher order cognitive function,¹⁷ thus, the use of functional near-infrared spectroscopy (fNIRS) to measure workload is a natural step.

fNIRS is an optical method that measures the changes in concentration of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbD). fNIRS provides independent measures of the chromophores present in HbO and HbD by utilizing at least two different wavelengths that are differentially absorbed.¹⁸ fNIRS utilizes wavelengths within the 600- to 950-nm near-infrared region in tissues where scattering is the main photon transport mechanism.¹⁹ Active neurons attract more blood flood flow to their region as they require additional energy.²⁰ Cortical neuronal activation increases cerebral oxygenation (neurovascular coupling) when specific stimuli are present, which make it possible to associate specific regions of the brain to cognitive functions.²¹ Previous studies have shown that by isolating the PFC, one can specifically look at mental workload.²² Another study revealed that when a person is stressed, the cortical activation in the PFC is reduced.²³ Hemodynamic responses from the PFC have been shown to be a reliable measurement in complex naturalistic tasks in terms of quantifying cognitive workload levels.²⁴ Using fNIRS to detect workload effects in the PFC, as a result of visual stimulation, has also been investigated.²⁵ Ayaz et al.²⁶ used fNIRS to detect the level of workload for participants piloting unmanned air vehicles. Solovey et al.²⁷ performed a multitasking assignment to emulate real-world environments as opposed to the standard $n$ -back task studies used to investigate mental workload.

In our study, we utilize fNIRS to address anomaly detection research by highlighting activation when anomalies are correctly detected and also missed while measuring performance under a variety of noise conditions. Behavioral data that include eye tracking, reaction time, and accuracy rate of target identification were also utilized in conjunction with fNIRS to aid in verifying the hemodynamic results. The study was undertaken specifically to quantify the effect of different noise stimuli on analysts’ performance and workload in anomaly detection by simulating a noisy work environment.

1.1.

Significance

Missed anomalies can have a devastating effect on ISR operations as they can provide significant information for numerous applications.²⁸ An understanding of how noisy work environments impact human analysts in anomaly detection could potentially lead to better performance by optimizing work environments. Most research in the past, when dealing with measuring workload in the brain, has been done with the use of electroencephalography (EEG) through monitoring brain activity. fNIRS has gained momentum as a brain imaging modality due to its ability to measure unique sets of physiological data (HbO and HbD) as well as lessen physical restrictions set on the user when compared to EEG systems.²⁹ Previous studies have shown that situations of high cognitive workload correlated to decreased task performance in the field of anomaly detection.¹⁶ fNIRS has the capability of monitoring neural activation in the PFC, which has shown to correlate with workload. Therefore, by deploying fNIRS, quantifiable data can be obtained that can assess the effect of noise-filled work environments on human analysts. Our main objective was to perform fNIRS in adults participating in visual search tasks in various noise-filled environments to simulate the effect of a noisy work environment of an ISR analyst. We hypothesize that noisy environments would have a negative effect on the participant in terms of performance due to the increase in workload, which would be reflected by an increase in PFC activity. Therefore, we anticipate that significant HbO and HbD concentrations will arise in the PFC, following evidence from previous studies.²²

2. Materials and Methods

2.1.

Instrumentation

A continuous wave, single phase, compact NIRScout imaging system (NIRx Medical Technologies LLC)—with eight 760- and 850-nm wavelength LED sources (power: $> 5 mW / wavelength$ ) and eight Si photodiode detectors (sensitivity and dynamic range: $< 1 pW$ and 90 dB)—was used to measure changes in HbO and HbD at a sampling rate of 7.81 Hz. Changes in concentration of both HbO and HbD were defined as the differences between an average baseline and a stimulus-induced visual target. Data were recorded using NIRStar 14.2 (NIRx Medical Technologies LLC). With the aid of a retaining cap, all sources and detectors (optodes) were arranged to overlie the PFC (Fig. 1).

Fig. 1

(a) Sources (black outline gray fill) and detectors (black outline white fill) layout based on 10–20 system to give a total of eight sources, eight detectors, and 23 channels. (b) Illustration of (i) task paradigm timing. A total of seven videos were used. (ii) Videos consisted of real-world scene with instruction to identify particular targets. (c) A zoomed-in view of the schematic shown in (a). Significant channels are indicated with asterisks and channel numbers. Channels 2, 9, and 15 were found to be significant for HbO. Channels 2, 8, 9, and 10 were significant for HbD.

2.2.

Experimental Protocol

The experiment was conducted at Wright State University (WSU) fNIRS Lab. The experiment consisted of seven different visual search tasks in the form of a live video feed that was obtained from Computer Vision Lab Walking Pedestrians dataset³⁰ and were implemented in various environments through the use of different noise stimuli via headphones. Every video was $\sim 4 \min$ in duration and contained six visual targets that consisted of four nonoverlapping and two overlapping targets. Examples of targets include a person pushing a stroller and a person wearing a red coat. The targets varied for each video by appearing at random time frames and were on screen for a variable amount of time. Therefore, each video was unique. The videos were randomized from subject to subject to eliminate any biasing, and a baseline of $\sim 1 \min$ occurred prior to the start of each video (Fig. 1).

A textbox was present in the upper right-hand corner for the entire duration of each video that contained all of the targets in the order they would appear on the screen. The subjects were instructed to press key “4” on the keyboard when the specified target entered and key “6” when the target exited. Prior to the start of the experiment, the participants were given instructions and a demonstration via a practice video. The noise stimuli were presented through audio files that consisted of realistic environmental noises, such as a telephone ringing and a jet flying overhead. All of the noise stimuli were $\sim 70 dB$ . Videos varied based on duration of noise stimuli and/or time of introduction of noise stimuli. The experiment consisted of one factor (occupational noise) and four levels (type of noise). The types of noise included: no noise, short-intermittent noise-12 and -60, and long-intermittent noise. The experimental design for each video is shown in Table 1.

Table 1

Video presentations were the same for all conditions, but the type of distraction noise was varied. Design was divided into: no noise, short-60, long, and short-12.

Video	Type of noise	Onset of noise
1	No noise	No delay
2	Short-intermittent-60	5 s prior to target
3	Short-intermittent-60	No delay
4	Long intermittent	5 s prior to target
5	Long intermittent	No delay
6	Short-intermittent-12	No delay
7	Short-intermittent-12	5 s prior to target

The short-intermittent noise-12 stimuli consist of 12 noise occurrences that are each 2 s in duration while the short-intermittent noise-60 is composed of 60 2-s noises. The long-intermittent noise stimulus contains 12 noise occurrences that are each 10 s in duration.

In collaboration with fNIRS, eye tracking, reaction time, and accuracy rate were utilized in the experiment. The eye-tracking metrics include target fixation count rate and target fixation duration rate. The target fixation and duration rates are the number and duration of fixations, respectively, divided by the duration of the specific target when it comes on screen. By dividing fixation count and duration by the duration of the target, the data can be normalized between targets due to the targets differing from video to video in terms of the time they are present on the screen. The reaction time is composed of the target-in and target-out latency. The target-in latency refers to the time that the participant presses key “4” when they identify that the target is entering the screen minus the time the target actually comes into the video. The target-out latency is the time when the participant presses key “6” when the target exits the screen minus the actual time the target exits the screen. The accuracy rate is composed of the number of missed targets and false alarms. Missed targets refer to a target entering or exiting the video and the participants fail to press the corresponding computer key. False alarms refer to when the participant pressed key “4” or “6” when there is no target on the screen. The eye tracking, reaction time, and accuracy measures are displayed in Table 2.

Table 2

Dependent variables with measure details for eye tracking, reaction time, and accuracy metrics.

Dependent variable	Measure
Target fixation count rate (per unit time)	(Number of target fixations)/(duration of target on screen)
Target fixation duration rate (per unit time)	(Target fixation duration)/(duration of target on screen)
Target-in latency	(Key “4” pressed time) – (duration of target when it comes on screen)
Target-out latency	(Key “6” pressed time) – (duration of target when it comes on screen)
Missed targets	Number of missed targets throughout the video
False alarms	Number of false alarms when pressing the “4” or “6” keys

2.3.

Participant and Preparation

A total of 10 participants [ $38.30 \pm 4.61 years$ , $mean \pm standard error$ (SE)] were recruited at WSU. Three were female ( $38.00 \pm 4.91 years$ ) and seven were male ( $38.43 \pm 4.89 years$ ), and all were right-hand dominant. The experimental protocol was approved by the institutional review board at WSU, and informed consent was obtained from each participant prior to involvement in the study.

Subjects were seated in front of a Tobii T120 Eye Tracker monitor with a screen size of 17 in. and resolution of $1280 \times 1024 pixels$ . After an optode retaining cap was placed on the subject’s head and centered, clear saline electrode gel was applied to enhance the signal quality by keeping the hair aside and improving optode contact with the skin. The gel application process allowed hair to be pushed and kept aside before inserting optodes into the cap. Measurements were taken in a quiet and darkened room. A large, fleece cap was placed over the subject’s head to further reduce any outside light that could possibly strike the detectors. Each participant was asked to focus on the eye-tracking monitor during data collection while remaining silent and as still as possible, aside from the hand-movement tasks associated with identifying the visual targets. Participants that had increased movement during the experiment were well-documented to correct for the motion during the preprocessing steps of data analysis.

2.4.

Data Analysis

Raw data were processed using the functions of Homer2³¹–MATLAB^®-based (The Mathworks, Inc.) analysis tool and used to extract relevant values, such as target and baseline concentrations. A 0.01- to 0.1-Hz bandpass filter was used, which allowed the elimination of low-frequency system drift and physiological noise, such as heart rate, without removing artifacts that may be stimulus induced. Filtered signals were translated to changes of hemoglobin concentration using the modified Beer–Lambert law. Differential pathlength factor values for this region and a mean age of 38.3 years were estimated to be 6.14 and 5.09 for wavelengths of 760 and 850 nm, respectively, based upon calculations from previous studies.³²^,³³ The filtered signals were visually inspected, and noisy data were removed to mitigate the effect of any motion artifacts that were not removed by the bandpass filter.

2.5.

Statistical Analysis

Quantitative statistical analysis of the mean hemoglobin value of the time points between the onset and exit of each respective nonoverlapping target subtracted by the mean of each target’s baseline was performed on all the optode channels. Subject heterogeneity and head sizes vary, and therefore recordings may not always come from the same specific region of interest. Here, baseline is defined as the average concentration during the 5-s period before each target. Mixed effects models were used to account for within-subject variability and repeated measures. The type of noise levels were compared for each individual channel. This was performed on the group levels, and all tests were two-sided and performed at the significance level $(α) = 0.05$ . JMP 11^® by SAS^® (SAS Institute Inc., 2013, Cary, North Carolina: SAS Institute Inc.) was used to perform statistical tests. All values are written as $mean \pm SE$ .

3. Results

From the 10 subjects that were tested, a total of eight subjects were used in the analysis as two subjects were excluded due to fNIRS system failure and noise-filled signals. After removing outliers and noisy data, the hemodynamic responses for both HbO and HbD were calculated for all of the channels (1 to 23). A representation of the block-averaged hemodynamic responses for a target response for all the participants and type of noise levels are shown in Fig. 2.

Fig. 2

An illustration of the block-averaged hemodynamic responses for a target response across all the participants and noise types. The error bars represent SE. HbO curves are asterisk lines and HbD curves are circular lines. The solid vertical line indicates the onset of the target. The dashed vertical line indicates the exit of the target. (a) Represents the no noise, (b) is the short-12, (c) is the short-60, and (d) is the long noise stimulus.

From the 23 channels that were analyzed, there were three statistically significant channels for HbO (channels 2, 9, and 15) and four significant channels (channels 2, 8, 9, and 10) for HbD. The significant channels are illustrated in Fig. 1. For HbO in channel 2, the long-intermittent noise ( $mean = - 0.11$ , $SE = 0.05 μ mol / L$ ) was significantly lower compared to short-60 ( $0.089 \pm 0.05 μ mol / L$ , $p - value = 0.026$ ), but there were no significant differences between the long, no noise ( $- 0.016 \pm 0.07 μ mol / L$ ), and short-12 ( $- 0.019 \pm 0.05 μ mol / L$ ) levels. The long noise level ( $- 0.12 \pm 0.06 μ mol / L$ ) for HbO in channel 9 was significantly lower than short-60 ( $0.15 \pm 0.06 μ mol / L$ , $p - value = 0.022$ ), but the long noise was not significantly different in comparison to the no noise ( $0.063 \pm 0.09 μ mol / L$ ) and short-12 ( $- 0.018 \pm 0.06 μ mol / L$ ) levels. In channel 15 for HbO, short-60 ( $0.098 \pm 0.08 μ mol / L$ , $p - value = 0.019$ ) was significantly higher in comparison to both the no noise ( $- 0.24 \pm 0.12 μ mol / L$ ) and long ( $- 0.17 \pm 0.08 μ mol / L$ ) levels, but there were no statistical differences between the short-60 and short-12 ( $0.018 \pm 0.08 μ mol / L$ ) noise stimuli. The box and whisker plots for the significant HbO channels are shown in Fig. 3.

Fig. 3

Mean HbO concentration changes ( $μ mol / L$ ) for significant channels across the different noise types. (a) The long noise level is significantly lower than the short-60 condition for HbO in channel 2. (b) In channel 9 for HbO, the long noise stimulus is significantly lower than the short-60 level. (c) The short-60 is significantly higher compared to both the long and no noise levels for HbO in channel 15.

For HbD in channel 2, the long-intermittent noise ( $0.12 \pm 0.05 μ mol / L$ , $p - value = 0.0060$ ) was significantly larger in comparison to both the short-12 ( $- 0.029 \pm 0.05 μ mol / L$ ) and short-60 ( $- 0.11 \pm 0.05 μ mol / L$ ) levels but not statistically different than the no noise condition ( $0.020 \pm 0.07 μ mol / L$ ). For HbD in channel 8, the long noise ( $0.10 \pm 0.04 μ mol / L$ , $p - value = 0.0082$ ) was significantly larger than the short-60 condition ( $- 0.095 \pm 0.04 μ mol / L$ ). There were no statistical differences between the long, no noise ( $- 0.0089 \pm 0.06 μ mol / L$ ), and short-12 ( $- 0.011 \pm 0.04 μ mol / L$ ) levels. The long-intermittent noise ( $0.077 \pm 0.03 μ mol / L$ , $p - value = 0.0014$ ) for HbD in channel 9 was significantly larger in comparison to both the no noise ( $- 0.072 \pm 0.05 μ mol / L$ ) and short-60 ( $- 0.080 \pm 0.0 μ mol / L$ ) levels. There were no statistical differences between the long and short-12 ( $- 0.011 \pm 0.03 μ mol / L$ ) noise stimuli. In channel 10 for HbD, the long noise stimulus ( $0.10 \pm 0.04 μ mol / L$ , $p - value = 0.0016$ ) was significantly larger than both the short-12 ( $- 0.016 \pm 0.04 μ mol / L$ ) and short-60 ( $- 0.088 \pm 0.04 μ mol / L$ ) noise levels. There were no statistical differences between the long and no noise ( $- 0.022 \pm 0.05 μ mol / L$ ) conditions. The box and whisker plots for the significant HbD channels are shown in Fig. 4.

Fig. 4

Mean HbD concentration changes ( $μ mol / L$ ) for significant channels across the different noise types. (a) The long level is significantly higher in comparison to both the short-12 and short-60 levels in channel 2 for HbD. (b) In channel 8 for HbD, the long noise is significantly higher compared to the short-60 level. (c) The long noise is significantly higher compared to both the no noise and short-60 levels for HbD in channel 9. (d) In HbD for channel 10, the long level was significantly higher compared to both the short-12 and short-60 stimuli.

Target-in and target-out latency was analyzed via paired student’s $t$ -test, and there was no statistical significance between the noise levels. However, short-12 had the highest mean for both target-in and target-out latency ( $6.57 \pm 0.90$ and $4.16 \pm 0.97$ , respectively) while short-60 ( $5.49 \pm 0.90$ and $2.52 \pm 0.97$ , respectively) had the lowest. There was also no statistical difference for both the false alarm rate and missed targets. However, short-12 ( $2.63 \pm 0.47$ ) and short-60 ( $1.69 \pm 0.47$ ) had higher mean false alarms compared to the long ( $1.38 \pm 0.47$ ) and no noise ( $1.13 \pm 0.67$ ) levels. For the missed targets, the short-12 ( $1.63 \pm 0.41$ ) and no noise ( $1.50 \pm 0.58$ ) levels had the highest average while the long ( $1.44 \pm 0.41$ ) and short-60 ( $mean = 1.06 \pm 0.41$ ) had the lowest.

Target fixation count rate and target fixation duration rate had statistical significance. In both cases, the mean of the long noise ( $1.12 \pm 0.05$ and $0.11 \pm 0.005$ , respectively) was the greatest compared to the other levels, and short-12 ( $0.92 \pm 0.05$ and $0.070 \pm 0.005$ , respectively) was the lowest. For the fixation count rate, the long-intermittent noise ( $p - value = 0.0047$ ) was significantly different than both the short-12 ( $p - value = 0.045$ ) and no noise levels. For the target fixation duration rate, the long noise ( $p - value < 0.0001$ ) was statistically different than the short-60, no noise, and short-12 ( $p - value < 0.0001$ ) levels.

4. Discussion and Conclusion

This study aimed to assess the fNIRS signal activation at the PFC in adults during visual search tasks in various noise-filled environments to simulate the workload of an ISR analyst. The salient findings of our study both partially support and go against our hypothesis that PFC activity would increase as a result of a noise stimulus during a visual target search task. The findings partially support our hypothesis as a result of HbO activation in channel 15 for short-60 being significantly higher in comparison to the no noise condition, hence indicating that this noise condition causes an increase in cognitive workload. The findings go against our hypothesis as a result of the no noise condition having a higher PFC activation compared to the long noise in channel 9 for HbD. Also, there were no significant differences between the no noise, long, and short-12 levels for HbO. This could be due to increased workload of the noise stimulus causing blood flow to be redirected to other portions of the brain, thus reducing the amount of blood flow to the PFC leading to no statistical difference for noise versus no noise. During the periods of multitasking, load sensitive brain regions can elicit either transient or sustained activation, which suggests that these regions could be involved in time-constrained activities such as memory updating. According to Wickens et al.,³⁴ “the resources on which this updating activity depends seem to be limited in their availability, and, when deployed in the service of one task, their availability to be of service to other tasks is reduced.” Due to the temporal aspect of these activations during multitasking, some of these neural stimulations could explain the decline in stimulation in one region (specifically, PFC) even though this region could be heavily involved. On the same note, increasing load on working memory and workload does not always result in a decrease in performance,³⁵ which could also explain why there were no significant findings with the reaction time and performance measures.

However, there were significant differences for HbO activation in channels 2, 9, and 15 due to short-60 being significantly higher in comparison to the long noise stimulus. Also, both short-12 and short-60 were found to have a significantly higher neural activation compared to the long noise stimulus in channels 2 and 10 for HbD. Therefore, the short-intermittent noises could mimic “alarms” that distract the participants attention away from their primary task (target identification) while the long noise is more continuous and could serve as “typical background noise” that the participants experience in their daily life. Furthermore, previous studies have indicated that HbO activation due to continuous noise is significantly lower than short-intermittent noise in regions of the PFC.³⁶37.38.^–³⁹ This implies that workload and working memory could be adversely affected by pulsated noises as these distractions can result in increased neuronal activation as it takes a larger cognitive effort to perform the task at hand. Pulsated noise also activates auditory regions of the brain to a larger degree compared to more continuous noises.³⁹^,⁴⁰ Therefore, pulsated noise habituates to a lesser degree compared to continuous noise due to it containing more information.³⁹

Stressors, such as time and noise, also play a factor in anomaly detection due to their influence on attention, executive function, and memory. Prior research has been conducted to assess the relationship between noise exposure and cognitive performance; however, the results have been ambiguous, as the effects of noise on performance have been found to be facilitative, detrimental, or even absent.¹⁶ Research has also shown that situations of high cognitive load correlate to increased error rates in anomaly detection by the human analyst.¹⁶ The target fixation duration rate, another indicator of cognitive workload, resulted in a significantly higher mean for the long-intermittent noise compared to the other levels. This suggests that the participants were more focused on the targets during the long noise as they entered and exited the screen. A previous study found that a decrease in target fixation rates could be indicative of “visual tunneling” on nontarget stimuli due to the participants having a reduced range of scanning. This reduced range of scanning could be the outcome of increased workload.⁴¹ This increased workload can account for the significant difference in HbO and HbD activation in the short-noise setting compared to the long noise.

With regards to fNIRS, the PFC serves a vital role as it is related to memory tasks.⁴² It is believed that the PFC promotes mental manipulations with regards to the central executive of working memory. Studying the PFC during periods of high working memory-intensive tasks is important because during this time, there is a change in the regional cerebral blood flow. This would indicate an increase in neuronal activation, in which fNIRS measures in the form of oxy-Hb concentration.⁴³ Herff et al.⁴² showed that fNIRS signals obtained from the PFC are indicators that can be utilized to quantify user workload. Bendall et al.⁴⁴ also demonstrated that NIRS is a well-suited technique to measure PFC activity during cognitive tasks. Specifically, previous fMRI studies have demonstrated that the posterior medial and dorsolateral PFCs are active in both working memory and workload tasks.⁴⁵46.47.^–⁴⁸ The posterior medial PFC has also been shown to play a key role in decision-making.⁴⁸^,⁴⁹ The significant channels in this study (Fig. 1) correspond to the previous findings as they are mainly congregated around the posterior medial PFC.

Overall, fNIRS is a promising domain, which can have major impacts in terms of military applications. Specifically, being able to “quantify” mental workload does not only impact anomaly detection but can play a huge role in influencing the probability of human error during remotely operated vehicle (ROV) operations, for example, where safety hinges on the operator’s ability to make instantaneous decisions.⁵⁰ A high workload can result in the operator making rash and ill-advised decisions, which can have catastrophic outcomes. According to the Office of Aerospace Medicine in the United States, human factors-related deficiencies are responsible for 21% to 67% of ROV accidents in the US Army, Navy, and Air Force.⁵¹ These are preventable, and fNIRS can play a major role in helping to bring these numbers down. By analyzing the effect of workload and environmental noise stressors via the utilization of fNIRS technology, the optimal environment for the ROV operators can be determined. This can possibly lead to an increase in work efficiency and a decrease in the number of human factor-related accidents.

Like any experiment, our study has several limitations. This study only investigated HbO and HbD activation in the PFC region; therefore, we were unable to measure other regions of the brain, such as the motor, visual or auditory cortices, which could experience significant changes in cerebral blood flow. Cap configurations for the former two cortices have been tested and optimized.⁵²^,⁵³ Signals from the motor cortex as a result of hand movements could be used as a regressor to clean up the data attained from the PFC. Correlation of the visual and auditory cortices with the PFC could further elucidate cognitive workload findings. Live video feed was used as the visual stimuli to simulate an ISR analyst task that could make one of the seven videos that the participant faced more challenging or easier than the others that could alter our results. fNIRS is unable to measure deep-brain regions due to experiencing limited sensitivity to hemodynamic changes in these areas.⁵⁴ Therefore, fNIRS can only measure cortical regions of the brain.⁵⁵ Additional fNIRS limitations include interferences from nontargeted chromophores (HbO and HbD), indefinite differential pathlength, and unknown scattering loss factor.⁵⁶ Future work will consist of investigating the overall trend of HbO and HbD during overlapping targets.

In conclusion, fNIRS has been shown to measure significant differences in both HbO and HbD activation in the PFC cortex between short- and long-intermittent noises during a visual search task. This suggests a difference in workload or working memory under different noise stimuli as certain regions of the brain could be heavily involved for the processing of one type of noise due to it acting like a distraction but not so much for another noise type. A more comprehensive study, including different noise amplitude and noise frequency, could lead to significant results in performance measures that could further explain the impact of noise on anomaly detection.

Disclosures

The authors have no relevant financial interests in the paper and no other potential conflicts of interest to disclose.

References

1. M. Fendley and S. Narayanan, “Decision aiding to overcome biases in object identification,” Adv. Hum. Comput. Interact. 2012, 1–7 (2012). http://dx.doi.org/10.1155/2012/790304 Google Scholar

2. A. C. Muller and S. Narayanan, “Cognitively-engineered multisensor image fusion for military applications,” Inf. Fusion 10(2), 137–149 (2009). http://dx.doi.org/10.1016/j.inffus.2008.08.008 Google Scholar

3. H. Abdi, Signal Detection Theory, Sage Publications Inc., Thousand Oaks, California (2007). Google Scholar

4. V. Chandola, A. Banerjee and V. Kumar, “Anomaly detection: a survey,” ACM Comput. Surv. 41(3), 1–58 (2009).ACSUEY0360-0300 http://dx.doi.org/10.1145/1541880 Google Scholar

5. A. M. Piasecki, M. E. Fendley and R. Warren, Improving Anomaly Detection Through Identification of Physiological Signatures of Unconscious Awareness, Springer International Publishing, Cham (2017). Google Scholar

6. R. Warren, R. Smith and A. K. Cybenko, “Use of Mahalanobis distance for detecting outliers and outlier clusters in markedly non-normal data: a vehicular traffic example,” Defense Technical Information Center, Wright-Patterson AFB, Dayton, Ohio (2011). Google Scholar

7. N. Kersten and E. Backe, “Occupational noise and myocardial infarction: considerations on the interrelation of noise with job demands,” Noise Health 17(75), 116–122 (2015). http://dx.doi.org/10.4103/1463-1741.153403 Google Scholar

8. P. R. Boyce, “User’s assessments of a landscaped office,” J. Archit. Res. 3(3), 44–62 (1974).JAPRER0738-0895 Google Scholar

9. F. J. Langdon, Modern Offices: A User Survey, (National Building Studies Research Paper 41), Her Majesty’s Stationery Office, London (1966). Google Scholar

10. P. Leather, D. Beale and L. Sullivan, “Noise, psychosocial stress and their interaction in the workplace,” J. Environ. Psychol. 23(2), 213–222 (2003).JEPSEO0272-4944 http://dx.doi.org/10.1016/S0272-4944(02)00082-8 Google Scholar

11. J. Nemecek and E. Grandjean, “Results of an ergonomic investigation of large-space offices,” Hum. Factors 15(2), 111–124 (1973).HUFAA60018-7208 http://dx.doi.org/10.1177/001872087301500203 Google Scholar

12. B. Wright et al., “Understanding noise stress-induced cognitive impairment in healthy adults and its implications for schizophrenia,” Noise Health 16(70), 166–176 (2014). http://dx.doi.org/10.4103/1463-1741.134917 Google Scholar

13. R. W. Hughes, “Auditory distraction: a duplex-mechanism account,” PsyCh J. 3(1), 30–41 (2014). http://dx.doi.org/10.1002/pchj.v3.1 Google Scholar

14. P. Sorqvist, “On interpretation of the effects of noise on cognitive performance: the fallacy of confusing the definition of an effect with the explanation of that effect,” Front. Psychol. 6, 754 (2015). http://dx.doi.org/10.3389/fpsyg.2015.00754 Google Scholar

15. R. Bell et al., “Habituation of the irrelevant sound effect: evidence for an attentional theory of short-term memory disruption,” J. Exp. Psychol. Learn. Mem. Cogn. 38(6), 1542–1557 (2012). http://dx.doi.org/10.1037/a0028459 Google Scholar

16. K. Irgens-Hansen et al., “Noise exposure and cognitive performance: a study on personnel on board Royal Norwegian Navy vessels,” Noise Health 17(78), 320–327 (2015). http://dx.doi.org/10.4103/1463-1741.165057 Google Scholar

17. N. Ramnani and A. M. Owen, “Anterior prefrontal cortex: insights into function from anatomy and neuroimaging,” Nat. Rev. Neurosci. 5(3), 184–194 (2004). http://dx.doi.org/10.1038/nrn1343 Google Scholar

18. S. Wijeakumar et al., “Validating an image-based fNIRS approach with fMRI and a working memory task,” NeuroImage 147, 204–218 (2017).NEIMEF1053-8119 http://dx.doi.org/10.1016/j.neuroimage.2016.12.007 Google Scholar

19. C. Mansouri and N. H. Kashou, “New window on optical brain imaging; medical development, simulations and applications,” in Selected Topics on Optical Fiber Technology INTECH Open Access Publisher (2012). http://dx.doi.org/10.5772/30609 Google Scholar

20. L.-D. Liao et al., “Neurovascular coupling: in vivo optical techniques for functional brain imaging,” Biomed. Eng. Online 12(1), 38 (2013). http://dx.doi.org/10.1186/1475-925X-12-38 Google Scholar

21. V. Quaresima, S. Bisconti and M. Ferrari, “A brief review on the use of functional near-infrared spectroscopy (fNIRS) for language imaging studies in human newborns and adults,” Brain Lang. 121(2), 79–89 (2012). http://dx.doi.org/10.1016/j.bandl.2011.03.009 Google Scholar

22. H. J. Foy, P. Runham and P. Chapman, “Prefrontal cortex activation and young driver behaviour: a fNIRS study,” PLoS One 11(5), e0156512 (2016).POLNCL1932-6203 http://dx.doi.org/10.1371/journal.pone.0156512 Google Scholar

23. F. Al-Shargie, T. Tong Boon and M. Kiguchi, “Mental stress grading based on fNIRS signals,” in Conf. Proc. IEEE Engineering in Medicine and Biology Society, pp. 5140–5143 (2016). http://dx.doi.org/10.1109/EMBC.2016.7591884 Google Scholar

24. A. Unni et al., “Brain activity measured with fNIRS for the prediction of cognitive workload,” in 6th IEEE Int. Conf. on Cognitive Infocommunications, pp. 349–354 (2015). http://dx.doi.org/10.1109/CogInfoCom.2015.7390617 Google Scholar

25. E. M. Peck et al., “Using fNIRS brain sensing to evaluate information visualization interfaces,” in Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, ACM, Paris, France (2013). Google Scholar

26. H. Ayaz et al., “Optical brain monitoring for operator training and mental workload assessment,” NeuroImage 59(1), 36–47 (2012).NEIMEF1053-8119 http://dx.doi.org/10.1016/j.neuroimage.2011.06.023 Google Scholar

27. E. T. Solovey et al., “Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines,” in Proc. of the 22nd Annual ACM Symp. on User Interface Software and Technology, ACM, Victoria, British Columbia, Canada (2009). Google Scholar

28. F. Li et al., “A novel approach for anomaly detection in data streams: fuzzy-statistical detection mode,” J. Intell. Fuzzy Syst. 30(5), 2611–2622 (2016).JIFSE21064-1246 http://dx.doi.org/10.3233/IFS-151910 Google Scholar

29. E. M. Peck et al., “Using fNIRS to measure mental workload in the real world,” in Advances in Physiological Computing, , S. H. Fairclough and K. Gilleade, Eds., pp. 117–139, Springer, London (2014). https://doi.org/10.1007/978-1-4471-6392-3_6 Google Scholar

30. S. Pellegrini et al., “You’ll never walk alone: modeling social behavior for multi-target tracking,” in IEEE 12th Int. Conf. on Computer Vision, pp. 261–268 (2009). http://dx.doi.org/10.1109/ICCV.2009.5459260 Google Scholar

31. T. J. Huppert et al., “HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain,” Appl. Opt. 48(10), D280–D298 (2009).APOPAI0003-6935 http://dx.doi.org/10.1364/AO.48.00D280 Google Scholar

32. A. Duncan et al., “Optical pathlength measurements on adult head, calf and forearm and the head of the newborn infant using phase resolved optical spectroscopy,” Phys. Med. Biol. 40(2), 295–304 (1995).PHMBA70031-9155 http://dx.doi.org/10.1088/0031-9155/40/2/007 Google Scholar

33. H. Zhao et al., “Maps of optical differential pathlength factor of human adult forehead, somatosensory motor and occipital regions at multi-wavelengths in NIR,” Phys. Med. Biol. 47(12), 2075–2093 (2002).PHMBA70031-9155 http://dx.doi.org/10.1088/0031-9155/47/12/306 Google Scholar

34. C. Wickens et al., “Performance of concurrent tasks: a psychophysiological analysis of the reciprocity of information-processing resources,” Science 221(4615), 1080–1082 (1983).SCIEAS0036-8075 http://dx.doi.org/10.1126/science.6879207 Google Scholar

35. G. R. Hockey, “Compensatory control in the regulation of human performance under stress and high workload: a cognitive-energetical framework,” Biol. Psychol. 45(1–3), 73–93 (1997). http://dx.doi.org/10.1016/S0301-0511(96)05223-4 Google Scholar

36. M. P. Harms and J. R. Melcher, “Sound repetition rate in the human auditory pathway: representations in the waveshape and amplitude of fMRI activation,” J. Neurophysiol. 88(3), 1433–1450 (2002).JONEA40022-3077 Google Scholar

37. A.-L. Giraud et al., “Representation of the temporal envelope of sounds in the human brain,” J. Neurophysiol. 84(3), 1588–1598 (2000).JONEA40022-3077 Google Scholar

38. E. Seifritz et al., “Spatiotemporal pattern of neural processing in the human auditory cortex,” Science 297(5587), 1706–1708 (2002).SCIEAS0036-8075 http://dx.doi.org/10.1126/science.1074355 Google Scholar

39. S. Haller et al., “Effect of fMRI acoustic noise on non-auditory working memory task: comparison between continuous and pulsed sound emitting EPI,” Magn. Reson. Mater. Phys. Biol. Med. 18(5), 263–271 (2005).MRBMEQ1352-8661 http://dx.doi.org/10.1007/s10334-005-0010-2 Google Scholar

40. A. Mazard et al., “Impact of fMRI acoustic noise on the functional anatomy of visual mental imagery,” J. Cognit. Neurosci. 14(2), 172–186 (2002).JCONEO0898-929X http://dx.doi.org/10.1162/089892902317236821 Google Scholar

41. Y.-F. Tsai et al., “Task performance and eye activity: predicting behavior relating to cognitive workload,” Aviat. Space Environ. Med. 78(5), B176–B185 (2007). Google Scholar

42. C. Herff et al., “Mental workload during n-back task-quantified in the prefrontal cortex using fNIRS,” Front. Hum. Neurosci. 7, 935 (2013). http://dx.doi.org/10.3389/fnhum.2013.00935 Google Scholar

43. Y. Hoshi et al., “Spatiotemporal characteristics of hemodynamic changes in the human lateral prefrontal cortex during working memory tasks,” NeuroImage 20(3), 1493–1504 (2003).NEIMEF1053-8119 http://dx.doi.org/10.1016/S1053-8119(03)00412-9 Google Scholar

44. R. C. Bendall, P. Eachus and C. Thompson, “A brief review of research using near-infrared spectroscopy to measure activation of the prefrontal cortex during emotional processing: the importance of experimental design,” Front. Hum. Neurosci. 10, 529 (2016). http://dx.doi.org/10.3389/fnhum.2016.00529 Google Scholar

45. E. Kirino et al., “Prefrontal activation evoked by infrequent target and novel stimuli in a visual target detection task: an event-related functional magnetic resonance imaging study,” J. Neurosci. 20(17), 6612–6618 (2000).JNRSDS0270-6474 Google Scholar

46. K. R. Ridderinkhof et al., “The role of the medial frontal cortex in cognitive control,” Science 306(5695), 443–447 (2004).SCIEAS0036-8075 http://dx.doi.org/10.1126/science.1100301 Google Scholar

47. S. A. Huettel, P. B. Mack and G. McCarthy, “Perceiving patterns in random series: dynamic processing of sequence in prefrontal cortex,” Nat. Neurosci. 5(5), 485–490 (2002).NANEFN1097-6256 http://dx.doi.org/10.1038/nn841 Google Scholar

48. V. Venkatraman et al., “Resolving response, decision, and strategic control: evidence for a functional topography in dorsomedial prefrontal cortex,” J. Neurosci. 29(42), 13158–13164 (2009). http://dx.doi.org/10.1523/JNEUROSCI.2708-09.2009 Google Scholar

49. M. M. Botvinick, J. D. Cohen and C. S. Carter, “Conflict monitoring and anterior cingulate cortex: an update,” Trends Cognit. Sci. 8(12), 539–546 (2004). http://dx.doi.org/10.1016/j.tics.2004.10.003 Google Scholar

50. G. Durantin et al., “Using near infrared spectroscopy and heart rate variability to detect mental overload,” Behav. Brain Res. 259, 16–23 (2014).BBREDI0166-4328 http://dx.doi.org/10.1016/j.bbr.2013.10.042 Google Scholar

51. K. W. Williams, “A summary of unmanned aircraft accident/incident data: human factors implications,” DTIC Document (2004). Google Scholar

52. N. H. Kashou et al., “Hand grasping and finger tapping induced similar fNIRS cortical responses,” Neurophotonics 3(2), 025006 (2016). http://dx.doi.org/10.1117/1.NPh.3.2.025006 Google Scholar

53. N. H. Kashou and B. M. Giacherio, “Optode placement effects on fNIRS of visual cortex,” Neurophotonics 3(2), 025005 (2016). http://dx.doi.org/10.1117/1.NPh.3.2.025005 Google Scholar

54. N. Liu et al., “Inferring deep-brain activity from cortical activity using functional near-infrared spectroscopy,” Biomed. Opt. Exp. 6(3), 1074–1089 (2015).BOEICL2156-7085 http://dx.doi.org/10.1364/BOE.6.001074 Google Scholar

55. N. Masataka, L. Perlovsky and K. Hiraki, “Near-infrared spectroscopy (NIRS) in functional research of prefrontal cortex,” Front. Hum. Neurosci. 9, 274 (2015). http://dx.doi.org/10.3389/fnhum.2015.00274 Google Scholar

56. A. Bakker et al., “Near-infrared spectroscopy, applied aspects of ultrasonography in humans,” in Applied Aspects of Ultrasonography in Humans, InTech (2012). http://dx.doi.org/110.5772/32493 Google Scholar

Biography

Ryan Gabbard received his master of science in biomedical engineering from Wright State University (WSU) in 2017 and completed an Air Force Research Laboratory (AFRL)/The Dayton Area Graduate Studies Institute (DAGSI) fellowship as a member of the Image Analysis Lab and Human Performance & Cognition Lab. Currently, he is a MD candidate at Boonshoft School of Medicine Class of 2021.

Mary Fendley is an associate professor at WSU. Her education includes a BA degree from Indiana University and a PhD in engineering. Her research interests are in the areas of cognitive engineering, modeling, decision analysis, and human factors, and she currently runs the Human Performance and Cognition Lab at WSU. She is a member of the Human Factors and Ergonomics Society and serves as a faculty advisor to the Society of Women Engineers.

Irfaan A. Dar received his Master of Science in biomedical engineering from WSU in 2015. Currently, he is continuing his studies as a PhD student at WSU as a member of the Image Analysis Lab (IAL) and as a research associate in the Neonatal Infant Feeding Disorders Program at Nationwide Children’s Hospital. His expertise includes biomedical signal processing, medical imaging, image processing, near-infrared spectroscopy, and hardware design.

Rik Warren (PhD, Cornell University) is a research psychologist in the Air Force Research Laboratory. He studies perception, cognition, and culture emphasizing agent-based modeling of attention, active search, and anomaly detection in natural environments. He is a National Research Council postdoctoral advisor and is on the boards of the International Journal of Aviation Psychology and Ecological Psychology. He edited Perception and Control of Self-Motion and is a fellow of the Psychonomic Society.

Nasser H. Kashou is an associate professor at WSU and an IEEE senior member. He received his PhD from Ohio State University’s Biomedical Engineering Department in 2008. His education also includes a Master of Science in electrical engineering in 2004. Currently, he is the director of the IAL at WSU. He has also established and directs the functional near-infrared spectroscopy lab.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Ryan D. Gabbard, Mary Fendley, Irfaan A. Dar, Rik Warren, Nasser H. Kashou, "Utilizing functional near-infrared spectroscopy for prediction of cognitive workload in noisy work environments," Neurophotonics 4(4), 041406 (18 August 2017). https://doi.org/10.1117/1.NPh.4.4.041406 Submission: Received 6 April 2017; Accepted 17 July 2017

Submission: Received 6 April 2017; Accepted 17 July 2017

Access the abstract

JOURNAL ARTICLE
9 PAGES

DOWNLOAD PDF + SAVE TO MY LIBRARY