1.

Introduction

Functional near-infrared spectroscopy (fNIRS) is a noninvasive optical technique for monitoring regional tissue oxygenation based on diffusion and absorption of near-infrared light photons in human tissue. Continuous-wave (cw) fNIRS provides measurements of concentration changes in oxy-, deoxy-, and total-hemoglobin species (${\mathrm{\Delta }[\mathrm{HbO}}_2]$, $\mathrm{\Delta }[\mathrm{Hb}]$, and $\mathrm{\Delta }[\mathrm{HbT}]$, respectively) in tissue with temporal sampling rate of on the order of 10 Hz.¹ Over the past three decades, fNIRS has been used in several brain imaging applications, including noninvasive imaging of cognitive tasks and brain functional activation,^1–4 and brain computer interface (BCI).⁵

Memory-based workload classification using fNIRS measurements has been demonstrated to be an ideal approach for a realistic adaptive BCI to measure human workload level.⁶ In this paper, we study the problem of classification of fNIRS corresponding to different conditions of an $n$-back task (i.e., subjects are required to continuously remember the last $n\in \{\mathrm{1,2},3,\dots \}$ of rapidly changing letters or numbers). We performed fNIRS measurements on prefrontal cortex (PFC), which has been found to be a relevant area for memory-related tasks by positron emission tomography and functional magnetic resonance imaging.^7,8 Most $n$-back classification studies in the literature are based on supervised methods on fNIRS signals in within-session and within-subject basis (i.e., within single trial of data acquisition on a single subject).^9–11 While those studies showed promising results, subject- and session-dependent systems are not realistic for an interface system that can adapt to different users with a wide range of physiological conditions. With the aim of use in BCI, workload classifications based on fNIRS data across experiment sessions (session-by-session alignment) and across subjects (subject-by-subject alignment) are necessary.

There are several challenges that hamper accurate workload classification using fNIRS data. We outline them below and propose methods to mitigate them.

The first challenge, which is the main focus of this paper, is to deal with session-by-session and subject-by-subject variations in classification of $n$-back tasks. These problems are related to what is referred to as domain adaptation in machine learning.^12–14 More specifically, data from different sessions or different subjects are referred to as belonging to different domains, and the changes in data distributions across different domains (the session or subject that the data belongs to) are considered as a domain shift.¹⁵ Due to this phenomenon, the knowledge we learned from one domain cannot be applied directly to another one. To address this problem, recent advances in the theory and methods of optimal transport (OT)¹⁶ and metric measure space alignment^17–19 could be used to align data with a known labeled $n$-back condition from one session or one subject to the unlabeled data from a different session within the same subject or from a different subject. Though OT has been applied for domain adaptation with potential performance,^20,21 it has some limitations when two sets of data used for alignment do not share the same metric space in which case a meaningful notion of distance between two spaces does not exist. For example, for session-by-session alignment, data from some of the fNIRS channels are removed from one of the two sessions due to a poor signal-to-noise ratio (SNR). This will cause two sessions’ data to be embedded in different dimensions in the two domains. A naïve solution is to remove the corresponding channels from the other session to guarantee that the two sessions have the same dimension. However, this has the disadvantage of causing loss of information. In this paper, we proposed that using Gromov–Wasserstein (G-W)^18,22 and fused Gromov–Wasserstein (FG-W) barycenter²³ would alleviate this problem and provide algorithms to align across domains for fNIRS $n$-back task classification.

The second challenge is motion artifacts in fNIRS signals. Motion artifacts in fNIRS are commonly due to the coupling changes of any source or detector from the scalp during the experiment. This causes sudden increases or decreases in measured light intensity and can affect the measured fNIRS signals. From a machine learning perspective, motion artifact detection and correction help remove any misleading correlation from the subject behavior (twitching, head movement, etc.) to what the classification model learns from fNIRS data. For example, a classification model may recognize when a subject presses a button as a requirement during the experiment by detecting spikes in the measured signals due to the subject’s head movement, instead of detecting real hemodynamic responses from the brain signals. A number of approaches, inspired by statistical signal processing methods such as adaptive filtering, independent component analysis (ICA), and time-frequency analysis, have been proposed to remove or correct for motion artifacts in fNIRS signals.^24–30 Most of these techniques either depend on the use of auxiliary reference signals (e.g., accelerometry, etc.) or extraoptical channels or require certain assumptions on the characteristics of motion artifacts and cleaned fNIRS signals. In this paper, we used an off-the-shelf method based on sparse optimization for automatic detection and removal of spikes and steps anomalies, namely transient artifact reduction algorithm (TARA).³¹ We will apply the method TARA in the hope of improving the classification accuracy of $n$-back tasks.

The main contributions of this paper to the classification of different $n$-back task conditions include: (1) applying G-W to align fNIRS data during each $n$-back task condition across different experimental sessions for every single subject (session-by-session alignment); (2) applying FG-W barycenter to align fNIRS data during each $n$-back condition between different subjects (subject-by-subject alignment); and (3) demonstrating that alignment accuracy could be improved by applying motion artifact removal with TARA as a preprocessing step on fNIRS data.

2.

Experiment

2.1.

Subjects and Experiment Design

Six healthy human subjects (one female, five males; age range: 23 to 54 years) participated in this study. The Tufts University Institutional Review Board approved the experimental protocol, and the subjects provided written informed consent prior to the experiment.

During the $n$-back task, subjects were instructed to watch a series of rapidly flashing random one-digit numbers (from 0 to 9) shown on a computer screen placed $\sim 50\mathrm{cm}$ in front of the subject. Subjects must continuously remember the last $n$ numbers ($n=0$, 1, 2, and 3) and were asked to press the space bar if the currently displayed number (target) matched the preceding $n$’th number. In the 0-back task, the subject pressed the space bar whenever numeral “0” appeared. With increasing $n$, the task difficulty is expected to increase, as the subjects must remember an increasing number of preceding digits and continuously shift the remembered sequence. The experiment was designed such that the targets appeared with 25% to 35% chance (i.e., 65% to 75% nontargets) in each task (chosen randomly). We measured the task performance by counting the number of missed targets (when the subject did not press the space bar for a target) and the number of wrong reactions (when the subject incorrectly identified a nontarget stimulus as a target).

Each subject performed a total of four separate experiment sessions in two days: two sessions per day, one in the morning shift (9 to 12 a.m.) and one in the afternoon shift (1 to 4 p.m.). The order of the $n$-back tasks was randomized among sessions, but the randomization order was kept the same among subjects (i.e., only four random sequences were used and each subject was shown each of the four after all of their sessions). A session started with 155 s of initial baseline with a countdown timer displayed on the screen. At the beginning of a task, an instruction was shown to inform the subject that the upcoming task was 0-, 1-, 2-, or 3-back. A task consisted of 100 displayed digits each lasting 2 s, during which the stimulus was displayed for 1.5 s and followed by a resting time of 0.5 s where a black screen was shown. Therefore, each task was a total of 200 s in length. Subsequently, the subject entered 30 s of baseline (rest) after finishing the task while the performance accuracy of the preceding task was displayed on the screen. This process was repeated for the four values of $n$. At the end of the fourth task, the subject rested for a 155-s baseline after which the experiment was completed. Figure 1(a) shows the experiment protocol. The entire experiment had a recording time of 20 min (four 200-s tasks, two 155-s baselines, and three 30-s rests in the middle).

Fig. 1

(a) Experimental design for $n$-back tasks. (b) fNIRS headset with eight sources and seven detectors to give a total of 20 channels at source–detector distance of 3 cm. (c) An enlarged view of the schematic in (b) showing positions of the 10-10 system (Fp1, Fpz, Fp2, AF7, AF3, AFz, AF4, AF8, F5, F3, F1, Fz, F2, F4, and F6) and the 10-5 system (AFp3, AFp4, AF5h, AFF1h, AFF2h, and AFF6h) covered by the sources and detectors.

2.3.

fNIRS Data Preprocessing by TARA

Measured fNIRS data were checked manually to remove those noisy channels contaminated by high frequency noise ($>1\mathrm{Hz}$). Examples of removed and retained channels from two subjects are shown in Fig. 2, and the numbers of remaining channels are reported in Table 4 in the Appendix. The whole session will be removed if more than 60% of channels are identified as noisy. To further remove motion artifacts from the retained channels, we used the TARA algorithm³¹ in which measured time series data are treated as a linear combination of a low-pass signal, motion artifacts, and white noise. The algorithm focuses on two types of motion artifacts: transient pulses (spike-like signals) and step discontinuities, and assumes both of them appear infrequently. A sparse optimization problem is then formulated to jointly estimate two types of motion artifacts. We refer the reader to Ref. 31 for more details. We used the code provided by the authors³⁴ and chose parameters for our fNIRS data as shown in Table 5 in the Appendix. Once the motion artifacts are detected, they can be removed from the original signal to obtain the cleaned data.

Fig. 2

Examples of removed and retained channels from two subjects (2 and 3). The first column shows the removed channels, the second column shows the retained channels. Time courses are shown for concentration changes in oxy-(${\mathrm{\Delta }[\mathrm{HbO}}_2]$, shown in orange) and deoxy-hemoglobin ($\mathrm{\Delta }[\mathrm{Hb}]$, shown in blue).

3.

Domain Adaptation for fNIRS

After the removal of the channels with poor SNR and motion artifacts, a small time duration $w$ is chosen as the window size to divide the remaining $n$-back data ($\mathrm{\Delta }[{\mathrm{HbO}}_2]$ and $\mathrm{\Delta }[\mathrm{Hb}]$) into $N$ nonoverlapping small segments. Here, we use $w=60$ samples ($\sim 8\mathrm{s}$). To concretely describe the proposed method, next we will set some notations that are used throughout the paper.

Notation: We will use lower-case boldface letters $\mathbf{x}$ to denote vectors and upper case bold-face letters $\mathbf{X}$ to denote matrices. Unless otherwise stated, unbolded lower case letters denote scalars. ${\{({\mathbf{X}}_{m,i}^s,y_{m,i}^s)\}}_{i=1}^N$ stands for the collection of segmented data set of subject $s$ in its $m_{th}$ session, where $N$ is the number of segments, integer $s\in [\mathrm{1,6}]$, and integer $m\in [\mathrm{1,4}]$. The $i$’th segment is denoted as ${\mathbf{X}}_{m,i}^s\in {\mathbb{R}}^{d\times w}$, where $d$ is the number of channels and $w$ is the window length. $y_{m,i}^s\in [\mathrm{0,3}]$ is the corresponding $n$-back task label for subject $s$ in session $m$ and segment $i$, ${\mathbf{y}}_m^s=\mathbf{vec}(y_{m,i}^s)$ is an $N$-dimensional vector of the label. The remaining notation will be introduced as needed.

4.

Results

4.1.

Subject Performance

Figure 3 shows summary of subject performance analysis with the average percentages of wrong and missed responses, respectively, across four sessions and six subjects for each $n$-back task condition. The difficulty level, in terms of the amounts of wrong and missed responses, increases significantly for the 3-back task as compared to other $n$-back tasks ($p<0.05$, paired-sample $t$-test). Next, the numbers of wrong and missed responses for the 2-back task in the four experiment sessions are significantly higher than those for the 1- and the 0-back tasks ($p<0.05$, paired-sample $t$-test). Finally, there was no significant difference in the difficulty level between the 0- and the 1-back task in terms of wrong and missed responses ($p>0.05$, paired-sample $t$-test).

Fig. 3

Summary of subject performance for the $n$-back task: average percentages of (a) wrong responses and (b) missed responses for $n$-back task conditions across subjects. Bars represent the means, and error bars represent standard errors across four experimental sessions.

4.2.

Peripheral Physiological Measurements

Figure 4 shows the examples of average time courses of changes in MAP and HR from three subjects (1, 2, and 4) across different measurement sessions of the 2-back task. We observe a greater variability in task-evoked changes in HR and MAP across subjects than across sessions. In particular, subjects 1 and 2 show negligible changes in MAP and HR during the task with respect to the initial baseline, with individual measurements from different sessions following the same trend. On the other hand, all the measurements from different sessions from subject 4 show totally different responses as compared to subjects 1 and 2. For subject 4, MAP increases during the middle of the task and then returns to the baseline in the last minute. The HR measurements from this subject feature an initial increase and immediate decrease at the onset of the task.

Fig. 4

Average changes in HR and MAP across all sections of 2-back task for three subjects (1, 2, and 4). The time traces are shown starting from 30 s before the task. Black dotted lines indicate time $t=0$ s. Solid green lines are the averages across sessions in $\mathrm{\Delta }\mathrm{MAP}$ and $\mathrm{\Delta }\mathrm{HR}$; standard errors of these averages are shown by the cyan shaded regions; solid gray lines depict the individual measurements.

4.3.

Effects of Motion Artifact Removal Using TARA

Figure 5 shows the effects of TARA in removing motion artifacts in the fNIRS signals ($\mathrm{\Delta }[\mathrm{Hb}]$). As shown in the figure, the original signal is contaminated by the motion artifacts with spikes and steps. After applying the TARA algorithm, most of the motion artifacts have been removed. As compared to applying a low-pass filter to the original signal, TARA does not bring any further distortion to the cleaned signal and is more effective at removing step artifacts. The effect of this motion artifacts’ removal as a preprocessing step before applying alignment algorithms is also shown in Table 1 and Fig. 8. An improvement for session-by-session alignment accuracy (by an average of $3\pm 3\%$ across six subjects; $p<0.005$, paired-sample $t$-test) and subject-by-subject alignment accuracy (by an average of $5\pm 2\%$ across six subjects; $p<0.0005$, paired-sample $t$-test) can be seen after applying TARA on fNIRS signals.

Fig. 5

Effect of using TARA on $\mathrm{\Delta }[\mathrm{Hb}]$ signal. (a) The original and cleaned signals. (b) The detected motion artifacts, including spike- and step-like features. The cleaned signal is obtained by subtracting these motion artifact features from the original signal. (c) The low-pass filtered signal of the original data. Distortion in the signal arising from the step discontinuity can still be observed from the low-pass filtered signal.

Table 1

Average session-by-session and subject-by-subject alignment accuracy (%) using G-W and FG-W, respectively, as compared with SVM, CNN, and RNN. G-W and FG-W barycenter alignment methods applied to both original data (org) and data cleaned by TARA algorithm, using data including and excluding data from subject 3. For other methods, only cleaned data were used as input. Averages and standard errors across all subjects are reported (Avg.).

		Sub 1	Sub 2	Sub 3	Sub 4	Sub 5	Sub 6	Avg.
Sess-by-sess	SVM	25	31	34	38	23	20	$29\pm 3$**
	CNN	$60\pm 7$	$37\pm 11$	$40\pm 12$	$58\pm 12$	$70\pm 7$	$52\pm 6$	$56\pm 4$**
	RNN	$68\pm 10$	$40\pm 11$	$43\pm 13$	$61\pm 14$	$70\pm 8$	$53\pm 11$	$58\pm 5$*
	G-W org	$54\pm 17$	$58\pm 12$	$48\pm 23$	$76\pm 12$	$75\pm 7$	$49\pm 26$	$68\pm 4$*
	G-W	$54\pm 8$	$63\pm 16$	$63\pm 21$	$76\pm 12$	$77\pm 6$	$62\pm 15$	$68\pm 4$
Sub-by-sub	SVM	25	39	30	36	47	43	$37\pm 3$**
	CNN	$45\pm 8$	$45\pm 10$	$50\pm 10$	$36\pm 10$	$38\pm 10$	$47\pm 11$	$44\pm 5$**
	RNN	$42\pm 6$	$49\pm 9$	$50\pm 10$	$41\pm 11$	$37\pm 9$	$49\pm 10$	$44\pm 2$**
	FG-W org	$51\pm 12$	$57\pm 10$	$48\pm 6$	$60\pm 18$	$57\pm 10$	$60\pm 16$	$53\pm 2$**
	FG-W	$55\pm 12$	$71\pm 16$	$50\pm 4$	$68\pm 18$	$67\pm 9$	$67\pm 16$	$55\pm 2$
	FG-W w/o Sub 3	$60\pm 3$	$77\pm 9$	N/A	$75\pm 9$	$69\pm 7$	$74\pm 9$	$64\pm 3$

^*p<0.005 compared to G-W or FG-W.

^**p<0.0005 compared to G-W or FG-W.

4.4.

Session-by-Session Alignment

A low-dimensional UMAP visualization of the alignment for two sessions’ data is shown in Fig. 6 for subject 4. In Fig. 6, the low-dimensional projection was generated individually from the distance matrix of each session’s data. Therefore, the positions of the two groups of sessions’ data are assigned randomly and their relative distances are not their true distances.

Fig. 6

Visualization of the alignment from session 1 to session 2 for subject 4. Circles indicate data from session 1 and triangles indicate the data from session 2. Four different colors represent 0- to 3-back experiments. Black lines indicate correct alignment and red lines indicate misalignment.

Figure 7 shows the confusion matrices of session-by-session alignment for four $n$-back tasks (0, 1, 2, and 3) of six subjects. Numbers reported in the confusion matrix are the average alignment accuracies of all the possible combinations of two out of all four sessions for each subject. Values in the main diagonal of each confusion matrix represent correct alignment between predicted and true labels, while the other values represent the misalignment results. Correct alignment results are significantly greater than chance level of 25% ($p<0.0001$, one-sample $t$-test). The averages and standard deviations of session-by-session alignment accuracy for six subjects are summarized in Table 1 and Fig. 8. For each subject, the value reported is calculated based on the alignment or prediction accuracy for all possible combination of session pairs. As compared to SVM, CNN, and RNN, alignment accuracy of G-W is greater by an average of $43\%\pm 5\%$, $7\%\pm 4\%$, and $5\%\pm 5\%$, respectively ($p<0.005$, paired-sample $t$-test).

Fig. 7

Confusion matrices of session-by-session and subject-by-subject alignments in six subjects. (a) Session-by-session alignment accuracy within each subject. Each number reported in each confusion matrix is the average accuracy from the alignment of every two separate sessions among four sessions. (b) Subject-by-subject alignment accuracy. Each number reported is the average accuracy from the alignment between one source subject to the other five target subjects.

Fig. 8

Average alignment accuracy (%) from six subjects. (a) Session-by-session alignment, values are shown for SVM, CNN, and RNN using cleaned data from TARA, G-W using original data, and cleaned data from TARA. (b) For subject-by-subject alignment, values are shown for SVM, CNN, and RNN using cleaned data from TARA, FG-W using original and cleaned data from TARA, and FG-W using cleaned data from TARA when data from subject 3 is excluded. Bars represent the means, and error bars represent standard errors.

5.

Discussion

In this study of six subjects, we showed that fNIRS signals measured from 20 channels on the PFC can be used to robustly discriminate subjects’ mental workload between different $n$-back task levels across sessions within one subject and across different subjects. One limitation of our study is a small number of subjects. However, with the current number of subjects (six subjects), this paper still achieved the goals of demonstrating: (1) an alignment accuracy greater than that of chance (25%) for the majority of session-session and subject-subject combinations; and (2) greater accuracies than that obtained from multiclass SVM-, CNN-, and RNN-based models. We thereby showed the potential of fNIRS as a modality for BCI and user state monitoring that can adapt to different users with various physiological states. Future works will address the extension of this study with a larger sample of subjects to further investigate the variability between sessions and subjects.

In regards to data preprocessing, we show that motion artifact removal in fNIRS signals is an important step for the following mental workload alignment. Specifically, we report that using TARA to remove motion artifacts from fNIRS signals increased alignment accuracy by an average of $3\%\pm 3\%$ for session-by-session alignment and by $5\%\pm 2\%$ for subject-by-subject alignment ($p<0.005$). Future work could include addressing different types of artifacts that could arise in fNIRS time series, which were not considered by TARA, such as oscillatory transients. In addition, possible future improvements in TARA may be to investigate an automatic way for selection of regularization and nonconvexity parameters in TARA algorithm across subjects.

We introduced two approaches, G-W and FG-W barycenter, for session-by-session and subject-by-subject alignment of mental workload during $n$-back task. We proved that our methods could be generalized across different sessions and subjects’ data. In particular, for session-by-session alignment, we used labeled fNIRS data with known $n$-back task conditions from one session to align with other unlabeled sessions from the same subject using the G-W method. We showed that most of the unlabeled sessions’ data could be mapped correctly to their true labels, with the alignment accuracy ranging from 54% to 77% (with 25% representing chance alignment). Meanwhile, with multi-class SVM and simplified CNN and RNN models, the $n$-back task classification accuracy was lower (by an average of $43\%\pm 5\%$, $7\%\pm 4\%$, and $5\%\pm 5\%$, respectively). Note that CNN and RNN required the same amount of data as the proposed methods for training, while SVM required more data (from more than one session) for training. Similarly, for subject-by-subject alignment, we used labeled data from one subject as the source data for alignment. Labels and structural information of the source data were combined to generate a new representation (i.e. the FG-W barycenter). Following the same routine as the session-by-session alignment, we were able to use the barycenter from the source subject to predict the labels for data from different sessions for other subjects, with the alignment accuracy from 50% to 71% (also with 25% representing chance alignment). From the corresponding SVM, RNN, and CNN methods, $n$-back task classification performance achieved lower accuracy than the FG-W method (by an average of 15% to 22%). Again, CNN and RNN were trained from data from one subject (source data), while SVM was trained from data from five subjects for classification. Moreover, our methods of G-W and FG-W do not require the two subsets of data used for alignment to have the same dimension. Thus, they do not require data interpolation due to removing noisy fNIRS signals as for CNN and RNN methods. However, we note that even though G-W and FG-W methods are free from the dimension requirement for data, they could not achieve satisfying results when a large amount of data is missing (e.g., in the case of subject 3 when around half of the channels were discarded in the preprocessing step).

We found relatively higher alignment results for session-by-session alignment (average of $68\%\pm 4\%$) than subject-by-subject alignment (average of $55\%\pm 2\%$). One source of variation in fNIRS data across experiment sessions and across different subjects could come from the variability in systemic physiology, as seen in the variability in the task-evoked changes in MAP and HR (see Sec. 4.2). We observed that the variability of these two physiological measurements are larger across subjects than across sessions. This may explain a greater accuracy results for session-by-session alignment than subject-by-subject alignment. Another source of variability across sessions and subjects may also come from the variation in fNIRS optode placement on the subject’s head. We anticipate the optode placement variation to be greater across subjects than across sessions due to different head geometry from different human subjects. From our results, we found that the new representation of the barycenter of the source subject still aligned well to data from other subjects even though subject-by-subject alignment was a more challenging problem. This is indicative that representations of different subjects may still share similar underlying structures even from different domains. Future work will explore generating barycenter from source data from multiple subjects’ information for subject-by-subject alignment to account for the across-subject variations in the barycenter.

Based on our alignment results shown in confusion matrices in Fig. 7, the misalignment in session-by-session and subject-by-subject alignments are relatively high between 0-back and 1-back, and between 2-back and 3-back tasks. In particular, the misalignment is the highest between 2- and 3-back tasks (when the 2-back task is the true label and the 3-back is the predicted label and vice versa), ranging from 19.8% to 43.5%. The second highest misalignment is between 0- and 1-back tasks, ranging from 6.8% to 31.8%. Similarly, for subject-by-subject alignment, the highest misalignment came from 0- and 1-back tasks, ranging from 15.2% to 46.5%. The second highest misalignment is between 2- and 3-back tasks, ranging from 14.2% to 38.9%. This gave us an idea of combining 0- with 1-back tasks, and 2- with 3-back tasks in the alignment. Substantial increases in alignment performance (by an average of $22\pm 2\%$ for session-by-session and $33\pm 3\%$ for subject-by-subject alignment) suggests that future works could study workload classification between rest to low workload level (0- and 1-back tasks) versus high workload level (2- and 3-back tasks).

Finally, single-distance cw fNIRS measurements of intensity from source-detector pairs at 3-cm distance were used in this study. These measurements have been known to be more sensitive to hemodynamic changes in superficial tissues (i.e., scalp and skull) than in the brain.⁵¹ Previous study⁵² has shown that tasked-evoked superficial artifacts may arise during brain activation task due to systemic changes in peripheral physiology rather than the cerebral hemodynamics. This also confirms the claim that variations in our alignment results across sessions and subjects could be partially due to variability in systemic physiological origins. For the purpose of our aim, it is desirable to increase the sensitivity of our measurements to brain tissue to probe hemodynamic changes associated with brain activation. One approach, namely the dual-slope method, involves a simple implementation of a certain arrangement of sources and detectors to localize sensitivity of NIRS measurements to a deeper region,⁵³ thus suppressing confounding signals from superficial tissue. This approach could also help remove instrumental drifts and motion artifacts from measured signals as dual-slopes are unaffected by changes in optical coupling. Future extensions of this work may involve implementing the dual-slope configuration in such experiments as those described here. Another approach to correct for extracerebral contamination is to acquire measurements in a multidistance arrangement to incorporate short ($<1\mathrm{cm}$, sensitive to extracerebral tissue only) and long ($>2.5\mathrm{cm}$, sensitive to both extracerebral and brain tissues) source–detector separations⁵⁴ and apply a processing method such as adaptive filtering⁵⁵ to remove global interference from systemic physiology from fNIRS measurements.

6.

Conclusions

To illustrate that fNIRS signals can be effectively used to identify subjects’ mental workload between different $n$-back task levels across different sessions and subjects, we proposed two domain adaptation methods, G-W and FG-W, for session-by-session and subject-by-subject alignments, respectively. The proposed methods can achieve alignment accuracies greater than the chance level of 25%. At the same time, the proposed methods do not require the same subset of fNIRS channels or further data interpolation for classification across all subjects and sessions as opposed to some other supervised methods such as CNN and RNN. This will alleviate the pressure from having to exclude fNIRS channels that were noisy in one session but not in others, or from having to interpolate the signals to replace those noisy channels. Besides adapting the domain adaptation method, we explored the effect of using the TARA signal processing algorithm for removing motion artifacts and found an improvement in the alignment accuracy results. In the future, we plan to explore the effect of our method on a larger sample of subjects and make it applicable for multiple source subjects.

7.

Appendix

Tables 3, 4, and 5 specify the structure of CNN, the number of retained channels for all subjects and the parameters used in TARA for motion artifact removal, respectively. “∗” stands for multiplication operator and ReLU means the rectified linear activation function.

Table 3

CNN architecture, where d = number of channels (20 in our case), w = number of time points (60 in our case), T1,T2 = length of time points after applying the filter and C = number of classes (four in our case).

Table 4

Number of retained channels for six subjects. The total number of channels is 20. “0” indicates when the particular session is removed.

Table 5

Values of TARA parameters (fc, cut-off frequency for the low-pass filter; d, order of the filter; θ and β, regularization parameters for TARA; σ, noise standard deviation). Parameter values were chosen differently for Δ[HbO2] and Δ[Hb] due to different noise level. For each subject, values of σ and the choice of β vary among sessions, as reported in square brackets “[]”. “-” indicates when TARA is not applied or when the session is removed.