Translator Disclaimer
22 October 2019 Co-localization of theta-band activity and hemodynamic responses during face perception: simultaneous electroencephalography and functional near-infrared spectroscopy recordings
Author Affiliations +
Abstract

Face-specific neural processes in the human brain have been localized to multiple anatomical structures and associated with diverse and dynamic social functions. The question of how various face-related systems and functions may be bound together remains an active area of investigation. We hypothesize that face processing may be associated with specific frequency band oscillations that serve to integrate distributed face processing systems. Using a multimodal imaging approach, including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), simultaneous signals were acquired during face and object picture viewing. As expected for face processing, hemodynamic activity in the right occipital face area (OFA) increased during face viewing compared to object viewing, and in a subset of participants, the expected N170 EEG response was observed for faces. Based on recently reported associations between the theta band and visual processing, we hypothesized that increased hemodynamic activity in a face processing area would also be associated with greater theta-band activity originating in the same area. Consistent with our hypothesis, theta-band oscillations were also localized to the right OFA for faces, whereas alpha- and beta-band oscillations were not. Together, these findings suggest that theta-band oscillations originating in the OFA may be part of the distributed face-specific processing mechanism.

1.

Introduction

The human face is a universally salient social stimulus that conveys uniquely valuable information to a viewer with the potential to influence the viewer’s actions, thoughts, and emotions. Many neuroimaging studies document that human neural systems differentiate faces from other visual stimuli. In particular, the fusiform face area (FFA) in the medial temporal lobe and the occipital face area (OFA) show increased hemodynamic responses to faces compared to objects and are thought to be part of a larger distributed network of face-specific areas.1,2 There is evidence that these face-selective areas tend to be right-lateralized.36 The OFA, while less well-studied than the FFA, is selective for faces and has been shown to be necessary for face perception.79 For example, lesions to the OFA are associated with prosopagnosia even when the FFA is intact.10 There is also some evidence to support a hierarchical model of face perception, in which visual input travels from the primary visual cortex first to the OFA, and then on to the FFA and to the right superior temporal sulcus.1,8,11 If true, this model suggests that the OFA plays an important role in the early stages of face processing.

Electroencephalography (EEG) studies also confirm the salience of face stimuli; for example, faces generate an N170 response, a negative amplitude event-related response that occurs 170  ms after the visual presentation of a face.12 Although a negative peak can be generated by other visual stimuli, the N170 response occurs faster and the amplitude is greater in response to faces relative to other object stimuli. The N170 is generally right-lateralized in neurotypical individuals13 and correlates with measures of social function.14 Although many studies have provided evidence that the N170 is a specific neuronal response related to the visual processing of faces,1518 the effect of faces on oscillatory activity from groups of neurons has been less studied. We aimed to extend these well-documented findings of the face-specific N170 response by investigating frequency band oscillations that co-localize to face-specific areas during face perception.

In addition to event-related responses, EEG signals include specific frequency bands, which are thought to represent oscillatory activity of the neurons. In general, synchronization of different frequency bands has been shown to be related to attention-switching, memory, and other cognitive functions.1923 In visual tasks, theta oscillations (4 to 8 Hz) have been associated with stimulus perception24 and stimuli with high emotional valence.25 Theta-band activity is often associated with memory processes, and theta oscillations originating in the hippocampus may enable retention during memory tasks.26,27 Additionally, theta power increases have been associated with memory in both visuospatial and verbal tasks.28 Theta oscillations show power increases during passive visual tasks29 and may play a role in error processing.30 In visual perception, theta oscillations are associated with basic features of the visual cue.31

In addition to their role as salient visual stimuli, faces also trigger emotional and motivational systems. Theta-band responses to faces have been associated with these higher-order features. For example, in face perception tasks, increases in theta power in frontal electrodes were associated with viewing familiar faces.32,33 Event-related desynchronization in the theta band in anterior electrodes has also been associated with emotional facial expressions, and this effect has been shown to be right-lateralized.34 Theta-band synchronization in the right parietal cortex and insula increases during implicit perception of angry and happy faces, whereas it localizes to the temporal lobe and bilateral prefrontal cortices during explicit perception.35 The same study found that emotional faces led to greater theta-band synchronization and individuals with higher sensitivity to emotional stimuli showed greater synchronization. Fearful faces have also been shown to increase theta-band power and increase the amplitude of the N170 effect.36

Synchronizations in theta activity have been used as a measure of functional connectivity between frontal areas in oddball studies.21 Few studies have investigated the source localization of theta-band activity but task-dependent sources in the frontal, parietal, and temporal lobes have been reported. One study examining theta-band activity (6.5 to 8 Hz) during rest in patients with epilepsy localized activity to the medial frontal and parietal lobes, specifically the precuneus, anterior and posterior cingulate, and medial frontal gyrus.37 The varied sources of theta-power increases have led Başar et al.29 to propose that theta oscillations may drive functional connections between a widely distributed system of areas.

In contrast, alpha oscillations have been associated almost exclusively with attention mechanisms. Decreased alpha power during higher cognitive tasks and increased beta oscillations (13 to 30 Hz) during sensorimotor tasks have been shown in experiments using multimodal stimuli.38 The low-frequency delta band (0.3 to 4 Hz) has been associated with salience detection of visual stimuli,39 although evidence for its involvement specifically in face perception has been mixed.38 Due to the association of the theta band with visual stimulation and the distributed, task-dependent nature of the anatomical sources of theta-band activity reported in the literature, we hypothesized that we would likely observe a spatially localized increase in theta-band oscillations in the OFA during a face perception task. We predicted that this localized increase in oscillatory activity would not be seen in the alpha and beta bands.

Simultaneous EEG measures of oscillatory activity and BOLD-related measures of face processing are not common practice due to the signal artifacts as a result of the rapid gradient switching in fMRI studies. However, functional near-infrared spectroscopy (fNIRS) acquires hemodynamic signals absent the high magnetic field. fNIRS is a neuroimaging method that uses absorption of near-infrared light to measure oxyhemoglobin (OxyHb) and deoxyhemoglobin (deOxyHb) concentrations in the cortex as a hemodynamic proxy for neural activity.4042 Until recently, fNIRS has mainly been used with infants, for whom fMRI studies are not an option. In the last decade, advances in signal processing, increased head coverage, and enhanced computational methods have made fNIRS a viable alternative to fMRI for investigating adult perception and cognition, particularly in natural conditions. Recent developments in the acquisition, processing, and 3-D representation of fNIRS signals allow for the presentation of hemodynamic signals that can be compared easily with those acquired using more traditional fMRI methods.43,44 EEG-fNIRS experiments are well-suited to acquire electrocortical and hemodynamic signals related to faces.

One goal of acquiring simultaneous EEG and fNIRS measurements includes the ability to relate the fast electrical signals of EEG with the slow, hemodynamic signals of fNIRS. One such study accomplished this by correlating the raw signals and identifying electrodes and fNIRS channel locations, in which this correlation was high.45 Other studies have correlated EEG signals with hemodynamic responses.4648 In epilepsy, aberrant EEG signals have been correlated with hemodynamic changes in areas associated with default mode activity.49 Theta-band activity, in particular, has been correlated with increased hemodynamic activity in task-positive networks as well as decreased activity in the default mode network with increasing cognitive demand.50,51 These findings bring us closer to a much-needed understanding of the relationship between the sources of these signals, and face perception serves as a case in which these two well-defined responses can be further related.

For this study, simultaneous EEG and fNIRS recordings were collected in order to compare the spatial localization of hemodynamic activity with band-specific electrocortical activity during perception of faces compared to objects. The area of interest for this study was the right OFA. This region is located posteriorly and laterally relative to the FFA, making activity from this area easily accessible using fNIRS. Source localizations associated with the alpha, beta, and theta bands of the EEG time course were also examined. Our aim is to relate spatial localizations of face processing based on both hemodynamic and electrocortical signals.

2.

Materials and Methods

2.1.

Participants

Twenty-two participants (mean age 27.6±6.6, 15 females, and 21 right-handed52) took part in the experiment. The mean laterality quotient from the Edinburgh Handedness Inventory was 68.8. All participants were previously screened using a right-handed finger-thumb tapping localizer task (see below). By self-report, participants had no known neurological or psychiatric conditions and had normal or corrected to normal vision. Participants provided written informed consent in accordance with guidelines approved by the Yale University Human Investigation Committee (HIC #1501015178) and were compensated for their participation. All data were obtained at Yale School of Medicine, New Haven, Connecticut.

2.2.

Screening Procedure

It has been well-documented that fNIRS signals are not obtainable from every individual, due to factors such as hair style, bone density, and skull thickness.5355 In order to ensure that we were able to obtain valid signals from each subject prior to their participation in the study, each participant underwent a screening using a finger-thumb tapping task. This ensured that any subsequent findings were based on viable signals. For this preliminary task, participants wore 80 optodes (40 emitters and 40 detectors) in the same 134-channel layout described below in Sec. 2.5. The numbers 1, 2, 3, or 4 appeared randomly on a screen and participants were asked to tap the corresponding finger depending on the number, with “1” for index finger, “2” for middle finger, “3” for ring finger, and “4” for pinky finger. The task was a block design, in which participants tapped 20 times in 15 s, followed by 15 s of rest, repeated 6 times for a total of 3 min. Hemodynamic data were analyzed using event-triggered averages over the six blocks, with each 30-s block constituting an event, and the amplitude of the response in each channel was used to generate a p value for that channel. If at least one fNIRS channel in the left motor cortex had a p value less than the threshold of p=0.001 for both oxyHb and deoxyHb signals, the participant was invited back at a later date to take part in the faces and object-viewing experiment described below. Based on a data set of 128 participants, we have found that 8% of subjects are excluded using this procedure.56

2.3.

Stimuli

Visual stimuli consisted of pictures of faces and objects obtained from standardized image sets. Face stimuli were all forward-facing images from the Face Place stimulus set (courtesy of Tarr, Center for the Neural Basis of Cognition and Department of Psychology, Carnegie Mellon University, funding provided by NSF Award No. 0339122).57 Object stimuli consisted of images of clothing, electronics, household items, and food acquired from the Bank of Standardized Stimuli set.58

2.4.

Paradigm

Each task block consisted of nine pictures of faces or objects, each of which was displayed on a monitor for 2 s (18 s total) (Fig. 1). The size of the presented images was the same for the faces and the objects. In each block, seven of the pictures were of one type, whereas two were of the other type (“oddball trials”). For example, the “face” blocks consisted of seven pictures of faces and two pictures of objects. The position of the two oddball trials in the block was random and was intended to reduce subject perceptions of a uniform block and possible adaptation effects. We have previously used this design in a Stroop task using gesture, in which task blocks included random foils of a different condition.59 The 18-s task blocks were interspersed with 15-s of rest during which the participant fixated on a crosshair. Each run consisted of three face blocks and three object blocks, which were randomized throughout the run. Each participant underwent two runs, with a short break between runs when the experimenter came in the room to check that the subject was still comfortable and wished to continue. Participants were instructed to focus on the center of the screen (visual angle of 0 deg). Each picture was presented once over the course of two runs.

Fig. 1

Experimental paradigm. One run consisted of six blocks. Each block contained two stimuli that were of a different category from the others in order to avoid the perception of a uniform stimulus block. Two runs were presented for a total of 54 faces and 54 objects. Individual face (F) and object (O) events were used for fNIRS and EEG ERP analyses, and blocks were used for frequency band source localization.

NPH_6_4_045002_f001.png

The experimental paradigm of blocks consisting of individual picture events was designed to optimize both the EEG and fNIRS analyses. Each face and object presentation was treated as an event in the event-related potential (ERP) EEG analysis and in the general linear model for the fNIRS analysis. Entire blocks were used in the frequency band source localization analyses, with “face blocks” consisting of mostly face pictures and “object blocks” consisting of mostly object pictures. Stimulus presentation, fNIRS signal acquisition, and EEG signal acquisition were synchronized using TTL triggers that were sent to all machines simultaneously.

2.5.

Functional Near-Infrared Spectroscopy Acquisition

Functional NIRS and EEG signals were acquired simultaneously using custom-made caps with interspersed optode and electrode holders [Fig. 2(a)]. The distance between optodes was 3 or 2.75 cm depending on the cap used to fit the participant’s head. Caps were placed such that the foremost optode holder was 1  cm above nasion and the backmost optode holder was on or below inion (see previous work for details).60 For a given participant, the cap that more closely fit these requirements was used. A Shimadzu LABNIRS system was used to collect fNIRS data at a sampling rate of 123 ms. Forty emitters and forty detectors (80 optodes total) were placed in the cap in a 134-channel layout covering frontal, parietal, temporal, and occipital lobes [Fig. 2(b)]. Each emitter transmitted three wavelengths of light, 780, 805, and 830 nm, and each detector measured the amount of light that was not absorbed. The amount of light absorbed by the blood was converted to concentrations of OxyHb and deOxyHb using a modified Beer–Lambert equation.61 A Patriot Polhemus digitizer (Polhemus, Colchester, Vermont) was used to record the position of EEG electrodes and fNIRS optodes as well as five anatomical locations (nasion, inion, Cz, left tragus, and right tragus) for each subject. See Supplemental Table S1 for median locations of each channel across the group with anatomical labels.

Fig. 2

(a) A participant wearing 32 EEG electrodes and 80 fNIRS optodes (obtained with permission). (b) Channel layout for one representative participant. Green dots represent the 134 fNIRS channels and red dots represent the 32 EEG electrodes.

NPH_6_4_045002_f002.png

2.6.

Electroencephalography Acquisition

A g.USBamp (g.tec medical engineering GmbH, Austria) system with two bioamplifiers and 32 electrodes was used to collect EEG data at a sampling rate of 256 Hz. Electrodes were arranged in a layout similar to the 10-10 system; however, exact positioning was limited by the location of the electrode holders, which were held rigid between the optode holders. Electrodes were placed as closely as possible to the following positions: Fp1, Fp2, AF3, AF4, F7, F3, Fz, F4, F8, PC5, PC1, PC2, PC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3, Pz, P4, P8, PO3, PO4, O1, Oz, and O2. Conducting gel was applied to each electrode to reduce resistance by ensuring contact between the electrodes and the scalp. As gel was applied, data were visualized using a bandpass filter to allow frequencies between 0.5 and 60 Hz. An ear clip was used as the reference electrode, and the ground electrode was placed on the forehead between AF3 and AF4.

2.7.

Data Analysis

2.7.1.

fNIRS analysis

The analysis methods used here have been described previously.59,62 Briefly, digitized optode locations were converted to Montreal Neurological Institute (MNI) coordinates using NIRS-SPM software,63 and channels from each participant were registered to group median channel locations. Both OxyHb and deOxyHb signals were analyzed, though studies have shown that deOxyHb signals, compared to OxyHb signals, show greater similarity with the blood oxygen level-dependent signal of fMRI40,61,64 and reduced susceptibility to systemic effects.6567 Signals were acquired every 0.123 s (8.13 Hz). Wavelet detrending was used to remove baseline drift using the algorithm provided by NIRS-SPM.63 Channels in which the root mean square of the raw signal was more than 10 times greater than the average signal were automatically identified and removed from the analysis. This process resulted in the exclusion of 7 channels total across all channels in all participants, i.e., 7 out of 2948 total channels were removed. A principal component analysis spatial filter was used to remove global components caused by non-neural factors, such as blood pressure, heart rate, and respiration.67,68 For the within-subject general linear model analysis, face and object neural events were convolved with the canonical hemodynamic response function and modeled to fit the data. Oddball stimuli were included in the relevant condition. For example, in the “face” condition, all face trials were modeled, including those that occurred during a predominantly “object” block. Linear interpolation was used to project the resulting beta values into MNI space based on each subject’s channel locations, and voxels that were >17 mm away from the surface of the brain were excluded. Second-level analysis was done using one-tailed t-tests in SPM8.

2.7.2.

Event-related potentials

EEG signals were preprocessed using EEGLAB v13.5.4b69 in MATLAB 2014a (Mathworks, Natick, Massachusetts), including high-pass filtering (1 Hz) and manual removal of channels, before being converted to a format compatible with SPM8 (Wellcome Trust, London, UK). Two types of channels exhibiting noise characteristic of poor contact with the scalp were rejected based on visual inspection: (1) signals whose amplitude exceeded ±100  μV and (2) signals that were completely flat with low-frequency drift. With these criteria, an average of 3 channels per person was removed and the signals from the surrounding channels were interpolated. Data were then divided into epochs from 100 to 700 ms relative to stimulus onset and averaged. As with the fNIRS analysis, oddball stimuli were also included in the average for the relevant condition. Event-triggered averages over the 10 parieto-occipital electrodes (P3, P4, P7, P8, PZ, PO3, PPO4, O1, O2, and Oz) for each participant were used to determine whether or not the data showed the N170 response.12,70 Subjects were considered to have an N170 response if a negative peak between 100 and 200 ms was observed after the onset of a face and preceded the negative peak for objects.12 Event-triggered average data from a single parietal electrode in a representative participant with an N170 response to faces is shown in Fig. 3 (inset) in contrast with event-triggered averaged hemodynamic responses from one channel in one representative participant for the face’s condition and for the object’s condition (Fig. 3).

Fig. 3

Comparison of time courses for the event-triggered average signals for the hemodynamic and EEG responses. The different time scales of the signals are shown using an inset. Representative event-triggered average hemodynamic signals from one participant in one channel (channel 134) for faces (oxyHb, solid red; deoxyHb, solid magenta) and for objects (oxyHb, blue dashed; deoxyHb, cyan dashed). Black dotted line marks end of 18-s task period. Solid vertical line marks the 600 ms time period of the inset EEG signal. Inset: representative event-triggered EEG response from one participant showing N170 response in the Pz electrode for faces (red) with a smaller negative peak for objects (blue) shifted to a longer latency. Dotted lines show the 100- to 200-ms time range during which the response was expected to occur.

NPH_6_4_045002_f003.png

2.7.3.

Frequency band source localization

Frequency band source localization was done for all 22 participants, whether or not they had shown an N170 ERP response to face stimuli. A “normal” mesh model for boundary element method (BEM) was generated using an MNI template, assuming 8196 current sources in the cortical layer. Co-registration to the template brain was based on the individually digitized coordinates for each of the 32 electrodes. A BEM was used to generate the forward model. The multiple sparse priors (MSP) algorithm in SPM871,72 was used in the inversion to identify the time course and spatial localization of the sources of the ERP response, without using source priors or restricting solutions. In the MSP algorithm, an empirical Bayes model is used to reconstruct the sources of the distributed electrical neural activity. This generated an image for each subject for faces and for objects. Group-level analysis was performed with one-tailed t-tests on the images generated by the difference between the face and object responses.

The source localization algorithm described above was used on the full time course, separated into task and rest blocks for faces and objects. The sources were localized for induced alpha band (8 to 13 Hz), beta band (13 to 30 Hz), and theta-band (4 to 8 Hz) oscillations during the 18-s task blocks and compared to the 15-s rest blocks. Frequency bands were obtained using wavelet decomposition function in SPM. Each block was labeled by block type based on the majority stimulus for that block, resulting in six events per run. For this analysis, oddball stimuli were not separated; i.e., if a block had mostly face stimuli, it was considered a face block despite having two object stimuli. For each frequency band, the analysis was run twice: once with the time range of the task block (0 to 18 s) and once with the time range of the rest blocks (18 to 33 s). This resulted in four images for each frequency band: the face task blocks (face block), object task blocks (object block), rest blocks following face blocks (face rest), and rest blocks following object blocks (object rest). For each subject, one-tailed t-tests were used to generate an image of the face block > face rest and object block > object rest contrasts. Group-level data were created from these images, and the contrast of face > object was generated using one-tailed t-tests in SPM8.

2.7.4.

Small volume analysis

Localization of activity in the right OFA was confirmed using Neurosynth.73 Neurosynth is an online meta-analysis tool that generates maps of activity based on previously published studies. An association image was generated using the search term “faces” from 864 studies in order to ensure that the image would encompass all face-specific areas. The cluster located in the right hemisphere and thresholded at p<0.05 was isolated. This cluster was then divided into two clusters: one in the fusiform gyrus consistent with the FFA [peak voxel at (42, 50, 24), n of voxels = 1154] and one in the occipital cortex consistent with the OFA [peak voxel at (44, 76, 14), n of voxels = 564]. A mask generated from the OFA cluster, located in lateral occipital cortex, was used as the volume of interest (VOI). The intersection of this OFA VOI and the whole-brain image of beta values generated from the first-level analysis was created for each subject. These individual subject images were combined using a second-level analysis in SPM, and the group result was rendered at a threshold of p<0.05. All images were rendered on a standardized brain using an MNI template.

3.

Results

All fNIRS and EEG frequency band source localization results presented are from analysis of all 22 participants.

3.1.

fNIRS Results

Event-triggered average plots are shown in Sec. 5, averaged across all participants for the faces > objects contrast, with OxyHb in red and deOxyHb in blue. For rendered images, deoxyHb signals are inverted so that overall hemodynamic activity is presented using the same color scheme in both oxyHb and deoxyHb figures. Yellow-red represents increases in hemodynamic activity, i.e., decreases in deOxyHb and increases in OxyHb concentrations. Using the deoxyHb signal, the small volume analysis revealed hemodynamic activity for the faces > objects contrast [p<0.05, Fig. 4(a)], with peak voxel at (32, 88, 16) (t=2.60, n of voxels = 52). According to the atlas,74 the label for this peak voxel was Broadmann Area 18, or visual association cortex (V2), consistent with the OFA. Using the oxyHb signal, the small volume analysis resulted in a cluster of activity with peak voxel at (32, 88, 16) (t=2.31, n of voxels = 55) [Fig. 4(b)]. As with the cluster found using the deoxyHb signal, the oxyHb cluster peak voxel was also in V2.

Fig. 4

Results of small volume analysis of activity localized to the right OFA (posterior view). Red represents increased activity for faces > objects contrast, p<0.05. Black circles indicate the right OFA VOI. Increased activity in this area is shown using the (a) deoxyHb activity, (b) oxyHb activity, and (c) theta-band source localization.

NPH_6_4_045002_f004.png

3.2.

ERP N170 Response

Ten subjects showed a negative peak between 100 and 200 ms after the onset of face stimuli in parieto-occipital electrodes that preceded a negative peak for object images. The event-triggered average for these ten subjects in these electrodes is shown in Fig. 5.

Fig. 5

Event-triggered average ERP responses in 10 posterior electrodes, averaged across the 10 participants exhibiting the N170 response. Graphs are organized in the rows that the electrodes were positioned on the head (left and right sides are labeled). Black dashed lines mark 100 and 200 ms after stimulus presentation. Red represents faces and blue represents objects. Shaded regions show standard deviation.

NPH_6_4_045002_f005.png

3.3.

Frequency Band Source Localization

Source localization of the theta-band oscillations for the face > object contrast revealed activity in the right OFA VOI [p<0.05, Fig. 4(c)], with peak voxel at (36, 70, 12) (t=2.84, n of voxels = 375). Localization of the alpha and beta band oscillations produced no positive activity in the VOI.

3.4.

Neurosynth Mask Overlap

To further confirm that all areas of increased activity were localized to the OFA, we report the overlap between the posterior, OFA cluster obtained from the Neurosynth “faces” map and the face > object contrasts using both fNIRS and EEG. When the overlap of voxels between the group results and the mask was calculated, 41.7% of the deoxyHb activity, 9.96% of the oxyHb activity, and 48.0% of the theta-band source localization overlapped with the Neurosynth “faces” mask. The right-lateralized OFA VOI overlaid with the fNIRS activity (both oxyHb and deoxyHb signals) and theta-band source localization results (all rendered at p<0.05) are shown in Fig. 6. Axial slices are shown using an MNI template in MRIcroGL.74

Fig. 6

Axial slices showing face > objects results overlaid on the OFA VOI from the neurosynth “faces” association map (yellow). Red indicates oxyHb activity, blue indicates deoxyHb activity, and cyan indicates theta-band source localization for this contrast at p<0.05. Purple reflects overlap of red (oxyHb) and blue (deOxyHb) activity. Green reflects overlap of cyan (theta band) and yellow (VOI). Lighter shades indicate areas of overlap.

NPH_6_4_045002_f006.png

4.

Discussion

Multimodal imaging approaches to simultaneously acquire fNIRS and EEG recordings have the potential to bridge the theoretical gap between fast neuronal responses and slower hemodynamic changes. Here, we apply the technical advantages of two integrated surface-mounted detectors, optodes for fNIRS signals and electrodes for EEG signals, to simultaneously acquire neural responses to faces across a time scale extending from milliseconds to seconds. In this study, a classical passive-viewing task was used to differentiate responses to faces and objects. EEG signals in the frequency of the theta band showed spatial localization to the right OFA during face presentation, whereas alpha and beta band activity was not localized to this area. Hemodynamic activity measured using fNIRS showed face-specific activity localized to the right OFA, consistent with prior fMRI findings. A subset of participants also demonstrated the classical N170 event-related EEG response to faces. This is the first study to our knowledge to use simultaneous electrocortical (EEG) and hemodynamic (fNIRS) signals to demonstrate the OFA as a common spatial source for hemodynamic activity and theta-band oscillations in face perception. Results are consistent with our hypothesis that theta oscillations localized to the OFA serve as a component of the face perception mechanism.

Prior studies using intracranial electrodes and fMRI have characterized the relationship between neuronal spike responses, local field potentials, and hemodynamic signals.75 Other simultaneous EEG/fMRI studies have also served to relate these two types of measures, often by correlating the two signals.4548 Importantly, here we do not directly relate the electrocortical and hemodynamic signals. Rather, we use source localization during a task with a known spatial response to show that both theta-band oscillations and hemodynamic responses originate in the same part of the brain during face processing. The OFA response to faces is well-established, suggesting that the similar theta-band oscillation co-localization to this area may be due to a common neuronal source for theta-band oscillations and hemodynamic signals. However, further research is necessary to directly relate to the two signals to confirm whether theta-band oscillations are responsive to faces and whether there is a direct relationship between these oscillations and the hemodynamic responses.

Although our findings show that theta oscillations and hemodynamic activity generated in response to face stimuli were localized to a common source, this was not observed with other electrocortical frequency bands. What could this common source for theta oscillations mean? One possibility is that theta oscillations serve as a signal that is sent out by the neurons in the OFA in preparation to interact with other areas of the brain. This is consistent with the integrated theta system proposed by Başar et al.,29 who suggested that the theta band originates in different parts of the brain in order to form functional connections between them during visual processing. Supporting this framework, a recent study showed that theta-band activity is correlated with the packaging of information about neuronal activity by layer II/III columnar cells, which provides short-range connectivity with nearby areas in the network.76 Face processing is a complex task that readies mechanisms for emotion processing, working memory, and communication.31,59,77 Although our study focused on the right OFA as a region of interest, face perception is known to involve a network of areas, including the FFA and the posterior superior temporal sulcus.1 This could suggest a bottom-up mechanism through which visual presentation of faces initiates theta oscillations of neurons in the OFA in preparation to form functional connections with other parts of the face processing network. Due to the passive task of the study, we do not measure the relationship between functional connectivity and theta oscillations. Future simultaneous EEG-fNIRS studies using more challenging cognitive tasks have the potential to investigate whether the theta band co-localizes with areas that are functionally connected to the OFA during cognitive tasks involving face perception.

Prior studies have also observed spatial co-localization for increases in theta-band oscillations and increases in hemodynamic activity. One recent study using simultaneous EEG and fMRI found that theta-band activity was negatively correlated with BOLD responses in parts of the default mode network during a memory task.50,78 As the cognitive demand of the task went up, BOLD responses in the default mode network decreased while the theta response band response localized to this area increased. In another study, increased cognitive demand in multiplication tasks resulted in greater fNIRS activity as well as greater theta-band power in frontal areas.51 Theta power has also been correlated with increased functional connectivity between parts of the default mode network during resting state,79 consistent with the idea that oscillations in the theta-band range may form the basis for functional connections between hemodynamically active areas.

Our findings, showing theta-band source localization to the same area as hemodynamic responses, may have been driven by the emotional and motivational response to faces, rather than lower level features of the face as a salient visual stimulus. In this study, neutral faces were used as the stimuli. It is possible that using angry, happy, or fearful faces would have increased the observed theta-band power or that the theta band may have localized to areas other than the OFA in accordance with prior studies.3436 If the emotional features of faces contributed to the observed results, different effects by gender may be expected.80 Due to limitations of sample size and the fact that the majority of our participants were female (15 out of 22 participants), we did not conduct an analysis of the effect of gender on the response. However, this is an interesting area for future research.

In our study, theta oscillations localized to the OFA and not the FFA, as shown by the group cluster of activity in Fig. 6. Haxby et al.1 have proposed that the OFA may serve as an early node in a hierarchical face-processing network and that information is relayed from the OFA to the FFA and the right superior temporal sulcus during visual face processing.8 Studies of the OFA using intracranial recordings and MEG source localization both provide support for this theory.11,81 In this study, using fNIRS and EEG, we record two temporally distinct processes, both driven by face-specific neuronal activity. Theta oscillations arise from the fast neuronal oscillatory activity, so theta localization to the OFA could be due to this area’s early role in the face perception process. It may be that theta-band oscillations originating in the OFA spread to other parts of the face-processing network, in accordance with Haxby’s hierarchical model. Alternatively, it could be that the spatial localization of the theta band in this study was limited due to the use of only 32 electrodes and that the use of more electrodes would have resulted in source localization to the FFA as well as the OFA. We further describe this limitation and interpretation below.

In contrast to the theta band, alpha and beta oscillations were not localized to the OFA during the face-viewing epochs. These bands may be less specific to face processing and might, therefore, not be upregulated during passive viewing. Alpha-band power is increased during rest and suppressed during visual perception and cognitive tasks.82 In vision studies, the anticipatory suppression of alpha oscillations is associated with attention and inhibitory control.83 Beta oscillations are associated with sensorimotor activity, and beta power may be driven by the salience of the stimuli.38 Although beta activity has been evoked in occipital electrodes during passive visual tasks, beta power is increased during integration of multimodal stimuli.84 Both alpha and beta desynchronization have been observed in response to emotional face stimuli; this is not detected for neutral stimuli, so information contained in these frequency bands may be related to emotion processing rather than neutral face perception.38 An alternative explanation may be that alpha and beta bands are more affected by high-frequency noise in the signal, preventing a localizable signal.

Although fusiform activity during face perception tasks is well-documented using fMRI, our results show that fNIRS can also be used to measure face-related activity from the OFA. Although this region does not overlap the well-described face-specific FFA, it also demonstrates preferential activity for faces.1,2,7,8 This is the first study to use simultaneous fNIRS and EEG in a face perception task, extending previous investigations of the hemodynamic and electrocortical effects of seeing a human face. The discovery that theta-band activity is co-localized to the same area as hemodynamic signals stimulated by faces suggests that theta-band oscillations may also be involved in face processing. Because the sources are task-specific and common to these temporally distinct processes, we speculate that the neural processes are linked. More generally, our findings generate hypotheses for future research about the possible relationship between theta-band source localization and hemodynamic activity during the processing of a face. We hypothesize that theta oscillations that originate in the OFA are transmitted to other face-specific areas and that this binding mechanism may lead to the recruitment of oxygenated blood to active areas in the face-processing network. The unmet need to understand the relationships between the two signals underlies the recent increased interest in creating viable EEG-fNIRS systems to collect electrocortical and hemodynamic data simultaneously. Future studies using simultaneous EEG-fNIRS are necessary to test the hypothesis that theta oscillations may relate to functional connectivity in the face processing network during cognitive tasks that involve face perception.

4.1.

Limitations

The penetration of functional NIRS is limited to the superficial gray matter, which restricts the cortical regions from which signals are acquired. This limits the brain regions sensitive to this technique, and findings of this study are restricted to fNIRS-compatible brain areas. Because of this, the area of interest for this study was the OFA, rather than the FFA.

Other limitations of the study arise from the simultaneous acquisition of both fNIRS and EEG signals. Custom-made caps with alternating electrode and optode holders in a grid pattern were used. Some electrodes were lifted away from the scalp when the optodes were placed. Although gel was used to bridge the space between the electrodes and the scalp, in some areas, this connection was not stable and the signals from some electrodes were unusable. Even electrodes from which data were included may have shown weaker signals than if the electrodes were flush against the scalp. Another limitation arose from the design of the experiment. Trials were presented in a mostly blocked design in order to maximize the opportunity to detect hemodynamic signals using fNIRS. This resulted in only 54 face trials and 54 object trials, whereas typically at least 100 trials are recommended to elicit early perceptual responses.85 The absence of interstimulus intervals (ISIs) between events may also have decreased sensitivity to evoked potentials. ISIs were not used in order in the design of the paradigm in order to maximize similarity to prior fMRI experiments. However, in the future, block designs incorporating ISIs may be the most efficient at eliciting both hemodynamic and event-related responses. These limitations, the suboptimal placement of electrodes, the relatively small number of picture presentations, and the constant interval between events, may explain our observation of an N170 response in only a subset of the participants. However, the main result reported here focused on the co-localization of the theta-band oscillations and the hemodynamic activity to the OFA specific to face perception. The lack of an observable N170 response was not considered an exclusion criterion for potential theta-band oscillations, therefore, all participants were included in the source localization analyses.

A final limitation of the study was the use of 32 electrodes for source localization analyses. It has been well-documented that a greater number of electrodes ensure greater spatial specificity of point-like sources.8688 One study has reported that the use of 32 or 64 electrodes results in the estimation of fewer sources in regions of interest (fewer true positives) as well as more sources outside the regions of interest (more false positives).89 In this study, we were interested specifically in the response to faces in face-selective areas such as the OFA. Because the source localization result was located in an area that we a priori expected to show activity in response to faces, we have more confidence that this activity is not a random false positive. Due to the trade-off between ease of experimental setup and spatial specificity, 32 electrodes have been used successfully for source localization in many studies.9092 However, the results of this and other studies that use this relatively small number of electrodes should be interpreted with caution.

5.

Appendix A: fNIRS Event-Triggered Average Plots

For visualization purposes, event-triggered average was performed in each channel on both the oxyhemoglobin and deoxyhemoglobin fNIRS data, for the Face-Objects contrast. Plots include activity averaged across all participants. Five channels that were within 3 cm of the peak voxel of the occipital face area volume-of-interest are shown circled in green.

Fig. 7

Event-triggered average plot of fNIRS signals for face-objects contrast, averaged across all 22 subjects. Red represents OxyHb, blue represents deOxyHb. Front, back, left, and right aspects of the head are labeled. Green circles highlight the subset of channels within 3 cm of the peak voxel of the OFA VOI in the majority of subjects.

NPH_6_4_045002_f007.png

Disclosures

The authors report no conflicts of interest.

Acknowledgments

This research was partially supported by the National Institute of Mental Health of the U.S. National Institutes of Health under Award Nos. R01MH107513 (PI JH), R01MH111629 (PI JH), R37 HD 090153, and F30MH116626 (PI SD); NIH Medical Scientist Training Program Training Grant No. T32GM007205; and the Japan Society for the Promotion of Science (JSPS) Grants in Aid for Scientific Research (KAKENHI) No. JP16K01520 (PI YO). The content is solely the responsibility of the authors and does not necessarily represent the official views of the U.S. National Institutes of Health. The data reported in this paper are available upon request.

References

1. 

J. V. Haxby, E. A. Hoffman and M. I. Gobbini, “The distributed human neural system for face perception,” Trends Cognit. Sci., 4 (6), 223 –233 (2000). https://doi.org/10.1016/S1364-6613(00)01482-0 Google Scholar

2. 

N. Kanwisher, J. McDermott and M. M. Chun, “The fusiform face area: a module in human extrastriate cortex specialized for face perception,” J. Neurosci., 17 (11), 4302 –4311 (1997). https://doi.org/10.1523/JNEUROSCI.17-11-04302.1997 JNRSDS 0270-6474 Google Scholar

3. 

H. A. Buchtel, “Left and right hemisphere contributions to physiognomic and verbal discrimination,” Neuropsychology, 15 (4), 597 –606 (2001). https://doi.org/10.1037/0894-4105.15.4.597 NEUPEG Google Scholar

4. 

E. De Renzi, “Prosopagnosia in two patients with CT scan evidence of damage confined to the right hemisphere,” Neuropsychologia, 24 (3), 385 –389 (1986). https://doi.org/10.1016/0028-3932(86)90023-0 NUPSA6 0028-3932 Google Scholar

5. 

R. D. Hilliard, “Hemispheric laterality effects on a facial recognition task in normal subjects,” Cortex, 9 (3), 246 –258 (1973). https://doi.org/10.1016/S0010-9452(73)80002-4 Google Scholar

6. 

M. Meng et al., “Lateralization of face processing in the human brain,” in Proc. R. Soc. B: Biol. Sci., (2012). Google Scholar

7. 

I. Gauthier et al., “The fusiform “face area” is part of a network that processes faces at the individual level,” J. Cognit. Neurosci., 12 (3), 495 –504 (2000). https://doi.org/10.1162/089892900562165 JCONEO 0898-929X Google Scholar

8. 

D. Pitcher, V. Walsh and B. Duchaine, “The role of the occipital face area in the cortical face perception network,” Exp. Brain Res., 209 (4), 481 –493 (2011). https://doi.org/10.1007/s00221-011-2579-1 EXBRAP 0014-4819 Google Scholar

9. 

B. Rossion, B. Hanseeuw and L. Dricot, “Defining face perception areas in the human brain: a large-scale factorial fMRI face localizer analysis,” Brain Cognit., 79 (2), 138 –157 (2012). https://doi.org/10.1016/j.bandc.2012.01.001 Google Scholar

10. 

B. Rossion et al., “A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing,” Brain, 126 (11), 2381 –2395 (2003). https://doi.org/10.1093/brain/awg241 BRAIAK 0006-8950 Google Scholar

11. 

N. Furl et al., “Cross-frequency power coupling between hierarchically organized face-selective areas,” Cereb. Cortex, 24 (9), 2409 –2420 (2013). https://doi.org/10.1093/cercor/bht097 53OPAV 1047-3211 Google Scholar

12. 

S. Bentin et al., “Electrophysiological studies of face perception in humans,” J. Cognit. Neurosci., 8 (6), 551 –565 (1996). https://doi.org/10.1162/jocn.1996.8.6.551 JCONEO 0898-929X Google Scholar

13. 

M. Behrmann, C. Thomas and K. Humphreys, “Seeing it differently: visual processing in autism,” Trends Cognit. Sci., 10 (6), 258 –264 (2006). https://doi.org/10.1016/j.tics.2006.05.001 Google Scholar

14. 

A. J. Naples et al., “Event-related potentials index neural response to eye contact,” Biol. Psychol., 127 18 –24 (2017). https://doi.org/10.1016/j.biopsycho.2017.04.006 Google Scholar

15. 

N. M. Corrigan et al., “An investigation of the relationship between fMRI and ERP source localized measurements of brain activity during face processing,” Brain Topogr., 22 (2), 83 –96 (2009). https://doi.org/10.1007/s10548-009-0086-5 BRTOEZ 0896-0267 Google Scholar

16. 

I. Deffke et al., “MEG/EEG sources of the 170-ms response to faces are co-localized in the fusiform gyrus,” Neuroimage, 35 (4), 1495 –1501 (2007). https://doi.org/10.1016/j.neuroimage.2007.01.034 NEIMEF 1053-8119 Google Scholar

17. 

M. J. Herrmann et al., “Source localization of early stages of face processing,” Brain Topogr., 18 (2), 77 –85 (2005). https://doi.org/10.1007/s10548-005-0277-7 BRTOEZ 0896-0267 Google Scholar

18. 

B. Sadeh et al., “Event-related potential and functional MRI measures of face-selectivity are highly correlated: a simultaneous ERP-fMRI investigation,” Hum. Brain Map., 31 (10), 1490 –1501 (2010). https://doi.org/10.1002/hbm.v31:10 Google Scholar

19. 

J. M. Walz et al., “Simultaneous EEG-fMRI reveals a temporal cascade of task-related and default-mode activations during a simple target detection task,” Neuroimage, 102 229 –239 (2014). https://doi.org/10.1016/j.neuroimage.2013.08.014 NEIMEF 1053-8119 Google Scholar

20. 

E. Visani et al., “Hemodynamic and EEG time-courses during unilateral hand movement in patients with cortical myoclonus. An EEG-fMRI and EEG-TD-fNIRS study,” Brain Topogr., 28 (6), 915 –925 (2015). https://doi.org/10.1007/s10548-014-0402-6 BRTOEZ 0896-0267 Google Scholar

21. 

J. Harper, S. M. Malone and W. G. Iacono, “Theta-and delta-band EEG network dynamics during a novelty oddball task,” Psychophysiology, 54 (11), 1590 –1605 (2017). https://doi.org/10.1111/psyp.12906 PSPHAF 0048-5772 Google Scholar

22. 

B. Güntekin and E. Basar, “Emotional face expressions are differentiated with brain oscillations,” Int. J. Psychophysiol., 64 (1), 91 –100 (2007). https://doi.org/10.1016/j.ijpsycho.2006.07.003 IJPSEE 0167-8760 Google Scholar

23. 

M.-P. Deiber et al., “Distinction between perceptual and attentional processing in working memory tasks: a study of phase-locked and induced oscillatory brain dynamics,” J. Cognit. Neurosci., 19 (1), 158 –172 (2007). https://doi.org/10.1162/jocn.2007.19.1.158 JCONEO 0898-929X Google Scholar

24. 

N. A. Busch, J. Dubois and R. VanRullen, “The phase of ongoing EEG oscillations predicts visual perception,” J. Neurosci., 29 (24), 7869 –7876 (2009). https://doi.org/10.1523/JNEUROSCI.0113-09.2009 JNRSDS 0270-6474 Google Scholar

25. 

L. Aftanas et al., “Affective picture processing: event-related synchronization within individually defined human theta band is modulated by valence dimension,” Neurosci. Lett., 303 (2), 115 –118 (2001). https://doi.org/10.1016/S0304-3940(01)01703-7 NELED5 0304-3940 Google Scholar

26. 

G. Buzsáki and A. Draguhn, “Neuronal oscillations in cortical networks,” Science, 304 (5679), 1926 –1929 (2004). https://doi.org/10.1126/science.1099745 SCIEAS 0036-8075 Google Scholar

27. 

E. Zion-Golumbic, M. Kutas and S. Bentin, “Neural dynamics associated with semantic and episodic memory for faces: evidence from multiple frequency bands,” J. Cognit. Neurosci., 22 (2), 263 –277 (2010). https://doi.org/10.1162/jocn.2009.21251 JCONEO 0898-929X Google Scholar

28. 

M. J. Kahana, “The cognitive correlates of human brain oscillations,” J. Neurosci., 26 (6), 1669 –1672 (2006). https://doi.org/10.1523/JNEUROSCI.3737-05c.2006 JNRSDS 0270-6474 Google Scholar

29. 

E. Başar, M. Schürmann and O. Sakowitz, “The selectively distributed theta system: functions,” Int. J. Psychophysiol., 39 (2–3), 197 –212 (2001). https://doi.org/10.1016/S0167-8760(00)00141-0 IJPSEE 0167-8760 Google Scholar

30. 

M. X. Cohen and S. van Gaal, “Subthreshold muscle twitches dissociate oscillatory neural signatures of conflicts from errors,” Neuroimage, 86 503 –513 (2014). https://doi.org/10.1016/j.neuroimage.2013.10.033 NEIMEF 1053-8119 Google Scholar

31. 

C. L. Zold and M. G. H. Shuler, “Theta oscillations in visual cortex emerge with experience to convey expected reward time and experienced reward rate,” J. Neurosci., 35 (26), 9603 –9614 (2015). https://doi.org/10.1523/JNEUROSCI.0296-15.2015 JNRSDS 0270-6474 Google Scholar

32. 

E. Başar et al., “Brain oscillations differentiate the picture of one’s own grandmother,” Int. J. Psychophysiol., 64 (1), 81 –90 (2007). https://doi.org/10.1016/j.ijpsycho.2006.07.002 IJPSEE 0167-8760 Google Scholar

33. 

M. Miyakoshi et al., “EEG evidence of face-specific visual self-representation,” Neuroimage, 50 (4), 1666 –1675 (2010). https://doi.org/10.1016/j.neuroimage.2010.01.030 NEIMEF 1053-8119 Google Scholar

34. 

M. Balconi and C. Lucchiari, “EEG correlates (event-related desynchronization) of emotional face elaboration: a temporal analysis,” Neurosci. Lett., 392 (1–2), 118 –123 (2006). https://doi.org/10.1016/j.neulet.2005.09.004 NELED5 0304-3940 Google Scholar

35. 

G. Knyazev, J. Y. Slobodskoj-Plusnin and A. Bocharov, “Event-related delta and theta synchronization during explicit and implicit emotion processing,” Neuroscience, 164 (4), 1588 –1600 (2009). https://doi.org/10.1016/j.neuroscience.2009.09.057 Google Scholar

36. 

D. Zhang et al., “Individual differences in detecting rapidly presented fearful faces,” PLoS One, 7 (11), e49517 (2012). https://doi.org/10.1371/journal.pone.0049517 POLNCL 1932-6203 Google Scholar

37. 

B. Clemens et al., “Theta EEG source localization using LORETA in partial epilepsy patients with and without medication,” Clin. Neurophysiol., 121 (6), 848 –858 (2010). https://doi.org/10.1016/j.clinph.2010.01.020 CNEUFU 1388-2457 Google Scholar

38. 

B. Güntekin and E. Başar, “A review of brain oscillations in perception of faces and emotional pictures,” Neuropsychologia, 58 33 –51 (2014). https://doi.org/10.1016/j.neuropsychologia.2014.03.014 NUPSA6 0028-3932 Google Scholar

39. 

G. G. Knyazev, “EEG delta oscillations as a correlate of basic homeostatic and motivational processes,” Neurosci. Biobehav. Rev., 36 (1), 677 –695 (2012). https://doi.org/10.1016/j.neubiorev.2011.10.002 NBREDE 0149-7634 Google Scholar

40. 

M. Ferrari and V. Quaresima, “A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application,” Neuroimage, 63 (2), 921 –935 (2012). https://doi.org/10.1016/j.neuroimage.2012.03.049 NEIMEF 1053-8119 Google Scholar

41. 

F. F. Jobsis, “Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters,” Science, 198 (4323), 1264 –1267 (1977). https://doi.org/10.1126/science.929199 SCIEAS 0036-8075 Google Scholar

42. 

F. Scholkmann et al., “A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology,” Neuroimage, 85 6 –27 (2014). https://doi.org/10.1016/j.neuroimage.2013.05.004 NEIMEF 1053-8119 Google Scholar

43. 

H.-D. Nguyen and K.-S. Hong, “Bundled-optode implementation for 3D imaging in functional near-infrared spectroscopy,” Biomed. Opt. Express, 7 (9), 3491 –3507 (2016). https://doi.org/10.1364/BOE.7.003491 BOEICL 2156-7085 Google Scholar

44. 

H.-D. Nguyen, K.-S. Hong and Y.-I. Shin, “Bundled-optode method in functional near-infrared spectroscopy,” PLoS One, 11 (10), e0165146 (2016). https://doi.org/10.1371/journal.pone.0165146 POLNCL 1932-6203 Google Scholar

45. 

F. Al-Shargie, T. B. Tang and M. Kiguchi, “Assessment of mental stress effects on prefrontal cortical activities using canonical correlation analysis: an fNIRS-EEG study,” Biomed. Opt. Express, 8 (5), 2583 –2598 (2017). https://doi.org/10.1364/BOE.8.002583 BOEICL 2156-7085 Google Scholar

46. 

M. Balconi, E. Grippa and M. E. Vanutelli, “What hemodynamic (fNIRS), electrophysiological (EEG) and autonomic integrated measures can tell us about emotional processing,” Brain Cognit., 95 67 –76 (2015). https://doi.org/10.1016/j.bandc.2015.02.001 Google Scholar

47. 

M. Muthalib et al., “Multimodal integration of fNIRS, fMRI and EEG neuroimaging,” Clin. Neurophysiol., 124 (10), 2060 –2062 (2013). https://doi.org/10.1016/j.clinph.2013.03.018 CNEUFU 1388-2457 Google Scholar

48. 

V. T. Nguyen, M. Breakspear and R. Cunnington, “Fusing concurrent EEG-fMRI with dynamic causal modeling: application to effective connectivity during face perception,” Neuroimage, 102 60 –70 (2014). https://doi.org/10.1016/j.neuroimage.2013.06.083 NEIMEF 1053-8119 Google Scholar

49. 

P. Pouliot et al., “Nonlinear hemodynamic responses in human epilepsy: a multimodal analysis with fNIRS-EEG and fMRI-EEG,” J. Neurosci. Methods, 204 (2), 326 –340 (2012). https://doi.org/10.1016/j.jneumeth.2011.11.016 JNMEDT 0165-0270 Google Scholar

50. 

J. A. Meltzer et al., “Individual differences in EEG theta and alpha dynamics during working memory correlate with fMRI responses across subjects,” Clin. Neurophysiol., 118 (11), 2419 –2436 (2007). https://doi.org/10.1016/j.clinph.2007.07.023 CNEUFU 1388-2457 Google Scholar

51. 

M. Soltanlou et al., “Increased arithmetic complexity is associated with domain-general but not domain-specific magnitude processing in children: a simultaneous fNIRS-EEG study,” Cognit. Affect. Behav. Neurosci., 17 (4), 724 –736 (2017). https://doi.org/10.3758/s13415-017-0508-x Google Scholar

52. 

R. C. Oldfield, “The assessment and analysis of handedness: the Edinburgh inventory,” Neuropsychologia, 9 (1), 97 –113 (1971). https://doi.org/10.1016/0028-3932(71)90067-4 NUPSA6 0028-3932 Google Scholar

53. 

E. A. Krall and B. Dawson-Hughes, “Heritable and life-style determinants of bone mineral density,” J. Bone Miner. Res., 8 (1), 1 –9 (1993). https://doi.org/10.1002/jbmr.5650080102 JBMREJ 0884-0431 Google Scholar

54. 

E. Okada and D. T. Delpy, “Near-infrared light propagation in an adult head model. II. Effect of superficial tissue thickness on the sensitivity of the near-infrared spectroscopy signal,” Appl. Opt., 42 (16), 2915 –2921 (2003). https://doi.org/10.1364/AO.42.002915 APOPAI 0003-6935 Google Scholar

55. 

A. Pellicer and M. del Carmen Bravo, “Near-infrared spectroscopy: a methodology-focused review,” Semin. Fetal and Neonatal Med., 16 (1), 42 –49 (2011). https://doi.org/10.1016/j.siny.2010.05.003 Google Scholar

56. 

J. A. Noah et al., “A screening procedure to determine eligibility for fNIRS neuroimaging studies,” Google Scholar

57. 

M. J. Tarr, “Face Place database,” www.face-place.org Google Scholar

58. 

M. B. Brodeur et al., “The Bank of Standardized Stimuli (BOSS), a new set of 480 normative photos of objects to be used as visual stimuli in cognitive research,” PLoS One, 5 (5), e10773 (2010). https://doi.org/10.1371/journal.pone.0010773 POLNCL 1932-6203 Google Scholar

59. 

J. A. Noah et al., “Neural correlates of conflict between gestures and words: a domain-specific role for a temporal-parietal complex,” PLoS One, 12 (3), e0173525 (2017). https://doi.org/10.1371/journal.pone.0173525 POLNCL 1932-6203 Google Scholar

60. 

J. A. Noah et al., “fMRI validation of fNIRS measurements during a naturalistic task,” J. Vis. Exp., 100 e52116 (2015). https://doi.org/10.3791/52116 Google Scholar

61. 

D. A. Boas et al., “Twenty Years of Functional Near-Infrared Spectroscopy: Introduction for the Special Issue,” Neuroimage, 85 (1), 1 –5 (2014). https://doi.org/10.1016/j.neuroimage.2013.11.033 Google Scholar

62. 

S. Dravida et al., “Comparison of oxyhemoglobin and deoxyhemoglobin signal reliability with and without global mean removal for digit manipulation motor tasks,” Neurophotonics, 5 (1), 011006 (2017). https://doi.org/10.1117/1.NPh.5.1.011006 Google Scholar

63. 

J. C. Ye et al., “NIRS-SPM: statistical parametric mapping for near-infrared spectroscopy,” Neuroimage, 44 (2), 428 –447 (2009). https://doi.org/10.1016/j.neuroimage.2008.08.036 NEIMEF 1053-8119 Google Scholar

64. 

D. A. Boas, A. M. Dale and M. A. Franceschini, “Diffuse optical imaging of brain activation: approaches to optimizing image sensitivity, resolution, and accuracy,” Neuroimage, 23 S275 –S288 (2004). https://doi.org/10.1016/j.neuroimage.2004.07.011 NEIMEF 1053-8119 Google Scholar

65. 

M. A. Franceschini et al., “Diffuse optical imaging of the whole head,” Neuroimage, 22 e2341 –e2342 (2004). Google Scholar

66. 

I. Tachtsidis and F. Scholkmann, “False positives and false negatives in functional near-infrared spectroscopy: issues, challenges, and the way forward,” Neurophotonics, 3 (3), 031405 (2016). https://doi.org/10.1117/1.NPh.3.3.031405 Google Scholar

67. 

X. Zhang, J. A. Noah and J. Hirsch, “Separation of the global and local components in functional near-infrared spectroscopy signals using principal component spatial filtering,” Neurophotonics, 3 (1), 015004 (2016). https://doi.org/10.1117/1.NPh.3.1.015004 Google Scholar

68. 

X. Zhang et al., “Signal processing of functional NIRS data acquired during overt speaking,” Neurophotonics, 4 (4), 041409 (2017). https://doi.org/10.1117/1.NPh.4.4.041409 Google Scholar

69. 

A. Delorme and S. Makeig, “EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis,” J. Neurosci. Methods, 134 (1), 9 –21 (2004). https://doi.org/10.1016/j.jneumeth.2003.10.009 JNMEDT 0165-0270 Google Scholar

70. 

R. J. Itier, M. Latinus and M. J. Taylor, “Face, eye and object early processing: what is the face specificity?,” Neuroimage, 29 (2), 667 –676 (2006). https://doi.org/10.1016/j.neuroimage.2005.07.041 NEIMEF 1053-8119 Google Scholar

71. 

K. Friston et al., “Multiple sparse priors for the M/EEG inverse problem,” Neuroimage, 39 (3), 1104 –1120 (2008). https://doi.org/10.1016/j.neuroimage.2007.09.048 NEIMEF 1053-8119 Google Scholar

72. 

V. Litvak et al., “EEG and MEG data analysis in SPM8,” Comput. Intell. Neurosci., 2011 1 –32 (2011). https://doi.org/10.1155/2011/852961 Google Scholar

73. 

T. Yarkoni et al., “Large-scale automated synthesis of human functional neuroimaging data,” Nature Methods, 8 (8), 665 –670 (2011). https://doi.org/10.1038/nmeth.1635 1548-7091 Google Scholar

74. 

C. Rorden and M. Brett, “Stereotaxic display of brain lesions,” Behav. Neurol., 12 (4), 191 –200 (2000). https://doi.org/10.1155/2000/421719 BNEUEI Google Scholar

75. 

N. K. Logothetis et al., “Neurophysiological investigation of the basis of the fMRI signal,” Nature, 412 (6843), 150 –157 (2001). https://doi.org/10.1038/35084005 Google Scholar

76. 

A.-L. Giraud and D. Poeppel, “Cortical oscillations and speech processing: emerging computational principles and operations,” Nat. Neurosci., 15 (4), 511 –517 (2012). https://doi.org/10.1038/nn.3063 NANEFN 1097-6256 Google Scholar

77. 

J. A. Brooks and J. B. Freeman, “Neuroimaging of person perception: a social-visual interface,” Neurosci. Lett., 693 40 –43 (2017). https://doi.org/10.1016/j.neulet.2017.12.046 NELED5 0304-3940 Google Scholar

78. 

R. Scheeringa et al., “Trial-by-trial coupling between EEG and BOLD identifies networks related to alpha and theta EEG power increases during working memory maintenance,” Neuroimage, 44 (3), 1224 –1238 (2009). https://doi.org/10.1016/j.neuroimage.2008.08.041 NEIMEF 1053-8119 Google Scholar

79. 

C. Chang et al., “EEG correlates of time-varying BOLD functional connectivity,” Neuroimage, 72 227 –236 (2013). https://doi.org/10.1016/j.neuroimage.2013.01.049 NEIMEF 1053-8119 Google Scholar

80. 

G. G. Knyazev, J. Y. Slobodskoj-Plusnin and A. V. Bocharov, “Gender differences in implicit and explicit processing of emotional facial expressions as revealed by event-related theta synchronization,” Emotion, 10 (5), 678 –687 (2010). https://doi.org/10.1037/a0019175 EBMOEN Google Scholar

81. 

W. Sato et al., “Rapid, high-frequency, and theta-coupled gamma oscillations in the inferior occipital gyrus during face processing,” Cortex, 60 52 –68 (2014). https://doi.org/10.1016/j.cortex.2014.02.024 Google Scholar

82. 

W. Klimesch, “EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis,” Brain Res. Rev., 29 (2–3), 169 –195 (1999). https://doi.org/10.1016/S0165-0173(98)00056-3 BRERD2 0165-0173 Google Scholar

83. 

J. J. Foxe, G. V. Simpson and S. P. Ahlfors, “Parieto-occipital ∼10 Hz activity reflects anticipatory state of visual attention mechanisms,” Neuroreport, 9 (17), 3929 –3933 (1998). https://doi.org/10.1097/00001756-199812010-00030 NERPEZ 0959-4965 Google Scholar

84. 

D. Senkowski et al., “Oscillatory beta activity predicts response speed during a multisensory audiovisual reaction time task: a high-density electrical mapping study,” Cereb. Cortex, 16 (11), 1556 –1565 (2005). https://doi.org/10.1093/cercor/bhj091 53OPAV 1047-3211 Google Scholar

85. 

G. F. Woodman, “A brief introduction to the use of event-related potentials in studies of perception and attention,” Attention, Percept. Psychophys., 72 (8), 2031 –2046 (2010). https://doi.org/10.3758/BF03196680 Google Scholar

86. 

T. Krings et al., “Accuracy of EEG dipole source localization using implanted sources in the human brain,” Clin. Neurophysiol., 110 (1), 106 –114 (1999). https://doi.org/10.1016/S0013-4694(98)00106-0 CNEUFU 1388-2457 Google Scholar

87. 

P. H. Laarne et al., “Effect of EEG electrode density on dipole localization accuracy using two realistically shaped skull resistivity models,” Brain Topogr., 12 (4), 249 –254 (2000). https://doi.org/10.1023/A:1023422504025 BRTOEZ 0896-0267 Google Scholar

88. 

C. M. Michel et al., “EEG source imaging,” Clin. Neurophysiol., 115 (10), 2195 –2222 (2004). https://doi.org/10.1016/j.clinph.2004.06.001 CNEUFU 1388-2457 Google Scholar

89. 

M. Hassan et al., “EEG source connectivity analysis: from dense array recordings to brain networks,” PLoS One, 9 (8), e105041 (2014). https://doi.org/10.1371/journal.pone.0105041 POLNCL 1932-6203 Google Scholar

90. 

G. G. Knyazev et al., “The default mode network and EEG alpha oscillations: an independent component analysis,” Brain Res., 1402 67 –79 (2011). https://doi.org/10.1016/j.brainres.2011.05.052 Google Scholar

91. 

F. Musso et al., “Spontaneous brain activity and EEG microstates. A novel EEG/fMRI analysis approach to explore resting-state networks,” Neuroimage, 52 (4), 1149 –1161 (2010). https://doi.org/10.1016/j.neuroimage.2010.01.093 NEIMEF 1053-8119 Google Scholar

92. 

F. Travis et al., “A self-referential default brain state: patterns of coherence, power, and eLORETA sources during eyes-closed rest and Transcendental Meditation practice,” Cognit. Process., 11 (1), 21 –30 (2010). https://doi.org/10.1007/s10339-009-0343-2 Google Scholar

Biography

Swethasri Dravida received her BS degree in mathematics and brain and cognitive sciences from MIT in 2013. She is a graduate student at Yale School of Medicine. Her current research interests include using functional near-infrared spectroscopy and EEG to study social interaction, especially in clinical contexts such as autism.

Yumie Ono received her PhD in electrical engineering from Waseda University in 2004. She is currently a professor at Meiji University in the Department of Electronics and Bioinformatics of School of Science and Technology. She is also the director of the Health Science and Medical Engineering Laboratory at Meiji University. Her research interests include functional brain imaging and development of biological measurement systems for medical applications.

J. Adam Noah received his PhD in biomedical sciences from Marshall University School of Medicine in 2003. He is an associate research scientist at Yale School of Medicine in the Department of Psychiatry and the Brain Function Laboratory. His research interests include functional near-infrared spectroscopy and integration of other multimodal and behavioral recording techniques for applications in communication and social interactions, neurofeedback, and cognitive neuroimaging.

Xian Zhang received his PhD in psychology and visual science from Columbia University, New York, USA, in 2003. He is an associate research scientist at the Brain Function Laboratory in the Department of Psychiatry of Yale School of Medicine. His research interests include computational neuroscience, signal processing, and neuroimaging technologies, such as EEG, fNIRS, and fMRI and their applications in psychiatry, vision science, social interactions, and decision making.

Joy Hirsch received her PhD in psychology and visual science from Columbia University and is now a professor of psychiatry and neurobiology at Yale School of Medicine, and a professor of neuroscience at the University College London. She is also the director of the Brain Function Laboratory at Yale University. Her research is focused on investigations of neural circuitry that underlies human social interactions using multimodal neuroimaging techniques including fNIRS, fMRI, EEG, eye-tracking, and behavioral measures. Prior to recruitment to Yale, she was a director of the fMRI research center at Columbia University.

© The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Swethasri Dravida, Yumie Ono, J. Adam Noah, Xian Z. Zhang, and Joy Hirsch "Co-localization of theta-band activity and hemodynamic responses during face perception: simultaneous electroencephalography and functional near-infrared spectroscopy recordings," Neurophotonics 6(4), 045002 (22 October 2019). https://doi.org/10.1117/1.NPh.6.4.045002
Received: 23 April 2019; Accepted: 12 September 2019; Published: 22 October 2019
JOURNAL ARTICLE
11 PAGES


SHARE
Advertisement
Advertisement
Back to Top