The functional magnetic resonance imaging (fMRI) researches on working memory have found that activation of cortical areas appeared dependent on memory load, and event-related potentials (ERP) studies have demonstrated that amplitudes of P300 decreased significantly when working memory load increased. However, the cortical activities related with P300 amplitudes under different memory loads remains unclear. Joint fMRI and EEG analysis which fusions the time and spatial information in simultaneous EEG-fMRI recording can reveal the regional activation at each ERP time point. In this paper, we first used wavelet transform to obtain the single-trial amplitudes of P300 caused by a digital N-back task in the simultaneous EEG-fMRI recording as the ERP feature sequences. Then the feature sequences in 1-back condition and 3-back condition were introduced into general linear model (GLM) separately as parametric modulations to compare the cortical activation under different memory loads. The results showed that the average amplitudes of P300 in 3-back significantly decreased than that in 1-back, and the activities induced by ERP feature sequences in 3-back also significantly decreased than that in the 1-back, including the insular, anterior cingulate cortex, right inferior frontal gyrus, and medial frontal gyrus, which were relevant to the storage, monitoring, and manipulation of information in working memory task. Moreover, the difference in the activation caused by ERP feature showed a positive correlation with the difference in behavioral performance. These findings demonstrated the locations of P300 amplitudes differences modulated by the memory load and its relationship with the behavioral performance.
The functional magnetic resonance imaging (fMRI) research on face processing have found that the significant
activation by face stimuli mainly locailized at the occipital temporal lobe especilly the fusiform gyrus. However, fMRI
cannot reflect the face processing as time changes. Event-related potential (ERP) can record electrophysiological
changes induced by neuronal activation in time, but spatial information is not well localized. Fusing fMRI and ERP data can perform that how the fMRI activation changes as time move at each ERP time point. Although most of fuse methods perform to analysis by constraint ERP or fMRI data, joint independent component analysis (jICA) method can equally use the ERP and fMRI data and simultaneously examine electrophysiologic and hemodynamic response. In this paper, we use jICA method to analysis two modalities in common data space in order to examine the dynamics of face stimuli response. The results showed that the ERP component N170 response associated with middle occipital gyrus, fusiform gyrus, inferior occipital gyrus, superior temporal gyrus and parahippocampa gyrus for face. Likewise, for non-face, the N170 component was mainly related to parahippocampa gyrus, middle occipital gyrus and inferior occipital gyrus. Further studying on the correlation of the localized ERP response and corresponding average ERP, it was also concluded that the spatial activations related to N170 response induced by face stimulus located in fusiform gyrus, and that induced by non-face stimulus located in parahippocampa gyrus. From the result, fusing fMRI and ERP data by jICA not only provides the time information on fMRI and the spatial source of ERP component, but also reflects spatiotemporal change during face processing.