Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Emotion self-regulation training in major depressive disorder using simultaneous real-time fMRI and EEG neurofeedback

Emotion self-regulation training in major depressive disorder using simultaneous real-time fMRI... Emotion self-regulation training in major depressive disorder using simultaneous real-time fMRI and EEG neurofeedback 1# 1,2 1 1,3# Vadim Zotev , Ahmad Mayeli , Masaya Misaki , Jerzy Bodurka 1 2 Laureate Institute for Brain Research, Tulsa, OK, USA; Electrical and Computer Engineering, University of Oklahoma, Tulsa, OK, USA; Stephenson School of Biomedical Engineering, University of Oklahoma, Norman, OK, USA Abstract: Simultaneous real-time fMRI and EEG neurofeedback (rtfMRI-EEG-nf) is an emerging neuromodulation approach, that enables simulta- neous volitional regulation of both hemodynamic (BOLD fMRI) and electrophysiological (EEG) brain activities. Here we report the first application of rtfMRI-EEG-nf for emotion self-regulation training in patients with major depressive disorder (MDD). In this proof-of-concept study, MDD pa- tients in the experimental group (n=16) used rtfMRI-EEG-nf during a happy emotion induction task to simultaneously upregulate two fMRI and two EEG activity measures relevant to MDD. The target measures included BOLD activities of the left amygdala (LA) and left rostral anterior cingulate cortex (rACC), and frontal EEG asymmetries in the alpha band (FAA, [7.5-12.5] Hz) and high-beta band (FBA, [21-30] Hz). MDD patients in the control group (n=8) were provided with sham feedback signals. An advanced procedure for improved real-time EEG-fMRI artifact correction was implemented. The experimental group participants demonstrated significant upregulation of the LA BOLD activity, FAA, and FBA during the rtfMRI-EEG-nf task, as well as significant enhancement in fMRI connectivity between the LA and left rACC. Average individual FAA changes dur- ing the rtfMRI-EEG-nf task positively correlated with depression and anhedonia severities, and negatively correlated with after-vs-before changes in depressed mood ratings. Temporal correlations between the FAA and FBA time courses and the LA BOLD activity were significantly enhanced dur- ing the rtfMRI-EEG-nf task. The experimental group participants reported significant mood improvements after the training. Our results suggest that the rtfMRI-EEG-nf may have potential for treatment of MDD. Keywords: depression, neurofeedback, emotion regulation, amygdala, rostral anterior cingulate cortex, EEG-fMRI, frontal EEG asymmetry Lemieux, 2010). In particular, relevant EEG measures can 1. Introduction represent different EEG frequency bands, while BOLD We have introduced simultaneous real-time fMRI and fMRI activity reflects cumulative metabolic energy de- EEG neurofeedback (rtfMRI-EEG-nf) – a non-invasive mands across the entire EEG spectrum. Second, rtfMRI- neuromodulation approach, that enables simultaneous vo- EEG-nf training may help to develop personalized mental litional regulation of both hemodynamic (BOLD fMRI) strategies that would reliably engage both the fMRI and and electrophysiological (EEG) brain activities (Zotev et EEG target brain activities at the same time and further al., 2014). It involves real-time integration of concurrent enhance their interactions. Such experimentally verified fMRI and EEG data streams to provide real-time fMRI mental strategies could then be employed during EEG-nf- neurofeedback (rtfMRI-nf) and EEG neurofeedback only training, which may provide a cost-effective, mobile, (EEG-nf) signals simultaneously to a participant inside the and long-term therapy in support of the rtfMRI-EEG-nf MRI scanner (Mano et al., 2017; Zotev et al., 2014). This training. Until now, rtfMRI-EEG-nf has only been used in multimodal neurofeedback approach holds two major proof-of-principle studies with healthy participants (Per- promises for treatment of neurological and psychiatric ronnet et al., 2017; Zotev et al., 2014). disorders. First, application of rtfMRI-EEG-nf may con- Here we report the first application of rtfMRI-EEG-nf ceivably have stronger therapeutic effects than standalone for emotion self-regulation training in a neuropsychiatric applications of either rtfMRI-nf (e.g. Thibault et al., 2018) population, specifically – in patients with major depres- or EEG-nf (e.g. Micoulaud-Franchi et al., 2015). The rea- sive disorder (MDD). During the rtfMRI-EEG-nf task, the son is that rtfMRI-EEG-nf can target disorder-specific participants induced happy emotion by retrieving and con- brain activity measures identified by two very different templating happy autobiographical memories, and, imaging modalities – fMRI and EEG (e.g. Mulert and simultaneously, learned to upregulate two rtfMRI-nf sig- ___________________ nals and two EEG-nf signals. The four neurofeedback Corresponding authors. E-mail: vzotev@laureateinstitute.org; signals represented four brain activity measures relevant jbodurka@laureateinstitute.org to MDD, as we explain below. The first rtfMRI-nf signal in our study is based on Ramot et al., 2017. We did not expect a significant mean BOLD activity of the left amygdala (LA) target region of fMRI activation of the left rACC during the rtfMRI-EEG- interest (ROI), as used in our previous rtfMRI-nf emotion nf task relative to rest, because the rACC is a part of the self-regulation studies with healthy participants (Zotev et default mode network, and its mean activation during the al., 2011), MDD patients (Young et al., 2014; Zotev et al., rtfMRI-nf training of the LA activity is relatively low 2016), and PTSD patients (Zotev et al., 2018b). The (Zotev et al., 2013). amygdala plays a fundamental role in emotion processing. The first EEG-nf signal in our study represents a In MDD, the amygdala exhibits blunted BOLD responses change in frontal alpha EEG asymmetry, which we abbre- to positive emotional stimuli (e.g. Price and Drevets, viate here as FAA. The FAA is defined as ln(P(right)) – 2012; Suslow et al., 2010; Victor et al., 2010). Upregula- ln(P(left)), where P is EEG power in the alpha frequency tion of the LA activity using the rtfMRI-nf during the band for corresponding (pre)frontal EEG channels on the positive emotion induction task based on retrieval of hap- right and on the left (e.g. F4 and F3). Because change in py autobiographical memories has been shown to correct alpha EEG power negatively correlates with cortical neu- this amygdala reactivity bias in MDD, and lead to signifi- ronal activation (e.g. Cook et al., 1998), a more positive cant reduction in depression severity (Young et al., 2017). FAA indicates a relatively stronger activation of the left A similar rtfMRI-nf training procedure in PTSD patients prefrontal regions. The FAA is commonly interpreted ac- yielded significant reductions in both PTSD severity and cording to the approach-withdrawal hypothesis, which comorbid depression severity (Zotev et al., 2018b). posits that activation of the left prefrontal regions is more The second rtfMRI-nf signal is based on BOLD activi- closely associated with approach motivation, while activa- ty of a target ROI in the left rostral anterior cingulate tion of the right prefrontal regions is associated with cortex (rACC). The rACC function is very important in avoidance motivation (e.g. Davidson, 1996; Harmon- MDD (e.g. Pizzagalli, 2011). Resting rACC activity, Jones and Gable, 2018; Spielberg et al., 2011, 2013). In- measured by PET prior to antidepressant treatment, has deed, approach-related emotional states, such as been shown to reliably predict MDD patients’ treatment happiness, are characterized by more positive FAA levels response (Mayberg et al., 1997; Pizzagalli, 2011). After than avoidance-related states, such as fear or disgust (e.g. treatment, MDD patients exhibit enhanced fMRI function- Davidson, 1996; Stewart et al., 2014). MDD patients and al connectivity between the rACC and the amygdala, both individuals with a history of depression show significantly at rest and during exposure to neutral and positive emo- lower FAA levels during an emotional task (either ap- tional images (Anand et al., 2005). We have demonstrated proach- or avoidance-related) than non-depressed that, in healthy participants, the rtfMRI-nf modulation of participants performing the same task (Stewart et al., the LA activity during happy emotion induction is accom- 2011, 2014). Resting-state FAA is also reduced in MDD panied by significant enhancement, across nf runs, in patients compared to non-depressed individuals (e.g. fMRI connectivity between the LA and the left rACC Smith et al., 2018; Thibodeau et al., 2006), though the (Zotev et al., 2011). Moreover, effective connectivity findings are less robust than those for an emotional chal- analyses suggest that the left rACC modulates activity of lenge (Stewart et al., 2014). The task-related FAA results the LA and several prefrontal regions during the rtfMRI-nf suggest that upregulation of FAA using EEG-nf during training (Zotev et al., 2013). In MDD patients, resting happy emotion induction would benefit MDD patients. fMRI connectivity between the LA and the left rACC Emotion regulation training with FAA-based EEG-nf has shows negative correlation with depression severity (Yuan been explored in several studies (e.g. Allen et al., 2001; et al., 2014). This connectivity is increased after the Baehr et al., 1997; Cavazza et al., 2014; Choi et al., 2011; rtfMRI-nf training of the LA activity (Yuan et al., 2014). Peeters et al., 2014; Quaedflieg et al., 2016; Rosenfeld et Collectively, these results suggest that enhancement in al., 1995). fMRI connectivity between the LA and the left rACC by Importantly, the FAA upregulation using the EEG-nf means of rtfMRI-nf during happy emotion induction in the present experimental design is consistent with the should be beneficial to MDD patients. LA BOLD activity upregulation using the rtfMRI-nf. Mo- Following this evidence, we included the rtfMRI-nf tivation is an important component of neurofeedback signal based on the left rACC activity to enable enhance- learning (e.g. Gaume et al., 2016). Because rtfMRI-nf ment in fMRI connectivity between the LA and the left training in general is a goal-oriented behavior, it requires rACC through simultaneous upregulation of the two nf approach motivation to be successful (e.g. Spielberg et al., signals. We chose this approach, because it is the simplest 2011, 2013). In our previous study, MDD patients, who way to enhance fMRI connectivity of the two brain re- underwent rtfMRI-nf training of the LA activity, showed gions, that does not involve computation of a correlation positive FAA changes, indicative of stronger approach coefficient for their fMRI waveforms. A conceptually sim- motivation, during the rtfMRI-nf task (Zotev et al., 2016). ilar ‘two-point’ technique was independently used by Moreover, mean FAA changes correlated with the amyg- dala BOLD laterality values. Temporal correlation be- and the FAA upregulation. The secondary outcome tween the FAA time course and the LA BOLD activity measures are the left rACC vs LA functional connectivity was significantly enhanced during the rtfMRI-nf task enhancement and the FBA upregulation. In addition to (Zotev et al., 2016). These observations suggest that the testing the two hypotheses, we conducted exploratory FAA and the LA BOLD activity can be modulated simul- analyses to examine associations between the target activi- taneously using the rtfMRI-EEG-nf. ty measures and the MDD patients’ depression severity, The second EEG-nf signal represents a change in anhedonia severity, and mood rating changes. frontal high-beta EEG asymmetry, abbreviated here as FBA. The FBA is defined as ln(P(left)) – ln(P(right)), 2. Methods where P is EEG power in the high-beta (beta3) band [21- 2.1. Participants 30] Hz for respective (pre)frontal EEG channels on the The study was conducted at the Laureate Institute for left and on the right (e.g. F3 and F4). Because cortical ac- Brain Research. It was approved by the Western tivation positively correlates with change in high-beta Institutional Review Board (IRB). All study procedures EEG power (Cook et al., 1998), a more positive FBA is were performed in accordance with the principles associated with a relatively stronger activation of the left expressed in the Declaration of Helsinki. prefrontal regions (similar to FAA). MDD patients, when Participants were recruited through online, newspaper, compared to healthy individuals, exhibit elevated resting radio, flyer, and other media advertisements. Individuals high-beta EEG activity in the right prefrontal regions, and were recruited, if they were currently depressed, but not deficient high-beta activity in the precuneus/posterior cin- currently taking psychiatric medication. All participants gulate (Pizzagalli et al., 2002). This finding suggests that underwent screening evaluations, including the Structured resting-state FBA is reduced in MDD. Paquette et al. em- Clinical Interview for the Diagnostic and Statistical ployed a high-beta EEG-nf for emotion self-regulation Manual of Mental Disorders, Fourth Edition (DSM-IV) training in MDD patients (Paquette et al., 2009). The (American Psychiatric Association, 2000) Axis I disorders. study showed that alleviation of MDD symptoms was as- The following exclusion criteria were applied: general sociated with reduction in resting high-beta EEG activity MRI exclusions, current pregnancy, psychosis, serious in the prefrontal cortex and increase in such activity in the suicidal ideation, major medical or neurological disorders, precuneus/posterior cingulate (Paquette et al., 2009). For exposure to any medication likely to influence cerebral responders in that study, the high-beta EEG activity reduc- function or blood flow within 3 weeks (8 weeks for tion was larger in the right prefrontal regions than in the fluoxetine), and meeting the DSM-IV criteria for drug or corresponding regions on the left (Paquette et al., 2009), alcohol abuse within the previous year or for lifetime indicating more positive resting-state FBA after the train- alcohol or drug dependence (except nicotine). Participants ing. This finding suggests that more positive FBA may be enrolled in the study met the criteria for MDD laid out in beneficial to MDD patients. We have already demonstrat- the DSM-IV. Most participants had recurrent MDD. They ed that healthy participants can learn to simultaneously provided a written informed consent as approved by the upregulate the LA BOLD activity and the FBA using IRB, and received monetary compensation. rtfMRI-EEG-nf while inducing happy emotion (Zotev et Twenty four unmedicated MDD patients completed an al., 2014). rtfMRI-EEG-nf training session. Prior to the session, the We conducted the proof-of-concept rtfMRI-EEG-nf participants underwent a psychological evaluation by a experiment, reported here, to test two main hypotheses. licensed psychiatrist. It included administration of the The first hypothesis was that MDD patients would be able following tests: the 21-item Hamilton Depression Rating to significantly increase the LA BOLD activity, the FAA, Scale (HDRS, Hamilton, 1960), the Montgomery-Asberg and the FBA using the rtfMRI-EEG-nf during happy emo- Depression Rating Scale (MADRS, Montgomery and tion induction. As part of this hypothesis, we also Asberg, 1979), the Snaith-Hamilton Pleasure Scale predicted that fMRI connectivity between the LA and the (SHAPS, Snaith et al., 1995), the Hamilton Anxiety left rACC would be significantly enhanced during the Rating Scale (HARS, Hamilton, 1959), the 20-item rtfMRI-EEG-nf task. The second hypothesis was that per- Toronto Alexithymia Scale (TAS-20, Bagby et al., 1994), formance of the rtfMRI-EEG-nf task would be and the Behavioral Inhibition System / Behavioral accompanied by significant enhancements in temporal Activation System scales (BIS/BAS, Carver and White, correlations between the FAA and FBA time courses and 1994). Both before and after the rtfMRI-EEG-nf session, the LA BOLD activity. Such enhancements would indi- the participants completed the Profile of Mood States cate that the corresponding EEG-nf and rtfMRI-nf signals were indeed upregulated together in real time, rather than (POMS, McNair et al., 1971), the State-Trait Anxiety Inventory (STAI, Spielberger et al., 1970), and the Visual independently one at a time (Zotev et al., 2014). The pri- Analogue Scale (VAS) with 10-point subscales for happy, mary outcome measures in our study are the LA activation Figure 1. Experimental paradigm for emotion self-regulation training using simultaneous real-time fMRI and EEG neurofeedback (rtfMRI-EEG- nf). A) Real-time GUI display screen for Happy Memories conditions with rtfMRI-EEG-nf. The four neurofeedback signals are displayed on the screen as four variable-height bars. The two EEG-nf signals on the left are based, respectively, on changes in frontal alpha EEG asymmetry (FAA, magenta) and frontal high-beta EEG asymmetry (FBA, purple). The two rtfMRI-nf signals on the right are based, respectively, on fMRI activities of the left amygdala (LA, red) and the left rostral anterior cingulate cortex (L rACC, orange). The bar heights are updated every 2 s. The black horizontal bar in the middle of the screen specifies a target level. It is raised from run to run. B) Experimental protocol consisted of six runs, each lasting 8 min 46 s. It included a Rest run, four rtfMRI-EEG-nf runs – Practice, Run 1, Run 2, Run 3, and a Transfer run without nf. The names of the five task runs are abbreviated in the text and figures as PR, R1, R2, R3, and TR, respectively. The task runs consisted of 40-s long blocks of Rest, Happy Memories, and Count conditions. The condition names are abbreviated as R, H, and C, respectively. No bars were displayed during the Rest and Count conditions, and during the entire Transfer run. restless, sad, anxious, irritated, drowsy, and alert states. simultaneously acquired EEG and fMRI data streams, Three MDD patients were in remission on the day of the described in Zotev et al., 2014. The neurofeedback experiment (HDRS ratings ≤ 7). BIS/BAS scores were information was displayed to a participant inside the unavailable for three participants out of 24. scanner on a projection screen via a multimodal graphical The participants were assigned to either an user interface (mGUI), depicted in Fig. 1A. The mGUI experimental group (EG) or a control group (CG) at 2:1 included four thermometer-style variable-height bars. The ratio in numbers, common in proof-of-concept rtfMRI-nf heights of these bars, updated every 2 s, represented the studies (Young et al., 2014). All the participants were four neurofeedback signals. Each bar height was also given identical instructions and were unaware of their indicated by a numeric value shown above that bar (Fig. group status. During the training session, participants in 1A). the EG (n=16, 13 females) received the rtfMRI-EEG-nf, The red rtfMRI-nf bar on the right represented BOLD based on their real-time EEG and fMRI brain activity fMRI activity of the left amygdala (LA) target ROI (Fig. measures. Participants in the CG (n=8, 4 females) were 2A). This spherical ROI with R=7 mm was centered at provided, without their knowledge, with sham feedback (−21, −5, −16) locus in the Talairach space (Talairach and signals, unrelated to brain activity. Because the Tournoux, 1988), as in our previous studies (Zotev et al., recruitment of unmedicated MDD patients was slow, the 2011, 2014, 2016). The orange rtfMRI-nf bar represented experiments for the EG were performed first, followed by fMRI activity of the left rACC target ROI (Fig. 2B). This the experiments for the CG. Psychological trait measures ROI, also with R=7 mm, was centered at (−3, 34, 5) locus, for the EG and CG participants, assessed before the which had exhibited significant enhancements in both rtfMRI-EEG-nf session, are reported in Supplementary functional and effective fMRI connectivities with the LA material (Table S1). There were no significant group during the rtfMRI-nf training in healthy participants differences in these measures. (Zotev et al., 2011, 2013). The magenta EEG-nf bar on the left represented a 2.2. Real-time fMRI and EEG neurofeedback change in relative alpha EEG asymmetry for channels F3 The rtfMRI-EEG-nf was implemented using the and F4 (Fig. 2C). The relative alpha asymmetry (A) was custom real-time control system for integration of defined as A = (P(F4) – P(F3)) / (P(F4) + P(F3)), where P activity. The sham feedback signals were computed, for each 40-s-long condition block, as random linear combinations of seven Legendre polynomials, as described previously (Zotev et al., 2018a). They were used to set heights of the four nf bars in real time (Fig. 1A). These signals’ waveforms were smooth, but randomly shaped, and they also varied randomly across condition blocks and across participants. 2.3. Experimental protocol The experimental protocol for training of emotion self- regulation using the rtfMRI-EEG-nf is illustrated in Fig. 1B. It has the same overall structure as the protocols we used previously (Zotev et al., 2011, 2014, 2016). The protocol included six EEG-fMRI runs (Fig. 1B), each lasting 8 min 46 s. During the Rest run, the participants were asked to relax and rest while looking at a fixation cross. The five task runs – the Practice run, Run 1, Run 2, Figure 2. Target regions of interest (ROIs) and EEG channels used to Run 3, and the Transfer run – consisted of alternating 40- provide the rtfMRI-EEG-nf. The ROIs were defined anatomically in the s-long blocks of Happy Memories, Count, and Rest co-planar stereotaxic atlas of the human brain by Talairach and conditions (Fig. 1B). Each condition was specified by Tournoux, and transformed to each participant’s individual fMRI image visual cues that included a color symbol at the center of space. A) Spherical 14-mm-diameter target ROI in the left amygdala (LA) region, projected onto the standard TT_N27 anatomical template in the screen and a text line at the top of the screen. For the the Talairach space. B) Spherical 14-mm-diatemer target ROI in the left Happy Memories with rtfMRI-EEG-nf condition blocks, rostral anterior cingulate cortex (L rACC) region. C) Frontal EEG the participants were asked to induce happy emotion by channels F3 (left) and F4 (right) used to provide the EEG-nf based on recalling happy autobiographical memories, while frontal EEG asymmetries in the alpha and high-beta EEG bands. Channel simultaneously trying to raise the levels of all four FCz was employed as reference. D) Custom modification of a standard MR-compatible 32-channel EEG cap to improve quality of the EEG-nf neurofeedback bars on the screen (Fig. 1A). For the Count during fMRI. The modified cap includes four wire contours with resistors condition blocks, the participants were instructed to for acquisition of reference artifact waveforms. mentally count back from 300 by subtracting a given integer. For the Rest condition blocks, the participants is EEG power in the alpha frequency band [7.5-12.5] Hz. were asked to relax and rest while looking at a fixation Note that normalized frontal alpha EEG asymmetry, cross. No bars were displayed during the Rest and Count commonly defined as FAA = ln(P(F4)) – ln(P(F3)), has a conditions, and during the Happy Memories conditions in Gaussian distribution, appropriate for statistical analyses. the Transfer run. However, its infinite variation range makes it less Prior to the experiment, the participants were asked to convenient for real-time applications. Therefore, we think of and write down three happy autobiographical employed a change in the relative asymmetry A as a target memories, keeping them confidential. The Practice run measure for EEG-nf, and used the FAA in offline data was included to give the participants an opportunity to analyses. The relative asymmetry A and the normalized become familiar with the rtfMRI-EEG-nf procedure and asymmetry FAA are related by the Fisher transform with evaluate emotional impact of the prepared happy factor ½ (Zotev et al., 2014). Similarly, the purple EEG-nf memories. During the four Happy Memories condition bar represented a change in relative high-beta EEG blocks in the Practice run, the instruction line at the top of asymmetry (B), defined as B = (P(F3) – P(F4)) / (P(F3) + the screen read “Happy – try 1st memory”, “Happy – try P(F4)), where P is EEG power in the high-beta frequency 2nd memory”, “Happy – try 3rd memory”, and “Happy – band [21-30] Hz. Normalized FBA = ln(P(F3)) – try best memory”, respectively. During the subsequent ln(P(F4)) was used in offline data analyses. Thus, the training runs (Run 1, Run 2, Run 3), the participants were rtfMRI-EEG-nf was designed to enable upregulation of encouraged to use any (and as many) memories that both the FAA and FBA simultaneously with upregulation helped them induce happy emotion and raise the rtfMRI- of BOLD activities of the LA and L rACC. EEG-nf bars. The instruction cue during the Happy For the control group (CG), the four actual Memories condition blocks in these runs was “Happy” neurofeedback signals were substituted, without the (Fig. 1A). The Transfer run without nf was included to participants’ knowledge, with sham feedback signals, evaluate whether the participants’ learned ability to which were computer generated and unrelated to any brain control the four target measures of brain activity generalized beyond the rtfMRI-EEG-nf training. During acquired with 0.2 ms temporal and 0.1 µ V measurement the Happy Memories condition blocks in the Transfer run, resolution (16-bit 5 kS/s sampling) in [0.016-250] Hz the instruction line read “As Happy as possible”. frequency band with respect to FCz reference. The ECG A target level for the rtfMRI-nf and EEG-nf signals waveform was acquired with 0.5 µ V resolution. BrainVision Recorder software was used for acquisition of was specified by the black horizontal bar above the red arrow in the middle of the mGUI screen (Fig. 1A). To raw EEG data, while BrainVision RecView software encourage the participants to improve their performance (Brain Products, GmbH) was employed for real-time from run to run, the target level was raised in a linear EEG-fMRI artifact correction as described below. fashion across the four nf runs. For the rtfMRI-nf signals, Technical details of the EEG-fMRI system setup, the target bar heights corresponded to 0.5%, 1.0%, 1.5%, configuration, and raw data acquisition were described and 2.0% fMRI percent signal changes for the Practice previously (Zotev et al., 2012). run, Run 1, Run 2, and Run 3, respectively. For the EEG- 2.5. Real-time EEG-fMRI artifact correction nf signals, the same bar heights corresponded to A and B relative asymmetry changes by 0.05, 0.10, 0.15, and 0.20, In the present study, we implemented a novel procedure for more efficient real-time EEG-fMRI artifact respectively. The display ranges were from −3% to +3% correction to improve quality of EEG-nf during fMRI. It for the fMRI-nf signals, and from −0.3 to +0.3 for the involved a special modification of a standard 32-channel EEG-nf signals. The Count conditions involved counting MR-compatible EEG cap (BrainCap-MR from back from 300 by subtracting 3, 4, 6, 7, and 9 for the Practice run, Run 1, Run 2, Run 3, and the Transfer run, EASYCAP, GmbH). The cap modification is shown in Fig. 2D and described in detail in Supplementary material respectively. After each experimental run with the Happy (S1.1). It enabled acquisition of four reference artifact Memories condition, a participant was asked to rate waveforms, which we refer to as R (t), R (t), R (t), and his/her performance on a scale from 0 (“not at all”) to 10 1 2 3 R (t), approximating cardioballistic (CB) and random- (“extremely”) by verbally answering two questions: “How 4 motion artifacts picked up by EEG channels F3 and F4. successful were you at recalling your happy memories?” and “How happy are you right now?”. The real-time procedure for EEG-fMRI artifact correction is depicted schematically in Fig. 3A. It is 2.4. MRI and EEG data acquisition implemented in BrainVision RecView software, which receives raw EEG data from BrainVision Recorder All experiments were conducted on the General Electric Discovery MR750 3T MRI scanner with a software in real time. The procedure includes three standard 8-channel receive-only head coil. A single-shot consecutive steps. First, the RecView MRI Artifact Filter is used to gradient echo EPI sequence with FOV/slice=240/2.9 mm, TR/TE=2000/30 ms, flip angle=90°, 34 axial slices per perform real-time average artifact subtraction (AAS) of volume, slice gap=0.5 mm, SENSE R=2 in the phase MR artifacts. The AAS method takes advantage of encoding (anterior-posterior) direction, acquisition matrix temporal periodicity of an fMRI pulse sequence 96×96, sampling bandwidth=250 kHz, was employed for (period=TR) and associated MR artifacts (Allen et al., fMRI. Each fMRI run included 263 EPI volumes (the first 2000). After the correction, the data are lowpass filtered at three EPI volumes were excluded from data analyses). 80 Hz (96 dB/octave) and downsampled to 250 S/s Physiological pulse oximetry and respiration waveforms sampling rate (4 ms interval). were recorded simultaneously with fMRI. The EPI images Second, real-time linear regression of CB and random- were reconstructed into a 128×128 matrix, resulting in motion artifacts (Masterton et al., 2007) is conducted 1.875×1.875×2.9 mm fMRI voxels. A T1-weighted 3D using the RecView Linear Derivation Filter. It is performed for channels F3 and F4 according to the MPRAGE sequence with FOV/slice=240/1.2 mm, TR/TE=5.0/1.9 ms, TD/TI=1400/725 ms, flip angle=10°, formulas: V (t,F3) = V(t,F3) – a R (t) – a R (t) – a R (t) – C 1 1 2 2 3 3 128 axial slices per slab, SENSE R=2, acquisition matrix a R (t), and V (t,F4) = V(t,F4) – b R (t) – b R (t) – b R (t) 4 4 C 1 1 2 2 3 3 – b R (t). Here, V (t,F3) and V (t,F4) are corrected 256×256, sampling bandwidth=31.2 kHz, scan time=4 4 4 C C min 58 s, was used for structural imaging. It provided waveforms for F3 and F4 after the regression, {R (t)}, high-resolution anatomical brain images with i=1…4, are the reference artifact waveforms, and {a }, {b }, i=1…4, are linear regression coefficients. The 0.94×0.94×1.2 mm voxels. i EEG recordings were performed simultaneously with coefficients are determined before each experimental run fMRI using a 32-channel MR-compatible EEG system as explained below. from Brain Products, GmbH. The system included one Third, the RecView Pulse Artifact Filter is used to BrainAmp MR plus amplifier. The EEG system’s clock carry out AAS of CB artifacts. The AAS in this case relies was synchronized with the MRI scanner’s 10 MHz clock on quasi-periodic nature of cardiac activity and related CB using the Brain Products’ SyncBox device. EEG data were artifacts (Allen et al., 1998). Cardiac epochs are includes the glmfit() function to solve general linear models (GLMs) fitting the reference artifact waveforms to the waveforms from channels F3 and F4: V(t,F3) = a R (t) + a R (t) + a R (t) + 1 1 2 2 3 3 a R (t) + e(t,F3), and V(t,F4) = b R (t) 4 4 1 1 + b R (t) + b R (t) + b R (t) + e(t,F4). 2 2 3 3 4 4 Here, e(t,F3) and e(t,F4) are neuronal and other signal components showing no correlations with the reference artifact waveforms. The fitting is carried out across the entire run. The GLM coefficients {a }, {b }, i=1…4, i i are then entered into the Linear Derivation Filter to enable the real- time artifact regression during the next experimental run. Thus, the real-time regression during the Practice run employs the coefficients determined from the data for the Rest run, the regression during Run 1 utilizes the coefficients computed from the data for the Practice run, and so on (Fig. 1B). Remarkably, the fact that the regression coefficients were determined from the preceding run’s data had little effect on the efficiency of the artifact regression in terms of signal variance reduction, as reported Figure 3. Real-time EEG-fMRI artifact correction procedure and its performance. A) The real- time procedure included three consecutive steps implemented in BrainVision RecView software: in Supplementary material (S1.2, Fig. average artifact subtraction (AAS) of MR artifacts; regression of reference artifact waveforms S1). This finding suggests that overall approximating cardioballistic (CB) and random-motion artifacts (channels F3 and F4 only); AAS properties of CB and random-motion of cardioballistic artifacts. B) Illustration of performance of the regression procedure across an artifacts did not generally change entire experimental run (left) and for a short time interval in the middle of that run (right). C) much from one run to the next. Performance of the regression procedure for channels F3 and F4 across all experimental (task) runs for all participants. Each histogram bar represents a probability of observing a change in Performance of the real-time signal variance within a given 1 dB-wide interval after the regression across one experimental artifact regression procedure is run. D) Illustration of topographic properties of residual CB and random-motion artifacts after demonstrated in Figs. 3B,C,D. Fig. 3B the real-time procedure (A). Topographies of independent components modeling such artifacts illustrates reduction in variance of an for a typical experimental run are shown. Note additional suppression of the artifacts for channels F3 and F4 compared to the surrounding EEG channels. artifact-contaminated EEG signal after the regression for a typical determined from the ECG waveform, and a moving experimental run. Histograms of signal variance changes average over 21 epochs is subtracted from each channel’s across all task runs for all participants in both groups data. Note that the linear regression procedure attenuates (24×5 = 120 runs) are shown in Fig. 3C. The mean signal CB and random-motion artifacts without any assumptions variance reductions after the real-time regression across about their temporal periodicity. Therefore, the linear one run are 5.3 dB for F3 and 5.1 dB for F4. Larger regression and the AAS reduce CB artifacts variance reductions are observed for participants with independently, enabling more efficient real-time CB stronger heart beats (taller and sharper R peaks in the artifact suppression for channels F3 and F4. ECG) leading to stronger CB artifacts in the alpha and The linear regression coefficients {a }, {b }, i=1…4, i i beta EEG bands. Fig. 3D shows topographies of residual are determined as follows. After the real-time application CB and random-motion artifacts from an independent of the MRI Artifact Filter, the data are bandpass filtered in component analysis (ICA) applied offline to representative [5-35] Hz frequency range (96 dB/octave) using the single-run EEG data after the real-time artifact correction RecView Frequency Filter, and saved to a file. After each procedure (Fig. 3A). The inclusion of the real-time artifact experimental run, a MATLAB script is executed offline. It regression for channels F3 and F4 led to an additional suppression of residual CB and random-motion artifacts every 2 s for a moving data interval of 2.048 s duration for these two channels compared to surrounding EEG with Hann window. The relative alpha EEG asymmetry A channels (Fig. 3D), for which only AAS of MR and CB and the relative high-beta EEG asymmetry B were artifacts were performed. Therefore, the described calculated as described above. The A and B values were approach can present a practical, intuitive, and easy-to-use sent via a TCP/IP socket to the mGUI software, where alternative to more advanced methods, such as real-time they were processed along with the corresponding fMRI ICA (e.g. Mayeli et al., 2016). signal values using a separate software thread. For each Happy Memories condition, a change in A was determined 2.6. Real-time data processing as a difference between the current A value and the Implementation of the rtfMRI-EEG-nf in the present baseline obtained by averaging A values for the preceding study was similar to that in our previous work (Zotev et Rest condition block (Fig. 1B). A moving average of the al., 2014), except that two rtfMRI-nf signals and two current and two preceding A changes was computed. This EEG-nf signals were computed and displayed to a moving average (multiplied by 10) was used to set the participant at the same time (Fig. 1A). height of the magenta EEG-nf bar (Fig. 1A) every 2 s. A Each rtfMRI-EEG-nf experiment began with change in B for each Happy Memories condition was acquisition of a high-resolution MPRAGE anatomical calculated in the same way, and its moving average brain image, followed by acquisition of a short EPI dataset (multiplied by 10) was used to set the height of the purple (5 volumes). The last volume of the EPI dataset was EEG-nf bar on the screen (Fig. 1A). The real-time A and B employed as a reference EPI volume defining the subject’s values and changes are compared to the corresponding A individual EPI space. The MPRAGE image was and B values and changes, determined in the offline EEG transformed to the Talairach space, and this data analysis, in Supplementary material (S1.3, Fig. S2). transformation was used as a template to transform the LA 2.7. fMRI data analysis and L rACC target ROIs from the Talairach space (Figs. 2A,B) to the individual EPI space. The resulting ROIs in Offline analysis of the fMRI data was performed in the EPI space contained approximately 140 voxels each. AFNI as described in detail in Supplementary material During the subsequent EEG-fMRI runs, the real-time (S1.4). The analysis involved fMRI pre-processing with plugin in AFNI (Cox, 1996; Cox and Hyde, 1997) was despiking, cardiorespiratory artifact correction (Glover et used to perform volume registration (Cox and al., 2000), slice timing correction, and volume registration. Jesmanowicz, 1999) of each acquired EPI volume to the A general linear model (GLM) fMRI activation analysis reference EPI volume and export mean values of fMRI with Happy Memories and Count block-stimulus signals for these two ROIs in real time. These fMRI signal conditions was applied to the preprocessed fMRI data. values were sent to the mGUI software via a TCP/IP Average GLM-based fMRI percent signal changes for the socket (Fig. 1 in Zotev et al., 2014). An rtfMRI signal for Happy Memories vs Rest condition contrast and for the the LA target ROI was computed for each Happy Happy Memories vs Count contrast were computed for the Memories condition as a percent signal change with LA and L rACC target ROIs (Figs. 2A,B) and used to respect to the baseline obtained by averaging the LA fMRI characterize the rtfMRI-nf performance. For exploratory signal values for the preceding 40-s-long Rest condition analyses, the amygdala BOLD laterality and the middle block (Fig. 1B). A moving average of the current and two frontal gyrus (MidFG) BOLD laterality were computed as preceding LA rtfMRI signal values was computed to described in Supplementary material (S1.4). reduce effects of fMRI noise and physiological artifacts. Statistical results in the present study were corrected This moving average was used to set the height of the red for multiple comparisons by controlling the False rtfMRI-nf bar (Fig. 1A) every TR=2 s. An rtfMRI signal Discovery Rate (FDR q). In whole-brain fMRI and EEG- for the L rACC target ROI was calculated in the same fMRI analyses, FDR correction was applied voxel-wise. way, and its moving average was used to set the height of 2.8. fMRI-based PPI analysis the orange rtfMRI-nf bar (Fig. 1A). To evaluate changes in the left amygdala fMRI During each EEG-fMRI run, the real-time correction of functional connectivity between experimental conditions, MR, random-motion, and CB artifacts was performed in the RecView software as described above (Fig. 3A). The we conducted a psychophysiological interaction (PPI) analysis (Friston et al., 1997; Gitelman et al., 2003), as corrected EEG data were exported in real time as data described in detail in Supplementary material (S1.5). The blocks of 8 ms duration via a TCP/IP socket to the EEG processing modules of the EEG-fMRI data integration analysis was based on fMRI time course for the LA seed ROI. This time course was used to define two PPI software (Fig. 1 in Zotev et al., 2014). These modules were written in Python and utilized NumPy functions. regressors: the fMRI-based PPI correlation regressor and the fMRI-based PPI interaction regressor for the Happy FFT power spectra for channels F3 and F4 were computed Memories vs Rest condition contrast (S1.5). We selected corresponding PPI regressors based on the EEG power this contrast a priori, because our earlier analyses sum, mentioned above. A single-subject EEG-based PPI (unpublished) had shown that the PPI interaction effects analysis for each run involved fitting a GLM model with for the LA time course were more significant for the these two PPI regressors (in addition to other fMRI Happy Memories vs Rest contrast than for the Happy regressors, see S1.7). For FBA, the PPI regressors were Memories vs Count contrast. A single-subject fMRI-based defined in a similar way, starting with the FBA time PPI analysis for each run involved fitting a GLM model course. with these two PPI regressors (in addition to other fMRI regressors, see S1.5). 3. Results 3.1. Emotional state changes 2.9. EEG data analysis Offline analysis of the EEG data was performed using The MDD patients’ mood ratings before and after the rtfMRI-EEG-nf session and statistics for the rating BrainVision Analyzer 2.1 software (Brain Products, changes are reported in Table 1. Five mood ratings most GmbH) as described in detail in Supplementary material (S1.6). Removal of EEG artifacts was based on the AAS relevant to the present study – POMS depression, confusion, total mood disturbance, STAI state anxiety, and (Allen et al., 1998, 2000) and independent component analysis (Bell and Sejnowski, 1995), implemented in VAS happiness – are included in the table. There were no Analyzer 2.1. Time-frequency analysis with Morlet significant EG vs CG group differences in these ratings before the session. Significant improvements in the mood wavelets was used to compute EEG power as a function of time and frequency. The upper alpha EEG frequency band ratings with medium effect sizes (d=−0.62, −0.73, −0.60, −0.57, and +0.59, respectively) were observed after the was defined individually for each participant as [IAF, rtfMRI-EEG-nf session for the EG (Table 1). The IAF+2] Hz, where IAF is the individual alpha peak frequency. The IAF was determined by inspection of corresponding mood improvements for the CG were non- significant with small effect sizes (Table 1). The EG vs average EEG spectra for the occipital and parietal EEG CG group differences in the mood rating changes were not channels across the Rest condition blocks in the four nf runs (Fig. 1B). The normalized FAA was computed as significant. The MDD patients’ memory-recall and happiness FAA = ln(P(F4)) – ln(P(F3)), where P is EEG power as a ratings, reported verbally during the experiment, are function of time in the individual upper alpha EEG band [IAF, IAF+2] Hz for a given channel (F3 or F4). In included in Supplementary material (S2.1, Fig. S4). Both ratings were higher for the EG than for the CG, with the addition to the FAA, a power-sum function ln(P(F4)) + EG vs CG group differences trending toward significance ln(P(F3)) was calculated for the upper alpha band. Similarly, the normalized FBA was computed as FBA = after correction, with large effect sizes (d=0.99 and 0.82, respectively, S2.1). ln(P(F3)) – ln(P(F4)), where P is EEG power as a function of time in the high-beta frequency band [21-30] 3.2. Neurofeedback performance Hz. A power-sum function was calculated for the same The main rtfMRI-EEG-nf performance characteristics channels for the high-beta band. Average FAA and FBA for the EG participants are exhibited in Figure 4. The changes between the Rest and Happy Memories results were obtained in offline EEG and fMRI data conditions were used to characterize the EEG-nf analyses. To characterize nf performance across the entire performance. nf training, we averaged individual-subject results across 2.10. EEG-based PPI analyses the four nf runs (Practice, Run 1, Run 2, Run 3), and compared their group mean to zero using a one-sample t- To investigate how temporal correlations between test (two-tailed). The corresponding significance (p-value) FAA (or FBA) and BOLD activity changed between experimental conditions, we performed PPI analyses and effect size (Cohen’s d) are included at the bottom of each figure after the NF notation. adapted for EEG-fMRI (Zotev et al., 2014, 2016, 2018a), According to Fig. 4, the EG participants were able to as described in detail in Supplementary material (S1.7). The FAA time course was used to define two PPI significantly increase the FAA during the rtfMRI-EEG-nf task compared to the Rest condition (NF: t(15)=3.21, regressors: the FAA-based PPI correlation regressor and the FAA-based PPI interaction regressor for the Happy p<0.006), with large effect size (d=0.80, 95% CI [0.23 1.36]). They also significantly upregulated the FBA (NF: Memories vs Count condition contrast (S1.7, Fig. S3). We t(15)=2.71, p<0.016), with medium effect size (d=0.68, chose this contrast a priori as in our previous studies (Zotev et al., 2014, 2016) to compare EEG-fMRI 95% CI [0.12 1.21]). Furthermore, the EG participants significantly increased BOLD activity of the LA target correlations between two cognitive tasks. The FAA-based PPI regressors were orthogonalized with respect to the ROI during the rtfMRI-EEG-nf task relative to the Rest baseline (NF: t(15)=3.21, p<0.006), with large effect size (d=0.80, 95% CI [0.23 1.36]). They also exhibited significant enhancement in fMRI connectivity between the LA and the L rACC target ROI (NF: t(15)=2.49, p<0.025), with medium effect size (d=0.62, 95% CI [0.08 1.15]). The fMRI connectivity changes were computed as the LA-based PPI interaction effects (Sec. 2.8) and averaged within the L rACC ROI. The average results across the nf runs (NF) in Fig. 4 remained significant after the multiple comparisons correction to account for testing the four quantities (FDR q<0.012, 0.021, 0.012, 0.025, respectively). There were no Figure 4. Main performance characteristics of the rtfMRI-EEG-nf training for the experimental group (EG). Each bar represents a group mean of average individual results for a given run. The significant differences in these activity error bars are standard errors of the mean (sem). The experimental runs and condition blocks are measures between the last nf training depicted schematically in Fig. 1B. The NF at the bottom of each figure refers to group statistics run (Run 3) and the Transfer run (p-value from a t-test and effect size d, both relative to zero) for the individual results averaged without nf, indicating transfer of the across the four nf runs (PR, R1, R2, R3). A) Average changes in frontal alpha EEG asymmetry (FAA) between the Happy Memories and Rest conditions (H vs R). FAA = ln(P(F4)) − learning effects (TR vs R3: ln(P(F3)), where P is EEG power in the individual upper alpha band. B) Average changes in t(15)=−0.24, p<0.817 for the FAA frontal high-beta EEG asymmetry (FBA) between the same conditions. FBA = ln(P(F3)) − changes; t(15)=−1.31, p<0.211 for the ln(P(F4)), where P is EEG power in the high-beta band. C) Average fMRI percent signal changes FBA changes; t(15)=−0.76, p<0.462 for the LA target ROI (Fig. 2A) for the Happy Memories conditions with respect to the Rest for the LA activations; t(15)=0.09, baseline (H vs R). D) Average changes in fMRI functional connectivity between the LA and L rACC target ROI (Fig. 2B) during the Happy Memories conditions compared to the Rest p<0.931 for the L rACC vs LA conditions (H vs R, psychophysiological interaction effect). connectivity changes). Figure 5 exhibits the corresponding activity measures for the CG. The average results across the four nf runs were non-significant with negative effects (Fig. 5). Importantly, the EG vs CG group differences either were significant or trended toward significance before correction (NF, EG vs CG: t(22)=2.14, p<0.044, d=0.93 for the FAA changes; t(22)=2.38, p<0.027, d=1.03 for the FBA changes; t(22)=1.84, p<0.080, d=0.79 for the LA activations; t(22)=2.82, p<0.010, d=1.22 for the L rACC vs LA connectivity changes). These group differences trended toward significance or remained significant after the multiple comparisons correction (FDR q<0.059, 0.054, 0.080, 0.040, Figure 5. Main performance characteristics for the control group (CG). Notations are the same respectively). as in Fig. 4. Figure 6. Average fMRI percent signal changes for the L rACC target ROI for the Happy Memories vs Rest condition contrast (H vs R) and for the Happy Memories vs Count condition contrast (H vs C). The black straight segments are linear fits across the four nf runs (PR, R1, R2, R3). The NF at the bottom of each figure refers to group statistics (p-value from a t-test and effect size d, both relative to zero) for the individual activity levels, corresponding to the H vs C contrast, averaged across the four nf runs. A) Results for the experimental group (EG). B) Results for the control group (CG). Figure 6 shows average fMRI percent signal changes relative to the Rest condition (H vs R), averaged across for the L rACC target ROI (Fig. 2B) for the Happy the four nf runs, showed significant positive correlations Memories vs Rest condition contrast (H vs R) and for the with the MDD patients’ MADRS depression severity Happy Memories vs Count condition contrast (H vs C) for ratings (r=0.52, p<0.039) and SHAPS anhedonia severity each group. Statistical analyses for these results are ratings (r=0.71, p<0.002). The corresponding correlation exploratory, because no hypotheses were made about with the HDRS depression severity ratings trended toward mean L rACC activations. The average L rACC activity significance (r=0.45, p<0.080). Among the four nf runs, levels during the rtfMRI-EEG-nf task across the four nf the correlation between the FAA changes and the MADRS runs for the EG were non-significant when compared to depression ratings was most pronounced for the Practice the Rest baseline (H vs R, NF: t(15)=−1.13, p<0.278, run (PR: r=0.65, p<0.007), i.e. at the beginning of the nf d=−0.28), and were significant when compared to the training. Count control task (H vs C, NF: t(15)=3.32, p<0.005, The same average FAA changes exhibited significant d=0.83). The EG vs CG group differences were not negative correlations with the after-vs-before changes in significant for either contrast. Interestingly, the L rACC POMS state depression ratings (r=−0.56, p<0.023) and activity levels showed positive linear trends across the POMS total mood disturbance ratings (r=−0.55, p<0.028). four nf runs for both contrasts for the EG (Fig. 6A). We The average FAA changes also showed significant computed a slope of a linear fit to the individual L rACC negative correlations with changes in POMS confusion activity levels across the four nf runs for each participant. ratings (r=−0.51, p<0.044) and STAI state anxiety ratings For the EG, the mean linear slopes were positive and (r=−0.50, p<0.049). Among the four nf runs, the negative trended toward significance for both contrasts (H vs R, correlation between the FAA changes and POMS slope: t(15)=2.12, p<0.051, d=0.53; H vs C, slope: depression changes was most pronounced for Run 3 (R3: t(15)=1.75, p<0.099, d=0.44). The EG vs CG group r=−0.64, p<0.008), i.e. at the end of the nf training. difference in the linear slopes was significant, with large All the correlation results in Fig. 7 remained significant effect size, for the Happy Memories vs Count contrast (H when they were controlled for the EG participants’ age vs C, slope difference: t(22)=2.41, p<0.025, d=1.04). and gender. For the CG, the correlations corresponding to those illustrated in Fig. 7 were not significant. 3.3. FAA changes vs psychological measures 3.4. Prefrontal BOLD laterality We conducted exploratory correlation analyses to examine associations between the target activity measures Whole-brain statistical maps of BOLD fMRI activity during the rtfMRI-EEG-nf training and individual during the rtfMRI-EEG-nf training for the EG are reported psychological metrics relevant to MDD. Significant in Supplementary material (S2.2, Fig. S5, Table S2). The associations were found only for the FAA changes. results demonstrate significant positive Happy Memories Results for the EG are exhibited in Figure 7. Correlations vs Count BOLD activity contrast for the left amygdala with trait measures are shown in Fig. 7A, and correlations region and many areas of the limbic system. They also with changes in state measures (after vs before the reveal pronounced BOLD laterality for parts of the MidFG session) are included in Fig. 7B. and superior frontal gyrus (SFG) (Fig. S5, Table S2). The FAA changes during the rtfMRI-EEG-nf task Figure 7. Correlations between changes in frontal alpha EEG asymmetry (FAA) during the rtfMRI-EEG-nf training and individual psychological measures. The individual FAA changes between the Rest and Happy Memories with rtfMRI-EEG-nf conditions (H vs R) were averaged across the four nf runs (PR, R1, R2, R3). The results are for the experimental group (EG), with each data point corresponding to one participant. A) Correlations between the FAA changes and severities of depression and anhedonia, assessed before the session. B) Correlations between the FAA changes and changes in state depression and total mood disturbance. The state measures were assessed both before and after the session, and their changes (after vs before) are included in the figures. Acronyms: MADRS – Montgomery-Asberg Depression Rating Scale, SHAPS – Snaith-Hamilton Pleasure Scale, POMS – Profile of Mood States. Figure 8. Illustration of BOLD laterality for middle frontal gyrus (MidFG) regions during the rtfMRI-EEG-nf training for the experimental group (EG) and its associations with individual psychological measures. A) The left and right MidFG ROIs as seen on the cortical surface. The ROIs were defined as described in the text. B) Average fMRI activity levels (percent signal changes) for the left and right MidFG ROIs, corresponding to the Happy Memories vs Rest condition contrast (H vs R) and to the Happy Memories vs Count contrast (H vs C). The individual fMRI activity levels were averaged across the four nf runs (PR, R1, R2, R3). C) Left: Correlation between the average individual MidFG laterality, i.e. the difference in fMRI activities between the left and right MidFG ROIs, for the Happy Memories vs Count contrast and after-vs-before changes in state depression. Right: Correlation between the average individual MidFG laterality and changes in total mood disturbance. POMS – Profile of Mood States. Whole-brain statistical maps for the EG vs CG group difference in the LA fMRI connectivity changes during the rtfMRI-EEG-nf training are reported in Supplementary material (S2.3, Fig. S6, Table S3). The maps revealed three loci in the rACC area, characterized by the most pronounced EG vs CG group differences (S2.3): (−8, 34, 7), (−9, 41, 5), and (3, 35, 9). All three loci were in close proximity to the center of the L rACC target ROI at (−3, 34, 5). For 10-mm-diameter ROIs, centered at these Figure 9. Correlations between changes in fMRI functional connectivity of the left loci, the EG vs CG group differences in fMRI amygdala with the left rACC during the rtfMRI-EEG-nf training and individual psychological measures. The fMRI connectivity changes for the target ROIs between connectivity changes with the LA were the Rest and Happy Memories with rtfMRI-EEG-nf conditions (H vs R, significant with large effect sizes (S2.3). psychophysiological interaction effect) are reported. The results are for the We performed exploratory correlation experimental group (EG), with each data point corresponding to one participant analyses to examine associations between the (however, n=13 for the BAS). The individual results were averaged across the four nf average changes in fMRI connectivity between runs (PR, R1, R2, R3). MADRS – Montgomery-Asberg Depression Rating Scale, BAS – Behavioral Activation System scale. the LA and the L rACC target ROI during the rtfMRI-EEG-nf training and individual We performed exploratory analyses of prefrontal psychological measures relevant to MDD. Figure 9 shows BOLD laterality to evaluate effects of the EEG-nf, such associations for the EG. The fMRI connectivity targeting the FAA and FBA, independently of the EEG changes for the Happy Memories condition relative to the data analysis. Results for the EG are exhibited in Figure 8. Rest condition (H vs R) were averaged across the four nf Because EEG electrodes F3 and F4 are situated above the runs. These changes exhibited negative and trending MidFG, we considered the left and right MidFG ROIs, toward significance correlation with the MADRS defined as described in Supplementary material (S1.4). depression severity ratings (r=−0.46, p<0.075) and Following the laterality pattern in Fig. S5, we limited the positive correlation with the BAS reward responsiveness ROIs to 42 ≤ z ≤ 57 mm. The resulting left and right ratings (r=0.52, p<0.070). For the CG, the corresponding MidFG ROIs are depicted in Fig. 8A. Mean BOLD correlations were not significant. activity levels for these ROIs, with individual results 3.6. EEG-fMRI correlations for the left amygdala averaged across the four nf runs, are shown in Fig. 8B. These activity levels were significantly higher for the left Figure 10 reports changes in temporal correlations MidFG ROI than for the right MidFG ROI (based on between time courses of the FAA and FBA and the LA paired t-test) both for the Happy Memories vs Rest BOLD activity during the rtfMRI-EEG-nf training. The condition contrast (H vs R: t(15)=3.28, p<0.005, d=0.82) correlation changes were computed as FAA-based or and for the Happy Memories vs Count contrast (H vs C: FBA-based PPI interaction effects (Sec. 2.10) and t(15)=4.46, p<0.0005, d=1.11). averaged within the LA ROI. We also conducted exploratory correlation analyses to Fig. 10A demonstrates that temporal correlation evaluate associations between the MidFG BOLD laterality between the FAA and the LA BOLD activity was during the rtfMRI-EEG-nf training and mood rating significantly enhanced during the rtfMRI-EEG-nf task changes. Results for the EG are shown in Fig. 8C. The compared to the Count condition for the EG (NF: average MidFG BOLD laterality for the Happy Memories t(15)=2.81, p<0.013), with medium effect size (d=0.70, vs Count contrast exhibited significant negative 95% CI [0.14 1.24]). There was no significant difference correlations with the after-vs-before changes in POMS in the mean PPI interaction effects between Run 3 and the state depression ratings (r=−0.58, p<0.019) and POMS Transfer run (TR vs R3: t(15)=0.22, p<0.833). Similarly, total mood disturbance ratings (r=−0.65, p<0.006). For the Fig. 10B indicates significant enhancement in temporal Happy Memories vs Rest contrast, the laterality also correlation between the FBA and the LA BOLD activity showed negative correlations with changes in these ratings during the rtfMRI-EEG-nf task for the EG (NF: (H vs R, POMS state depression change: r=−0.48, t(15)=2.51, p<0.024), with medium effect size (d=0.63, p<0.061; POMS total mood disturbance change: r=−0.57, 95% CI [0.08 1.16]). However, the PPI effect was p<0.020). For the CG, the results corresponding to those negligible during the Transfer run in this case. For the CG, in Figs. 8B,C were not significant. the FAA-based PPI interaction effect was negative yet small (Fig. 10C), while the FBA-based PPI interaction 3.5. Amygdala-rACC connectivity enhancement effect was negative with large effect size (Fig. 10D). target ROI was positive and trended toward significance for the EG (NF: t(15)=2.03, p<0.061, d=0.51). The EG vs CG group difference in the FAA- based PPI interaction effects for the L rACC ROI also trended toward significance (t(22)=1.86, p<0.076, d=0.81). 3.7. EEG-fMRI correlations across the brain Figure 11 exhibits whole-brain statistical maps for the FAA-based PPI interaction effect corresponding to the Happy Memories vs Count condition contrast for the EG. The PPI interaction results from twelve EG participants were included in the group analysis. The other four cases were considered outliers based on the low amygdala BOLD laterality (Fig. Figure 10. Changes in temporal correlations between frontal EEG asymmetries and BOLD fMRI activity of the left amygdala during the rtfMRI-EEG-nf training. The correlation changes S7). For each participant, the PPI between Happy Memories and Count conditions (H vs C) were evaluated in EEG-fMRI interaction maps were averaged for psychophysiological interaction (PPI) analyses, based on time courses of frontal alpha EEG three nf runs (out of four) with the asymmetry (FAA) and frontal high-beta EEG asymmetry (FBA). Each bar represents a group most positive individual amygdala mean of the individual PPI interaction values for a given run, averaged within the LA ROI. The BOLD laterality values. Statistical error bars are standard errors of the mean (sem). The NF at the bottom of each figure refers to group statistics (p-value from a t-test and effect size d, both relative to zero) for the individual results for the FAA-based PPI results averaged across the four nf runs (PR, R1, R2, R3). A) Average values of the FAA-based interaction effect are summarized in PPI interaction effect for the LA ROI for the experimental group (EG). B) Average values of the Table 2. The maps in Fig. 11 are FDR FBA-based PPI interaction effect for the LA ROI for the EG. C) Average values of the FAA- corrected with q<0.04 threshold, and based PPI interaction effect for the control group (CG). D) Average values of the FBA-based PPI interaction effect for the CG. the data in Table 2 – with q<0.02 threshold. Whole-brain statistical Importantly, the EG vs CG group differences in the PPI maps for the FBA-based PPI interaction effect, obtained in interaction effects for the LA ROI were significant with a similar way for the same contrast for the EG, are large effect sizes for both the FAA (t(22)=2.32, p<0.030, reported in Supplementary material (S2.5, Fig. S8, Table d=1.00) and the FBA (t(22)=3.90, p<0.001, d=1.69). S4). Average individual values of the FAA-based PPI Figure 12 compares the FAA- and FBA-based PPI interaction effect for the LA ROI across the four nf runs interaction effects from Fig. 11 and Fig. S8 for the left for the EG exhibited significant positive correlation with amygdala and its vicinity. The PPI results demonstrate the corresponding average values of the amygdala BOLD that both the FAA and FBA exhibited enhanced temporal laterality (NF: r=0.53, p<0.035), as illustrated in correlations, during the rtfMRI-EEG-nf task, with BOLD Supplementary Fig. S7A. When the individual results activities of the left amygdala and large brain networks. were averaged across three nf runs (out of four) These results are discussed in detail below. characterized by the most positive amygdala BOLD laterality values, the correlation was more significant 4. Discussion (NF*: r=0.61, p<0.012), as shown in Fig. S7B. The In this paper, we reported the first, proof-of-concept amygdala BOLD laterality (‘LA−RA’) was computed as application of the simultaneous real-time fMRI and EEG described in Supplementary material (S1.4). It can be neurofeedback (rtfMRI-EEG-nf) for emotion self- viewed as a performance measure characterizing target- regulation training in patients with a neuropsychiatric specific effects of the LA-based rtfMRI-nf procedure, as disorder, specifically, major depressive disorder (MDD). explained in Supplementary material (S2.4, see also Zotev This is also the first neurofeedback study in which et al., 2016). participants had an opportunity to simultaneously regulate Consistent with the results for the LA in Fig. 10A, the two rtfMRI-nf signals and two EEG-nf signals. average FAA-based PPI interaction effect for the L rACC Figure 11. Enhancement in temporal correlation between frontal alpha EEG asymmetry (FAA) and BOLD activity during the rtfMRI-EEG-nf training. Statistical maps of the FAA-based PPI interaction effect for the Happy Memories vs Count condition contrast (H vs C) are shown for the experimental group (EG). The maps are voxel-wise FDR corrected and projected onto the standard TT_N27 anatomical template, with 3 mm separation between axial slices. The number adjacent to each slice indicates the z coordinate in mm. The left hemisphere (L) is to the reader’s right. Peak t-statistics values for the FAA-based PPI interaction effect and the corresponding locations are specified in Table 2. Furthermore, we implemented the advanced real-time 4.2. Neurofeedback performance EEG-fMRI artifact correction procedure that made EEG- nf during fMRI practical and efficient. During the rtfMRI-EEG-nf training, the MDD patients in the EG learned to significantly increase BOLD activity 4.1. Emotional state changes of the LA (Fig. 4C) and significantly upregulate the FAA Following the rtfMRI-EEG-nf session, the MDD (Figs. 4A). These are the primary outcome measures in patients in the EG showed significant mood our study. The EG participants also achieved significant improvements, including significant reductions in state enhancement in fMRI connectivity between the LA and depression, confusion, total mood disturbance, and state the L rACC (Fig. 4D) and significant upregulation of the anxiety, as well as significant increase in state happiness FBA (Fig. 4B), which are the secondary outcome (Table 1). These improvements, characterized by medium measures. These results support the first of the two main hypotheses in our study (Sec. 1). Importantly, the EG vs effect sizes, suggest that the rtfMRI-EEG-nf training may be beneficial to MDD patients. The significant reduction CG group differences in these four measures either were in confusion may indicate that the EG participants were significant or trended toward significance (Sec. 3.2), able to develop a better grasp of the experimental indicating that effects of the rtfMRI-EEG-nf were specific procedure as the training continued, despite the relative to the EG and different from those of the sham feedback complexity of the rtfMRI-EEG-nf task. During the for the CG. The left rACC activity levels exhibited experiment, the EG participants rated their abilities to positive linear trends across the four nf runs for the EG recall autobiographical memories and feel happiness (Fig. 6). higher, than the CG participants (Supplementary Fig. S4). In our previous study, which combined rtfMRI-nf of These group differences, with large effect sizes, suggest the LA activity with passive EEG (Zotev et al., 2016), the that the rtfMRI-EEG-nf provided information more effect size for the LA activation was large (d=0.87, 95% consistent with the Happy Memories task than the sham CI [0.21 1.50]), while the effect size for the associated feedback. FAA increase was small (d=0.45, 95% CI [−0.14 1.01]). In the present work, both effects sizes were large (d=0.80, 4.4. Prefrontal BOLD laterality Performance of the rtfMRI-EEG-nf task was associated with pronounced laterality of BOLD activations for the dorsal PFC regions (Supplementary Fig. S5). BOLD activity levels during the rtfMRI-EEG-nf task (relative to the Count condition) were more positive for the MidFG and SFG areas on the left, compared to the corresponding MidFG and SFG areas on the right (Fig. S5). The most positive BOLD contrast t-score was observed for the left MidFG (BA 8) at (−40, 20, 47), while the most negative contrast t-score occurred in the right SFG (BA 8) at (27, Figure 12. Comparison of the FAA- and FBA-based PPI interaction effects for the left amygdala region. A) FAA-based PPI interaction 17, 49) (Table S2). These locations are parts of the left effects from Fig. 11. B) FBA-based PPI interaction effects from and right DLPFC, respectively. Furthermore, EEG Supplementary Fig. S8. The green crosshairs (x=−21 mm, z=−16 electrodes F3 and F4 are situated above the MidFG and mm) correspond to the center of the LA target ROI. BA 8. Thus, the significant positive MidFG BOLD laterality, illustrated in Fig. 8B, is consistent with the 95% CI [0.23 1.36], Figs. 4A,C). This comparison (which significant positive FAA and FBA changes for channels should be taken with caution because of the wide F3 and F4 (Figs. 4A,B). The negative correlations confidence intervals) suggests potential benefits of the between the average MidFG BOLD laterality values and rtfMRI-EEG-nf compared to the rtfMRI-nf alone. the after-vs-before changes in the mood ratings (Fig. 8C, 4.3. FAA changes vs psychological measures Sec. 3.4) are consistent with the negative correlations between the average FAA changes and the same mood The average individual FAA changes during the rating changes (Fig. 7B). Therefore, the prefrontal BOLD rtfMRI-EEG-nf task for the EG showed significant laterality effects independently confirm the EEG positive correlations with the MDD patients’ depression asymmetry effects observed in the EEG data analyses. and anhedonia severities (Fig. 7A). These findings are consistent with those reported in our study of EEG 4.5. Amygdala-rACC connectivity enhancement correlates of the amygdala rtfMRI-nf (Zotev et al., 2016). Modulation of the left rACC BOLD activity MDD patients exhibit lower FAA levels compared to non- simultaneously with that of the LA (Fig. 1A) enabled (or, depressed individuals, particularly during an emotional at least, was consistent with) enhancement in the two challenge (Stewart et al., 2011, 2014). The more positive regions’ fMRI functional connectivity (Fig. 4D). This FAA changes during the rtfMRI-EEG-nf task in the effect was specific to the rtfMRI-EEG-nf, as evidenced by patients with more severe depression suggest the potential the significant EG vs CG group differences in fMRI for correction of the FAA deficiencies specific to MDD connectivities of the LA with three loci close to the center (Zotev et al., 2016). This reasoning is explained in more of the L rACC target ROI (Supplementary Fig. S6B). In detail in the follow-up study (Zotev and Bodurka, 2020). future studies, an rtfMRI-nf based on an actual fMRI It is supported by the observed mood improvements: the connectivity metric, such as the Pearson’s correlation MDD patients, who achieved more positive average FAA coefficient, could be implemented and used together with changes during the rtfMRI-EEG-nf task, showed stronger the rtfMRI-nf of the amygdala activity. The enhancement reductions in state depression and total mood disturbance in fMRI connectivity between the LA and the left rACC after the training (Fig. 7B). during the rtfMRI-EEG-nf task for the EG showed The positive correlations between the FAA changes negative association with the MDD patients’ depression and the depression and anhedonia severities in Fig. 7A are severity and positive association with reward not as pronounced as those in our previous work (Zotev et responsiveness (Fig. 9). This means that a stronger al., 2016). The primary reason is that the FAA in the interaction between these two regions during positive present study was explicitly modulated via the EEG-nf. emotion induction with rtfMRI-EEG-nf should be Consequently, the FAA changes depended on individual beneficial to MDD patients. This observation is consistent EEG-nf performance, including a participant’s attention to with results of the previous studies that emphasized the the FAA-based EEG-nf signal and effort to regulate it. important role of the rACC in emotion regulation and Indeed, the positive correlation between the FAA changes modulation on the amygdala activity (e.g. Etkin et al., and the depression severity was most pronounced for the 2006; Pizzagalli, 2011; Zotev et al., 2013). Practice run, when the participants were first exposed to the rtfMRI-EEG-nf, and became weaker as the training 4.6. EEG-fMRI correlations for the left amygdala continued (Sec. 3.3). The FAA-based PPI interaction results for the LA ROI The results in our study demonstrate involvement of (Fig. 10A) demonstrate that temporal correlation between the left premotor cortex (PMC), specifically the precentral the FAA time course (convolved with the HRF) and the gyrus (PrecG), BA 6 in performance of the rtfMRI-EEG- LA BOLD activity was significantly stronger during the nf task. A local maximum of the FAA-based PPI rtfMRI-EEG-nf task than during the control condition. interaction effect is observed near the border of the left Similarly, the FBA-based PPI interaction results (Fig. PrecG and MidFG at (−42, −4, 44) (Fig. 11, Table 2). This 10B) indicate significant enhancement in temporal locus is relatively close to the location of the maximum of correlation between the FBA time course and the LA the corresponding BOLD activity contrast in the left BOLD activity. These EEG-based PPI interactions had MidFG at (−40, 20, 47) (Fig. S5, Table S2), which is also medium effect sizes for the EG, and large effect sizes near the anterior boundary of the PrecG. Furthermore, the when compared to those for the CG (Sec. 3.6). Therefore, EG vs CG group difference in the LA connectivity the EG participants were able, on the average, to enhancement during the rtfMRI-EEG-nf task was upregulate the FAA and FBA together with the LA prominent in nearby regions of the left MidFG at (−53, 8, activity. These results support the second of the two main 36) and PrecG at (−51, 1, 33) (Fig. S6A, Table S3). hypotheses in our study (Sec. 1). Collectively, these findings point to mutually consistent Interestingly, the average individual FAA-based PPI roles of the left DLPFC and the adjacent area of the left interaction effects for the LA showed significant positive PMC during the rtfMRI-EEG-nf task. correlation with the average individual amygdala BOLD The last observation is not surprising, because the laterality (Supplementary material S2.4, Fig. S7). This anterior (rostral) PMC has strong interconnections with means that the EG participants, who were more successful the prefrontal cortex (e.g. Chouinard and Paus, 2006; at upregulating BOLD activity of the target amygdala Hanakawa et al., 2003). A recent meta-analysis of rtfMRI- region (LA) relative to the non-target region (RA), were nf studies with various target regions revealed consistent also more successful at doing so simultaneously with fMRI activations of bilateral DLPFC areas extending to increasing the FAA-based EEG-nf signal. This PMC (Emmert et al., 2016). We hypothesize that the left observation confirms the connection between the FAA DLPFC in our study is involved in mental strategy changes and the amygdala BOLD laterality we reported implementation, while the left PMC is involved in previously (Zotev et al. 2016). observation and control of the variable-height nf bars. From this point of view, the enhanced temporal 4.7. fMRI correlates of the FAA modulation correlation between the FAA and the left PMC activity The whole-brain maps of the FAA-based PPI (Fig. 11, Table 2) suggests that the FAA modulation was interaction effect (Fig. 11) demonstrate that temporal closely associated with direct regulation of the FAA-based correlation between the FAA and BOLD activity was EEG-nf signal. significantly enhanced, during the rtfMRI-EEG-nf task for Interestingly, resting-state EEG source imaging studies the EG, not only for the left amygdala, but also for the have suggested that motivation is related to activities of large brain network. Note that the FAA- and FBA-based both the DLPFC and the PMC. Stronger reward bias in PPI regressors were orthogonalized with respect to the healthy individuals is associated with reduced upper alpha corresponding regressors based on the EEG power sums (alpha2) EEG activity (i.e. stronger activation) in the left for channels F3 and F4. Therefore, a positive PPI MidFG, left SFG, and left PrecG (BA 6) (Pizzagalli et al., interaction effect for a given region means that its BOLD 2005). In MDD patients, resting alpha EEG source activity increased simultaneously with activation of laterality index shows negative correlations with cortical areas contributing to EEG signal measured by F3, depression severity for both the MidFG and the PrecG and decreased simultaneously with deactivation of areas regions (Smith et al., 2018). It is suggested that the PMC contributing to EEG signal measured by F4. activity may “facilitate mobilization of the body for The results in Fig. 11 show significant positive PPI approach-motivated behaviors” (Smith et al., 2018). In the interaction effects for the corresponding left and right hierarchical model of approach/avoidance motivation by DLPFC regions (MidFG, BA 9), with maxima at (−36, 16, Sprielberg et al., 2013, the left DLPFC instantiates 24) and (49, 18, 24), respectively (Fig. 11, Table 2). These approach motivation at the strategic level, while the left results are consistent with the common view of frontal PMC subserves it at the tactical level. EEG asymmetry as reflecting activation of the left DLPFC In the left amygdala area, the main statistical maximum and deactivation of the right DLPFC (and vice versa). for the FAA-based PPI interaction effect is observed in the Within the approach-avoidance framework, these effects superficial (SF) subdivision of the amygdala at (−17, −5, are interpreted as indicative of enhanced approach −17) (Fig. 12A, Table 2). This finding is consistent with motivation and reduced avoidance motivation, that in our previous study (Zotev et al., 2016), which respectively. showed that the same PPI effect had the maximum in the SF subdivision at (−17, −3, −16) (Table 2 therein). Two maximize simultaneous engagement of both fMRI and additional maxima are found in the laterobasal (LB) EEG target brain activities. Second, the study participants amygdala subdivision at (−21, −6, −19) and (−21, −6, had, on the average, moderate depression (Table S1). −11) (Fig. 12A, Table 2). Compared to the LB, the SF Recruitment of more unmedicated MDD patients with subdivision is more closely involved in processing severe depression will help to elucidate effects of the reward-related and socially relevant information, as well rtfMRI-EEG-nf that are specific to MDD. Third, the sham as in modulation of approach-avoidance behavior (Bzdok feedback signals, provided to the control group et al., 2013). participants, were computer generated and unrelated to brain activity. Further research on rtfMRI-EEG-nf will 4.8. fMRI correlates of the FBA modulation benefit from a more realistic sham feedback, utilizing The FBA-based PPI interaction effects are most actual real-time fMRI and EEG data. pronounced along the cortical midline and the cingulate gyrus (Supplementary material S2.5, Fig. S8, Table S4). 5. Conclusion Elevated high-beta activity in these areas, often with some Our simultaneous real-time fMRI and EEG lateralization to the right, is associated with anxiety (e.g. neurofeedback (rtfMRI-EEG-nf) procedure provided Zotev and Bodurka, 2020, and references therein). proof-of-concept demonstration of intended target Significant FBA-based PPI interaction effects are also engagements and modulatory effects on recruited brain found for many regions involved in autobiographical circuitry dynamics. Furthermore, we observed enhanced memory retrieval, including the hippocampus, the temporal correlations of the target EEG and fMRI activity extended areas of the parahippocampal gyrus, the anterior measures, clearly indicating the ability of both thalamus, the precuneus, the posterior cingulate, the neurofeedback modalities to capture common aspects of lingual gyrus (involved in visual memory), and others neuronal activity. In our opinion, the rtfMRI-EEG-nf is (Fig. S8, Table S4). For the PMC areas, the PPI effects are worth implementation efforts, because it is a powerful less pronounced than those for the FAA. These findings approach to influence brain activity in a more suggest that the FBA modulation during the rtfMRI-EEG- experimentally controllable fashion and investigate nf task might have been more closely associated with resulting changes in spatial and temporal brain dynamics. variations in anxiety and activity of the autobiographical Our study suggests that the rtfMRI-EEG-nf can benefit memory system than with direct regulation of the FBA- depressed individuals and may have potential for based EEG-nf signal. Indeed, high-beta EEG activity is treatment of MDD. The described rtfMRI-EEG-nf relevant to the autobiographical memory function and implementation with two rtfMRI-nf and two EEG-nf limbic functions in general (e.g. Cannon et al., 2005; signals is an advanced and versatile neuromodulation tool. Paquette et al., 2009). Efficient mental strategies and imaginative experimental In the left amygdala region, the main statistical designs will be needed to take full advantage of the maximum for the FBA-based PPI interaction effect is opportunities it offers. Ultimately, effectiveness of the observed in the LB amygdala subdivision at (−28, −5, rtfMRI-EEG-nf compared to either of the individual −10) (Fig. 12B, Table S4). This result is consistent with neurofeedback modalities will have to be demonstrated. that in our previous work (Zotev et al., 2014), which showed that the same PPI effect was more pronounced in Conflict of interest the LB subdivision of the left amygdala (Fig. 4 therein). The authors declare that the research was conducted in Therefore, while the FAA temporal variations during the the absence of any commercial or financial relationships rtfMRI-EEG-nf task exhibited enhanced correlations with that could be construed as a potential conflict of interest. BOLD activities of both the SF and LB amygdala Funding subdivisions, the FBA variations showed enhanced correlation mainly with activity of the LB subdivision. This work was supported by the Laureate Institute for Brain Research and the William K. Warren Foundation, 4.9. Study limitations and in part by the P20 GM121312 award from National The reported study has several limitations. First, the Institute of General Medical Sciences, National Institutes experimental protocol with happy emotion induction of Health. based on recall of happy autobiographical memories was Acknowledgments adopted from our earlier studies on the amygdala rtfMRI- nf (Zotev et al., 2011; also Young et al., 2014), and was We would like to thank Dr. Tracy Warbrick and Dr. not optimized for the rtfMRI-EEG-nf. In future studies, Brett Bays of Brain Products, GmbH for their continued mental strategies and training procedures will have to be help, inspired teaching, and excellent technical support. developed specifically for the rtfMRI-EEG-nf to D.E.J., et al., 2016. Meta-analysis of real-time fMRI neurofeedback References studies using individual participant data: how is brain regulation Allen, J.J.B., Harmon-Jones, E., Cavender, J.H. (2001). Manipulation mediated? NeuroImage 124, 806-812. of frontal EEG asymmetry through biofeedback alters self-reported Etkin, A., Egner, T., Peraza, D.M., Kandel, E.R., Hirsch, J., 2006. emotional responses and facial EMG. Psychophysiology, 38, 685- Resolving emotional conflict: a role for the rostral anterior cingulate cortex in modulating activity in the amygdala. Neuron 51, Allen, P.J., Polizzi, G., Krakow, K., Fish, D.R., Lemieux, L., 1998. 871-882. Identification of EEG events in the MRI scanner: the problem of Friston, K.J., Buechel, C., Fink, G.R., Morris, J., Rolls, E., Dolan, R.J., pulse artifact and a method for its subtraction. NeuroImage 8, 229- 1997. Psychophysiological and modulatory interactions in neuroimaging. NeuroImage 6, 218-229. Allen, P.J., Josephs, O., Turner, R., 2000. A method for removing Gaume, A., Vialatte, A., Mora-Sánchez, A., Ramdani, C., Vialatte, imaging artifact from continuous EEG recorded during functional F.B., 2016. A psychoengineering paradigm for the neurocognitive MRI. NeuroImage 12, 230-239. mechanisms of biofeedback and neurofeedback. Neuroscience and American Psychiatric Association, 2000. Diagnostic and Statistical Biobehavioral Reviews 68, 891-910. Manual of Mental Disorders, 4th ed. Text Rev. (DSM-IV-TR). Gitelman, D.R., Penny, W.D., Ashburner, J., Friston, K.J., 2003. American Psychiatric Press, Washington, DC. Modeling regional and psychophysiologic interactions in fMRI: the Anand, A., Li, Y., Wang, Y., Wu, J., Gao, S., Bukhari, L., et al., 2005. importance of hemodynamic deconvolution. NeuroImage 19, 200- Antidepressant effect on connectivity of the mood-regulating circuit: an fMRI study. Neuropsychopharmacology 30, 1334-1344. Glover, G.H., Li, T.Q., Ress, D., 2000. Image-based method for Bagby, R.M., Parker, J.D.A., Taylor, G.J., 1994. The twenty-item retrospective correction of physiological motion effects in fMRI: Toronto Alexithymia Scale – I. Item selection and cross-validation RETROICOR. Magnetic Resonance in Medicine 44, 162-167. of the factor structure. Journal of Psychosomatic Research 38, 23- Hamilton, M., 1959. The assessment of anxiety states by rating. British Journal of Medical Psychology 32, 50-55. Baehr, E., Rosenfeld, J.P., Baehr, R., 1997. The clinical use of an alpha Hamilton, M., 1960. A rating scale for depression. Journal of asymmetry protocol in the neurofeedback treatment of depression: Neurology, Neurosurgery, and Psychiatry 23, 56-62. two case studies. Journal of Neurotherapy 2, 10-23. Hanakawa, T., Immisch, I., Toma, K., Dimyan, M.A., van Gelderen, P., Bell, A.J., Sejnowski, T.J., 1995. An information-maximization Hallett, M., 2003. Functional properties of brain areas associated approach to blind separation and blind deconvolution. Neural with motor execution and imagery. Journal of Neurophysiology 89, Computation 7, 1129-1159. 989-1002. Bzdok, D., Laird, A.R., Zilles, K., Fox, P.T., Eickhoff, S.B., 2013. An Harmon-Jones, E., Gable, P.A., 2018. On the role of asymmetric frontal investigation of the structural, connectional, and functional cortical activity in approach and withdrawal motivation: un updated subspecialization in the human amygdala. Human Brain Mapping review of the evidence. Psychophysiology 55, e12879. 34, 3247-3266. Mano, M., Lécuyer, A., Bannier, E., Perronnet, L., Noorzadeh, S., Cannon, R., Lubar, J., Thornton, K., Wilson, S., Congedo, M., 2005. Barillot, C., 2017. How to build a hybrid neurofeedback platform Limbic beta activation and LORETA: can hippocampal and related combining EEG and fMRI. Frontiers in Neuroscience 11, 140. limbic activity be recorded and changes visualized using LORETA Masterton, R.A.J., Abbott, D.F., Fleming, S.W., Jackson, G.D., 2007. in an affective memory condition? Journal of Neurotherapy 8, 5-24. Measurement and reduction of motion and ballistocardiogram Carver, C.S., White, T.L., 1994. Behavioral inhibition, behavioral artefacts from simultaneous EEG and fMRI recordings. activation, and affective responses to impending reward and NeuroImage 37, 202-211. punishment: the BIS/BAS scales. Journal of Personality and Social Mayberg, H.S., Brannan, S.K., Mahurin, R.K., Jerabek, P.A., Psychology 67, 319-333. Brickman, J.S., Tekell, J.L., et al., 1997. Cingulate function in Cavazza, M., Aranyi, G., Charles, F., Porteous, J., Gilroy, S., Klovatch, depression: a potential predictor of treatment response. I., et al., 2014. Towards empathic neurofeedback for interactive NeuroReport 8, 1057-1061. storytelling. OpenAccess Series in Informatics 41, 42-60. Mayeli, A., Zotev, V., Refai, H., Bodurka, J., 2016. Real-time EEG Choi, S.W., Chi, S.E., Chung, S.Y., Kim, J.W., Ahn, C.Y., Kim, H.T., artifact correction during fMRI using ICA. Journal of Neuroscience 2011. Is alpha wave neurofeedback effective with randomized Methods, 274, 27-37. clinical trials in depression? A pilot study. Neuropsychobiology 63, McNair, D.M., Lorr, M., Droppleman, L.F., 1971. Profile of Mood 43-51. States. Educational and Industrial Testing Service, San Diego, CA. Chouinard, P.A., Paus, T., 2006. The primary motor and premotor areas Micoulaud-Franchi, J.A., McGonigal, A., Lopez, R., Daudet, C., of the human cerebral cortex. The Neuroscientist 12, 143-152. Kotwas, I., Bartolomei, F., 2015. Electroencephalographic Cook, I.A., O’Hara, R., Uijtdehaage, S.H.J., Mandelkern, M., Leuchter, neurofeedback: level of evidence in mental and brain disorders and A.F., 1998. Assessing the accuracy of topographic EEG mapping suggestions for good clinical practice. Neurophysiologie Clinique for determining local brain function. Electroencephalography and 45, 423-433. Clinical Neurophysiology 107, 408-414. Montgomery, S.A., Asberg, M., 1979. A new depression scale designed Cox, R.W., 1996. AFNI: software for analysis and visualization of to be sensitive to change. British Journal of Psychiatry 134, 382- functional magnetic resonance neuroimages. Computers and Biomedical Research 29, 162-173. Mulert, C., Lemieux, L. (Eds.), 2010. EEG-fMRI: Physiological Basis, Cox, R.W., Hyde, J.S., 1997. Software tools for analysis and Technique, and Applications. Springer-Verlag, Berlin Heidelberg. visualization of fMRI data. NMR in Biomedicine 10, 171-178. Paquette, V., Beauregard, M., Beaulieu-Prevost, D., 2009. Effect of a Cox, R.W., Jesmanowicz, A., 1999. Real-time 3D image registration psychoneurotherapy on brain electromagnetic tomography in for functional MRI. Magnetic Resonance in Medicine 42, 1014- individuals with major depressive disorder. Psychiatry Research: Neuroimaging 174, 231-239. Davidson, R.J., 1996. Cerebral asymmetry, emotion, and affective Peeters, F., Oehlen, M., Ronner, J., van Os, J., Lousberg, R., 2014. style. In: Davidson, R.J., Hugdahl, K. (Eds.), Brain Asymmetry. Neurofeedback as a treatment for major depressive disorder – a The MIT Press, Cambridge, MA, pp. 361-387. pilot study. PLoS ONE 9, e91837. Emmert, K., Kopel, R., Sulzer, J., Brühl, A.B., Berman, B.D., Linden, Perronnet, L., Lécuyer, A., Mano, M., Bannier, E., Lotte, F., Clerc, M., et al., 2017. Unimodal versus bimodal EEG-fMRI neurofeedback General Psychiatry 67, 1128-1138. of a motor imagery task. Frontiers in Human Neuroscience 11, 193. Young, K.D., Zotev, V., Phillips, R., Misaki, M., Yuan, H., Drevets, Pizzagalli, D.A., 2011. Frontocingulate dysfunction in depression: W.D., et al., 2014. Real-time fMRI neurofeedback training of toward biomarkers of treatment response. amygdala activity in patients with major depressive disorder. PLoS Neuropsychopharmacology 36, 183-206. ONE 9, e88785. Pizzagalli, D.A., Nitschke, J.B., Oakes, T.R., Hendrick, A.M., Horras, Young, K.D., Siegle, G.J., Zotev, V., Phillips, R., Misaki, M., Yuan, K.A., Larson, C.L., et al., 2002. Brain electrical tomography in H., et al., 2017. Randomized clinical trial of real-time fMRI depression: the importance of symptom severity, anxiety, and amygdala neurofeedback for major depressive disorder: effects on melancholic features. Biological Psychiatry 52, 73-85. symptoms and autobiographical memory recall. American Journal Pizzagalli, D.A., Sherwood, R.J., Henriques, J.B., Davidson, R.J., of Psychiatry 174, 748-755. 2005. Frontal brain asymmetry and reward responsiveness: a Yuan, H., Young, K.D., Phillips, R., Zotev, V., Misaki, M., Bodurka, source-localization study. Psychological Science 16, 805-813. J., 2014. Resting-state functional connectivity modulation and Price, J.L., Drevets, W.C., 2012. Neural circuits underlying the sustained changes after real-time functional magnetic resonance pathophysiology of mood disorders. Trends in Cognitive Sciences imaging neurofeedback in depression. Brain Connectivity 4, 690- 16, 61-71. 701. Quaedflieg, C.W.E.M., Smulders, F.T.Y., Meyer, T., Peeters, F., Zotev, V., Krueger, F., Phillips, R., Alvarez, R.P., Simmons, W.K., Merckelbach, H., Smeets, T., 2016. The validity of individual Bellgowan, P., et al., 2011. Self-regulation of amygdala activation frontal alpha asymmetry EEG neurofeedback. Social Cognitive and using real-time fMRI neurofeedback. PLoS ONE 6, e24522. Affective Neuroscience 11, 33-43. Zotev, V., Yuan, H., Phillips, R., Bodurka, J., 2012. EEG-assisted Ramot, M., Kimmich, S., Gonzalez-Castillo, J., Roopchansingh, V., retrospective motion correction for fMRI: E-REMCOR. Popal, H., White, E., et al., 2017. Direct modulation of aberrant NeuroImage 63, 698-712. brain network connectivity through real-time neurofeedback. eLife Zotev, V., Phillips, R., Young, K.D., Drevets, W.C., Bodurka, J., 2013. 6, e28974. Prefrontal control of the amygdala during real-time fMRI Rosenfeld, J.P., Cha, G., Blair, T., Gotlib, I.H., 1995. Operant neurofeedback training of emotion regulation. PLoS ONE 8, (biofeedback) control of left-right frontal alpha power differences: e79184. potential neurotherapy for affective disorders. Biofeedback and Zotev, V., Phillips, R., Yuan, H., Misaki, M., Bodurka, J., 2014. Self- Self-Regulation 20, 241-258. regulation of human brain activity using simultaneous real-time Smith, E.E., Cavanagh, J.F., Allen, J.J.B., 2018. Intracranial source fMRI and EEG neurofeedback. NeuroImage 85, 985-995. activity (eLORETA) related to scalp-level asymmetry scores and Zotev, V., Yuan, H., Misaki, M., Phillips, R., Young, K.D., Feldner, depression status. Psychophysiology 55, e13019. M.T., et al., 2016. Correlation between amygdala BOLD activity Snaith, R.P., Hamilton, M., Morley, S., Humayan, A., Hargreaves, D., and frontal EEG asymmetry during real-time fMRI neurofeedback Trigwell, P., 1995. A scale for the assessment of hedonic tone: the training in patients with depression. NeuroImage: Clinical 11, 224- Snaith-Hamilton Pleasure Scale. British Journal of Psychiatry 167, 238. 99-103. Zotev, V., Misaki, M., Phillips, R., Wong, CK, Bodurka, J., 2018a. Spielberg, J.M., Miller, G.A., Engels, A.S., Herrington, J.D., Sutton, Real-time fMRI neurofeedback of the mediodorsal and anterior B.P., Banich, M.T., Heller, W., 2011. Trait approach and avoidance thalamus enhances correlation between thalamic BOLD activity motivation: lateralized neural activity associated with executive and alpha EEG rhythm. Human Brain Mapping 39, 1024-1042. function. NeuroImage 54, 661-670. Zotev, V., Phillips, R., Misaki, M., Wong, C.K., Wurfel, B.E., Krueger, Spielberg, J.M., Heller, W., Miller, G.A., 2013. Hierarchical brain F., et al., 2018b. Real-time fMRI neurofeedback training of the networks active in approach and avoidance goal pursuit. Frontiers amygdala activity with simultaneous EEG in veterans with combat- in Human Neuroscience 7, 284. related PTSD. NeuroImage: Clinical 19, 106-121. Spielberger, C.D., Gorsuch, R.L., Lushene, R.E., 1970. Test Manual for Zotev, V., Bodurka, J., 2020. Effects of simultaneous real-time fMRI the State Trait Anxiety Inventory. Consulting Psychologists Press, and EEG neurofeedback in major depressive disorder evaluated Palo Alto, CA. with brain electromagnetic tomography. Preprint Stewart, J.L. Coan, J.A., Towers, D.N., Allen, J.J.B., 2011. Frontal https://arxiv.org/abs/2006.01923 EEG asymmetry during emotional challenge differentiates individuals with and without lifetime major depressive disorder. Journal of Affective Disorders 129, 167-174. Stewart, J.L. Coan, J.A., Towers, D.N., Allen, J.J.B., 2014. Resting and task-elicited prefrontal EEG alpha asymmetry in depression: support for the capability model. Psychophysiology 51, 446-455. Suslow, T., Konrad, C., Kugel, H., Rumstadt, D., Zwitserlood, P., Schöning, S., et al., 2010. Automatic mood-congruent amygdala responses to masked facial expressions in major depression. Biological Psychiatry 67, 155-160. Talairach, J., Tournoux, P., 1988. Co-Planar Stereotaxic Atlas of the Human Brain. Thieme Medical Publishers, New York, NY. Thibault, R.T., MacPherson, A., Lifshitz, M., Roth, R.R., Raz, A., 2018. Neurofeedback with fMRI: a critical systematic review. NeuroImage 172, 786-807. Thibodeau, R., Jorgensen, R.S., Kim, S., 2006. Depression, anxiety, and resting frontal EEG asymmetry: a meta-analytic review. Journal of Abnormal Psychology 115, 715-729. Victor, T., Furey, M.L., Fromm, S.J., Öhman, A., Drevets, W.C., 2010. Relationship between amygdala responses to masked faces and mood state and treatment in major depressive disorder. Archives of Table 1. Participants’ emotional state measures before and after the rtfMRI-EEG-nf session. Emotional states were assessed using the Profile of Mood States (POMS), the State-Trait Anxiety Inventory (STAI), and the Visual Analogue Scale (VAS). Before After Effect Change Change Measure mean (SD) mean (SD) size (d) t-score# p-value [q] Experimental group (EG, n=16) POMS Depression 15.4 (14.0) 7.75 (10.1) −0.62 −2.49 0.025 [0.039]* Confusion 10.7 (5.91) 7.25 (4.22) −0.73 −2.91 0.011 [0.039]* Total mood disturbance 46.1 (39.9) 26.8 (28.0) −0.60 −2.39 0.030 [0.039]* STAI State anxiety 44.9 (12.2) 40.4 (10.5) −0.57 −2.26 0.039 [0.039]* VAS Happiness 4.56 (2.31) 5.94 (1.53) +0.59 +2.36 0.033 [0.039]* Control group (CG, n=8) POMS Depression 23.9 (17.4) 19.6 (13.4) −0.42 −1.18 0.276 [0.856] Confusion 11.4 (5.21) 10.4 (4.21) −0.29 −0.83 0.436 [0.856] Total mood disturbance 65.8 (40.4) 59.0 (34.8) −0.21 −0.58 0.580 [0.856] STAI State anxiety 53.8 (12.0) 53.3 (9.51) −0.04 −0.11 0.915 [0.915] VAS Happiness 2.38 (1.85) 2.75 (2.25) +0.15 +0.42 0.685 [0.856] # t(15) for the EG, t(7) for the CG, two-tailed. * FDR q<0.05 for the five tests. Table 2. Psychophysiological interaction effect, based on the time course of frontal alpha EEG asymmetry (FAA), for the Happy vs Count condition contrast for the experimental group (EG). Late- x, y, z Region t-score rality (mm) Frontal lobe Medial frontal gyrus (BA 6) L −3, −5, 54 17.6 Superior frontal gyrus (BA 6) L −21, −5, 65 12.1 Precentral gyrus (BA 4) L −39, −15, 42 11.5 Precentral gyrus (BA 6) R 49, −11, 26 10.3 Medial frontal gyrus (BA 11) R 2, 29, −14 9.51 Inferior frontal gyrus (BA 46) R 39, 33, 10 9.28 Inferior frontal gyrus (BA 44) L −51, 10, 12 8.89 Precentral gyrus (BA 4) R 45, −11, 51 8.86 Inferior frontal gyrus (BA 47) R 37, 16, −4 8.71 Middle frontal gyrus (BA 9) R 49, 18, 24 8.23 Precentral / mid. frontal gyrus (BA 6) L −42, −4, 44 7.81 Middle frontal gyrus (BA 9) L −36, 16, 24 6.33 Temporal lobe Superior temporal gyrus (BA 22) L −51, 1, 5 11.1 Superior temporal gyrus (BA 38) L −47, 15, −18 8.73 Superior temporal gyrus (BA 39) R 53, −53, 12 8.53 Middle temporal gyrus (BA 19/22) L −38, −59, 12 8.34 Superior temporal gyrus (BA 22) R 50, −15, 6 7.90 Superior temporal gyrus (BA 39) L −51, −55, 14 7.02 Transverse temporal gyrus (BA 41) L −29, −29, 8 6.95 Parietal lobe Inferior parietal lobule (BA 40) R 59, −43, 24 12.6 Precuneus (BA 7) R 5, −35, 44 10.2 Precuneus (BA 31/18) L −17, −68, 21 10.1 Precuneus (BA 7) R 1, −55, 48 9.57 Postcentral gyrus (BA 4) R 19, −33, 62 8.49 Postcentral gyrus (BA 3) L −27, −31, 60 8.40 Precuneus (BA 31) R 1, −48, 33 7.90 Limbic lobe Cingulate gyrus (BA 24) L −1, 2, 44 13.1 Amygdala / parahipp. gyrus L −17, −5, −17 8.73 Parahippocampal gyrus (BA 28) R 21, −13, −18 8.70 Amygdala / uncus L −21, −6, −19 7.86 Posterior cingulate (BA 29) L −12, −47, 18 7.85 Amygdala / parahipp. gyrus L −21, −6, −11 7.73 Uncus (BA 36) R 21, −3, −32 7.53 Parahippocampal gyrus (BA 28) L −21, −19, −8 6.94 Subcallosal gyrus (BA 25) L −9, 15, −14 6.72 Sub-lobar Thalamus, laterodorsal R 13, −19, 16 9.49 Claustrum R 33, −7, −4 9.38 Insula (BA 13) R 44, 0, 4 9.12 Thalamus, mediodorsal L −3, −13, 6 9.10 Thalamus, mediodorsal R 3, −15, 2 8.80 Culmen L −1, −57, −22 8.58 Declive L −19, −65, −18 7.47 Caudate body R 17, −9, 20 6.82 FDR q<0.02 for |t|>6.0; BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates. Supplementary material quantities. A few outlier points (included in the statistics computation) out of total n=120 points in each plot S1.1. Modified MR-compatible EEG cap correspond to situations when a current run is characterized by a very drastic (usually one per run) head To achieve more efficient real-time EEG-fMRI artifact motion, not present during the previous run, or vice versa. suppression, we modified a 32-channel BrainCap-MR The linear fits to the data for F3 and F4 have slopes of (EASYCAP, GmbH) as shown in Fig. 2D. In the modified 0.96 and 0.98, respectively (Fig. S1). Therefore, the use of cap, four EEG channels out of 31 – FC1, FC2, TP9, TP10 the regression coefficients optimized for the previous run – were re-purposed for acquisition of reference artifact yields signal variance reduction that is 96-98%, on the waveforms, which we refer to as R (t), R (t), R (t), and 1 2 3 average, of the variance reduction achieved with the R (t), instead of EEG activity. For each of these channels, regression coefficients optimized for the current run. the lead was disconnected from its electrode, and These results justify our proposed implementation of the connected to one end of a wire contour. The other end of linear regression procedure for improved real-time EEG- the contour was connected to the Ref electrode (FCz, blue) fMRI artifact correction, described in Sec. 2.5. via a 50 kOhm resistor. Geometries of the four contours were optimized so that electromotive forces (EMFs), S1.3. Reliability of the real-time EEG artifact correction induced in the contours during head movements in the Figure S2 compares the EEG-nf target measures MRI scanner’s main field, approximate cardioballistic computed in real time and the corresponding measures (CB) and random-motion artifacts picked up by EEG determined in offline EEG data analysis. The relative channels F3 and F4. The two shorter contours (brown frontal EEG asymmetries A and B were computed every 2 wires in Fig. 2D) followed the leads of channels F3 and s (Sec. 2.6) following the real-time EEG-fMRI artifact F4, respectively, then looped around the Gnd electrode correction procedure (Sec. 2.5), and their values were (AFz, black), and connected to the Ref via the resistors. saved to a file during each run. The real-time A and B The two longer contours (orange wires in Fig. 2D) also data, reported in Fig. S2, were taken from these files. The followed the leads of F3 and F4, looped around electrodes offline EEG data processing was applied to the raw EEG Fp1 and Fp2, respectively, then around the Gnd, and data recorded during fMRI. It included average artifact connected to the Ref through the resistors. The resistors subtraction (AAS) for MR artifacts, followed by AAS for were non-magnetic non-inductive surface mount thin film CB artifacts, and exclusion of bad intervals, as described resistors (Vishay PNM1206-50KBCT-ND, 50k, 0.1%, in detail below (S1.6). The offline A and B data in Fig. S2 0.4W). After the cap had been placed and aligned on a were computed after such processing. participant’s head, the wire contours were fixed tightly to Mean real-time A values for the Happy Memories the cap’s fabric with adhesive tape (3M Durapore). This conditions in each run and the corresponding mean offline EEG cap modification enabled acquisition of the four A values are compared in Fig. S2A. The results are pooled reference artifact waveforms, along with 27 EEG across five task runs for all participants. Fig. S2B waveforms and one ECG waveform, using the standard compares mean real-time A changes and mean offline A 32-channel system configuration. Importantly, the use of changes between the Rest and Happy Memories the modified EEG cap did not affect quality of structural conditions. Similar plots for the B values and B changes or functional MRI brain images. are shown in Figs. S2C and S2D. The correlations in Figs. S1.2. Performance of real-time EEG artifact regression S2A-D are highly significant with large effect sizes (r>0.5). These results suggest that the real-time EEG-nf Figure S1 evaluates effectiveness of the real-time target measures were sufficiently reliable, both for the regression of the CB and motion artifacts (Sec. 2.5, Fig. 3) alpha band (A) and for the high-beta band (B). The with the linear regression coefficients {a }, {b }, i=1…4, i i differences between the real-time and offline data in Fig. based on the previous run as opposed to the current run. S2 can be attributed to the following factors. First, the The actual real-time signal variance changes for channels offline AAS procedures for MR and CB artifacts are more F3 and F4 (same as in Fig. 3C) after the regression accurate than the corresponding real-time AAS procedure with the coefficients for a current run procedures, because they involve careful visual inspection determined by fitting the data for the previous run are of the artifact patterns across an entire run and manual shown along the y-axis in Fig. S1. The corresponding adjustment of the correction parameters (semi-automatic variance changes after an offline regression procedure correction mode). Second, the real-time artifact regression with the optimum regression coefficients for a current run procedure (Fig. 3B) can reduce some CB artifacts that are shown along the x-axis. The results are pooled across cannot be efficiently suppressed by the offline AAS, e.g. five task runs for all participants. The plots in Fig. S1 due to large variations in CB artifacts’ temporal profiles. demonstrate high correlation (r=0.98) between the two S1.4. fMRI data analysis To compare BOLD activity levels for the left and right amygdala, we considered amygdala BOLD laterality, i.e. a Offline analysis of the fMRI data was performed in difference in mean GLM-based fMRI percent signal AFNI (Cox, 1996; Cox and Hyde, 1997). Pre-processing changes between the LA and RA ROIs for each contrast, of single-subject fMRI data included time series despiking run, and participant. The LA and RA ROIs in this case using the 3dDespike AFNI program with -localedit option. were defined as the left and right amygdala regions It was followed by correction of cardiorespiratory artifacts specified in the AFNI implementation of the Talairach- using the AFNI implementation of the RETROICOR Tournoux brain atlas. Similarly, we computed BOLD method (Glover et al., 2000). Further fMRI pre-processing laterality for the middle frontal gyrus (MidFG) as a involved slice timing correction and volume registration difference in mean GLM-based fMRI percent signal of all EPI volumes acquired in the experiment using the changes between the left and right MidFG ROIs. The 3dvolreg AFNI program with two-pass registration. The ROIs were defined as the left and right middle frontal last volume of the short EPI dataset, acquired immediately gyrus regions specified in the AFNI implementation of the after the high-resolution anatomical MPRAGE brain Talairach-Tournoux atlas, and limited to a selected slab image (Sec. 2.6), was used as the registration base. along z-axis (z ≤ z ≤ z ). 1 2 To enable transformation of the fMRI data to the Talairach space, the Talairach transform was first S1.5. fMRI-based PPI analysis performed for each subject’s high-resolution anatomical To evaluate changes in the left amygdala fMRI MPRAGE brain image. The image was subjected to functional connectivity between experimental conditions, explicit skull-stripping using the 3dSkullStrip AFNI we conducted a psychophysiological interaction (PPI) program with -blur_fwhm option, and then transformed analysis (Friston et al., 1997; Gitelman et al., 2003). The towards the standard TT_N27 template in the Talairach analysis was based on fMRI time course for an LA seed space using the @auto_tlrc AFNI program. ROI. The seed ROI was defined as the left amygdala The fMRI activation analysis was performed according region specified in the AFNI implementation of the to the standard general linear model (GLM) approach. It Talairach-Tournoux brain atlas. (We used the anatomical was conducted for each of the five task fMRI runs (Fig. amygdala ROI, because the spherical LA target ROI (Fig. 1B) using the 3dDeconvolve AFNI program. The GLM 2A) includes some voxels outside the amygdala proper). model included two block-design stimulus condition The seed ROI was transformed to each subject’s terms, Happy Memories and Count, represented by the individual EPI space. In addition, 10-mm-diameter ROIs standard block-stimulus regressors in AFNI. A general were defined within the left and right frontal white matter linear test (-gltsym) term was included to compute the (WM) and within the left and right ventricle cerebrospinal Happy vs Count contrast. Nuisance covariates included fluid (CSF) using the individual high-resolution the six fMRI motion parameters and five polynomial anatomical brain image in the Talairach space (S1.4), and terms for modeling the baseline. To further reduce effects also transformed to the EPI space. The pre-processed of residual motion artifacts, the fMRI data and motion fMRI data and the six fMRI motion parameters were parameters were lowpass Fourier filtered at 0.1 Hz prior to bandpass filtered between 0.01 and 0.08 Hz using the the GLM analysis. GLM β coefficients were computed for 3dTproject AFNI program. The 3dmaskave AFNI each voxel, and average percent signal changes for Happy program was then used to compute average fMRI time vs Rest, Count vs Rest, and Happy vs Count contrasts courses for the LA, WM, and CSF ROIs. The LA seed were obtained by dividing the corresponding β values ROI time course was employed as the fMRI-based PPI (×100%) by the β value for the constant baseline term. correlation regressor. A PPI interaction regressor was The resulting fMRI percent signal change maps for each defined for the Happy Memories vs Rest condition run were transformed to the Talairach space by means of contrast as follows. A [Happy−Rest] contrast function was the @auto_tlrc AFNI program. The individual high- defined to be equal +1 for the Happy Memories condition resolution anatomical brain image in the Talairach space blocks, −1 for the preceding Rest condition blocks, and 0 was used as the transformation template. The maps were for all other condition blocks (Fig. 1B). The LA time re-sampled to 2×2×2 mm isotropic voxel size. course was detrended, using the 3dTproject AFNI Average individual BOLD activity levels were program, with respect to the time courses of the six fMRI computed in the offline analysis for the LA and L rACC motion parameters (together with the same time courses target ROIs, exhibited in Figs. 2A,B. The voxel-wise shifted by one TR), the time courses for the WM and CSF fMRI percent signal change data from the GLM analysis, ROIs, and five polynomial terms. It was then deconvolved transformed to the Talairach space, were averaged within using the 3dTfitter AFNI program to estimate a time these ROIs and used as GLM-based measures of these course of the underlying neuronal activity. This estimated regions’ BOLD activities. ‘neuronal’ time course was multiplied by the [Happy−Rest] contrast function, and convolved with the artifact to be subtracted was defined, for each EEG same hemodynamic response function (HRF, ‘Cox channel, by a moving average over 21 cardiac periods. special’) using the waver AFNI program. The resulting Cardiac periods with strong random-motion artifacts were waveform was employed as the fMRI-based PPI not included in the CB correction. interaction regressor for the Happy Memories vs Rest Following the MR and CB artifact removal, the EEG condition contrast. data from the five task runs (Fig. 1B) were concatenated A single-subject fMRI-based PPI analysis involved to form a single dataset. The data were carefully fitting a GLM model with the two PPI regressors using the examined, and intervals exhibiting significant motion or 3dDeconvolve AFNI program. The GLM design matrix instrumental artifacts were marked manually as “bad for each task run included four stimulus regressors, intervals” and excluded from the analysis. The signals sixteen covariates of no interest, and five polynomial from the four reference artifact channels were taken into terms for modeling the baseline. The stimulus regressors account to more reliably identify data intervals affected by included the fMRI-based PPI interaction regressor, the random head motions and distinguish them from intervals fMRI-based PPI correlation regressor, the Happy exhibiting neuronal activity (e.g. theta). Channel FCz was Memories block-stimulus regressor, and the Count block- kept as the EEG reference throughout the analysis. stimulus regressor. The last two regressors were the An independent component analysis (ICA) was standard block-design fMRI regressors in AFNI performed over the entire dataset with exclusion of the corresponding to the stimulus waveforms with 40-s-long bad intervals. This approach ensured that independent condition blocks (Fig. 1B). The covariates of no interest components (ICs) corresponding to various artifacts were included time courses of the six fMRI motion parameters, identified and removed in a consistent manner across all time courses of the same parameters shifted by one TR, five runs. The Infomax ICA algorithm (Bell and time courses of the left and right WM ROIs, and time Sejnowski, 1995), implemented in BrainVision Analyzer courses of the left and right ventricle CSF ROIs. Each 2.1, was applied to the data from 27 EEG channels and single-subject PPI analysis produced GLM-based R - yielded 27 ICs. Time courses, spectra, topographies, and statistics and t-statistics maps for the fMRI-based PPI kurtosis values of all the ICs were carefully analyzed to interaction and correlation terms for each run. These identify various artifacts, as well as EEG signals of statistics were used to compute voxel-wise PPI interaction neuronal origin. After all the ICs had been classified, an and correlation values. The resulting maps were subjected inverse ICA transform was applied to remove the to the Fisher r-to-z normalization, transformed to the identified artifacts from the EEG data. Because many Talairach space, re-sampled to 2×2×2 mm isotropic voxel artifacts had already been removed using the ICA, the data size, and spatially smoothed using isotropic Gaussian blur were examined again, and new bad intervals were defined with FWHM = 5 mm. The single-subject fMRI-based PPI to exclude remaining artifacts. interaction maps were submitted to whole-brain group PPI A time-frequency analysis was performed to compute analyses that employed the 3dttest++ AFNI program. EEG power for each channel as a function of time and frequency. The continuous wavelet transform with Morlet S1.6. EEG data analysis wavelets, implemented in BrainVision Analyzer 2.1, was Offline analysis of the EEG data was performed in applied to obtain EEG signal power in [0.5-30] Hz BrainVision Analyzer 2.1 (Brain Products, GmbH). frequency range with 0.5 Hz frequency resolution and 4 Removal of MR and cardioballistic (CB) artifacts was ms temporal sampling. An EEG power as a function of based on the average artifact subtraction (AAS) method time was then computed for each band of interest. (Allen et al., 1998, 2000) implemented in the Analyzer. S1.7. EEG-based PPI analyses The MR artifact template was defined using MRI slice markers recorded with the EEG data. After the MR artifact To investigate how temporal correlations between removal, the data were bandpass filtered between 0.5 and FAA (or FBA) and BOLD activity changed between 80 Hz (48 dB/octave) and downsampled to 250 S/s experimental conditions, we performed PPI analyses sampling rate (4 ms interval). The fMRI slice selection adapted for EEG-fMRI (Zotev et al., 2014, 2016, 2018a). frequency (17 Hz) and its harmonics were removed by The analyses followed the standard fMRI-based PPI band rejection filtering. The MR artifact removal was analysis approach (Friston et al., 1997; Gitelman et al., performed for the 27 EEG channels, the ECG channel, and 2003), except that the initial deconvolution step, used to the 4 reference artifact channels (re-purposed FC1, FC2, estimate an underlying neuronal activity from an fMRI TP9, TP10). Removal of CB artifacts and follow-up time course, was skipped, and the actual EEG activity analyses were conducted for the 27 EEG channels only. time course was employed. The EEG-based PPI analyses The CB artifact template was determined from the cardiac were conducted separately for the FAA and FBA time waveform recorded by the ECG channel, and the CB courses. EEG-based PPI regressors for the FAA were defined as interaction maps using the 3dttest++ AFNI program. The illustrated in Figure S3. The FAA values, computed with 4 analyses included two covariates: the participants’ ms temporal resolution for each experimental run, were MADRS depression severity ratings and average averaged for 200-ms-long time bins. The resulting individual values of the PPI interaction effect for a WM mask. The WM mask was defined for each participant as waveform was linearly detrended and orthogonalized with respect to the Happy Memories and Count stimulus follows. The individual high-resolution anatomical brain waveforms (Fig. 1B) using the glmfit() MATLAB image in the Talairach space (S1.4) was thresholded to program. This procedure removed variations in mean FAA select WM regions only, and the resulting mask was levels across the conditions to focus the analysis on multiplied by the standard WM mask in the Talairach temporal FAA variations around the means. The FAA space (TT_wm+tlrc). The mask was then re-sampled to time course was then converted to z-scores across each 2×2×2 mm voxels, and subjected to erosion by one voxel run. The HRF (‘Cox special’) was calculated with 200 ms to improve its separation from gray matter. The resulting sampling using the waver AFNI program. Convolution of individual WM mask in the Talairach space contained, on the z(FAA) time course with the HRF by means of the average, ~10000 voxels. The EEG-based PPI interaction conv() MATLAB program yielded a regressor, which we values, averaged within this WM mask, were used as a employed as the EEG-based PPI correlation regressor covariate vector in the group analyses to better account for (Fig. S3A). An EEG-based PPI interaction regressor was spurious PPI interaction effects. Such effects could be defined for the Happy Memories vs Count condition caused, e.g., by residual motion artifacts in the contrast as follows. A [Happy−Count] contrast function simultaneously acquired EEG and fMRI data. was set to be equal +1 for the Happy Memories condition S2.1. Verbal self-report performance ratings blocks, −1 for the Count condition blocks, and 0 for the Figure S4 exhibits average memory recall and Rest condition blocks (Fig. S3B). The z(FAA) time course was first multiplied by the [Happy−Count] contrast happiness ratings for each group. The two ratings were reported verbally by each participant after each function, and then convolved with the HRF (Fig. S3C). The resulting waveform was used as the EEG-based PPI experimental run (except Rest). The EG vs CG group differences in the individual ratings averaged across the interaction regressor for the Happy Memories vs Count four nf runs trended toward significance after correction condition contrast. The two PPI regressors – correlation and interaction – were sub-sampled to the middle time both for the memory recall ratings (EG vs CG, NF: t(22)=2.29, p<0.032, q<0.064, d=0.99) and the happiness points of fMRI volumes. Two additional PPI regressors ratings (EG vs CG, NF: t(22)=1.90, p<0.071, q<0.071, were defined in the same way using the power-sum function instead of the FAA. Prior to its inclusion in the d=0.82). For the Transfer run, group differences were significant after correction for both ratings (Memory- GLM model, the FAA-based PPI correlation regressor was recall, EG vs CG, TR: t(22)=2.61, p<0.016, q<0.043, linearly detrended and orthogonalized with respect to the corresponding power-sum-based PPI correlation regressor. d=1.13; Happiness, EG vs CG, TR: t(22)=2.39, p<0.026, q<0.043, d=1.04; corrected for the four tests). Similarly, the FAA-based PPI interaction regressor was linearly detrended and orthogonalized with respect to the S2.2. BOLD fMRI activity across the brain power-sum-based PPI interaction regressor. This Figure S5 exhibits whole-brain statistical maps of procedure ensured that the two EEG-based PPI regressors BOLD fMRI activity during the rtfMRI-EEG-nf training specifically reflect temporal variations in the FAA rather for the EG participants. The maps correspond to the than variations in the average power for the two channels. Happy Memories vs Count condition contrast (H vs C). For the FBA, the PPI regressors were defined in a similar The individual-subject fMRI percent signal change maps way starting with the FBA time course. were averaged for the four nf runs (PR, R1, R2, R3). The A single-subject EEG-based PPI analysis involved group mean was compared to zero using a one-sample t- fitting a GLM model using the 3dDeconvolve AFNI test (df=15, two-tailed). The statistical results are program. The design matrix for each task run had the summarized in Table S2. The maps in Fig. S5 are FDR same structure as described above for the fMRI-based PPI corrected with q<0.05 threshold, and the data in Table S2 analysis, except that the two EEG-based PPI regressors – – with q<0.01 threshold. The results demonstrate interaction and correlation – were used instead of the significant positive BOLD activity contrast for the left fMRI-based PPI regressors. The resulting PPI interaction amygdala region and many areas of the limbic system. and correlation maps were subjected to the Fisher They also reveal pronounced BOLD laterality for large normalization, Talairach transform, re-sampling, and parts of the middle frontal gyrus and superior frontal gyrus spatial smoothing as described above. Whole-brain group (Fig. S5, Table S2). PPI analyses were conducted for the EEG-based PPI S2.3. Amygdala fMRI connectivity changes Similar to the results for the FAA (Fig. 11), the whole- brain maps of the FBA-based PPI interaction effect (Fig. Figure S6 shows whole-brain statistical maps for the S8) demonstrate that performance of the rtfMRI-EEG-nf EG vs CG group difference in the LA fMRI connectivity task was associated with enhancement in temporal changes during the rtfMRI-EEG-nf training. The statistics correlations between the FBA and BOLD activities of the are summarized in Table S3. The results reveal three loci large brain network. In particular, significant positive in the rACC area, characterized by the most pronounced FBA-based PPI interaction effects are observed for the EG vs CG group differences: (−8, 34, 7) with t=4.41, (−9, corresponding left and right DLPFC regions (MidFG, BA 41, 5) with t=4.04, and (3, 35, 9) with t=3.34 (Table S3). 9), with maxima at (−29, 35, 32) and (31, 41, 28), These loci are pointed by green arrows in Fig. S6B. The respectively (Fig. S8, Table S4). These maxima are EG vs CG group differences in fMRI connectivity changes located closer to the medial plane, than the similar between the LA and 10-mm-diameter spherical ROIs maxima for the FAA-based PPI effect (Fig. 11, Table 2). centered at these locations were significant with large The results in Fig. S8 show the most pronounced FBA- effect sizes: t(22)=3.31, p<0.003, d=1.43 for the (−8, 34, based PPI interaction effects along the cortical midline 7) centered ROI; t(22)=3.31, p<0.003, d=1.43 for the (−9, and the cingulate gyrus, including the anterior and 41, 5) centered ROI; and t(22)=3.06, p<0.006, d=1.32 for posterior cingulate (Fig. S8, Table S4). The statistical the (3, 35, 9) centered ROI. The average connectivity maximum is at (−19, 12, 42) in the left SFG (BA 8) and changes between the LA and the right rACC ROI centered the effect extends down to the ACC (BA 32). Elevated at (3, 35, 9) showed significant negative correlation with high-beta activity in these areas, often with some the MADRS depression severity ratings for the EG lateralization to the right, is associated with anxiety (e.g. (r=−0.55, p<0.027). Zotev and Bodurka, 2020 and references therein). S2.4. Amygdala BOLD laterality interpretation Significant FBA-based PPI interaction effects are also found for many regions involved in autobiographical The amygdala BOLD laterality (‘LA−RA’) is a memory retrieval, including the hippocampus, the relevant metric for assessing target-specific effects of the extended areas of the parahippocampal gyrus, the anterior rtfMRI-nf procedure aimed at upregulating the LA BOLD thalamus, the precuneus (BA 31), the posterior cingulate activity for the following reasons (Zotev et al., 2016). (BA 29), the lingual gyrus (involved in visual memory), First, both the LA and RA are activated by happy emotion and others (Fig. S8, Table S4). For the PMC areas (PrecG, induction, as evidenced by substantial BOLD activations BA 6), the FBA-based PPI interaction effects are less of the RA in our studies that used rtfMRI-nf of the LA pronounced than the corresponding FAA-based effects. activity (e.g. Zotev et al., 2011, 2016; Young et al., 2014). Second, emotional side effects that accompany performance of the difficult neurofeedback task (e.g. confusion, frustration, anxiety, etc.) conceivably involve activations of both the LA and RA. Third, due to the relative proximity of the LA and RA, their apparent BOLD activity levels may be affected by similar fMRI artifacts (e.g. signal losses due to magnetic susceptibility variations). Because only the LA BOLD activity (and not that of the RA) is explicitly upregulated using the rtfMRI- nf in our studies, the amygdala BOLD laterality is more sensitive to target-specific effects of the rtfMRI-nf procedure. S2.5. fMRI correlates of the FBA modulation Figure S8 shows whole-brain statistical maps for the FBA-based PPI interaction effect for the Happy Memories vs Count contrast for the EG. The group analysis was conducted in the same way as described for the FAA. Statistical results for the FBA-based PPI interaction effect are summarized in Table S4. The maps in Fig. S8 are voxel-wise FDR corrected at q<0.07 level, and the data in Table S4 – at q<0.05 level. Figure S1. Effectiveness of the real-time EEG-fMRI artifact regression procedure with the regression coefficients {ai} and {bi} determined by fitting the data for the previous run (y-axis), compared to the effectiveness of a similar procedure with the optimum coefficients determined by fitting the data for the same (current) run (x-axis). Results for 120 experimental runs (24 participants, 5 task runs) are pooled together in each plot. Figure S2. Comparison of relative frontal EEG asymmetry measures A (alpha band) and B (high-beta band), determined for the EEG data after the real-time processing and for the same data after the offline EEG data processing. Results for 120 experimental runs (24 participants, 5 task runs) are pooled together in each plot. A) Mean A values for the Happy Memories conditions in each run. B) Mean A changes between the Rest and Happy Memories conditions (H vs R) in each run. C) Mean B values for the Happy Memories conditions. D) Mean B changes between the Rest and Happy Memories conditions. Figure S3. Definition of regressors for EEG-based psychophysiological interaction (PPI) analysis of fMRI data. A) Convolution of a time course of the FAA (converted to z-scores) with the HRF yields an EEG-based PPI correlation regressor. B) Contrast function for the Happy Memories (H) versus Count (C) conditions for one experimental run. C) Convolution of the FAA time course, multiplied by the contrast function, with the HRF yields an EEG-based PPI interaction regressor for the Happy Memories vs Count condition contrast. Figure S4. Average memory recall and happiness ratings reported by the participants after each experimental run. EG – experimental group, CG – control group. The error bars are standard errors of the mean (sem). The NF refers to group difference statistics (p-value from an independent- samples t-test and the corresponding effect size d) for the individual ratings averaged across the four nf runs (PR, R1, R2, R3). Figure S5. Statistical maps of BOLD fMRI activity, corresponding to the Happy Memories vs Count condition contrast, during the rtfMRI-EEG- nf training for the experimental group (EG). The individual fMRI activity results were averaged across the four nf runs (PR, R1, R2, R3). The maps are voxel-wise FDR corrected and projected onto the standard TT_N27 anatomical template in the Talairach space, with 3 mm separation between axial slices. The number adjacent to each slice indicates the z coordinate in mm. Following the radiological notation, the left hemisphere (L) is shown to the reader’s right. The t-statistics maxima and minima and the corresponding locations are specified in Table S2. Figure S6. A) Statistical maps of the experimental vs control group difference (EG vs CG) in the left amygdala fMRI functional connectivity changes during the rtfMRI-EEG-nf training. The fMRI connectivity changes between the Rest and Happy Memories conditions (H vs R) were evaluated in the psychophysiological interaction (PPI) analysis, based on the LA time course. The individual PPI interaction results were averaged across the four nf runs (PR, R1, R2, R3). The maps are projected onto the standard TT_N27 anatomical template in the Talairach space. The number adjacent to each slice indicates the z coordinate in mm. Peak t-statistics values and the corresponding locations are specified in Table S3. B) Loci in the rACC area, which exhibited the largest EG vs CG group differences, are pointed by green arrows: (−8, 34, 7), (−9, 41, 5), and (3, 35, 9). The green crosshairs mark the center of the L rACC target ROI. Figure S7. Correlations between average values of the amygdala BOLD laterality (‘LA−RA’) and the corresponding average values of the FAA- based PPI interaction effect for the LA ROI for the experimental group (EG). Both quantities correspond to the Happy Memories vs Count (H vs C) condition contrast. A) The individual results were averaged across the four nf runs (PR, R1, R2, R3). B) The individual results were averaged for three nf runs (out of four) with the most positive amygdala BOLD laterality values. Figure S8. Enhancement in temporal correlation between frontal high-beta EEG asymmetry (FBA) and BOLD activity during the rtfMRI-EEG- nf training. Statistical maps of the FBA-based PPI interaction effect for the Happy Memories vs Count condition contrast (H vs C) are shown for the experimental group (EG). The maps are voxel-wise FDR corrected and projected onto the standard TT_N27 anatomical template, with 3 mm separation between axial slices. The number adjacent to each slice indicates the z coordinate in mm. The left hemisphere (L) is to the reader’s right. Peak t-statistics values for the FBA-based PPI interaction effect and the corresponding locations are specified in Table S4. Table S1. Psychological trait measures for the study participants. Psychological traits were assessed before the rtfMRI-EEG-nf session using the Hamilton Depression Rating Scale (HDRS), the Montgomery-Asberg Depression Rating Scale (MADRS), the Snaith-Hamilton Pleasure Scale (SHAPS), the Hamilton Anxiety Rating Scale (HARS), the State-Trait Anxiety Inventory (STAI), the Toronto Alexithymia Scale (TAS-20), and the Behavioral Inhibition System / Behavioral Activation System scales (BIS/BAS). Experimental Control group, Difference Measure group, mean (SD) mean (SD) t-score [p]# Participants 16 8 Age (years) 32 (11) 34 (7) −0.67 [0.510] HDRS 14.4 (7.0) 15.1 (4.9) −0.25 [0.807] MADRS 19.6 (10.7) 20.5 (5.7) −0.23 [0.821] SHAPS 27.1 (6.7) 31.5 (5.8) −1.58 [0.129] HARS 13.2 (7.5) 16.1 (6.4) −0.95 [0.355] STAI Trait anxiety 56.9 (9.9) 59.6 (9.6) −0.65 [0.522] TAS-20 Total alexithymia 53.5 (14.4) 61.8 (12.2) −1.38 [0.180] BAS Reward responsiveness 14.9 (3.3) 16.3 (2.4) −0.99 [0.337] BIS 22.4 (3.6) 24.1 (4.1) −1.02 [0.319] # t(22), but t(19) for BIS/BAS; p – two-tailed, uncorrected Table S2. BOLD fMRI activity, corresponding to the Happy Memories vs Count condition contrast, during the rtfMRI-EEG-nf training for the experimental group (EG). Late- x, y, z Region t-score rality (mm) Frontal lobe Inferior frontal gyrus (BA 47) L −33, 17, −18 8.26 Middle frontal gyrus (BA 8) L −40, 20, 47 7.21 Inferior frontal gyrus (BA 47) L −43, 29, −5 7.09 Superior frontal gyrus (BA 8) L −7, 49, 48 6.90 Superior frontal gyrus (BA 10) L −7, 60, 24 6.71 Inferior frontal gyrus (BA 13/47) R 29, 14, −10 6.69 Superior frontal gyrus (BA 6) L −11, 13, 62 6.66 Superior frontal gyrus (BA 8) R 27, 17, 49 −6.59 Superior frontal gyrus (BA 6) L −9, 31, 57 6.36 Temporal lobe Middle temporal gyrus (BA 37) R 49, −59, 5 9.14 Superior temporal gyrus (BA 38) R 39, 5, −28 7.90 Middle temporal gyrus (BA 37) L −42, −67, 11 6.65 Inferior temporal gyrus (BA 37) L −55, −45, −16 −5.66 Middle temporal gyrus (BA 20) R 53, −35, −15 −5.35 Parietal lobe Postcentral gyrus (BA 2) L −55, −23, 44 −10.9 Precuneus (BA 7/19) R 23, −81, 35 6.07 Postcentral gyrus (BA 3) R 45, −19, 54 −5.50 Inferior parietal lobule (BA 40) L −46, −33, 45 −5.29 Inferior parietal lobule (BA 39) R 35, −63, 42 −5.24 Occipital lobe Middle occipital gyrus (BA 19/18) R 39, −81, 2 9.70 Middle occipital gyrus (BA 19) L −39, −77, 2 8.26 Lingual gyrus (BA 17) L −19, −86, 1 6.69 Cuneus (BA 18) L −11, −79, 21 −5.64 Limbic lobe Parahippocampal gyrus (BA 36) R 25, −31, −14 7.07 Anterior cingulate (BA 32) L −7, 49, −2 6.62 Amygdala / parahipp. gyrus L −29, −1, −14 6.55 Amygdala / uncus L −21, −3, −22 6.13 Cingulate gyrus (BA 31) R 7, −39, 34 −6.10 Hippocampus / parahipp. gyrus L −28, −24, −9 6.00 Cingulate gyrus (BA 31) L −6, −49, 28 5.36 Sub-lobar Red nucleus L −7, −19, −10 8.43 Insula (BA 13) R 37, −13, 12 −7.12 Insula (BA 13) L −39, −9, 6 −6.07 Thalamus, mediodorsal R 2, −9, 2 5.76 Thalamus, mediodorsal L −1, −14, 8 5.53 Culmen R 13, −57, −6 −5.41 FDR q<0.01 for |t|>5.0; BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates. Table S3. Experimental vs control group difference in the left amygdala fMRI functional connectivity changes between the Rest and Happy Memories conditions (H vs R) during the rtfMRI-EEG-nf training. Late- x, y, z Region t-score rality (mm) Frontal lobe Inferior frontal gyrus (BA 9) R 57, 13, 32 5.73 Middle frontal gyrus (BA 9) L −53, 8, 36 4.63 Superior frontal gyrus (BA 6) L −7, 33, 54 −4.56 Precentral gyrus (BA 6) L −51, 1, 33 4.49 Middle frontal gyrus (BA 9/46) L −39, 19, 24 4.42 Superior frontal gyrus (BA 6) L −15, −7, 64 3.68 Inferior frontal gyrus (BA 44) L −55, 5, 14 3.67 Medial frontal gyrus (BA 11) L −3, 61, −14 −3.64 Superior frontal gyrus (BA 6) R 9, 37, 52 −3.47 Temporal lobe Fusiform gyrus (BA 20) L −43, −31, −24 5.22 Superior temporal gyrus (BA 22) L −67, −35, 6 −3.82 Superior temporal gyrus (BA 21) R 55, −13, 0 3.31 Parietal lobe Postcentral gyrus (BA 2) L −51, −19, 28 4.37 Inferior parietal lobule (BA 40) L −47, −37, 38 3.28 Occipital lobe Middle occipital gyrus (BA 18) R 31, −91, 2 −4.01 Limbic lobe Anterior cingulate (BA 24) L −8, 34, 7 4.41 Parahippocampal gyrus (BA 37) L −33, −39, −4 4.36 Anterior cingulate (BA 32) L −9, 41, 5 4.04 Cingulate gyrus (BA 24) R 11, 1, 38 4.01 Uncus (BA 36) L −21, −9, −30 3.62 Anterior cingulate (BA 24) R 3, 35, 9 3.34 Sub-lobar Subthalamic nucleus R 15, −13, −4 7.39 Red nucleus L −7, −25, −6 4.44 Insula (BA 13) R 29, 21, 4 4.29 Insula (BA 13/47) L −31, 15, 4 4.15 Thalamus, ventroposterolateral R 17, −19, 6 3.95 Claustrum L −31, −11, 14 3.91 Putamen R 23, −9, 8 3.79 p<0.05, uncorr. BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates Table S4. Psychophysiological interaction effect, based on the time course of frontal high-beta EEG asymmetry (FBA), for the Happy vs Count contrast for the experimental group (EG). Late- x, y, z Region t-score rality (mm) Frontal lobe Superior frontal gyrus (BA 8/32) L −19, 12, 42 11.8 Medial frontal gyrus (BA 6) L −5, 35, 36 10.2 Superior frontal gyrus (BA 8) R 3, 29, 50 9.58 Inferior frontal gyrus (BA 44) L −51, 6, 13 8.75 Medial frontal gyrus (BA 6) L −1, 3, 52 7.93 Middle frontal gyrus (BA 9) L −29, 35, 32 7.79 Precentral gyrus (BA 4) L −29, −21, 58 7.52 Middle frontal gyrus (BA 9) R 31, 41, 28 7.30 Precentral gyrus (BA 6) R 27, −9, 52 6.23 Medial frontal gyrus (BA 10) L −9, 55, 6 6.12 Temporal lobe Middle temporal gyrus (BA 39) L −37, −69, 18 7.91 Superior temporal gyrus (BA 41) R 49, −37, 10 7.58 Middle temporal gyrus (BA 20) L −45, −5, −20 7.42 Superior temporal gyrus (BA 22) L −63, −41, 6 6.88 Middle temporal gyrus (BA 21) R 59, −7, −12 6.37 Parietal lobe Precuneus (BA 31) L −13, −59, 24 9.79 Precuneus (BA 31) R 15, −63, 20 8.16 Occipital lobe Lingual gyrus (BA 18) L −9, −79, −6 7.20 Limbic lobe Anterior cingulate (BA 33) L −9, 19, 16 10.3 Posterior cingulate (BA 29) R 3, −39, 20 9.53 Anterior cingulate (BA 32) L −13, 33, −2 9.36 Parahippocampal gyrus (BA 30) R 17, −37, 4 8.40 Parahippocampal gyrus (BA 36) R 24, −33, −14 8.19 Cingulate gyrus (BA 32) L −1, 21, 28 7.94 Subcallosal gyrus (BA 47) R 20, 11, −12 7.65 Cingulate gyrus (BA 31) L −10, −25, 34 7.61 Hippocampus / parahipp. gyrus L −27, −23, −10 7.41 Hippocampus / parahipp. gyrus L −27, −33, −3 7.29 Amygdala / parahipp. gyrus L −28, −5, −10 6.94 Parahippocampal gyrus (BA 28) L −19, −23, −18 6.58 Sub-lobar Thalamus, ventral anterior R 15, −9, 15 9.12 Declive R 33, −69, −17 9.12 Putamen L −23, −13, 8 9.10 Culmen of vermis R 1, −61, 2 8.64 Claustrum R 29, 7, −6 8.14 Caudate body L −16, 5, 18 7.87 Declive L −39, −61, −20 7.52 Thalamus, anterior L −7, −11, 15 7.50 Thalamus, mediodorsal L −3, −10, 10 7.21 Insula (BA 13) L −39, −1, 14 6.86 FDR q<0.05 for |t|>6.0; BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Quantitative Biology arXiv (Cornell University)

Emotion self-regulation training in major depressive disorder using simultaneous real-time fMRI and EEG neurofeedback

Loading next page...
 
/lp/arxiv-cornell-university/emotion-self-regulation-training-in-major-depressive-disorder-using-xiW3INQ00t
ISSN
2213-1582
eISSN
ARCH-3345
DOI
10.1016/j.nicl.2020.102331
Publisher site
See Article on Publisher Site

Abstract

Emotion self-regulation training in major depressive disorder using simultaneous real-time fMRI and EEG neurofeedback 1# 1,2 1 1,3# Vadim Zotev , Ahmad Mayeli , Masaya Misaki , Jerzy Bodurka 1 2 Laureate Institute for Brain Research, Tulsa, OK, USA; Electrical and Computer Engineering, University of Oklahoma, Tulsa, OK, USA; Stephenson School of Biomedical Engineering, University of Oklahoma, Norman, OK, USA Abstract: Simultaneous real-time fMRI and EEG neurofeedback (rtfMRI-EEG-nf) is an emerging neuromodulation approach, that enables simulta- neous volitional regulation of both hemodynamic (BOLD fMRI) and electrophysiological (EEG) brain activities. Here we report the first application of rtfMRI-EEG-nf for emotion self-regulation training in patients with major depressive disorder (MDD). In this proof-of-concept study, MDD pa- tients in the experimental group (n=16) used rtfMRI-EEG-nf during a happy emotion induction task to simultaneously upregulate two fMRI and two EEG activity measures relevant to MDD. The target measures included BOLD activities of the left amygdala (LA) and left rostral anterior cingulate cortex (rACC), and frontal EEG asymmetries in the alpha band (FAA, [7.5-12.5] Hz) and high-beta band (FBA, [21-30] Hz). MDD patients in the control group (n=8) were provided with sham feedback signals. An advanced procedure for improved real-time EEG-fMRI artifact correction was implemented. The experimental group participants demonstrated significant upregulation of the LA BOLD activity, FAA, and FBA during the rtfMRI-EEG-nf task, as well as significant enhancement in fMRI connectivity between the LA and left rACC. Average individual FAA changes dur- ing the rtfMRI-EEG-nf task positively correlated with depression and anhedonia severities, and negatively correlated with after-vs-before changes in depressed mood ratings. Temporal correlations between the FAA and FBA time courses and the LA BOLD activity were significantly enhanced dur- ing the rtfMRI-EEG-nf task. The experimental group participants reported significant mood improvements after the training. Our results suggest that the rtfMRI-EEG-nf may have potential for treatment of MDD. Keywords: depression, neurofeedback, emotion regulation, amygdala, rostral anterior cingulate cortex, EEG-fMRI, frontal EEG asymmetry Lemieux, 2010). In particular, relevant EEG measures can 1. Introduction represent different EEG frequency bands, while BOLD We have introduced simultaneous real-time fMRI and fMRI activity reflects cumulative metabolic energy de- EEG neurofeedback (rtfMRI-EEG-nf) – a non-invasive mands across the entire EEG spectrum. Second, rtfMRI- neuromodulation approach, that enables simultaneous vo- EEG-nf training may help to develop personalized mental litional regulation of both hemodynamic (BOLD fMRI) strategies that would reliably engage both the fMRI and and electrophysiological (EEG) brain activities (Zotev et EEG target brain activities at the same time and further al., 2014). It involves real-time integration of concurrent enhance their interactions. Such experimentally verified fMRI and EEG data streams to provide real-time fMRI mental strategies could then be employed during EEG-nf- neurofeedback (rtfMRI-nf) and EEG neurofeedback only training, which may provide a cost-effective, mobile, (EEG-nf) signals simultaneously to a participant inside the and long-term therapy in support of the rtfMRI-EEG-nf MRI scanner (Mano et al., 2017; Zotev et al., 2014). This training. Until now, rtfMRI-EEG-nf has only been used in multimodal neurofeedback approach holds two major proof-of-principle studies with healthy participants (Per- promises for treatment of neurological and psychiatric ronnet et al., 2017; Zotev et al., 2014). disorders. First, application of rtfMRI-EEG-nf may con- Here we report the first application of rtfMRI-EEG-nf ceivably have stronger therapeutic effects than standalone for emotion self-regulation training in a neuropsychiatric applications of either rtfMRI-nf (e.g. Thibault et al., 2018) population, specifically – in patients with major depres- or EEG-nf (e.g. Micoulaud-Franchi et al., 2015). The rea- sive disorder (MDD). During the rtfMRI-EEG-nf task, the son is that rtfMRI-EEG-nf can target disorder-specific participants induced happy emotion by retrieving and con- brain activity measures identified by two very different templating happy autobiographical memories, and, imaging modalities – fMRI and EEG (e.g. Mulert and simultaneously, learned to upregulate two rtfMRI-nf sig- ___________________ nals and two EEG-nf signals. The four neurofeedback Corresponding authors. E-mail: vzotev@laureateinstitute.org; signals represented four brain activity measures relevant jbodurka@laureateinstitute.org to MDD, as we explain below. The first rtfMRI-nf signal in our study is based on Ramot et al., 2017. We did not expect a significant mean BOLD activity of the left amygdala (LA) target region of fMRI activation of the left rACC during the rtfMRI-EEG- interest (ROI), as used in our previous rtfMRI-nf emotion nf task relative to rest, because the rACC is a part of the self-regulation studies with healthy participants (Zotev et default mode network, and its mean activation during the al., 2011), MDD patients (Young et al., 2014; Zotev et al., rtfMRI-nf training of the LA activity is relatively low 2016), and PTSD patients (Zotev et al., 2018b). The (Zotev et al., 2013). amygdala plays a fundamental role in emotion processing. The first EEG-nf signal in our study represents a In MDD, the amygdala exhibits blunted BOLD responses change in frontal alpha EEG asymmetry, which we abbre- to positive emotional stimuli (e.g. Price and Drevets, viate here as FAA. The FAA is defined as ln(P(right)) – 2012; Suslow et al., 2010; Victor et al., 2010). Upregula- ln(P(left)), where P is EEG power in the alpha frequency tion of the LA activity using the rtfMRI-nf during the band for corresponding (pre)frontal EEG channels on the positive emotion induction task based on retrieval of hap- right and on the left (e.g. F4 and F3). Because change in py autobiographical memories has been shown to correct alpha EEG power negatively correlates with cortical neu- this amygdala reactivity bias in MDD, and lead to signifi- ronal activation (e.g. Cook et al., 1998), a more positive cant reduction in depression severity (Young et al., 2017). FAA indicates a relatively stronger activation of the left A similar rtfMRI-nf training procedure in PTSD patients prefrontal regions. The FAA is commonly interpreted ac- yielded significant reductions in both PTSD severity and cording to the approach-withdrawal hypothesis, which comorbid depression severity (Zotev et al., 2018b). posits that activation of the left prefrontal regions is more The second rtfMRI-nf signal is based on BOLD activi- closely associated with approach motivation, while activa- ty of a target ROI in the left rostral anterior cingulate tion of the right prefrontal regions is associated with cortex (rACC). The rACC function is very important in avoidance motivation (e.g. Davidson, 1996; Harmon- MDD (e.g. Pizzagalli, 2011). Resting rACC activity, Jones and Gable, 2018; Spielberg et al., 2011, 2013). In- measured by PET prior to antidepressant treatment, has deed, approach-related emotional states, such as been shown to reliably predict MDD patients’ treatment happiness, are characterized by more positive FAA levels response (Mayberg et al., 1997; Pizzagalli, 2011). After than avoidance-related states, such as fear or disgust (e.g. treatment, MDD patients exhibit enhanced fMRI function- Davidson, 1996; Stewart et al., 2014). MDD patients and al connectivity between the rACC and the amygdala, both individuals with a history of depression show significantly at rest and during exposure to neutral and positive emo- lower FAA levels during an emotional task (either ap- tional images (Anand et al., 2005). We have demonstrated proach- or avoidance-related) than non-depressed that, in healthy participants, the rtfMRI-nf modulation of participants performing the same task (Stewart et al., the LA activity during happy emotion induction is accom- 2011, 2014). Resting-state FAA is also reduced in MDD panied by significant enhancement, across nf runs, in patients compared to non-depressed individuals (e.g. fMRI connectivity between the LA and the left rACC Smith et al., 2018; Thibodeau et al., 2006), though the (Zotev et al., 2011). Moreover, effective connectivity findings are less robust than those for an emotional chal- analyses suggest that the left rACC modulates activity of lenge (Stewart et al., 2014). The task-related FAA results the LA and several prefrontal regions during the rtfMRI-nf suggest that upregulation of FAA using EEG-nf during training (Zotev et al., 2013). In MDD patients, resting happy emotion induction would benefit MDD patients. fMRI connectivity between the LA and the left rACC Emotion regulation training with FAA-based EEG-nf has shows negative correlation with depression severity (Yuan been explored in several studies (e.g. Allen et al., 2001; et al., 2014). This connectivity is increased after the Baehr et al., 1997; Cavazza et al., 2014; Choi et al., 2011; rtfMRI-nf training of the LA activity (Yuan et al., 2014). Peeters et al., 2014; Quaedflieg et al., 2016; Rosenfeld et Collectively, these results suggest that enhancement in al., 1995). fMRI connectivity between the LA and the left rACC by Importantly, the FAA upregulation using the EEG-nf means of rtfMRI-nf during happy emotion induction in the present experimental design is consistent with the should be beneficial to MDD patients. LA BOLD activity upregulation using the rtfMRI-nf. Mo- Following this evidence, we included the rtfMRI-nf tivation is an important component of neurofeedback signal based on the left rACC activity to enable enhance- learning (e.g. Gaume et al., 2016). Because rtfMRI-nf ment in fMRI connectivity between the LA and the left training in general is a goal-oriented behavior, it requires rACC through simultaneous upregulation of the two nf approach motivation to be successful (e.g. Spielberg et al., signals. We chose this approach, because it is the simplest 2011, 2013). In our previous study, MDD patients, who way to enhance fMRI connectivity of the two brain re- underwent rtfMRI-nf training of the LA activity, showed gions, that does not involve computation of a correlation positive FAA changes, indicative of stronger approach coefficient for their fMRI waveforms. A conceptually sim- motivation, during the rtfMRI-nf task (Zotev et al., 2016). ilar ‘two-point’ technique was independently used by Moreover, mean FAA changes correlated with the amyg- dala BOLD laterality values. Temporal correlation be- and the FAA upregulation. The secondary outcome tween the FAA time course and the LA BOLD activity measures are the left rACC vs LA functional connectivity was significantly enhanced during the rtfMRI-nf task enhancement and the FBA upregulation. In addition to (Zotev et al., 2016). These observations suggest that the testing the two hypotheses, we conducted exploratory FAA and the LA BOLD activity can be modulated simul- analyses to examine associations between the target activi- taneously using the rtfMRI-EEG-nf. ty measures and the MDD patients’ depression severity, The second EEG-nf signal represents a change in anhedonia severity, and mood rating changes. frontal high-beta EEG asymmetry, abbreviated here as FBA. The FBA is defined as ln(P(left)) – ln(P(right)), 2. Methods where P is EEG power in the high-beta (beta3) band [21- 2.1. Participants 30] Hz for respective (pre)frontal EEG channels on the The study was conducted at the Laureate Institute for left and on the right (e.g. F3 and F4). Because cortical ac- Brain Research. It was approved by the Western tivation positively correlates with change in high-beta Institutional Review Board (IRB). All study procedures EEG power (Cook et al., 1998), a more positive FBA is were performed in accordance with the principles associated with a relatively stronger activation of the left expressed in the Declaration of Helsinki. prefrontal regions (similar to FAA). MDD patients, when Participants were recruited through online, newspaper, compared to healthy individuals, exhibit elevated resting radio, flyer, and other media advertisements. Individuals high-beta EEG activity in the right prefrontal regions, and were recruited, if they were currently depressed, but not deficient high-beta activity in the precuneus/posterior cin- currently taking psychiatric medication. All participants gulate (Pizzagalli et al., 2002). This finding suggests that underwent screening evaluations, including the Structured resting-state FBA is reduced in MDD. Paquette et al. em- Clinical Interview for the Diagnostic and Statistical ployed a high-beta EEG-nf for emotion self-regulation Manual of Mental Disorders, Fourth Edition (DSM-IV) training in MDD patients (Paquette et al., 2009). The (American Psychiatric Association, 2000) Axis I disorders. study showed that alleviation of MDD symptoms was as- The following exclusion criteria were applied: general sociated with reduction in resting high-beta EEG activity MRI exclusions, current pregnancy, psychosis, serious in the prefrontal cortex and increase in such activity in the suicidal ideation, major medical or neurological disorders, precuneus/posterior cingulate (Paquette et al., 2009). For exposure to any medication likely to influence cerebral responders in that study, the high-beta EEG activity reduc- function or blood flow within 3 weeks (8 weeks for tion was larger in the right prefrontal regions than in the fluoxetine), and meeting the DSM-IV criteria for drug or corresponding regions on the left (Paquette et al., 2009), alcohol abuse within the previous year or for lifetime indicating more positive resting-state FBA after the train- alcohol or drug dependence (except nicotine). Participants ing. This finding suggests that more positive FBA may be enrolled in the study met the criteria for MDD laid out in beneficial to MDD patients. We have already demonstrat- the DSM-IV. Most participants had recurrent MDD. They ed that healthy participants can learn to simultaneously provided a written informed consent as approved by the upregulate the LA BOLD activity and the FBA using IRB, and received monetary compensation. rtfMRI-EEG-nf while inducing happy emotion (Zotev et Twenty four unmedicated MDD patients completed an al., 2014). rtfMRI-EEG-nf training session. Prior to the session, the We conducted the proof-of-concept rtfMRI-EEG-nf participants underwent a psychological evaluation by a experiment, reported here, to test two main hypotheses. licensed psychiatrist. It included administration of the The first hypothesis was that MDD patients would be able following tests: the 21-item Hamilton Depression Rating to significantly increase the LA BOLD activity, the FAA, Scale (HDRS, Hamilton, 1960), the Montgomery-Asberg and the FBA using the rtfMRI-EEG-nf during happy emo- Depression Rating Scale (MADRS, Montgomery and tion induction. As part of this hypothesis, we also Asberg, 1979), the Snaith-Hamilton Pleasure Scale predicted that fMRI connectivity between the LA and the (SHAPS, Snaith et al., 1995), the Hamilton Anxiety left rACC would be significantly enhanced during the Rating Scale (HARS, Hamilton, 1959), the 20-item rtfMRI-EEG-nf task. The second hypothesis was that per- Toronto Alexithymia Scale (TAS-20, Bagby et al., 1994), formance of the rtfMRI-EEG-nf task would be and the Behavioral Inhibition System / Behavioral accompanied by significant enhancements in temporal Activation System scales (BIS/BAS, Carver and White, correlations between the FAA and FBA time courses and 1994). Both before and after the rtfMRI-EEG-nf session, the LA BOLD activity. Such enhancements would indi- the participants completed the Profile of Mood States cate that the corresponding EEG-nf and rtfMRI-nf signals were indeed upregulated together in real time, rather than (POMS, McNair et al., 1971), the State-Trait Anxiety Inventory (STAI, Spielberger et al., 1970), and the Visual independently one at a time (Zotev et al., 2014). The pri- Analogue Scale (VAS) with 10-point subscales for happy, mary outcome measures in our study are the LA activation Figure 1. Experimental paradigm for emotion self-regulation training using simultaneous real-time fMRI and EEG neurofeedback (rtfMRI-EEG- nf). A) Real-time GUI display screen for Happy Memories conditions with rtfMRI-EEG-nf. The four neurofeedback signals are displayed on the screen as four variable-height bars. The two EEG-nf signals on the left are based, respectively, on changes in frontal alpha EEG asymmetry (FAA, magenta) and frontal high-beta EEG asymmetry (FBA, purple). The two rtfMRI-nf signals on the right are based, respectively, on fMRI activities of the left amygdala (LA, red) and the left rostral anterior cingulate cortex (L rACC, orange). The bar heights are updated every 2 s. The black horizontal bar in the middle of the screen specifies a target level. It is raised from run to run. B) Experimental protocol consisted of six runs, each lasting 8 min 46 s. It included a Rest run, four rtfMRI-EEG-nf runs – Practice, Run 1, Run 2, Run 3, and a Transfer run without nf. The names of the five task runs are abbreviated in the text and figures as PR, R1, R2, R3, and TR, respectively. The task runs consisted of 40-s long blocks of Rest, Happy Memories, and Count conditions. The condition names are abbreviated as R, H, and C, respectively. No bars were displayed during the Rest and Count conditions, and during the entire Transfer run. restless, sad, anxious, irritated, drowsy, and alert states. simultaneously acquired EEG and fMRI data streams, Three MDD patients were in remission on the day of the described in Zotev et al., 2014. The neurofeedback experiment (HDRS ratings ≤ 7). BIS/BAS scores were information was displayed to a participant inside the unavailable for three participants out of 24. scanner on a projection screen via a multimodal graphical The participants were assigned to either an user interface (mGUI), depicted in Fig. 1A. The mGUI experimental group (EG) or a control group (CG) at 2:1 included four thermometer-style variable-height bars. The ratio in numbers, common in proof-of-concept rtfMRI-nf heights of these bars, updated every 2 s, represented the studies (Young et al., 2014). All the participants were four neurofeedback signals. Each bar height was also given identical instructions and were unaware of their indicated by a numeric value shown above that bar (Fig. group status. During the training session, participants in 1A). the EG (n=16, 13 females) received the rtfMRI-EEG-nf, The red rtfMRI-nf bar on the right represented BOLD based on their real-time EEG and fMRI brain activity fMRI activity of the left amygdala (LA) target ROI (Fig. measures. Participants in the CG (n=8, 4 females) were 2A). This spherical ROI with R=7 mm was centered at provided, without their knowledge, with sham feedback (−21, −5, −16) locus in the Talairach space (Talairach and signals, unrelated to brain activity. Because the Tournoux, 1988), as in our previous studies (Zotev et al., recruitment of unmedicated MDD patients was slow, the 2011, 2014, 2016). The orange rtfMRI-nf bar represented experiments for the EG were performed first, followed by fMRI activity of the left rACC target ROI (Fig. 2B). This the experiments for the CG. Psychological trait measures ROI, also with R=7 mm, was centered at (−3, 34, 5) locus, for the EG and CG participants, assessed before the which had exhibited significant enhancements in both rtfMRI-EEG-nf session, are reported in Supplementary functional and effective fMRI connectivities with the LA material (Table S1). There were no significant group during the rtfMRI-nf training in healthy participants differences in these measures. (Zotev et al., 2011, 2013). The magenta EEG-nf bar on the left represented a 2.2. Real-time fMRI and EEG neurofeedback change in relative alpha EEG asymmetry for channels F3 The rtfMRI-EEG-nf was implemented using the and F4 (Fig. 2C). The relative alpha asymmetry (A) was custom real-time control system for integration of defined as A = (P(F4) – P(F3)) / (P(F4) + P(F3)), where P activity. The sham feedback signals were computed, for each 40-s-long condition block, as random linear combinations of seven Legendre polynomials, as described previously (Zotev et al., 2018a). They were used to set heights of the four nf bars in real time (Fig. 1A). These signals’ waveforms were smooth, but randomly shaped, and they also varied randomly across condition blocks and across participants. 2.3. Experimental protocol The experimental protocol for training of emotion self- regulation using the rtfMRI-EEG-nf is illustrated in Fig. 1B. It has the same overall structure as the protocols we used previously (Zotev et al., 2011, 2014, 2016). The protocol included six EEG-fMRI runs (Fig. 1B), each lasting 8 min 46 s. During the Rest run, the participants were asked to relax and rest while looking at a fixation cross. The five task runs – the Practice run, Run 1, Run 2, Figure 2. Target regions of interest (ROIs) and EEG channels used to Run 3, and the Transfer run – consisted of alternating 40- provide the rtfMRI-EEG-nf. The ROIs were defined anatomically in the s-long blocks of Happy Memories, Count, and Rest co-planar stereotaxic atlas of the human brain by Talairach and conditions (Fig. 1B). Each condition was specified by Tournoux, and transformed to each participant’s individual fMRI image visual cues that included a color symbol at the center of space. A) Spherical 14-mm-diameter target ROI in the left amygdala (LA) region, projected onto the standard TT_N27 anatomical template in the screen and a text line at the top of the screen. For the the Talairach space. B) Spherical 14-mm-diatemer target ROI in the left Happy Memories with rtfMRI-EEG-nf condition blocks, rostral anterior cingulate cortex (L rACC) region. C) Frontal EEG the participants were asked to induce happy emotion by channels F3 (left) and F4 (right) used to provide the EEG-nf based on recalling happy autobiographical memories, while frontal EEG asymmetries in the alpha and high-beta EEG bands. Channel simultaneously trying to raise the levels of all four FCz was employed as reference. D) Custom modification of a standard MR-compatible 32-channel EEG cap to improve quality of the EEG-nf neurofeedback bars on the screen (Fig. 1A). For the Count during fMRI. The modified cap includes four wire contours with resistors condition blocks, the participants were instructed to for acquisition of reference artifact waveforms. mentally count back from 300 by subtracting a given integer. For the Rest condition blocks, the participants is EEG power in the alpha frequency band [7.5-12.5] Hz. were asked to relax and rest while looking at a fixation Note that normalized frontal alpha EEG asymmetry, cross. No bars were displayed during the Rest and Count commonly defined as FAA = ln(P(F4)) – ln(P(F3)), has a conditions, and during the Happy Memories conditions in Gaussian distribution, appropriate for statistical analyses. the Transfer run. However, its infinite variation range makes it less Prior to the experiment, the participants were asked to convenient for real-time applications. Therefore, we think of and write down three happy autobiographical employed a change in the relative asymmetry A as a target memories, keeping them confidential. The Practice run measure for EEG-nf, and used the FAA in offline data was included to give the participants an opportunity to analyses. The relative asymmetry A and the normalized become familiar with the rtfMRI-EEG-nf procedure and asymmetry FAA are related by the Fisher transform with evaluate emotional impact of the prepared happy factor ½ (Zotev et al., 2014). Similarly, the purple EEG-nf memories. During the four Happy Memories condition bar represented a change in relative high-beta EEG blocks in the Practice run, the instruction line at the top of asymmetry (B), defined as B = (P(F3) – P(F4)) / (P(F3) + the screen read “Happy – try 1st memory”, “Happy – try P(F4)), where P is EEG power in the high-beta frequency 2nd memory”, “Happy – try 3rd memory”, and “Happy – band [21-30] Hz. Normalized FBA = ln(P(F3)) – try best memory”, respectively. During the subsequent ln(P(F4)) was used in offline data analyses. Thus, the training runs (Run 1, Run 2, Run 3), the participants were rtfMRI-EEG-nf was designed to enable upregulation of encouraged to use any (and as many) memories that both the FAA and FBA simultaneously with upregulation helped them induce happy emotion and raise the rtfMRI- of BOLD activities of the LA and L rACC. EEG-nf bars. The instruction cue during the Happy For the control group (CG), the four actual Memories condition blocks in these runs was “Happy” neurofeedback signals were substituted, without the (Fig. 1A). The Transfer run without nf was included to participants’ knowledge, with sham feedback signals, evaluate whether the participants’ learned ability to which were computer generated and unrelated to any brain control the four target measures of brain activity generalized beyond the rtfMRI-EEG-nf training. During acquired with 0.2 ms temporal and 0.1 µ V measurement the Happy Memories condition blocks in the Transfer run, resolution (16-bit 5 kS/s sampling) in [0.016-250] Hz the instruction line read “As Happy as possible”. frequency band with respect to FCz reference. The ECG A target level for the rtfMRI-nf and EEG-nf signals waveform was acquired with 0.5 µ V resolution. BrainVision Recorder software was used for acquisition of was specified by the black horizontal bar above the red arrow in the middle of the mGUI screen (Fig. 1A). To raw EEG data, while BrainVision RecView software encourage the participants to improve their performance (Brain Products, GmbH) was employed for real-time from run to run, the target level was raised in a linear EEG-fMRI artifact correction as described below. fashion across the four nf runs. For the rtfMRI-nf signals, Technical details of the EEG-fMRI system setup, the target bar heights corresponded to 0.5%, 1.0%, 1.5%, configuration, and raw data acquisition were described and 2.0% fMRI percent signal changes for the Practice previously (Zotev et al., 2012). run, Run 1, Run 2, and Run 3, respectively. For the EEG- 2.5. Real-time EEG-fMRI artifact correction nf signals, the same bar heights corresponded to A and B relative asymmetry changes by 0.05, 0.10, 0.15, and 0.20, In the present study, we implemented a novel procedure for more efficient real-time EEG-fMRI artifact respectively. The display ranges were from −3% to +3% correction to improve quality of EEG-nf during fMRI. It for the fMRI-nf signals, and from −0.3 to +0.3 for the involved a special modification of a standard 32-channel EEG-nf signals. The Count conditions involved counting MR-compatible EEG cap (BrainCap-MR from back from 300 by subtracting 3, 4, 6, 7, and 9 for the Practice run, Run 1, Run 2, Run 3, and the Transfer run, EASYCAP, GmbH). The cap modification is shown in Fig. 2D and described in detail in Supplementary material respectively. After each experimental run with the Happy (S1.1). It enabled acquisition of four reference artifact Memories condition, a participant was asked to rate waveforms, which we refer to as R (t), R (t), R (t), and his/her performance on a scale from 0 (“not at all”) to 10 1 2 3 R (t), approximating cardioballistic (CB) and random- (“extremely”) by verbally answering two questions: “How 4 motion artifacts picked up by EEG channels F3 and F4. successful were you at recalling your happy memories?” and “How happy are you right now?”. The real-time procedure for EEG-fMRI artifact correction is depicted schematically in Fig. 3A. It is 2.4. MRI and EEG data acquisition implemented in BrainVision RecView software, which receives raw EEG data from BrainVision Recorder All experiments were conducted on the General Electric Discovery MR750 3T MRI scanner with a software in real time. The procedure includes three standard 8-channel receive-only head coil. A single-shot consecutive steps. First, the RecView MRI Artifact Filter is used to gradient echo EPI sequence with FOV/slice=240/2.9 mm, TR/TE=2000/30 ms, flip angle=90°, 34 axial slices per perform real-time average artifact subtraction (AAS) of volume, slice gap=0.5 mm, SENSE R=2 in the phase MR artifacts. The AAS method takes advantage of encoding (anterior-posterior) direction, acquisition matrix temporal periodicity of an fMRI pulse sequence 96×96, sampling bandwidth=250 kHz, was employed for (period=TR) and associated MR artifacts (Allen et al., fMRI. Each fMRI run included 263 EPI volumes (the first 2000). After the correction, the data are lowpass filtered at three EPI volumes were excluded from data analyses). 80 Hz (96 dB/octave) and downsampled to 250 S/s Physiological pulse oximetry and respiration waveforms sampling rate (4 ms interval). were recorded simultaneously with fMRI. The EPI images Second, real-time linear regression of CB and random- were reconstructed into a 128×128 matrix, resulting in motion artifacts (Masterton et al., 2007) is conducted 1.875×1.875×2.9 mm fMRI voxels. A T1-weighted 3D using the RecView Linear Derivation Filter. It is performed for channels F3 and F4 according to the MPRAGE sequence with FOV/slice=240/1.2 mm, TR/TE=5.0/1.9 ms, TD/TI=1400/725 ms, flip angle=10°, formulas: V (t,F3) = V(t,F3) – a R (t) – a R (t) – a R (t) – C 1 1 2 2 3 3 128 axial slices per slab, SENSE R=2, acquisition matrix a R (t), and V (t,F4) = V(t,F4) – b R (t) – b R (t) – b R (t) 4 4 C 1 1 2 2 3 3 – b R (t). Here, V (t,F3) and V (t,F4) are corrected 256×256, sampling bandwidth=31.2 kHz, scan time=4 4 4 C C min 58 s, was used for structural imaging. It provided waveforms for F3 and F4 after the regression, {R (t)}, high-resolution anatomical brain images with i=1…4, are the reference artifact waveforms, and {a }, {b }, i=1…4, are linear regression coefficients. The 0.94×0.94×1.2 mm voxels. i EEG recordings were performed simultaneously with coefficients are determined before each experimental run fMRI using a 32-channel MR-compatible EEG system as explained below. from Brain Products, GmbH. The system included one Third, the RecView Pulse Artifact Filter is used to BrainAmp MR plus amplifier. The EEG system’s clock carry out AAS of CB artifacts. The AAS in this case relies was synchronized with the MRI scanner’s 10 MHz clock on quasi-periodic nature of cardiac activity and related CB using the Brain Products’ SyncBox device. EEG data were artifacts (Allen et al., 1998). Cardiac epochs are includes the glmfit() function to solve general linear models (GLMs) fitting the reference artifact waveforms to the waveforms from channels F3 and F4: V(t,F3) = a R (t) + a R (t) + a R (t) + 1 1 2 2 3 3 a R (t) + e(t,F3), and V(t,F4) = b R (t) 4 4 1 1 + b R (t) + b R (t) + b R (t) + e(t,F4). 2 2 3 3 4 4 Here, e(t,F3) and e(t,F4) are neuronal and other signal components showing no correlations with the reference artifact waveforms. The fitting is carried out across the entire run. The GLM coefficients {a }, {b }, i=1…4, i i are then entered into the Linear Derivation Filter to enable the real- time artifact regression during the next experimental run. Thus, the real-time regression during the Practice run employs the coefficients determined from the data for the Rest run, the regression during Run 1 utilizes the coefficients computed from the data for the Practice run, and so on (Fig. 1B). Remarkably, the fact that the regression coefficients were determined from the preceding run’s data had little effect on the efficiency of the artifact regression in terms of signal variance reduction, as reported Figure 3. Real-time EEG-fMRI artifact correction procedure and its performance. A) The real- time procedure included three consecutive steps implemented in BrainVision RecView software: in Supplementary material (S1.2, Fig. average artifact subtraction (AAS) of MR artifacts; regression of reference artifact waveforms S1). This finding suggests that overall approximating cardioballistic (CB) and random-motion artifacts (channels F3 and F4 only); AAS properties of CB and random-motion of cardioballistic artifacts. B) Illustration of performance of the regression procedure across an artifacts did not generally change entire experimental run (left) and for a short time interval in the middle of that run (right). C) much from one run to the next. Performance of the regression procedure for channels F3 and F4 across all experimental (task) runs for all participants. Each histogram bar represents a probability of observing a change in Performance of the real-time signal variance within a given 1 dB-wide interval after the regression across one experimental artifact regression procedure is run. D) Illustration of topographic properties of residual CB and random-motion artifacts after demonstrated in Figs. 3B,C,D. Fig. 3B the real-time procedure (A). Topographies of independent components modeling such artifacts illustrates reduction in variance of an for a typical experimental run are shown. Note additional suppression of the artifacts for channels F3 and F4 compared to the surrounding EEG channels. artifact-contaminated EEG signal after the regression for a typical determined from the ECG waveform, and a moving experimental run. Histograms of signal variance changes average over 21 epochs is subtracted from each channel’s across all task runs for all participants in both groups data. Note that the linear regression procedure attenuates (24×5 = 120 runs) are shown in Fig. 3C. The mean signal CB and random-motion artifacts without any assumptions variance reductions after the real-time regression across about their temporal periodicity. Therefore, the linear one run are 5.3 dB for F3 and 5.1 dB for F4. Larger regression and the AAS reduce CB artifacts variance reductions are observed for participants with independently, enabling more efficient real-time CB stronger heart beats (taller and sharper R peaks in the artifact suppression for channels F3 and F4. ECG) leading to stronger CB artifacts in the alpha and The linear regression coefficients {a }, {b }, i=1…4, i i beta EEG bands. Fig. 3D shows topographies of residual are determined as follows. After the real-time application CB and random-motion artifacts from an independent of the MRI Artifact Filter, the data are bandpass filtered in component analysis (ICA) applied offline to representative [5-35] Hz frequency range (96 dB/octave) using the single-run EEG data after the real-time artifact correction RecView Frequency Filter, and saved to a file. After each procedure (Fig. 3A). The inclusion of the real-time artifact experimental run, a MATLAB script is executed offline. It regression for channels F3 and F4 led to an additional suppression of residual CB and random-motion artifacts every 2 s for a moving data interval of 2.048 s duration for these two channels compared to surrounding EEG with Hann window. The relative alpha EEG asymmetry A channels (Fig. 3D), for which only AAS of MR and CB and the relative high-beta EEG asymmetry B were artifacts were performed. Therefore, the described calculated as described above. The A and B values were approach can present a practical, intuitive, and easy-to-use sent via a TCP/IP socket to the mGUI software, where alternative to more advanced methods, such as real-time they were processed along with the corresponding fMRI ICA (e.g. Mayeli et al., 2016). signal values using a separate software thread. For each Happy Memories condition, a change in A was determined 2.6. Real-time data processing as a difference between the current A value and the Implementation of the rtfMRI-EEG-nf in the present baseline obtained by averaging A values for the preceding study was similar to that in our previous work (Zotev et Rest condition block (Fig. 1B). A moving average of the al., 2014), except that two rtfMRI-nf signals and two current and two preceding A changes was computed. This EEG-nf signals were computed and displayed to a moving average (multiplied by 10) was used to set the participant at the same time (Fig. 1A). height of the magenta EEG-nf bar (Fig. 1A) every 2 s. A Each rtfMRI-EEG-nf experiment began with change in B for each Happy Memories condition was acquisition of a high-resolution MPRAGE anatomical calculated in the same way, and its moving average brain image, followed by acquisition of a short EPI dataset (multiplied by 10) was used to set the height of the purple (5 volumes). The last volume of the EPI dataset was EEG-nf bar on the screen (Fig. 1A). The real-time A and B employed as a reference EPI volume defining the subject’s values and changes are compared to the corresponding A individual EPI space. The MPRAGE image was and B values and changes, determined in the offline EEG transformed to the Talairach space, and this data analysis, in Supplementary material (S1.3, Fig. S2). transformation was used as a template to transform the LA 2.7. fMRI data analysis and L rACC target ROIs from the Talairach space (Figs. 2A,B) to the individual EPI space. The resulting ROIs in Offline analysis of the fMRI data was performed in the EPI space contained approximately 140 voxels each. AFNI as described in detail in Supplementary material During the subsequent EEG-fMRI runs, the real-time (S1.4). The analysis involved fMRI pre-processing with plugin in AFNI (Cox, 1996; Cox and Hyde, 1997) was despiking, cardiorespiratory artifact correction (Glover et used to perform volume registration (Cox and al., 2000), slice timing correction, and volume registration. Jesmanowicz, 1999) of each acquired EPI volume to the A general linear model (GLM) fMRI activation analysis reference EPI volume and export mean values of fMRI with Happy Memories and Count block-stimulus signals for these two ROIs in real time. These fMRI signal conditions was applied to the preprocessed fMRI data. values were sent to the mGUI software via a TCP/IP Average GLM-based fMRI percent signal changes for the socket (Fig. 1 in Zotev et al., 2014). An rtfMRI signal for Happy Memories vs Rest condition contrast and for the the LA target ROI was computed for each Happy Happy Memories vs Count contrast were computed for the Memories condition as a percent signal change with LA and L rACC target ROIs (Figs. 2A,B) and used to respect to the baseline obtained by averaging the LA fMRI characterize the rtfMRI-nf performance. For exploratory signal values for the preceding 40-s-long Rest condition analyses, the amygdala BOLD laterality and the middle block (Fig. 1B). A moving average of the current and two frontal gyrus (MidFG) BOLD laterality were computed as preceding LA rtfMRI signal values was computed to described in Supplementary material (S1.4). reduce effects of fMRI noise and physiological artifacts. Statistical results in the present study were corrected This moving average was used to set the height of the red for multiple comparisons by controlling the False rtfMRI-nf bar (Fig. 1A) every TR=2 s. An rtfMRI signal Discovery Rate (FDR q). In whole-brain fMRI and EEG- for the L rACC target ROI was calculated in the same fMRI analyses, FDR correction was applied voxel-wise. way, and its moving average was used to set the height of 2.8. fMRI-based PPI analysis the orange rtfMRI-nf bar (Fig. 1A). To evaluate changes in the left amygdala fMRI During each EEG-fMRI run, the real-time correction of functional connectivity between experimental conditions, MR, random-motion, and CB artifacts was performed in the RecView software as described above (Fig. 3A). The we conducted a psychophysiological interaction (PPI) analysis (Friston et al., 1997; Gitelman et al., 2003), as corrected EEG data were exported in real time as data described in detail in Supplementary material (S1.5). The blocks of 8 ms duration via a TCP/IP socket to the EEG processing modules of the EEG-fMRI data integration analysis was based on fMRI time course for the LA seed ROI. This time course was used to define two PPI software (Fig. 1 in Zotev et al., 2014). These modules were written in Python and utilized NumPy functions. regressors: the fMRI-based PPI correlation regressor and the fMRI-based PPI interaction regressor for the Happy FFT power spectra for channels F3 and F4 were computed Memories vs Rest condition contrast (S1.5). We selected corresponding PPI regressors based on the EEG power this contrast a priori, because our earlier analyses sum, mentioned above. A single-subject EEG-based PPI (unpublished) had shown that the PPI interaction effects analysis for each run involved fitting a GLM model with for the LA time course were more significant for the these two PPI regressors (in addition to other fMRI Happy Memories vs Rest contrast than for the Happy regressors, see S1.7). For FBA, the PPI regressors were Memories vs Count contrast. A single-subject fMRI-based defined in a similar way, starting with the FBA time PPI analysis for each run involved fitting a GLM model course. with these two PPI regressors (in addition to other fMRI regressors, see S1.5). 3. Results 3.1. Emotional state changes 2.9. EEG data analysis Offline analysis of the EEG data was performed using The MDD patients’ mood ratings before and after the rtfMRI-EEG-nf session and statistics for the rating BrainVision Analyzer 2.1 software (Brain Products, changes are reported in Table 1. Five mood ratings most GmbH) as described in detail in Supplementary material (S1.6). Removal of EEG artifacts was based on the AAS relevant to the present study – POMS depression, confusion, total mood disturbance, STAI state anxiety, and (Allen et al., 1998, 2000) and independent component analysis (Bell and Sejnowski, 1995), implemented in VAS happiness – are included in the table. There were no Analyzer 2.1. Time-frequency analysis with Morlet significant EG vs CG group differences in these ratings before the session. Significant improvements in the mood wavelets was used to compute EEG power as a function of time and frequency. The upper alpha EEG frequency band ratings with medium effect sizes (d=−0.62, −0.73, −0.60, −0.57, and +0.59, respectively) were observed after the was defined individually for each participant as [IAF, rtfMRI-EEG-nf session for the EG (Table 1). The IAF+2] Hz, where IAF is the individual alpha peak frequency. The IAF was determined by inspection of corresponding mood improvements for the CG were non- significant with small effect sizes (Table 1). The EG vs average EEG spectra for the occipital and parietal EEG CG group differences in the mood rating changes were not channels across the Rest condition blocks in the four nf runs (Fig. 1B). The normalized FAA was computed as significant. The MDD patients’ memory-recall and happiness FAA = ln(P(F4)) – ln(P(F3)), where P is EEG power as a ratings, reported verbally during the experiment, are function of time in the individual upper alpha EEG band [IAF, IAF+2] Hz for a given channel (F3 or F4). In included in Supplementary material (S2.1, Fig. S4). Both ratings were higher for the EG than for the CG, with the addition to the FAA, a power-sum function ln(P(F4)) + EG vs CG group differences trending toward significance ln(P(F3)) was calculated for the upper alpha band. Similarly, the normalized FBA was computed as FBA = after correction, with large effect sizes (d=0.99 and 0.82, respectively, S2.1). ln(P(F3)) – ln(P(F4)), where P is EEG power as a function of time in the high-beta frequency band [21-30] 3.2. Neurofeedback performance Hz. A power-sum function was calculated for the same The main rtfMRI-EEG-nf performance characteristics channels for the high-beta band. Average FAA and FBA for the EG participants are exhibited in Figure 4. The changes between the Rest and Happy Memories results were obtained in offline EEG and fMRI data conditions were used to characterize the EEG-nf analyses. To characterize nf performance across the entire performance. nf training, we averaged individual-subject results across 2.10. EEG-based PPI analyses the four nf runs (Practice, Run 1, Run 2, Run 3), and compared their group mean to zero using a one-sample t- To investigate how temporal correlations between test (two-tailed). The corresponding significance (p-value) FAA (or FBA) and BOLD activity changed between experimental conditions, we performed PPI analyses and effect size (Cohen’s d) are included at the bottom of each figure after the NF notation. adapted for EEG-fMRI (Zotev et al., 2014, 2016, 2018a), According to Fig. 4, the EG participants were able to as described in detail in Supplementary material (S1.7). The FAA time course was used to define two PPI significantly increase the FAA during the rtfMRI-EEG-nf task compared to the Rest condition (NF: t(15)=3.21, regressors: the FAA-based PPI correlation regressor and the FAA-based PPI interaction regressor for the Happy p<0.006), with large effect size (d=0.80, 95% CI [0.23 1.36]). They also significantly upregulated the FBA (NF: Memories vs Count condition contrast (S1.7, Fig. S3). We t(15)=2.71, p<0.016), with medium effect size (d=0.68, chose this contrast a priori as in our previous studies (Zotev et al., 2014, 2016) to compare EEG-fMRI 95% CI [0.12 1.21]). Furthermore, the EG participants significantly increased BOLD activity of the LA target correlations between two cognitive tasks. The FAA-based PPI regressors were orthogonalized with respect to the ROI during the rtfMRI-EEG-nf task relative to the Rest baseline (NF: t(15)=3.21, p<0.006), with large effect size (d=0.80, 95% CI [0.23 1.36]). They also exhibited significant enhancement in fMRI connectivity between the LA and the L rACC target ROI (NF: t(15)=2.49, p<0.025), with medium effect size (d=0.62, 95% CI [0.08 1.15]). The fMRI connectivity changes were computed as the LA-based PPI interaction effects (Sec. 2.8) and averaged within the L rACC ROI. The average results across the nf runs (NF) in Fig. 4 remained significant after the multiple comparisons correction to account for testing the four quantities (FDR q<0.012, 0.021, 0.012, 0.025, respectively). There were no Figure 4. Main performance characteristics of the rtfMRI-EEG-nf training for the experimental group (EG). Each bar represents a group mean of average individual results for a given run. The significant differences in these activity error bars are standard errors of the mean (sem). The experimental runs and condition blocks are measures between the last nf training depicted schematically in Fig. 1B. The NF at the bottom of each figure refers to group statistics run (Run 3) and the Transfer run (p-value from a t-test and effect size d, both relative to zero) for the individual results averaged without nf, indicating transfer of the across the four nf runs (PR, R1, R2, R3). A) Average changes in frontal alpha EEG asymmetry (FAA) between the Happy Memories and Rest conditions (H vs R). FAA = ln(P(F4)) − learning effects (TR vs R3: ln(P(F3)), where P is EEG power in the individual upper alpha band. B) Average changes in t(15)=−0.24, p<0.817 for the FAA frontal high-beta EEG asymmetry (FBA) between the same conditions. FBA = ln(P(F3)) − changes; t(15)=−1.31, p<0.211 for the ln(P(F4)), where P is EEG power in the high-beta band. C) Average fMRI percent signal changes FBA changes; t(15)=−0.76, p<0.462 for the LA target ROI (Fig. 2A) for the Happy Memories conditions with respect to the Rest for the LA activations; t(15)=0.09, baseline (H vs R). D) Average changes in fMRI functional connectivity between the LA and L rACC target ROI (Fig. 2B) during the Happy Memories conditions compared to the Rest p<0.931 for the L rACC vs LA conditions (H vs R, psychophysiological interaction effect). connectivity changes). Figure 5 exhibits the corresponding activity measures for the CG. The average results across the four nf runs were non-significant with negative effects (Fig. 5). Importantly, the EG vs CG group differences either were significant or trended toward significance before correction (NF, EG vs CG: t(22)=2.14, p<0.044, d=0.93 for the FAA changes; t(22)=2.38, p<0.027, d=1.03 for the FBA changes; t(22)=1.84, p<0.080, d=0.79 for the LA activations; t(22)=2.82, p<0.010, d=1.22 for the L rACC vs LA connectivity changes). These group differences trended toward significance or remained significant after the multiple comparisons correction (FDR q<0.059, 0.054, 0.080, 0.040, Figure 5. Main performance characteristics for the control group (CG). Notations are the same respectively). as in Fig. 4. Figure 6. Average fMRI percent signal changes for the L rACC target ROI for the Happy Memories vs Rest condition contrast (H vs R) and for the Happy Memories vs Count condition contrast (H vs C). The black straight segments are linear fits across the four nf runs (PR, R1, R2, R3). The NF at the bottom of each figure refers to group statistics (p-value from a t-test and effect size d, both relative to zero) for the individual activity levels, corresponding to the H vs C contrast, averaged across the four nf runs. A) Results for the experimental group (EG). B) Results for the control group (CG). Figure 6 shows average fMRI percent signal changes relative to the Rest condition (H vs R), averaged across for the L rACC target ROI (Fig. 2B) for the Happy the four nf runs, showed significant positive correlations Memories vs Rest condition contrast (H vs R) and for the with the MDD patients’ MADRS depression severity Happy Memories vs Count condition contrast (H vs C) for ratings (r=0.52, p<0.039) and SHAPS anhedonia severity each group. Statistical analyses for these results are ratings (r=0.71, p<0.002). The corresponding correlation exploratory, because no hypotheses were made about with the HDRS depression severity ratings trended toward mean L rACC activations. The average L rACC activity significance (r=0.45, p<0.080). Among the four nf runs, levels during the rtfMRI-EEG-nf task across the four nf the correlation between the FAA changes and the MADRS runs for the EG were non-significant when compared to depression ratings was most pronounced for the Practice the Rest baseline (H vs R, NF: t(15)=−1.13, p<0.278, run (PR: r=0.65, p<0.007), i.e. at the beginning of the nf d=−0.28), and were significant when compared to the training. Count control task (H vs C, NF: t(15)=3.32, p<0.005, The same average FAA changes exhibited significant d=0.83). The EG vs CG group differences were not negative correlations with the after-vs-before changes in significant for either contrast. Interestingly, the L rACC POMS state depression ratings (r=−0.56, p<0.023) and activity levels showed positive linear trends across the POMS total mood disturbance ratings (r=−0.55, p<0.028). four nf runs for both contrasts for the EG (Fig. 6A). We The average FAA changes also showed significant computed a slope of a linear fit to the individual L rACC negative correlations with changes in POMS confusion activity levels across the four nf runs for each participant. ratings (r=−0.51, p<0.044) and STAI state anxiety ratings For the EG, the mean linear slopes were positive and (r=−0.50, p<0.049). Among the four nf runs, the negative trended toward significance for both contrasts (H vs R, correlation between the FAA changes and POMS slope: t(15)=2.12, p<0.051, d=0.53; H vs C, slope: depression changes was most pronounced for Run 3 (R3: t(15)=1.75, p<0.099, d=0.44). The EG vs CG group r=−0.64, p<0.008), i.e. at the end of the nf training. difference in the linear slopes was significant, with large All the correlation results in Fig. 7 remained significant effect size, for the Happy Memories vs Count contrast (H when they were controlled for the EG participants’ age vs C, slope difference: t(22)=2.41, p<0.025, d=1.04). and gender. For the CG, the correlations corresponding to those illustrated in Fig. 7 were not significant. 3.3. FAA changes vs psychological measures 3.4. Prefrontal BOLD laterality We conducted exploratory correlation analyses to examine associations between the target activity measures Whole-brain statistical maps of BOLD fMRI activity during the rtfMRI-EEG-nf training and individual during the rtfMRI-EEG-nf training for the EG are reported psychological metrics relevant to MDD. Significant in Supplementary material (S2.2, Fig. S5, Table S2). The associations were found only for the FAA changes. results demonstrate significant positive Happy Memories Results for the EG are exhibited in Figure 7. Correlations vs Count BOLD activity contrast for the left amygdala with trait measures are shown in Fig. 7A, and correlations region and many areas of the limbic system. They also with changes in state measures (after vs before the reveal pronounced BOLD laterality for parts of the MidFG session) are included in Fig. 7B. and superior frontal gyrus (SFG) (Fig. S5, Table S2). The FAA changes during the rtfMRI-EEG-nf task Figure 7. Correlations between changes in frontal alpha EEG asymmetry (FAA) during the rtfMRI-EEG-nf training and individual psychological measures. The individual FAA changes between the Rest and Happy Memories with rtfMRI-EEG-nf conditions (H vs R) were averaged across the four nf runs (PR, R1, R2, R3). The results are for the experimental group (EG), with each data point corresponding to one participant. A) Correlations between the FAA changes and severities of depression and anhedonia, assessed before the session. B) Correlations between the FAA changes and changes in state depression and total mood disturbance. The state measures were assessed both before and after the session, and their changes (after vs before) are included in the figures. Acronyms: MADRS – Montgomery-Asberg Depression Rating Scale, SHAPS – Snaith-Hamilton Pleasure Scale, POMS – Profile of Mood States. Figure 8. Illustration of BOLD laterality for middle frontal gyrus (MidFG) regions during the rtfMRI-EEG-nf training for the experimental group (EG) and its associations with individual psychological measures. A) The left and right MidFG ROIs as seen on the cortical surface. The ROIs were defined as described in the text. B) Average fMRI activity levels (percent signal changes) for the left and right MidFG ROIs, corresponding to the Happy Memories vs Rest condition contrast (H vs R) and to the Happy Memories vs Count contrast (H vs C). The individual fMRI activity levels were averaged across the four nf runs (PR, R1, R2, R3). C) Left: Correlation between the average individual MidFG laterality, i.e. the difference in fMRI activities between the left and right MidFG ROIs, for the Happy Memories vs Count contrast and after-vs-before changes in state depression. Right: Correlation between the average individual MidFG laterality and changes in total mood disturbance. POMS – Profile of Mood States. Whole-brain statistical maps for the EG vs CG group difference in the LA fMRI connectivity changes during the rtfMRI-EEG-nf training are reported in Supplementary material (S2.3, Fig. S6, Table S3). The maps revealed three loci in the rACC area, characterized by the most pronounced EG vs CG group differences (S2.3): (−8, 34, 7), (−9, 41, 5), and (3, 35, 9). All three loci were in close proximity to the center of the L rACC target ROI at (−3, 34, 5). For 10-mm-diameter ROIs, centered at these Figure 9. Correlations between changes in fMRI functional connectivity of the left loci, the EG vs CG group differences in fMRI amygdala with the left rACC during the rtfMRI-EEG-nf training and individual psychological measures. The fMRI connectivity changes for the target ROIs between connectivity changes with the LA were the Rest and Happy Memories with rtfMRI-EEG-nf conditions (H vs R, significant with large effect sizes (S2.3). psychophysiological interaction effect) are reported. The results are for the We performed exploratory correlation experimental group (EG), with each data point corresponding to one participant analyses to examine associations between the (however, n=13 for the BAS). The individual results were averaged across the four nf average changes in fMRI connectivity between runs (PR, R1, R2, R3). MADRS – Montgomery-Asberg Depression Rating Scale, BAS – Behavioral Activation System scale. the LA and the L rACC target ROI during the rtfMRI-EEG-nf training and individual We performed exploratory analyses of prefrontal psychological measures relevant to MDD. Figure 9 shows BOLD laterality to evaluate effects of the EEG-nf, such associations for the EG. The fMRI connectivity targeting the FAA and FBA, independently of the EEG changes for the Happy Memories condition relative to the data analysis. Results for the EG are exhibited in Figure 8. Rest condition (H vs R) were averaged across the four nf Because EEG electrodes F3 and F4 are situated above the runs. These changes exhibited negative and trending MidFG, we considered the left and right MidFG ROIs, toward significance correlation with the MADRS defined as described in Supplementary material (S1.4). depression severity ratings (r=−0.46, p<0.075) and Following the laterality pattern in Fig. S5, we limited the positive correlation with the BAS reward responsiveness ROIs to 42 ≤ z ≤ 57 mm. The resulting left and right ratings (r=0.52, p<0.070). For the CG, the corresponding MidFG ROIs are depicted in Fig. 8A. Mean BOLD correlations were not significant. activity levels for these ROIs, with individual results 3.6. EEG-fMRI correlations for the left amygdala averaged across the four nf runs, are shown in Fig. 8B. These activity levels were significantly higher for the left Figure 10 reports changes in temporal correlations MidFG ROI than for the right MidFG ROI (based on between time courses of the FAA and FBA and the LA paired t-test) both for the Happy Memories vs Rest BOLD activity during the rtfMRI-EEG-nf training. The condition contrast (H vs R: t(15)=3.28, p<0.005, d=0.82) correlation changes were computed as FAA-based or and for the Happy Memories vs Count contrast (H vs C: FBA-based PPI interaction effects (Sec. 2.10) and t(15)=4.46, p<0.0005, d=1.11). averaged within the LA ROI. We also conducted exploratory correlation analyses to Fig. 10A demonstrates that temporal correlation evaluate associations between the MidFG BOLD laterality between the FAA and the LA BOLD activity was during the rtfMRI-EEG-nf training and mood rating significantly enhanced during the rtfMRI-EEG-nf task changes. Results for the EG are shown in Fig. 8C. The compared to the Count condition for the EG (NF: average MidFG BOLD laterality for the Happy Memories t(15)=2.81, p<0.013), with medium effect size (d=0.70, vs Count contrast exhibited significant negative 95% CI [0.14 1.24]). There was no significant difference correlations with the after-vs-before changes in POMS in the mean PPI interaction effects between Run 3 and the state depression ratings (r=−0.58, p<0.019) and POMS Transfer run (TR vs R3: t(15)=0.22, p<0.833). Similarly, total mood disturbance ratings (r=−0.65, p<0.006). For the Fig. 10B indicates significant enhancement in temporal Happy Memories vs Rest contrast, the laterality also correlation between the FBA and the LA BOLD activity showed negative correlations with changes in these ratings during the rtfMRI-EEG-nf task for the EG (NF: (H vs R, POMS state depression change: r=−0.48, t(15)=2.51, p<0.024), with medium effect size (d=0.63, p<0.061; POMS total mood disturbance change: r=−0.57, 95% CI [0.08 1.16]). However, the PPI effect was p<0.020). For the CG, the results corresponding to those negligible during the Transfer run in this case. For the CG, in Figs. 8B,C were not significant. the FAA-based PPI interaction effect was negative yet small (Fig. 10C), while the FBA-based PPI interaction 3.5. Amygdala-rACC connectivity enhancement effect was negative with large effect size (Fig. 10D). target ROI was positive and trended toward significance for the EG (NF: t(15)=2.03, p<0.061, d=0.51). The EG vs CG group difference in the FAA- based PPI interaction effects for the L rACC ROI also trended toward significance (t(22)=1.86, p<0.076, d=0.81). 3.7. EEG-fMRI correlations across the brain Figure 11 exhibits whole-brain statistical maps for the FAA-based PPI interaction effect corresponding to the Happy Memories vs Count condition contrast for the EG. The PPI interaction results from twelve EG participants were included in the group analysis. The other four cases were considered outliers based on the low amygdala BOLD laterality (Fig. Figure 10. Changes in temporal correlations between frontal EEG asymmetries and BOLD fMRI activity of the left amygdala during the rtfMRI-EEG-nf training. The correlation changes S7). For each participant, the PPI between Happy Memories and Count conditions (H vs C) were evaluated in EEG-fMRI interaction maps were averaged for psychophysiological interaction (PPI) analyses, based on time courses of frontal alpha EEG three nf runs (out of four) with the asymmetry (FAA) and frontal high-beta EEG asymmetry (FBA). Each bar represents a group most positive individual amygdala mean of the individual PPI interaction values for a given run, averaged within the LA ROI. The BOLD laterality values. Statistical error bars are standard errors of the mean (sem). The NF at the bottom of each figure refers to group statistics (p-value from a t-test and effect size d, both relative to zero) for the individual results for the FAA-based PPI results averaged across the four nf runs (PR, R1, R2, R3). A) Average values of the FAA-based interaction effect are summarized in PPI interaction effect for the LA ROI for the experimental group (EG). B) Average values of the Table 2. The maps in Fig. 11 are FDR FBA-based PPI interaction effect for the LA ROI for the EG. C) Average values of the FAA- corrected with q<0.04 threshold, and based PPI interaction effect for the control group (CG). D) Average values of the FBA-based PPI interaction effect for the CG. the data in Table 2 – with q<0.02 threshold. Whole-brain statistical Importantly, the EG vs CG group differences in the PPI maps for the FBA-based PPI interaction effect, obtained in interaction effects for the LA ROI were significant with a similar way for the same contrast for the EG, are large effect sizes for both the FAA (t(22)=2.32, p<0.030, reported in Supplementary material (S2.5, Fig. S8, Table d=1.00) and the FBA (t(22)=3.90, p<0.001, d=1.69). S4). Average individual values of the FAA-based PPI Figure 12 compares the FAA- and FBA-based PPI interaction effect for the LA ROI across the four nf runs interaction effects from Fig. 11 and Fig. S8 for the left for the EG exhibited significant positive correlation with amygdala and its vicinity. The PPI results demonstrate the corresponding average values of the amygdala BOLD that both the FAA and FBA exhibited enhanced temporal laterality (NF: r=0.53, p<0.035), as illustrated in correlations, during the rtfMRI-EEG-nf task, with BOLD Supplementary Fig. S7A. When the individual results activities of the left amygdala and large brain networks. were averaged across three nf runs (out of four) These results are discussed in detail below. characterized by the most positive amygdala BOLD laterality values, the correlation was more significant 4. Discussion (NF*: r=0.61, p<0.012), as shown in Fig. S7B. The In this paper, we reported the first, proof-of-concept amygdala BOLD laterality (‘LA−RA’) was computed as application of the simultaneous real-time fMRI and EEG described in Supplementary material (S1.4). It can be neurofeedback (rtfMRI-EEG-nf) for emotion self- viewed as a performance measure characterizing target- regulation training in patients with a neuropsychiatric specific effects of the LA-based rtfMRI-nf procedure, as disorder, specifically, major depressive disorder (MDD). explained in Supplementary material (S2.4, see also Zotev This is also the first neurofeedback study in which et al., 2016). participants had an opportunity to simultaneously regulate Consistent with the results for the LA in Fig. 10A, the two rtfMRI-nf signals and two EEG-nf signals. average FAA-based PPI interaction effect for the L rACC Figure 11. Enhancement in temporal correlation between frontal alpha EEG asymmetry (FAA) and BOLD activity during the rtfMRI-EEG-nf training. Statistical maps of the FAA-based PPI interaction effect for the Happy Memories vs Count condition contrast (H vs C) are shown for the experimental group (EG). The maps are voxel-wise FDR corrected and projected onto the standard TT_N27 anatomical template, with 3 mm separation between axial slices. The number adjacent to each slice indicates the z coordinate in mm. The left hemisphere (L) is to the reader’s right. Peak t-statistics values for the FAA-based PPI interaction effect and the corresponding locations are specified in Table 2. Furthermore, we implemented the advanced real-time 4.2. Neurofeedback performance EEG-fMRI artifact correction procedure that made EEG- nf during fMRI practical and efficient. During the rtfMRI-EEG-nf training, the MDD patients in the EG learned to significantly increase BOLD activity 4.1. Emotional state changes of the LA (Fig. 4C) and significantly upregulate the FAA Following the rtfMRI-EEG-nf session, the MDD (Figs. 4A). These are the primary outcome measures in patients in the EG showed significant mood our study. The EG participants also achieved significant improvements, including significant reductions in state enhancement in fMRI connectivity between the LA and depression, confusion, total mood disturbance, and state the L rACC (Fig. 4D) and significant upregulation of the anxiety, as well as significant increase in state happiness FBA (Fig. 4B), which are the secondary outcome (Table 1). These improvements, characterized by medium measures. These results support the first of the two main hypotheses in our study (Sec. 1). Importantly, the EG vs effect sizes, suggest that the rtfMRI-EEG-nf training may be beneficial to MDD patients. The significant reduction CG group differences in these four measures either were in confusion may indicate that the EG participants were significant or trended toward significance (Sec. 3.2), able to develop a better grasp of the experimental indicating that effects of the rtfMRI-EEG-nf were specific procedure as the training continued, despite the relative to the EG and different from those of the sham feedback complexity of the rtfMRI-EEG-nf task. During the for the CG. The left rACC activity levels exhibited experiment, the EG participants rated their abilities to positive linear trends across the four nf runs for the EG recall autobiographical memories and feel happiness (Fig. 6). higher, than the CG participants (Supplementary Fig. S4). In our previous study, which combined rtfMRI-nf of These group differences, with large effect sizes, suggest the LA activity with passive EEG (Zotev et al., 2016), the that the rtfMRI-EEG-nf provided information more effect size for the LA activation was large (d=0.87, 95% consistent with the Happy Memories task than the sham CI [0.21 1.50]), while the effect size for the associated feedback. FAA increase was small (d=0.45, 95% CI [−0.14 1.01]). In the present work, both effects sizes were large (d=0.80, 4.4. Prefrontal BOLD laterality Performance of the rtfMRI-EEG-nf task was associated with pronounced laterality of BOLD activations for the dorsal PFC regions (Supplementary Fig. S5). BOLD activity levels during the rtfMRI-EEG-nf task (relative to the Count condition) were more positive for the MidFG and SFG areas on the left, compared to the corresponding MidFG and SFG areas on the right (Fig. S5). The most positive BOLD contrast t-score was observed for the left MidFG (BA 8) at (−40, 20, 47), while the most negative contrast t-score occurred in the right SFG (BA 8) at (27, Figure 12. Comparison of the FAA- and FBA-based PPI interaction effects for the left amygdala region. A) FAA-based PPI interaction 17, 49) (Table S2). These locations are parts of the left effects from Fig. 11. B) FBA-based PPI interaction effects from and right DLPFC, respectively. Furthermore, EEG Supplementary Fig. S8. The green crosshairs (x=−21 mm, z=−16 electrodes F3 and F4 are situated above the MidFG and mm) correspond to the center of the LA target ROI. BA 8. Thus, the significant positive MidFG BOLD laterality, illustrated in Fig. 8B, is consistent with the 95% CI [0.23 1.36], Figs. 4A,C). This comparison (which significant positive FAA and FBA changes for channels should be taken with caution because of the wide F3 and F4 (Figs. 4A,B). The negative correlations confidence intervals) suggests potential benefits of the between the average MidFG BOLD laterality values and rtfMRI-EEG-nf compared to the rtfMRI-nf alone. the after-vs-before changes in the mood ratings (Fig. 8C, 4.3. FAA changes vs psychological measures Sec. 3.4) are consistent with the negative correlations between the average FAA changes and the same mood The average individual FAA changes during the rating changes (Fig. 7B). Therefore, the prefrontal BOLD rtfMRI-EEG-nf task for the EG showed significant laterality effects independently confirm the EEG positive correlations with the MDD patients’ depression asymmetry effects observed in the EEG data analyses. and anhedonia severities (Fig. 7A). These findings are consistent with those reported in our study of EEG 4.5. Amygdala-rACC connectivity enhancement correlates of the amygdala rtfMRI-nf (Zotev et al., 2016). Modulation of the left rACC BOLD activity MDD patients exhibit lower FAA levels compared to non- simultaneously with that of the LA (Fig. 1A) enabled (or, depressed individuals, particularly during an emotional at least, was consistent with) enhancement in the two challenge (Stewart et al., 2011, 2014). The more positive regions’ fMRI functional connectivity (Fig. 4D). This FAA changes during the rtfMRI-EEG-nf task in the effect was specific to the rtfMRI-EEG-nf, as evidenced by patients with more severe depression suggest the potential the significant EG vs CG group differences in fMRI for correction of the FAA deficiencies specific to MDD connectivities of the LA with three loci close to the center (Zotev et al., 2016). This reasoning is explained in more of the L rACC target ROI (Supplementary Fig. S6B). In detail in the follow-up study (Zotev and Bodurka, 2020). future studies, an rtfMRI-nf based on an actual fMRI It is supported by the observed mood improvements: the connectivity metric, such as the Pearson’s correlation MDD patients, who achieved more positive average FAA coefficient, could be implemented and used together with changes during the rtfMRI-EEG-nf task, showed stronger the rtfMRI-nf of the amygdala activity. The enhancement reductions in state depression and total mood disturbance in fMRI connectivity between the LA and the left rACC after the training (Fig. 7B). during the rtfMRI-EEG-nf task for the EG showed The positive correlations between the FAA changes negative association with the MDD patients’ depression and the depression and anhedonia severities in Fig. 7A are severity and positive association with reward not as pronounced as those in our previous work (Zotev et responsiveness (Fig. 9). This means that a stronger al., 2016). The primary reason is that the FAA in the interaction between these two regions during positive present study was explicitly modulated via the EEG-nf. emotion induction with rtfMRI-EEG-nf should be Consequently, the FAA changes depended on individual beneficial to MDD patients. This observation is consistent EEG-nf performance, including a participant’s attention to with results of the previous studies that emphasized the the FAA-based EEG-nf signal and effort to regulate it. important role of the rACC in emotion regulation and Indeed, the positive correlation between the FAA changes modulation on the amygdala activity (e.g. Etkin et al., and the depression severity was most pronounced for the 2006; Pizzagalli, 2011; Zotev et al., 2013). Practice run, when the participants were first exposed to the rtfMRI-EEG-nf, and became weaker as the training 4.6. EEG-fMRI correlations for the left amygdala continued (Sec. 3.3). The FAA-based PPI interaction results for the LA ROI The results in our study demonstrate involvement of (Fig. 10A) demonstrate that temporal correlation between the left premotor cortex (PMC), specifically the precentral the FAA time course (convolved with the HRF) and the gyrus (PrecG), BA 6 in performance of the rtfMRI-EEG- LA BOLD activity was significantly stronger during the nf task. A local maximum of the FAA-based PPI rtfMRI-EEG-nf task than during the control condition. interaction effect is observed near the border of the left Similarly, the FBA-based PPI interaction results (Fig. PrecG and MidFG at (−42, −4, 44) (Fig. 11, Table 2). This 10B) indicate significant enhancement in temporal locus is relatively close to the location of the maximum of correlation between the FBA time course and the LA the corresponding BOLD activity contrast in the left BOLD activity. These EEG-based PPI interactions had MidFG at (−40, 20, 47) (Fig. S5, Table S2), which is also medium effect sizes for the EG, and large effect sizes near the anterior boundary of the PrecG. Furthermore, the when compared to those for the CG (Sec. 3.6). Therefore, EG vs CG group difference in the LA connectivity the EG participants were able, on the average, to enhancement during the rtfMRI-EEG-nf task was upregulate the FAA and FBA together with the LA prominent in nearby regions of the left MidFG at (−53, 8, activity. These results support the second of the two main 36) and PrecG at (−51, 1, 33) (Fig. S6A, Table S3). hypotheses in our study (Sec. 1). Collectively, these findings point to mutually consistent Interestingly, the average individual FAA-based PPI roles of the left DLPFC and the adjacent area of the left interaction effects for the LA showed significant positive PMC during the rtfMRI-EEG-nf task. correlation with the average individual amygdala BOLD The last observation is not surprising, because the laterality (Supplementary material S2.4, Fig. S7). This anterior (rostral) PMC has strong interconnections with means that the EG participants, who were more successful the prefrontal cortex (e.g. Chouinard and Paus, 2006; at upregulating BOLD activity of the target amygdala Hanakawa et al., 2003). A recent meta-analysis of rtfMRI- region (LA) relative to the non-target region (RA), were nf studies with various target regions revealed consistent also more successful at doing so simultaneously with fMRI activations of bilateral DLPFC areas extending to increasing the FAA-based EEG-nf signal. This PMC (Emmert et al., 2016). We hypothesize that the left observation confirms the connection between the FAA DLPFC in our study is involved in mental strategy changes and the amygdala BOLD laterality we reported implementation, while the left PMC is involved in previously (Zotev et al. 2016). observation and control of the variable-height nf bars. From this point of view, the enhanced temporal 4.7. fMRI correlates of the FAA modulation correlation between the FAA and the left PMC activity The whole-brain maps of the FAA-based PPI (Fig. 11, Table 2) suggests that the FAA modulation was interaction effect (Fig. 11) demonstrate that temporal closely associated with direct regulation of the FAA-based correlation between the FAA and BOLD activity was EEG-nf signal. significantly enhanced, during the rtfMRI-EEG-nf task for Interestingly, resting-state EEG source imaging studies the EG, not only for the left amygdala, but also for the have suggested that motivation is related to activities of large brain network. Note that the FAA- and FBA-based both the DLPFC and the PMC. Stronger reward bias in PPI regressors were orthogonalized with respect to the healthy individuals is associated with reduced upper alpha corresponding regressors based on the EEG power sums (alpha2) EEG activity (i.e. stronger activation) in the left for channels F3 and F4. Therefore, a positive PPI MidFG, left SFG, and left PrecG (BA 6) (Pizzagalli et al., interaction effect for a given region means that its BOLD 2005). In MDD patients, resting alpha EEG source activity increased simultaneously with activation of laterality index shows negative correlations with cortical areas contributing to EEG signal measured by F3, depression severity for both the MidFG and the PrecG and decreased simultaneously with deactivation of areas regions (Smith et al., 2018). It is suggested that the PMC contributing to EEG signal measured by F4. activity may “facilitate mobilization of the body for The results in Fig. 11 show significant positive PPI approach-motivated behaviors” (Smith et al., 2018). In the interaction effects for the corresponding left and right hierarchical model of approach/avoidance motivation by DLPFC regions (MidFG, BA 9), with maxima at (−36, 16, Sprielberg et al., 2013, the left DLPFC instantiates 24) and (49, 18, 24), respectively (Fig. 11, Table 2). These approach motivation at the strategic level, while the left results are consistent with the common view of frontal PMC subserves it at the tactical level. EEG asymmetry as reflecting activation of the left DLPFC In the left amygdala area, the main statistical maximum and deactivation of the right DLPFC (and vice versa). for the FAA-based PPI interaction effect is observed in the Within the approach-avoidance framework, these effects superficial (SF) subdivision of the amygdala at (−17, −5, are interpreted as indicative of enhanced approach −17) (Fig. 12A, Table 2). This finding is consistent with motivation and reduced avoidance motivation, that in our previous study (Zotev et al., 2016), which respectively. showed that the same PPI effect had the maximum in the SF subdivision at (−17, −3, −16) (Table 2 therein). Two maximize simultaneous engagement of both fMRI and additional maxima are found in the laterobasal (LB) EEG target brain activities. Second, the study participants amygdala subdivision at (−21, −6, −19) and (−21, −6, had, on the average, moderate depression (Table S1). −11) (Fig. 12A, Table 2). Compared to the LB, the SF Recruitment of more unmedicated MDD patients with subdivision is more closely involved in processing severe depression will help to elucidate effects of the reward-related and socially relevant information, as well rtfMRI-EEG-nf that are specific to MDD. Third, the sham as in modulation of approach-avoidance behavior (Bzdok feedback signals, provided to the control group et al., 2013). participants, were computer generated and unrelated to brain activity. Further research on rtfMRI-EEG-nf will 4.8. fMRI correlates of the FBA modulation benefit from a more realistic sham feedback, utilizing The FBA-based PPI interaction effects are most actual real-time fMRI and EEG data. pronounced along the cortical midline and the cingulate gyrus (Supplementary material S2.5, Fig. S8, Table S4). 5. Conclusion Elevated high-beta activity in these areas, often with some Our simultaneous real-time fMRI and EEG lateralization to the right, is associated with anxiety (e.g. neurofeedback (rtfMRI-EEG-nf) procedure provided Zotev and Bodurka, 2020, and references therein). proof-of-concept demonstration of intended target Significant FBA-based PPI interaction effects are also engagements and modulatory effects on recruited brain found for many regions involved in autobiographical circuitry dynamics. Furthermore, we observed enhanced memory retrieval, including the hippocampus, the temporal correlations of the target EEG and fMRI activity extended areas of the parahippocampal gyrus, the anterior measures, clearly indicating the ability of both thalamus, the precuneus, the posterior cingulate, the neurofeedback modalities to capture common aspects of lingual gyrus (involved in visual memory), and others neuronal activity. In our opinion, the rtfMRI-EEG-nf is (Fig. S8, Table S4). For the PMC areas, the PPI effects are worth implementation efforts, because it is a powerful less pronounced than those for the FAA. These findings approach to influence brain activity in a more suggest that the FBA modulation during the rtfMRI-EEG- experimentally controllable fashion and investigate nf task might have been more closely associated with resulting changes in spatial and temporal brain dynamics. variations in anxiety and activity of the autobiographical Our study suggests that the rtfMRI-EEG-nf can benefit memory system than with direct regulation of the FBA- depressed individuals and may have potential for based EEG-nf signal. Indeed, high-beta EEG activity is treatment of MDD. The described rtfMRI-EEG-nf relevant to the autobiographical memory function and implementation with two rtfMRI-nf and two EEG-nf limbic functions in general (e.g. Cannon et al., 2005; signals is an advanced and versatile neuromodulation tool. Paquette et al., 2009). Efficient mental strategies and imaginative experimental In the left amygdala region, the main statistical designs will be needed to take full advantage of the maximum for the FBA-based PPI interaction effect is opportunities it offers. Ultimately, effectiveness of the observed in the LB amygdala subdivision at (−28, −5, rtfMRI-EEG-nf compared to either of the individual −10) (Fig. 12B, Table S4). This result is consistent with neurofeedback modalities will have to be demonstrated. that in our previous work (Zotev et al., 2014), which showed that the same PPI effect was more pronounced in Conflict of interest the LB subdivision of the left amygdala (Fig. 4 therein). The authors declare that the research was conducted in Therefore, while the FAA temporal variations during the the absence of any commercial or financial relationships rtfMRI-EEG-nf task exhibited enhanced correlations with that could be construed as a potential conflict of interest. BOLD activities of both the SF and LB amygdala Funding subdivisions, the FBA variations showed enhanced correlation mainly with activity of the LB subdivision. This work was supported by the Laureate Institute for Brain Research and the William K. Warren Foundation, 4.9. Study limitations and in part by the P20 GM121312 award from National The reported study has several limitations. First, the Institute of General Medical Sciences, National Institutes experimental protocol with happy emotion induction of Health. based on recall of happy autobiographical memories was Acknowledgments adopted from our earlier studies on the amygdala rtfMRI- nf (Zotev et al., 2011; also Young et al., 2014), and was We would like to thank Dr. Tracy Warbrick and Dr. not optimized for the rtfMRI-EEG-nf. In future studies, Brett Bays of Brain Products, GmbH for their continued mental strategies and training procedures will have to be help, inspired teaching, and excellent technical support. developed specifically for the rtfMRI-EEG-nf to D.E.J., et al., 2016. Meta-analysis of real-time fMRI neurofeedback References studies using individual participant data: how is brain regulation Allen, J.J.B., Harmon-Jones, E., Cavender, J.H. (2001). Manipulation mediated? NeuroImage 124, 806-812. of frontal EEG asymmetry through biofeedback alters self-reported Etkin, A., Egner, T., Peraza, D.M., Kandel, E.R., Hirsch, J., 2006. emotional responses and facial EMG. Psychophysiology, 38, 685- Resolving emotional conflict: a role for the rostral anterior cingulate cortex in modulating activity in the amygdala. Neuron 51, Allen, P.J., Polizzi, G., Krakow, K., Fish, D.R., Lemieux, L., 1998. 871-882. Identification of EEG events in the MRI scanner: the problem of Friston, K.J., Buechel, C., Fink, G.R., Morris, J., Rolls, E., Dolan, R.J., pulse artifact and a method for its subtraction. NeuroImage 8, 229- 1997. Psychophysiological and modulatory interactions in neuroimaging. NeuroImage 6, 218-229. Allen, P.J., Josephs, O., Turner, R., 2000. A method for removing Gaume, A., Vialatte, A., Mora-Sánchez, A., Ramdani, C., Vialatte, imaging artifact from continuous EEG recorded during functional F.B., 2016. A psychoengineering paradigm for the neurocognitive MRI. NeuroImage 12, 230-239. mechanisms of biofeedback and neurofeedback. Neuroscience and American Psychiatric Association, 2000. Diagnostic and Statistical Biobehavioral Reviews 68, 891-910. Manual of Mental Disorders, 4th ed. Text Rev. (DSM-IV-TR). Gitelman, D.R., Penny, W.D., Ashburner, J., Friston, K.J., 2003. American Psychiatric Press, Washington, DC. Modeling regional and psychophysiologic interactions in fMRI: the Anand, A., Li, Y., Wang, Y., Wu, J., Gao, S., Bukhari, L., et al., 2005. importance of hemodynamic deconvolution. NeuroImage 19, 200- Antidepressant effect on connectivity of the mood-regulating circuit: an fMRI study. Neuropsychopharmacology 30, 1334-1344. Glover, G.H., Li, T.Q., Ress, D., 2000. Image-based method for Bagby, R.M., Parker, J.D.A., Taylor, G.J., 1994. The twenty-item retrospective correction of physiological motion effects in fMRI: Toronto Alexithymia Scale – I. Item selection and cross-validation RETROICOR. Magnetic Resonance in Medicine 44, 162-167. of the factor structure. Journal of Psychosomatic Research 38, 23- Hamilton, M., 1959. The assessment of anxiety states by rating. British Journal of Medical Psychology 32, 50-55. Baehr, E., Rosenfeld, J.P., Baehr, R., 1997. The clinical use of an alpha Hamilton, M., 1960. A rating scale for depression. Journal of asymmetry protocol in the neurofeedback treatment of depression: Neurology, Neurosurgery, and Psychiatry 23, 56-62. two case studies. Journal of Neurotherapy 2, 10-23. Hanakawa, T., Immisch, I., Toma, K., Dimyan, M.A., van Gelderen, P., Bell, A.J., Sejnowski, T.J., 1995. An information-maximization Hallett, M., 2003. Functional properties of brain areas associated approach to blind separation and blind deconvolution. Neural with motor execution and imagery. Journal of Neurophysiology 89, Computation 7, 1129-1159. 989-1002. Bzdok, D., Laird, A.R., Zilles, K., Fox, P.T., Eickhoff, S.B., 2013. An Harmon-Jones, E., Gable, P.A., 2018. On the role of asymmetric frontal investigation of the structural, connectional, and functional cortical activity in approach and withdrawal motivation: un updated subspecialization in the human amygdala. Human Brain Mapping review of the evidence. Psychophysiology 55, e12879. 34, 3247-3266. Mano, M., Lécuyer, A., Bannier, E., Perronnet, L., Noorzadeh, S., Cannon, R., Lubar, J., Thornton, K., Wilson, S., Congedo, M., 2005. Barillot, C., 2017. How to build a hybrid neurofeedback platform Limbic beta activation and LORETA: can hippocampal and related combining EEG and fMRI. Frontiers in Neuroscience 11, 140. limbic activity be recorded and changes visualized using LORETA Masterton, R.A.J., Abbott, D.F., Fleming, S.W., Jackson, G.D., 2007. in an affective memory condition? Journal of Neurotherapy 8, 5-24. Measurement and reduction of motion and ballistocardiogram Carver, C.S., White, T.L., 1994. Behavioral inhibition, behavioral artefacts from simultaneous EEG and fMRI recordings. activation, and affective responses to impending reward and NeuroImage 37, 202-211. punishment: the BIS/BAS scales. Journal of Personality and Social Mayberg, H.S., Brannan, S.K., Mahurin, R.K., Jerabek, P.A., Psychology 67, 319-333. Brickman, J.S., Tekell, J.L., et al., 1997. Cingulate function in Cavazza, M., Aranyi, G., Charles, F., Porteous, J., Gilroy, S., Klovatch, depression: a potential predictor of treatment response. I., et al., 2014. Towards empathic neurofeedback for interactive NeuroReport 8, 1057-1061. storytelling. OpenAccess Series in Informatics 41, 42-60. Mayeli, A., Zotev, V., Refai, H., Bodurka, J., 2016. Real-time EEG Choi, S.W., Chi, S.E., Chung, S.Y., Kim, J.W., Ahn, C.Y., Kim, H.T., artifact correction during fMRI using ICA. Journal of Neuroscience 2011. Is alpha wave neurofeedback effective with randomized Methods, 274, 27-37. clinical trials in depression? A pilot study. Neuropsychobiology 63, McNair, D.M., Lorr, M., Droppleman, L.F., 1971. Profile of Mood 43-51. States. Educational and Industrial Testing Service, San Diego, CA. Chouinard, P.A., Paus, T., 2006. The primary motor and premotor areas Micoulaud-Franchi, J.A., McGonigal, A., Lopez, R., Daudet, C., of the human cerebral cortex. The Neuroscientist 12, 143-152. Kotwas, I., Bartolomei, F., 2015. Electroencephalographic Cook, I.A., O’Hara, R., Uijtdehaage, S.H.J., Mandelkern, M., Leuchter, neurofeedback: level of evidence in mental and brain disorders and A.F., 1998. Assessing the accuracy of topographic EEG mapping suggestions for good clinical practice. Neurophysiologie Clinique for determining local brain function. Electroencephalography and 45, 423-433. Clinical Neurophysiology 107, 408-414. Montgomery, S.A., Asberg, M., 1979. A new depression scale designed Cox, R.W., 1996. AFNI: software for analysis and visualization of to be sensitive to change. British Journal of Psychiatry 134, 382- functional magnetic resonance neuroimages. Computers and Biomedical Research 29, 162-173. Mulert, C., Lemieux, L. (Eds.), 2010. EEG-fMRI: Physiological Basis, Cox, R.W., Hyde, J.S., 1997. Software tools for analysis and Technique, and Applications. Springer-Verlag, Berlin Heidelberg. visualization of fMRI data. NMR in Biomedicine 10, 171-178. Paquette, V., Beauregard, M., Beaulieu-Prevost, D., 2009. Effect of a Cox, R.W., Jesmanowicz, A., 1999. Real-time 3D image registration psychoneurotherapy on brain electromagnetic tomography in for functional MRI. Magnetic Resonance in Medicine 42, 1014- individuals with major depressive disorder. Psychiatry Research: Neuroimaging 174, 231-239. Davidson, R.J., 1996. Cerebral asymmetry, emotion, and affective Peeters, F., Oehlen, M., Ronner, J., van Os, J., Lousberg, R., 2014. style. In: Davidson, R.J., Hugdahl, K. (Eds.), Brain Asymmetry. Neurofeedback as a treatment for major depressive disorder – a The MIT Press, Cambridge, MA, pp. 361-387. pilot study. PLoS ONE 9, e91837. Emmert, K., Kopel, R., Sulzer, J., Brühl, A.B., Berman, B.D., Linden, Perronnet, L., Lécuyer, A., Mano, M., Bannier, E., Lotte, F., Clerc, M., et al., 2017. Unimodal versus bimodal EEG-fMRI neurofeedback General Psychiatry 67, 1128-1138. of a motor imagery task. Frontiers in Human Neuroscience 11, 193. Young, K.D., Zotev, V., Phillips, R., Misaki, M., Yuan, H., Drevets, Pizzagalli, D.A., 2011. Frontocingulate dysfunction in depression: W.D., et al., 2014. Real-time fMRI neurofeedback training of toward biomarkers of treatment response. amygdala activity in patients with major depressive disorder. PLoS Neuropsychopharmacology 36, 183-206. ONE 9, e88785. Pizzagalli, D.A., Nitschke, J.B., Oakes, T.R., Hendrick, A.M., Horras, Young, K.D., Siegle, G.J., Zotev, V., Phillips, R., Misaki, M., Yuan, K.A., Larson, C.L., et al., 2002. Brain electrical tomography in H., et al., 2017. Randomized clinical trial of real-time fMRI depression: the importance of symptom severity, anxiety, and amygdala neurofeedback for major depressive disorder: effects on melancholic features. Biological Psychiatry 52, 73-85. symptoms and autobiographical memory recall. American Journal Pizzagalli, D.A., Sherwood, R.J., Henriques, J.B., Davidson, R.J., of Psychiatry 174, 748-755. 2005. Frontal brain asymmetry and reward responsiveness: a Yuan, H., Young, K.D., Phillips, R., Zotev, V., Misaki, M., Bodurka, source-localization study. Psychological Science 16, 805-813. J., 2014. Resting-state functional connectivity modulation and Price, J.L., Drevets, W.C., 2012. Neural circuits underlying the sustained changes after real-time functional magnetic resonance pathophysiology of mood disorders. Trends in Cognitive Sciences imaging neurofeedback in depression. Brain Connectivity 4, 690- 16, 61-71. 701. Quaedflieg, C.W.E.M., Smulders, F.T.Y., Meyer, T., Peeters, F., Zotev, V., Krueger, F., Phillips, R., Alvarez, R.P., Simmons, W.K., Merckelbach, H., Smeets, T., 2016. The validity of individual Bellgowan, P., et al., 2011. Self-regulation of amygdala activation frontal alpha asymmetry EEG neurofeedback. Social Cognitive and using real-time fMRI neurofeedback. PLoS ONE 6, e24522. Affective Neuroscience 11, 33-43. Zotev, V., Yuan, H., Phillips, R., Bodurka, J., 2012. EEG-assisted Ramot, M., Kimmich, S., Gonzalez-Castillo, J., Roopchansingh, V., retrospective motion correction for fMRI: E-REMCOR. Popal, H., White, E., et al., 2017. Direct modulation of aberrant NeuroImage 63, 698-712. brain network connectivity through real-time neurofeedback. eLife Zotev, V., Phillips, R., Young, K.D., Drevets, W.C., Bodurka, J., 2013. 6, e28974. Prefrontal control of the amygdala during real-time fMRI Rosenfeld, J.P., Cha, G., Blair, T., Gotlib, I.H., 1995. Operant neurofeedback training of emotion regulation. PLoS ONE 8, (biofeedback) control of left-right frontal alpha power differences: e79184. potential neurotherapy for affective disorders. Biofeedback and Zotev, V., Phillips, R., Yuan, H., Misaki, M., Bodurka, J., 2014. Self- Self-Regulation 20, 241-258. regulation of human brain activity using simultaneous real-time Smith, E.E., Cavanagh, J.F., Allen, J.J.B., 2018. Intracranial source fMRI and EEG neurofeedback. NeuroImage 85, 985-995. activity (eLORETA) related to scalp-level asymmetry scores and Zotev, V., Yuan, H., Misaki, M., Phillips, R., Young, K.D., Feldner, depression status. Psychophysiology 55, e13019. M.T., et al., 2016. Correlation between amygdala BOLD activity Snaith, R.P., Hamilton, M., Morley, S., Humayan, A., Hargreaves, D., and frontal EEG asymmetry during real-time fMRI neurofeedback Trigwell, P., 1995. A scale for the assessment of hedonic tone: the training in patients with depression. NeuroImage: Clinical 11, 224- Snaith-Hamilton Pleasure Scale. British Journal of Psychiatry 167, 238. 99-103. Zotev, V., Misaki, M., Phillips, R., Wong, CK, Bodurka, J., 2018a. Spielberg, J.M., Miller, G.A., Engels, A.S., Herrington, J.D., Sutton, Real-time fMRI neurofeedback of the mediodorsal and anterior B.P., Banich, M.T., Heller, W., 2011. Trait approach and avoidance thalamus enhances correlation between thalamic BOLD activity motivation: lateralized neural activity associated with executive and alpha EEG rhythm. Human Brain Mapping 39, 1024-1042. function. NeuroImage 54, 661-670. Zotev, V., Phillips, R., Misaki, M., Wong, C.K., Wurfel, B.E., Krueger, Spielberg, J.M., Heller, W., Miller, G.A., 2013. Hierarchical brain F., et al., 2018b. Real-time fMRI neurofeedback training of the networks active in approach and avoidance goal pursuit. Frontiers amygdala activity with simultaneous EEG in veterans with combat- in Human Neuroscience 7, 284. related PTSD. NeuroImage: Clinical 19, 106-121. Spielberger, C.D., Gorsuch, R.L., Lushene, R.E., 1970. Test Manual for Zotev, V., Bodurka, J., 2020. Effects of simultaneous real-time fMRI the State Trait Anxiety Inventory. Consulting Psychologists Press, and EEG neurofeedback in major depressive disorder evaluated Palo Alto, CA. with brain electromagnetic tomography. Preprint Stewart, J.L. Coan, J.A., Towers, D.N., Allen, J.J.B., 2011. Frontal https://arxiv.org/abs/2006.01923 EEG asymmetry during emotional challenge differentiates individuals with and without lifetime major depressive disorder. Journal of Affective Disorders 129, 167-174. Stewart, J.L. Coan, J.A., Towers, D.N., Allen, J.J.B., 2014. Resting and task-elicited prefrontal EEG alpha asymmetry in depression: support for the capability model. Psychophysiology 51, 446-455. Suslow, T., Konrad, C., Kugel, H., Rumstadt, D., Zwitserlood, P., Schöning, S., et al., 2010. Automatic mood-congruent amygdala responses to masked facial expressions in major depression. Biological Psychiatry 67, 155-160. Talairach, J., Tournoux, P., 1988. Co-Planar Stereotaxic Atlas of the Human Brain. Thieme Medical Publishers, New York, NY. Thibault, R.T., MacPherson, A., Lifshitz, M., Roth, R.R., Raz, A., 2018. Neurofeedback with fMRI: a critical systematic review. NeuroImage 172, 786-807. Thibodeau, R., Jorgensen, R.S., Kim, S., 2006. Depression, anxiety, and resting frontal EEG asymmetry: a meta-analytic review. Journal of Abnormal Psychology 115, 715-729. Victor, T., Furey, M.L., Fromm, S.J., Öhman, A., Drevets, W.C., 2010. Relationship between amygdala responses to masked faces and mood state and treatment in major depressive disorder. Archives of Table 1. Participants’ emotional state measures before and after the rtfMRI-EEG-nf session. Emotional states were assessed using the Profile of Mood States (POMS), the State-Trait Anxiety Inventory (STAI), and the Visual Analogue Scale (VAS). Before After Effect Change Change Measure mean (SD) mean (SD) size (d) t-score# p-value [q] Experimental group (EG, n=16) POMS Depression 15.4 (14.0) 7.75 (10.1) −0.62 −2.49 0.025 [0.039]* Confusion 10.7 (5.91) 7.25 (4.22) −0.73 −2.91 0.011 [0.039]* Total mood disturbance 46.1 (39.9) 26.8 (28.0) −0.60 −2.39 0.030 [0.039]* STAI State anxiety 44.9 (12.2) 40.4 (10.5) −0.57 −2.26 0.039 [0.039]* VAS Happiness 4.56 (2.31) 5.94 (1.53) +0.59 +2.36 0.033 [0.039]* Control group (CG, n=8) POMS Depression 23.9 (17.4) 19.6 (13.4) −0.42 −1.18 0.276 [0.856] Confusion 11.4 (5.21) 10.4 (4.21) −0.29 −0.83 0.436 [0.856] Total mood disturbance 65.8 (40.4) 59.0 (34.8) −0.21 −0.58 0.580 [0.856] STAI State anxiety 53.8 (12.0) 53.3 (9.51) −0.04 −0.11 0.915 [0.915] VAS Happiness 2.38 (1.85) 2.75 (2.25) +0.15 +0.42 0.685 [0.856] # t(15) for the EG, t(7) for the CG, two-tailed. * FDR q<0.05 for the five tests. Table 2. Psychophysiological interaction effect, based on the time course of frontal alpha EEG asymmetry (FAA), for the Happy vs Count condition contrast for the experimental group (EG). Late- x, y, z Region t-score rality (mm) Frontal lobe Medial frontal gyrus (BA 6) L −3, −5, 54 17.6 Superior frontal gyrus (BA 6) L −21, −5, 65 12.1 Precentral gyrus (BA 4) L −39, −15, 42 11.5 Precentral gyrus (BA 6) R 49, −11, 26 10.3 Medial frontal gyrus (BA 11) R 2, 29, −14 9.51 Inferior frontal gyrus (BA 46) R 39, 33, 10 9.28 Inferior frontal gyrus (BA 44) L −51, 10, 12 8.89 Precentral gyrus (BA 4) R 45, −11, 51 8.86 Inferior frontal gyrus (BA 47) R 37, 16, −4 8.71 Middle frontal gyrus (BA 9) R 49, 18, 24 8.23 Precentral / mid. frontal gyrus (BA 6) L −42, −4, 44 7.81 Middle frontal gyrus (BA 9) L −36, 16, 24 6.33 Temporal lobe Superior temporal gyrus (BA 22) L −51, 1, 5 11.1 Superior temporal gyrus (BA 38) L −47, 15, −18 8.73 Superior temporal gyrus (BA 39) R 53, −53, 12 8.53 Middle temporal gyrus (BA 19/22) L −38, −59, 12 8.34 Superior temporal gyrus (BA 22) R 50, −15, 6 7.90 Superior temporal gyrus (BA 39) L −51, −55, 14 7.02 Transverse temporal gyrus (BA 41) L −29, −29, 8 6.95 Parietal lobe Inferior parietal lobule (BA 40) R 59, −43, 24 12.6 Precuneus (BA 7) R 5, −35, 44 10.2 Precuneus (BA 31/18) L −17, −68, 21 10.1 Precuneus (BA 7) R 1, −55, 48 9.57 Postcentral gyrus (BA 4) R 19, −33, 62 8.49 Postcentral gyrus (BA 3) L −27, −31, 60 8.40 Precuneus (BA 31) R 1, −48, 33 7.90 Limbic lobe Cingulate gyrus (BA 24) L −1, 2, 44 13.1 Amygdala / parahipp. gyrus L −17, −5, −17 8.73 Parahippocampal gyrus (BA 28) R 21, −13, −18 8.70 Amygdala / uncus L −21, −6, −19 7.86 Posterior cingulate (BA 29) L −12, −47, 18 7.85 Amygdala / parahipp. gyrus L −21, −6, −11 7.73 Uncus (BA 36) R 21, −3, −32 7.53 Parahippocampal gyrus (BA 28) L −21, −19, −8 6.94 Subcallosal gyrus (BA 25) L −9, 15, −14 6.72 Sub-lobar Thalamus, laterodorsal R 13, −19, 16 9.49 Claustrum R 33, −7, −4 9.38 Insula (BA 13) R 44, 0, 4 9.12 Thalamus, mediodorsal L −3, −13, 6 9.10 Thalamus, mediodorsal R 3, −15, 2 8.80 Culmen L −1, −57, −22 8.58 Declive L −19, −65, −18 7.47 Caudate body R 17, −9, 20 6.82 FDR q<0.02 for |t|>6.0; BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates. Supplementary material quantities. A few outlier points (included in the statistics computation) out of total n=120 points in each plot S1.1. Modified MR-compatible EEG cap correspond to situations when a current run is characterized by a very drastic (usually one per run) head To achieve more efficient real-time EEG-fMRI artifact motion, not present during the previous run, or vice versa. suppression, we modified a 32-channel BrainCap-MR The linear fits to the data for F3 and F4 have slopes of (EASYCAP, GmbH) as shown in Fig. 2D. In the modified 0.96 and 0.98, respectively (Fig. S1). Therefore, the use of cap, four EEG channels out of 31 – FC1, FC2, TP9, TP10 the regression coefficients optimized for the previous run – were re-purposed for acquisition of reference artifact yields signal variance reduction that is 96-98%, on the waveforms, which we refer to as R (t), R (t), R (t), and 1 2 3 average, of the variance reduction achieved with the R (t), instead of EEG activity. For each of these channels, regression coefficients optimized for the current run. the lead was disconnected from its electrode, and These results justify our proposed implementation of the connected to one end of a wire contour. The other end of linear regression procedure for improved real-time EEG- the contour was connected to the Ref electrode (FCz, blue) fMRI artifact correction, described in Sec. 2.5. via a 50 kOhm resistor. Geometries of the four contours were optimized so that electromotive forces (EMFs), S1.3. Reliability of the real-time EEG artifact correction induced in the contours during head movements in the Figure S2 compares the EEG-nf target measures MRI scanner’s main field, approximate cardioballistic computed in real time and the corresponding measures (CB) and random-motion artifacts picked up by EEG determined in offline EEG data analysis. The relative channels F3 and F4. The two shorter contours (brown frontal EEG asymmetries A and B were computed every 2 wires in Fig. 2D) followed the leads of channels F3 and s (Sec. 2.6) following the real-time EEG-fMRI artifact F4, respectively, then looped around the Gnd electrode correction procedure (Sec. 2.5), and their values were (AFz, black), and connected to the Ref via the resistors. saved to a file during each run. The real-time A and B The two longer contours (orange wires in Fig. 2D) also data, reported in Fig. S2, were taken from these files. The followed the leads of F3 and F4, looped around electrodes offline EEG data processing was applied to the raw EEG Fp1 and Fp2, respectively, then around the Gnd, and data recorded during fMRI. It included average artifact connected to the Ref through the resistors. The resistors subtraction (AAS) for MR artifacts, followed by AAS for were non-magnetic non-inductive surface mount thin film CB artifacts, and exclusion of bad intervals, as described resistors (Vishay PNM1206-50KBCT-ND, 50k, 0.1%, in detail below (S1.6). The offline A and B data in Fig. S2 0.4W). After the cap had been placed and aligned on a were computed after such processing. participant’s head, the wire contours were fixed tightly to Mean real-time A values for the Happy Memories the cap’s fabric with adhesive tape (3M Durapore). This conditions in each run and the corresponding mean offline EEG cap modification enabled acquisition of the four A values are compared in Fig. S2A. The results are pooled reference artifact waveforms, along with 27 EEG across five task runs for all participants. Fig. S2B waveforms and one ECG waveform, using the standard compares mean real-time A changes and mean offline A 32-channel system configuration. Importantly, the use of changes between the Rest and Happy Memories the modified EEG cap did not affect quality of structural conditions. Similar plots for the B values and B changes or functional MRI brain images. are shown in Figs. S2C and S2D. The correlations in Figs. S1.2. Performance of real-time EEG artifact regression S2A-D are highly significant with large effect sizes (r>0.5). These results suggest that the real-time EEG-nf Figure S1 evaluates effectiveness of the real-time target measures were sufficiently reliable, both for the regression of the CB and motion artifacts (Sec. 2.5, Fig. 3) alpha band (A) and for the high-beta band (B). The with the linear regression coefficients {a }, {b }, i=1…4, i i differences between the real-time and offline data in Fig. based on the previous run as opposed to the current run. S2 can be attributed to the following factors. First, the The actual real-time signal variance changes for channels offline AAS procedures for MR and CB artifacts are more F3 and F4 (same as in Fig. 3C) after the regression accurate than the corresponding real-time AAS procedure with the coefficients for a current run procedures, because they involve careful visual inspection determined by fitting the data for the previous run are of the artifact patterns across an entire run and manual shown along the y-axis in Fig. S1. The corresponding adjustment of the correction parameters (semi-automatic variance changes after an offline regression procedure correction mode). Second, the real-time artifact regression with the optimum regression coefficients for a current run procedure (Fig. 3B) can reduce some CB artifacts that are shown along the x-axis. The results are pooled across cannot be efficiently suppressed by the offline AAS, e.g. five task runs for all participants. The plots in Fig. S1 due to large variations in CB artifacts’ temporal profiles. demonstrate high correlation (r=0.98) between the two S1.4. fMRI data analysis To compare BOLD activity levels for the left and right amygdala, we considered amygdala BOLD laterality, i.e. a Offline analysis of the fMRI data was performed in difference in mean GLM-based fMRI percent signal AFNI (Cox, 1996; Cox and Hyde, 1997). Pre-processing changes between the LA and RA ROIs for each contrast, of single-subject fMRI data included time series despiking run, and participant. The LA and RA ROIs in this case using the 3dDespike AFNI program with -localedit option. were defined as the left and right amygdala regions It was followed by correction of cardiorespiratory artifacts specified in the AFNI implementation of the Talairach- using the AFNI implementation of the RETROICOR Tournoux brain atlas. Similarly, we computed BOLD method (Glover et al., 2000). Further fMRI pre-processing laterality for the middle frontal gyrus (MidFG) as a involved slice timing correction and volume registration difference in mean GLM-based fMRI percent signal of all EPI volumes acquired in the experiment using the changes between the left and right MidFG ROIs. The 3dvolreg AFNI program with two-pass registration. The ROIs were defined as the left and right middle frontal last volume of the short EPI dataset, acquired immediately gyrus regions specified in the AFNI implementation of the after the high-resolution anatomical MPRAGE brain Talairach-Tournoux atlas, and limited to a selected slab image (Sec. 2.6), was used as the registration base. along z-axis (z ≤ z ≤ z ). 1 2 To enable transformation of the fMRI data to the Talairach space, the Talairach transform was first S1.5. fMRI-based PPI analysis performed for each subject’s high-resolution anatomical To evaluate changes in the left amygdala fMRI MPRAGE brain image. The image was subjected to functional connectivity between experimental conditions, explicit skull-stripping using the 3dSkullStrip AFNI we conducted a psychophysiological interaction (PPI) program with -blur_fwhm option, and then transformed analysis (Friston et al., 1997; Gitelman et al., 2003). The towards the standard TT_N27 template in the Talairach analysis was based on fMRI time course for an LA seed space using the @auto_tlrc AFNI program. ROI. The seed ROI was defined as the left amygdala The fMRI activation analysis was performed according region specified in the AFNI implementation of the to the standard general linear model (GLM) approach. It Talairach-Tournoux brain atlas. (We used the anatomical was conducted for each of the five task fMRI runs (Fig. amygdala ROI, because the spherical LA target ROI (Fig. 1B) using the 3dDeconvolve AFNI program. The GLM 2A) includes some voxels outside the amygdala proper). model included two block-design stimulus condition The seed ROI was transformed to each subject’s terms, Happy Memories and Count, represented by the individual EPI space. In addition, 10-mm-diameter ROIs standard block-stimulus regressors in AFNI. A general were defined within the left and right frontal white matter linear test (-gltsym) term was included to compute the (WM) and within the left and right ventricle cerebrospinal Happy vs Count contrast. Nuisance covariates included fluid (CSF) using the individual high-resolution the six fMRI motion parameters and five polynomial anatomical brain image in the Talairach space (S1.4), and terms for modeling the baseline. To further reduce effects also transformed to the EPI space. The pre-processed of residual motion artifacts, the fMRI data and motion fMRI data and the six fMRI motion parameters were parameters were lowpass Fourier filtered at 0.1 Hz prior to bandpass filtered between 0.01 and 0.08 Hz using the the GLM analysis. GLM β coefficients were computed for 3dTproject AFNI program. The 3dmaskave AFNI each voxel, and average percent signal changes for Happy program was then used to compute average fMRI time vs Rest, Count vs Rest, and Happy vs Count contrasts courses for the LA, WM, and CSF ROIs. The LA seed were obtained by dividing the corresponding β values ROI time course was employed as the fMRI-based PPI (×100%) by the β value for the constant baseline term. correlation regressor. A PPI interaction regressor was The resulting fMRI percent signal change maps for each defined for the Happy Memories vs Rest condition run were transformed to the Talairach space by means of contrast as follows. A [Happy−Rest] contrast function was the @auto_tlrc AFNI program. The individual high- defined to be equal +1 for the Happy Memories condition resolution anatomical brain image in the Talairach space blocks, −1 for the preceding Rest condition blocks, and 0 was used as the transformation template. The maps were for all other condition blocks (Fig. 1B). The LA time re-sampled to 2×2×2 mm isotropic voxel size. course was detrended, using the 3dTproject AFNI Average individual BOLD activity levels were program, with respect to the time courses of the six fMRI computed in the offline analysis for the LA and L rACC motion parameters (together with the same time courses target ROIs, exhibited in Figs. 2A,B. The voxel-wise shifted by one TR), the time courses for the WM and CSF fMRI percent signal change data from the GLM analysis, ROIs, and five polynomial terms. It was then deconvolved transformed to the Talairach space, were averaged within using the 3dTfitter AFNI program to estimate a time these ROIs and used as GLM-based measures of these course of the underlying neuronal activity. This estimated regions’ BOLD activities. ‘neuronal’ time course was multiplied by the [Happy−Rest] contrast function, and convolved with the artifact to be subtracted was defined, for each EEG same hemodynamic response function (HRF, ‘Cox channel, by a moving average over 21 cardiac periods. special’) using the waver AFNI program. The resulting Cardiac periods with strong random-motion artifacts were waveform was employed as the fMRI-based PPI not included in the CB correction. interaction regressor for the Happy Memories vs Rest Following the MR and CB artifact removal, the EEG condition contrast. data from the five task runs (Fig. 1B) were concatenated A single-subject fMRI-based PPI analysis involved to form a single dataset. The data were carefully fitting a GLM model with the two PPI regressors using the examined, and intervals exhibiting significant motion or 3dDeconvolve AFNI program. The GLM design matrix instrumental artifacts were marked manually as “bad for each task run included four stimulus regressors, intervals” and excluded from the analysis. The signals sixteen covariates of no interest, and five polynomial from the four reference artifact channels were taken into terms for modeling the baseline. The stimulus regressors account to more reliably identify data intervals affected by included the fMRI-based PPI interaction regressor, the random head motions and distinguish them from intervals fMRI-based PPI correlation regressor, the Happy exhibiting neuronal activity (e.g. theta). Channel FCz was Memories block-stimulus regressor, and the Count block- kept as the EEG reference throughout the analysis. stimulus regressor. The last two regressors were the An independent component analysis (ICA) was standard block-design fMRI regressors in AFNI performed over the entire dataset with exclusion of the corresponding to the stimulus waveforms with 40-s-long bad intervals. This approach ensured that independent condition blocks (Fig. 1B). The covariates of no interest components (ICs) corresponding to various artifacts were included time courses of the six fMRI motion parameters, identified and removed in a consistent manner across all time courses of the same parameters shifted by one TR, five runs. The Infomax ICA algorithm (Bell and time courses of the left and right WM ROIs, and time Sejnowski, 1995), implemented in BrainVision Analyzer courses of the left and right ventricle CSF ROIs. Each 2.1, was applied to the data from 27 EEG channels and single-subject PPI analysis produced GLM-based R - yielded 27 ICs. Time courses, spectra, topographies, and statistics and t-statistics maps for the fMRI-based PPI kurtosis values of all the ICs were carefully analyzed to interaction and correlation terms for each run. These identify various artifacts, as well as EEG signals of statistics were used to compute voxel-wise PPI interaction neuronal origin. After all the ICs had been classified, an and correlation values. The resulting maps were subjected inverse ICA transform was applied to remove the to the Fisher r-to-z normalization, transformed to the identified artifacts from the EEG data. Because many Talairach space, re-sampled to 2×2×2 mm isotropic voxel artifacts had already been removed using the ICA, the data size, and spatially smoothed using isotropic Gaussian blur were examined again, and new bad intervals were defined with FWHM = 5 mm. The single-subject fMRI-based PPI to exclude remaining artifacts. interaction maps were submitted to whole-brain group PPI A time-frequency analysis was performed to compute analyses that employed the 3dttest++ AFNI program. EEG power for each channel as a function of time and frequency. The continuous wavelet transform with Morlet S1.6. EEG data analysis wavelets, implemented in BrainVision Analyzer 2.1, was Offline analysis of the EEG data was performed in applied to obtain EEG signal power in [0.5-30] Hz BrainVision Analyzer 2.1 (Brain Products, GmbH). frequency range with 0.5 Hz frequency resolution and 4 Removal of MR and cardioballistic (CB) artifacts was ms temporal sampling. An EEG power as a function of based on the average artifact subtraction (AAS) method time was then computed for each band of interest. (Allen et al., 1998, 2000) implemented in the Analyzer. S1.7. EEG-based PPI analyses The MR artifact template was defined using MRI slice markers recorded with the EEG data. After the MR artifact To investigate how temporal correlations between removal, the data were bandpass filtered between 0.5 and FAA (or FBA) and BOLD activity changed between 80 Hz (48 dB/octave) and downsampled to 250 S/s experimental conditions, we performed PPI analyses sampling rate (4 ms interval). The fMRI slice selection adapted for EEG-fMRI (Zotev et al., 2014, 2016, 2018a). frequency (17 Hz) and its harmonics were removed by The analyses followed the standard fMRI-based PPI band rejection filtering. The MR artifact removal was analysis approach (Friston et al., 1997; Gitelman et al., performed for the 27 EEG channels, the ECG channel, and 2003), except that the initial deconvolution step, used to the 4 reference artifact channels (re-purposed FC1, FC2, estimate an underlying neuronal activity from an fMRI TP9, TP10). Removal of CB artifacts and follow-up time course, was skipped, and the actual EEG activity analyses were conducted for the 27 EEG channels only. time course was employed. The EEG-based PPI analyses The CB artifact template was determined from the cardiac were conducted separately for the FAA and FBA time waveform recorded by the ECG channel, and the CB courses. EEG-based PPI regressors for the FAA were defined as interaction maps using the 3dttest++ AFNI program. The illustrated in Figure S3. The FAA values, computed with 4 analyses included two covariates: the participants’ ms temporal resolution for each experimental run, were MADRS depression severity ratings and average averaged for 200-ms-long time bins. The resulting individual values of the PPI interaction effect for a WM mask. The WM mask was defined for each participant as waveform was linearly detrended and orthogonalized with respect to the Happy Memories and Count stimulus follows. The individual high-resolution anatomical brain waveforms (Fig. 1B) using the glmfit() MATLAB image in the Talairach space (S1.4) was thresholded to program. This procedure removed variations in mean FAA select WM regions only, and the resulting mask was levels across the conditions to focus the analysis on multiplied by the standard WM mask in the Talairach temporal FAA variations around the means. The FAA space (TT_wm+tlrc). The mask was then re-sampled to time course was then converted to z-scores across each 2×2×2 mm voxels, and subjected to erosion by one voxel run. The HRF (‘Cox special’) was calculated with 200 ms to improve its separation from gray matter. The resulting sampling using the waver AFNI program. Convolution of individual WM mask in the Talairach space contained, on the z(FAA) time course with the HRF by means of the average, ~10000 voxels. The EEG-based PPI interaction conv() MATLAB program yielded a regressor, which we values, averaged within this WM mask, were used as a employed as the EEG-based PPI correlation regressor covariate vector in the group analyses to better account for (Fig. S3A). An EEG-based PPI interaction regressor was spurious PPI interaction effects. Such effects could be defined for the Happy Memories vs Count condition caused, e.g., by residual motion artifacts in the contrast as follows. A [Happy−Count] contrast function simultaneously acquired EEG and fMRI data. was set to be equal +1 for the Happy Memories condition S2.1. Verbal self-report performance ratings blocks, −1 for the Count condition blocks, and 0 for the Figure S4 exhibits average memory recall and Rest condition blocks (Fig. S3B). The z(FAA) time course was first multiplied by the [Happy−Count] contrast happiness ratings for each group. The two ratings were reported verbally by each participant after each function, and then convolved with the HRF (Fig. S3C). The resulting waveform was used as the EEG-based PPI experimental run (except Rest). The EG vs CG group differences in the individual ratings averaged across the interaction regressor for the Happy Memories vs Count four nf runs trended toward significance after correction condition contrast. The two PPI regressors – correlation and interaction – were sub-sampled to the middle time both for the memory recall ratings (EG vs CG, NF: t(22)=2.29, p<0.032, q<0.064, d=0.99) and the happiness points of fMRI volumes. Two additional PPI regressors ratings (EG vs CG, NF: t(22)=1.90, p<0.071, q<0.071, were defined in the same way using the power-sum function instead of the FAA. Prior to its inclusion in the d=0.82). For the Transfer run, group differences were significant after correction for both ratings (Memory- GLM model, the FAA-based PPI correlation regressor was recall, EG vs CG, TR: t(22)=2.61, p<0.016, q<0.043, linearly detrended and orthogonalized with respect to the corresponding power-sum-based PPI correlation regressor. d=1.13; Happiness, EG vs CG, TR: t(22)=2.39, p<0.026, q<0.043, d=1.04; corrected for the four tests). Similarly, the FAA-based PPI interaction regressor was linearly detrended and orthogonalized with respect to the S2.2. BOLD fMRI activity across the brain power-sum-based PPI interaction regressor. This Figure S5 exhibits whole-brain statistical maps of procedure ensured that the two EEG-based PPI regressors BOLD fMRI activity during the rtfMRI-EEG-nf training specifically reflect temporal variations in the FAA rather for the EG participants. The maps correspond to the than variations in the average power for the two channels. Happy Memories vs Count condition contrast (H vs C). For the FBA, the PPI regressors were defined in a similar The individual-subject fMRI percent signal change maps way starting with the FBA time course. were averaged for the four nf runs (PR, R1, R2, R3). The A single-subject EEG-based PPI analysis involved group mean was compared to zero using a one-sample t- fitting a GLM model using the 3dDeconvolve AFNI test (df=15, two-tailed). The statistical results are program. The design matrix for each task run had the summarized in Table S2. The maps in Fig. S5 are FDR same structure as described above for the fMRI-based PPI corrected with q<0.05 threshold, and the data in Table S2 analysis, except that the two EEG-based PPI regressors – – with q<0.01 threshold. The results demonstrate interaction and correlation – were used instead of the significant positive BOLD activity contrast for the left fMRI-based PPI regressors. The resulting PPI interaction amygdala region and many areas of the limbic system. and correlation maps were subjected to the Fisher They also reveal pronounced BOLD laterality for large normalization, Talairach transform, re-sampling, and parts of the middle frontal gyrus and superior frontal gyrus spatial smoothing as described above. Whole-brain group (Fig. S5, Table S2). PPI analyses were conducted for the EEG-based PPI S2.3. Amygdala fMRI connectivity changes Similar to the results for the FAA (Fig. 11), the whole- brain maps of the FBA-based PPI interaction effect (Fig. Figure S6 shows whole-brain statistical maps for the S8) demonstrate that performance of the rtfMRI-EEG-nf EG vs CG group difference in the LA fMRI connectivity task was associated with enhancement in temporal changes during the rtfMRI-EEG-nf training. The statistics correlations between the FBA and BOLD activities of the are summarized in Table S3. The results reveal three loci large brain network. In particular, significant positive in the rACC area, characterized by the most pronounced FBA-based PPI interaction effects are observed for the EG vs CG group differences: (−8, 34, 7) with t=4.41, (−9, corresponding left and right DLPFC regions (MidFG, BA 41, 5) with t=4.04, and (3, 35, 9) with t=3.34 (Table S3). 9), with maxima at (−29, 35, 32) and (31, 41, 28), These loci are pointed by green arrows in Fig. S6B. The respectively (Fig. S8, Table S4). These maxima are EG vs CG group differences in fMRI connectivity changes located closer to the medial plane, than the similar between the LA and 10-mm-diameter spherical ROIs maxima for the FAA-based PPI effect (Fig. 11, Table 2). centered at these locations were significant with large The results in Fig. S8 show the most pronounced FBA- effect sizes: t(22)=3.31, p<0.003, d=1.43 for the (−8, 34, based PPI interaction effects along the cortical midline 7) centered ROI; t(22)=3.31, p<0.003, d=1.43 for the (−9, and the cingulate gyrus, including the anterior and 41, 5) centered ROI; and t(22)=3.06, p<0.006, d=1.32 for posterior cingulate (Fig. S8, Table S4). The statistical the (3, 35, 9) centered ROI. The average connectivity maximum is at (−19, 12, 42) in the left SFG (BA 8) and changes between the LA and the right rACC ROI centered the effect extends down to the ACC (BA 32). Elevated at (3, 35, 9) showed significant negative correlation with high-beta activity in these areas, often with some the MADRS depression severity ratings for the EG lateralization to the right, is associated with anxiety (e.g. (r=−0.55, p<0.027). Zotev and Bodurka, 2020 and references therein). S2.4. Amygdala BOLD laterality interpretation Significant FBA-based PPI interaction effects are also found for many regions involved in autobiographical The amygdala BOLD laterality (‘LA−RA’) is a memory retrieval, including the hippocampus, the relevant metric for assessing target-specific effects of the extended areas of the parahippocampal gyrus, the anterior rtfMRI-nf procedure aimed at upregulating the LA BOLD thalamus, the precuneus (BA 31), the posterior cingulate activity for the following reasons (Zotev et al., 2016). (BA 29), the lingual gyrus (involved in visual memory), First, both the LA and RA are activated by happy emotion and others (Fig. S8, Table S4). For the PMC areas (PrecG, induction, as evidenced by substantial BOLD activations BA 6), the FBA-based PPI interaction effects are less of the RA in our studies that used rtfMRI-nf of the LA pronounced than the corresponding FAA-based effects. activity (e.g. Zotev et al., 2011, 2016; Young et al., 2014). Second, emotional side effects that accompany performance of the difficult neurofeedback task (e.g. confusion, frustration, anxiety, etc.) conceivably involve activations of both the LA and RA. Third, due to the relative proximity of the LA and RA, their apparent BOLD activity levels may be affected by similar fMRI artifacts (e.g. signal losses due to magnetic susceptibility variations). Because only the LA BOLD activity (and not that of the RA) is explicitly upregulated using the rtfMRI- nf in our studies, the amygdala BOLD laterality is more sensitive to target-specific effects of the rtfMRI-nf procedure. S2.5. fMRI correlates of the FBA modulation Figure S8 shows whole-brain statistical maps for the FBA-based PPI interaction effect for the Happy Memories vs Count contrast for the EG. The group analysis was conducted in the same way as described for the FAA. Statistical results for the FBA-based PPI interaction effect are summarized in Table S4. The maps in Fig. S8 are voxel-wise FDR corrected at q<0.07 level, and the data in Table S4 – at q<0.05 level. Figure S1. Effectiveness of the real-time EEG-fMRI artifact regression procedure with the regression coefficients {ai} and {bi} determined by fitting the data for the previous run (y-axis), compared to the effectiveness of a similar procedure with the optimum coefficients determined by fitting the data for the same (current) run (x-axis). Results for 120 experimental runs (24 participants, 5 task runs) are pooled together in each plot. Figure S2. Comparison of relative frontal EEG asymmetry measures A (alpha band) and B (high-beta band), determined for the EEG data after the real-time processing and for the same data after the offline EEG data processing. Results for 120 experimental runs (24 participants, 5 task runs) are pooled together in each plot. A) Mean A values for the Happy Memories conditions in each run. B) Mean A changes between the Rest and Happy Memories conditions (H vs R) in each run. C) Mean B values for the Happy Memories conditions. D) Mean B changes between the Rest and Happy Memories conditions. Figure S3. Definition of regressors for EEG-based psychophysiological interaction (PPI) analysis of fMRI data. A) Convolution of a time course of the FAA (converted to z-scores) with the HRF yields an EEG-based PPI correlation regressor. B) Contrast function for the Happy Memories (H) versus Count (C) conditions for one experimental run. C) Convolution of the FAA time course, multiplied by the contrast function, with the HRF yields an EEG-based PPI interaction regressor for the Happy Memories vs Count condition contrast. Figure S4. Average memory recall and happiness ratings reported by the participants after each experimental run. EG – experimental group, CG – control group. The error bars are standard errors of the mean (sem). The NF refers to group difference statistics (p-value from an independent- samples t-test and the corresponding effect size d) for the individual ratings averaged across the four nf runs (PR, R1, R2, R3). Figure S5. Statistical maps of BOLD fMRI activity, corresponding to the Happy Memories vs Count condition contrast, during the rtfMRI-EEG- nf training for the experimental group (EG). The individual fMRI activity results were averaged across the four nf runs (PR, R1, R2, R3). The maps are voxel-wise FDR corrected and projected onto the standard TT_N27 anatomical template in the Talairach space, with 3 mm separation between axial slices. The number adjacent to each slice indicates the z coordinate in mm. Following the radiological notation, the left hemisphere (L) is shown to the reader’s right. The t-statistics maxima and minima and the corresponding locations are specified in Table S2. Figure S6. A) Statistical maps of the experimental vs control group difference (EG vs CG) in the left amygdala fMRI functional connectivity changes during the rtfMRI-EEG-nf training. The fMRI connectivity changes between the Rest and Happy Memories conditions (H vs R) were evaluated in the psychophysiological interaction (PPI) analysis, based on the LA time course. The individual PPI interaction results were averaged across the four nf runs (PR, R1, R2, R3). The maps are projected onto the standard TT_N27 anatomical template in the Talairach space. The number adjacent to each slice indicates the z coordinate in mm. Peak t-statistics values and the corresponding locations are specified in Table S3. B) Loci in the rACC area, which exhibited the largest EG vs CG group differences, are pointed by green arrows: (−8, 34, 7), (−9, 41, 5), and (3, 35, 9). The green crosshairs mark the center of the L rACC target ROI. Figure S7. Correlations between average values of the amygdala BOLD laterality (‘LA−RA’) and the corresponding average values of the FAA- based PPI interaction effect for the LA ROI for the experimental group (EG). Both quantities correspond to the Happy Memories vs Count (H vs C) condition contrast. A) The individual results were averaged across the four nf runs (PR, R1, R2, R3). B) The individual results were averaged for three nf runs (out of four) with the most positive amygdala BOLD laterality values. Figure S8. Enhancement in temporal correlation between frontal high-beta EEG asymmetry (FBA) and BOLD activity during the rtfMRI-EEG- nf training. Statistical maps of the FBA-based PPI interaction effect for the Happy Memories vs Count condition contrast (H vs C) are shown for the experimental group (EG). The maps are voxel-wise FDR corrected and projected onto the standard TT_N27 anatomical template, with 3 mm separation between axial slices. The number adjacent to each slice indicates the z coordinate in mm. The left hemisphere (L) is to the reader’s right. Peak t-statistics values for the FBA-based PPI interaction effect and the corresponding locations are specified in Table S4. Table S1. Psychological trait measures for the study participants. Psychological traits were assessed before the rtfMRI-EEG-nf session using the Hamilton Depression Rating Scale (HDRS), the Montgomery-Asberg Depression Rating Scale (MADRS), the Snaith-Hamilton Pleasure Scale (SHAPS), the Hamilton Anxiety Rating Scale (HARS), the State-Trait Anxiety Inventory (STAI), the Toronto Alexithymia Scale (TAS-20), and the Behavioral Inhibition System / Behavioral Activation System scales (BIS/BAS). Experimental Control group, Difference Measure group, mean (SD) mean (SD) t-score [p]# Participants 16 8 Age (years) 32 (11) 34 (7) −0.67 [0.510] HDRS 14.4 (7.0) 15.1 (4.9) −0.25 [0.807] MADRS 19.6 (10.7) 20.5 (5.7) −0.23 [0.821] SHAPS 27.1 (6.7) 31.5 (5.8) −1.58 [0.129] HARS 13.2 (7.5) 16.1 (6.4) −0.95 [0.355] STAI Trait anxiety 56.9 (9.9) 59.6 (9.6) −0.65 [0.522] TAS-20 Total alexithymia 53.5 (14.4) 61.8 (12.2) −1.38 [0.180] BAS Reward responsiveness 14.9 (3.3) 16.3 (2.4) −0.99 [0.337] BIS 22.4 (3.6) 24.1 (4.1) −1.02 [0.319] # t(22), but t(19) for BIS/BAS; p – two-tailed, uncorrected Table S2. BOLD fMRI activity, corresponding to the Happy Memories vs Count condition contrast, during the rtfMRI-EEG-nf training for the experimental group (EG). Late- x, y, z Region t-score rality (mm) Frontal lobe Inferior frontal gyrus (BA 47) L −33, 17, −18 8.26 Middle frontal gyrus (BA 8) L −40, 20, 47 7.21 Inferior frontal gyrus (BA 47) L −43, 29, −5 7.09 Superior frontal gyrus (BA 8) L −7, 49, 48 6.90 Superior frontal gyrus (BA 10) L −7, 60, 24 6.71 Inferior frontal gyrus (BA 13/47) R 29, 14, −10 6.69 Superior frontal gyrus (BA 6) L −11, 13, 62 6.66 Superior frontal gyrus (BA 8) R 27, 17, 49 −6.59 Superior frontal gyrus (BA 6) L −9, 31, 57 6.36 Temporal lobe Middle temporal gyrus (BA 37) R 49, −59, 5 9.14 Superior temporal gyrus (BA 38) R 39, 5, −28 7.90 Middle temporal gyrus (BA 37) L −42, −67, 11 6.65 Inferior temporal gyrus (BA 37) L −55, −45, −16 −5.66 Middle temporal gyrus (BA 20) R 53, −35, −15 −5.35 Parietal lobe Postcentral gyrus (BA 2) L −55, −23, 44 −10.9 Precuneus (BA 7/19) R 23, −81, 35 6.07 Postcentral gyrus (BA 3) R 45, −19, 54 −5.50 Inferior parietal lobule (BA 40) L −46, −33, 45 −5.29 Inferior parietal lobule (BA 39) R 35, −63, 42 −5.24 Occipital lobe Middle occipital gyrus (BA 19/18) R 39, −81, 2 9.70 Middle occipital gyrus (BA 19) L −39, −77, 2 8.26 Lingual gyrus (BA 17) L −19, −86, 1 6.69 Cuneus (BA 18) L −11, −79, 21 −5.64 Limbic lobe Parahippocampal gyrus (BA 36) R 25, −31, −14 7.07 Anterior cingulate (BA 32) L −7, 49, −2 6.62 Amygdala / parahipp. gyrus L −29, −1, −14 6.55 Amygdala / uncus L −21, −3, −22 6.13 Cingulate gyrus (BA 31) R 7, −39, 34 −6.10 Hippocampus / parahipp. gyrus L −28, −24, −9 6.00 Cingulate gyrus (BA 31) L −6, −49, 28 5.36 Sub-lobar Red nucleus L −7, −19, −10 8.43 Insula (BA 13) R 37, −13, 12 −7.12 Insula (BA 13) L −39, −9, 6 −6.07 Thalamus, mediodorsal R 2, −9, 2 5.76 Thalamus, mediodorsal L −1, −14, 8 5.53 Culmen R 13, −57, −6 −5.41 FDR q<0.01 for |t|>5.0; BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates. Table S3. Experimental vs control group difference in the left amygdala fMRI functional connectivity changes between the Rest and Happy Memories conditions (H vs R) during the rtfMRI-EEG-nf training. Late- x, y, z Region t-score rality (mm) Frontal lobe Inferior frontal gyrus (BA 9) R 57, 13, 32 5.73 Middle frontal gyrus (BA 9) L −53, 8, 36 4.63 Superior frontal gyrus (BA 6) L −7, 33, 54 −4.56 Precentral gyrus (BA 6) L −51, 1, 33 4.49 Middle frontal gyrus (BA 9/46) L −39, 19, 24 4.42 Superior frontal gyrus (BA 6) L −15, −7, 64 3.68 Inferior frontal gyrus (BA 44) L −55, 5, 14 3.67 Medial frontal gyrus (BA 11) L −3, 61, −14 −3.64 Superior frontal gyrus (BA 6) R 9, 37, 52 −3.47 Temporal lobe Fusiform gyrus (BA 20) L −43, −31, −24 5.22 Superior temporal gyrus (BA 22) L −67, −35, 6 −3.82 Superior temporal gyrus (BA 21) R 55, −13, 0 3.31 Parietal lobe Postcentral gyrus (BA 2) L −51, −19, 28 4.37 Inferior parietal lobule (BA 40) L −47, −37, 38 3.28 Occipital lobe Middle occipital gyrus (BA 18) R 31, −91, 2 −4.01 Limbic lobe Anterior cingulate (BA 24) L −8, 34, 7 4.41 Parahippocampal gyrus (BA 37) L −33, −39, −4 4.36 Anterior cingulate (BA 32) L −9, 41, 5 4.04 Cingulate gyrus (BA 24) R 11, 1, 38 4.01 Uncus (BA 36) L −21, −9, −30 3.62 Anterior cingulate (BA 24) R 3, 35, 9 3.34 Sub-lobar Subthalamic nucleus R 15, −13, −4 7.39 Red nucleus L −7, −25, −6 4.44 Insula (BA 13) R 29, 21, 4 4.29 Insula (BA 13/47) L −31, 15, 4 4.15 Thalamus, ventroposterolateral R 17, −19, 6 3.95 Claustrum L −31, −11, 14 3.91 Putamen R 23, −9, 8 3.79 p<0.05, uncorr. BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates Table S4. Psychophysiological interaction effect, based on the time course of frontal high-beta EEG asymmetry (FBA), for the Happy vs Count contrast for the experimental group (EG). Late- x, y, z Region t-score rality (mm) Frontal lobe Superior frontal gyrus (BA 8/32) L −19, 12, 42 11.8 Medial frontal gyrus (BA 6) L −5, 35, 36 10.2 Superior frontal gyrus (BA 8) R 3, 29, 50 9.58 Inferior frontal gyrus (BA 44) L −51, 6, 13 8.75 Medial frontal gyrus (BA 6) L −1, 3, 52 7.93 Middle frontal gyrus (BA 9) L −29, 35, 32 7.79 Precentral gyrus (BA 4) L −29, −21, 58 7.52 Middle frontal gyrus (BA 9) R 31, 41, 28 7.30 Precentral gyrus (BA 6) R 27, −9, 52 6.23 Medial frontal gyrus (BA 10) L −9, 55, 6 6.12 Temporal lobe Middle temporal gyrus (BA 39) L −37, −69, 18 7.91 Superior temporal gyrus (BA 41) R 49, −37, 10 7.58 Middle temporal gyrus (BA 20) L −45, −5, −20 7.42 Superior temporal gyrus (BA 22) L −63, −41, 6 6.88 Middle temporal gyrus (BA 21) R 59, −7, −12 6.37 Parietal lobe Precuneus (BA 31) L −13, −59, 24 9.79 Precuneus (BA 31) R 15, −63, 20 8.16 Occipital lobe Lingual gyrus (BA 18) L −9, −79, −6 7.20 Limbic lobe Anterior cingulate (BA 33) L −9, 19, 16 10.3 Posterior cingulate (BA 29) R 3, −39, 20 9.53 Anterior cingulate (BA 32) L −13, 33, −2 9.36 Parahippocampal gyrus (BA 30) R 17, −37, 4 8.40 Parahippocampal gyrus (BA 36) R 24, −33, −14 8.19 Cingulate gyrus (BA 32) L −1, 21, 28 7.94 Subcallosal gyrus (BA 47) R 20, 11, −12 7.65 Cingulate gyrus (BA 31) L −10, −25, 34 7.61 Hippocampus / parahipp. gyrus L −27, −23, −10 7.41 Hippocampus / parahipp. gyrus L −27, −33, −3 7.29 Amygdala / parahipp. gyrus L −28, −5, −10 6.94 Parahippocampal gyrus (BA 28) L −19, −23, −18 6.58 Sub-lobar Thalamus, ventral anterior R 15, −9, 15 9.12 Declive R 33, −69, −17 9.12 Putamen L −23, −13, 8 9.10 Culmen of vermis R 1, −61, 2 8.64 Claustrum R 29, 7, −6 8.14 Caudate body L −16, 5, 18 7.87 Declive L −39, −61, −20 7.52 Thalamus, anterior L −7, −11, 15 7.50 Thalamus, mediodorsal L −3, −10, 10 7.21 Insula (BA 13) L −39, −1, 14 6.86 FDR q<0.05 for |t|>6.0; BA – Brodmann areas; L – left; R – right; x, y, z – Talairach coordinates.

Journal

Quantitative BiologyarXiv (Cornell University)

Published: Sep 12, 2019

References