Information

Connection problem with PsychoPy to EGI netstation for EEG experiment

Connection problem with PsychoPy to EGI netstation for EEG experiment

I posted the issue to the PsychoPy forum already, but was wondering if I could get more input from StackExchange.

So I'm having difficulty establishing connection between the Psychopy running testing machine and the machine with the actual netstation installed. I numbered the summary questions at bottom!

I have the basic following code below, but I'm assuming I am inputting a wrong IPv4 address or port number for the net station.

from __future__ import absolute_import, division, print_function from psychopy import locale_setup, sound, gui, visual, core, data, event, logging, clock from psychopy.constants import (NOT_STARTED, STARTED, PLAYING, PAUSED, STOPPED, FINISHED, PRESSED, RELEASED, FOREVER) import numpy as np # whole numpy lib is available, prepend 'np.' from numpy import (sin, cos, tan, log, log10, pi, average, sqrt, std, deg2rad, rad2deg, linspace, asarray) from numpy.random import random, randint, normal, shuffle import os # handy system and path functions import sys # to get file system encoding # Setup for EGI import egi.simple as egi # for testing purposes #import egi.simple as egi ms_localtime = egi.ms_localtime # gives local time in ms ns = egi.Netstation() ns.connect('ipv4 address of the testingmachine ', 'port number of netstation machine') # change/ check by Network Utility?

And I get the following error:

File "/Users/abc/Documents/PsychoPy/cfv/cfv_egi.py", line 281, in ns.BeginSession() File "/Users/abc/Documents/PsychoPy/cfv/egi/simple.py", line 799, in BeginSession self._socket.write( message ) File "/Users/abc/Documents/PsychoPy/cfv/egi/socket_wrapper.py", line 58, in write self._connection.write( data ) AttributeError: Socket instance has no attribute '_connection' 6.9777 WARNING When I change the port to another one with a server, it gives me the below: Traceback (most recent call last): File "/Users/abc/Documents/PsychoPy/cfv/cfv_egi.py", line 281, in ns.BeginSession() File "/Users/abc/Documents/PsychoPy/cfv/egi/simple.py", line 804, in BeginSession return self.GetServerResponse() File "/Users/abc/Documents/PsychoPy/cfv/egi/simple.py", line 783, in GetServerResponse raise Eggog( "unexpected character code returned from server: '%s'" % (code, ) ) egi.simple.Eggog: unexpected character code returned from server: 'R'

So summarizing my question,

  1. Is it correct that I am supposed to input the IPv4 address of the NetStation machine?

  2. How do I figure out what port number I need to input? Do I use the port numbers listed from using “Network Utility” app in the mac?

  3. When running the psychopy code that makes me connect to the machine, what needs to be setup in the Netstation computer? ie. do I need to have netstation software running?

Thank you. Any help or pointers would be appreciated!


Discussion

This study was conducted to investigate brain responses to high speeds of simulated forward motion from optic flow. The stimulus was a road simulated by poles moving from near the center of the screen and out toward the edges of the screen, creating a realistic simulation of an optic flow field. Scalp potentials in two parietal channels of interest were investigated for the three different driving speeds (low, medium, and high) of forward motion and a time-frequency analysis was performed.

We found significant differences between the three forward motion speeds in parietal channels P3 and P4 in the VEP analyses. Peak latency significantly increased as speed increased, while amplitude decreased as speed increased. The time-frequency analysis showed alpha de-synchronizations in response to forward motion, while the static condition showed alpha synchronizations. Alpha de-synchronizations were not significantly different between the three forward motion speeds. The parietal midline (PM) source showed significant differences between alpha de-synchronizations in response to forward motion and alpha synchronizations in response to the static control condition.

Our results, showing that peak latency increased with the speed of forward motion but that peak amplitude was inversely related to the speed, are different from the findings in a study using considerably lower speeds (Maruyama et al., 2002), and are the opposite of what Heinrich (2007) in a review article concluded to be the most common finding. However, for high speeds of motion there are a number of studies that corroborate the present findings. An MEG study measuring neural responses to light spot motion onset with a wide range of motion speeds, reported a decrease and subsequent increase in latencies as a function of speed of motion (Kawakami et al., 2002). In another study, Maunsell and Van Essen (1983) reported that in the macaque monkey, most speed sensitive neurons in the MT area have a preference for relatively low motion speeds. Thus, with fewer neurons preferring higher speeds, peak amplitude is likely to decrease with motion speed at the high end of the scale. Amplitude reflects the number of synchronously active neurons (Elul, 1972 Pfurtscheller and Lopes da Silva, 1999). Low amplitude indicates fewer neurons firing in synchrony, and therefore fewer neurons attuned to the particular condition. Kawakami et al. (2002) argued that amplitude change is related to the size of the neuronal population responding to the stimuli. The present findings showed that amplitude decreased with increasing speed of forward motion, suggesting that most of the neurons in the motion sensitive area were attuned to the lowest speed. Indeed, Liu and Newsome (2003) found that neurons are clustered according to preference of speed. Thus, these studies together with the present findings provide evidence that motion speeds over a certain magnitude give rise to increasing latencies and decreasing amplitudes.

Earlier studies on visual motion perception with adults have generally reported N2 latencies of approximately 150� ms (Kuba and Kubová, 1992 van der Meer et al., 2008), while this study reported latencies of up to 292 ms. These longer latencies could indicate that the optic flow pattern specifying forward motion at driving speeds used in the present study was more challenging than previously used.

From a life-span developmental perspective, it has been argued that increased latencies in response to visual motion reflect slower information processing (Langrová et al., 2006 van der Meer et al., 2008). The increased latencies in the current study might be due to the increased amount of information contained in the high speed condition and, as a result, the participants might have perceived it as more complex than the lower speeds. This could explain why high speeds result in slower information processing compared to lower speeds. The increased latencies in response to the higher speeds might also reflect that high speeds are less familiar than lower speeds, as humans do not encounter these speeds as often in the real world. A long response time has been argued to be a result of a lack of, or less specialized, neuronal networks (Howard et al., 1996 Johnson, 2000 Dubois et al., 2006) and it has been suggested that an increase in processing speed might be the result of an individual's experience with different environmental stimuli (Mukherjee et al., 2002 Agyei et al., 2015). White matter myelination increases with experience and increases from infancy to adulthood, and improved white matter tracts have been shown to be related to improved processing speed (Mukherjee et al., 2002 Fields, 2008). This might indicate that participants had more experience with the lower speeds which resulted in lower latencies. Demanding tasks are thought to put a greater workload on the system (Strayer and Drews, 2007 Allison and Polich, 2008) and these have been shown to decrease the magnitude of the response. Our findings showed this decrease in amplitude in response to the higher optic flow speeds as compared to the lower speed. This might indicate that the visual system when dealing with higher speeds of simulated forward visual motion is put under greater strain and, therefore, displays lower amplitudes. The higher optic flow density at the higher speeds of simulated forward visual motion might be more demanding for the visual system.

Latencies in channel P4 showed a significantly greater increase with motion speed compared to channel P3. This finding suggests that the right hemisphere is more involved in the processing of optic flow. Right lateralization for motion stimuli was also found in a study by de Jong et al. (1994), who found that the right latero-posterior precuneus (superior parietal lobe) was an important contributor to the processing of optic flow, fed by area V3.

The current study found latency and amplitude changes in the N2 component as a function of speed changes in simulated forward visual motion from optic flow. The N2 component has been shown to respond to visual motion stimuli that can induce vection in participants. These N2 components have also been found in response to vection-inducing stimuli as reported by Keshavarz and Berti (2014). They found the highest N2 amplitudes in O1 and O2 in response to a translational moving stimulus with a moving periphery and stationary center. However, the authors stated that the stimuli used were too short (2.5𠄳.5 s) to induce vection during EEG recordings, but argued that their N2 findings could indicate the initial steps in vection processing. Vection ratings were obtained using longer visual motion presentations (45 s) after the EEG recordings, and the results showed highest vection ratings for the stationary center and moving surround stimulus. A study by Thilo et al. (2003) found a decrease in the N70 component in occipital electrodes O1, Oz, and O2 in response to perceived rotational self-motion with participant reports of vection. These studies have not investigated speed changes in optic flow, but suggested that EEG research can help find objective markers of vection (Palmisano et al., 2015). The current study, however, used a stimulus presentation of 1 s, which is too short to induce any sensation of vection (Palmisano et al., 2015), and had a static scene of 3 s in between every motion condition to avoid motion adaptation (Heinrich, 2007). In addition, we did not ask whether or not the participants were experiencing vection. This is necessary since vection is a (conscious) subjective sensation of self-motion (Palmisano et al., 2015), and participant report is crucial for establishing whether perception of vection has occurred. Further studies should use longer stimulus presentations and gather subjective reports of vection during the EEG recordings. This could determine whether or not changes in motion speed have any effect on perceived vection, and how this affects the N2 component of visual motion.

In addition to VEP analysis, a time-frequency analysis was carried out to study changes in brain oscillations in response to visual motion. In the TSE analysis the static control condition was compared with the three different speeds in the motion condition. The present findings showed alpha-band de-synchronizations in several parietal and occipital sources in response to visual motion. However, significant differences were mainly found between alpha-band synchronizations and de-synchronizations in the PM source. There were no significant differences between the three speeds of visual motion in the TSE analysis. This is probably because the same optic flow pattern was shown, only moving at three different speeds. So it may be that the motion sensitive areas of the brain are responding to visual motion, irrespective of speed. Differences lie in the perceived speeds of motion, not in the environment the participants find themselves in, and the differences between the three speeds are only seen in the VEP analysis of the N2 component. If the alpha de-synchronizations reflect the general processing of visual motion (Pfurtscheller et al., 1994), then VEP latency and amplitude reflect the processing time and load, respectively.

Occipital and parietal de-synchronizations in alpha-band frequencies are thought to reflect an activated state (Pfurtscheller et al., 1994 Klimesch, 1999), and this fits well with the present findings. There was a long alpha de-synchronization in the PM source in response to visual motion, followed by synchronization in response to the static condition, which is related to a deactivated or resting period. Pfurtscheller et al. (1994) reported two different alpha bands: a lower one reflecting visual processing situated in occipital areas, and a higher one reflecting cognitive processes and attention situated in parietal areas. The present participants' alpha de-synchronizations were, however, too variable and the data not accurate enough to identify whether they were higher or lower alpha waves. Coupled with the findings of visually evoked N2 components, the induced oscillations very likely indicate visual motion processing. These findings seem to be in line with several other studies (Pfurtscheller et al., 1994 Schürmann et al., 1997 Klimesch, 1999) reporting alpha de-synchronizations in connection with visual stimulation.

In conclusion, we demonstrated that for high speeds of simulated forward motion, peak latency increases whereas peak amplitude decreases with speed. This suggests that for driving speeds, lower speeds are perceived as less demanding, or that more experience with lower speeds results in shorter latencies and higher amplitudes. With fewer neurons attuned to higher visual speeds, the motion sensitive areas of the adult brain appear to be less attuned to relatively high motion speeds of up to 75 km/h. These findings have implications for road traffic safety with adult perceivers taking longer to respond to, and having fewer neurons specialized for, higher driving speeds. In addition, significant differences between alpha de-synchronizations in response to visual motion and alpha synchronizations in the static condition were found in the parietal midline (PM) source. We suggest that the alpha de-synchronizations reflect an activated state related to the visual processing of simulated forward motion, whereas the alpha synchronizations in response to the static condition reflect a deactivated resting period.


Frequency Bands

As is apparent from visual inspection, EEG is a complex signal composed of multiple frequencies oscillating simultaneously. In the brain, lower frequencies show greater power than higher frequencies, with power decreasing as the frequency increases. The majority of brain activity occurs at frequencies under 100 Hz. The frequency spectrum of the brain is divided into different bands of activity based around a center frequency. These bands and their ranges are, in increasing frequency order: delta (<4 Hz), theta (4–7 Hz), alpha (8–13 Hz), beta (14–30 Hz), and gamma (>35 Hz). Alpha activity has more power than beta activity, which has more power than gamma activity. Of these, alpha is perhaps the easiest to see, appearing as large rhythmic waves visible to the naked eye in an EEG signal. It should be noted that while the center frequency of each band never changes (e.g., alpha always straddles 10 Hz), the boundaries of the bands are imprecise. For example, some may report alpha as 8–13 Hz or beta as 15–30 Hz.

These EEG frequency bands are associated with psychological phenomena. Delta is often associated with drowsiness, sleep, and states of altered consciousness. Theta appears to serve as a carrier wave for and modulator of the other oscillations and is associated with the cessation of pleasurable activity. Alpha activity is associated with attention and inhibitory control in the brain. Alpha activity is most prominent during relaxation and is inversely related to brain activity, as indicated by PET and fMRI studies (Cook, O'Hara, Uijtdehaage, Mandelkern, & Leuchter, 1998 Goldman, Stern, Engel, & Cohen, 2002 ). Interruption of alpha, known as alpha blocking, occurs during cognitive tasks. During alpha blocking, alpha waves are replaced with higher frequency, lower amplitude beta waves. Beta activity occurs when one is alert and is related to the regulation of processing states. Finally, gamma activity is associated with object maintenance, memory, and a variety of cognitive processes.

The EEG signal is collected in the time domain and must be converted to the frequency domain before these bands can be analyzed. This is done using the Fourier transform. Fourier posited that a time series may be represented by the sum of series of sine waves and a coefficient corresponding to how much of that sine wave is needed to reconstruct the original signal. In reverse, multiplying these sine waves by their associated coefficient and adding them together will reproduce the starting waveform. These principles form the basis of frequency analysis. In EEG research, a Fourier transform is used to decompose the EEG into a series of frequency coefficients that represent the amount of power at each frequency needed to reconstruct the original waveform. Practically, this labels the amount of power at each frequency and therefore allows examination of EEG frequency bands. Stern, Ray, and Quigley ( 2001 ) describe the Fourier transform as comparing a “template” of frequencies (the sine waves) to an existing EEG signal to see how closely it matches the template. A power spectrum allows researchers to visually represent all the frequencies present in the dataset, collapsed over time. More complex versions of this analysis are used to examine time-locked frequency variations in the waveform.

A related procedure is a coherence analysis, which provides information about how the signal from two given electrodes co-vary at a certain frequency. In other words, a coherence analysis describes how an EEG signal at “each of two electrodes is related to each electrode” (Stern et al., 2001, p. 91). A coherence analysis allows researchers to investigate the extent to which frequencies at different electrode sites are synchronous.


Online function reference available

Thanks to the clever and enthusiastic work of Tobias Wolf, the help texts of Psychtoolbox are now also available online:

The same link is available from the top page of the Wiki as “Function reference”.

The reference is mostly automatically generated by a clever Python script that we will occassionally run over the whole ‘beta’ distribution. The script automatically extracts, formats, hyperlinks and uploads the help content from all Psychtoolbox M-Files and the online help of most MEX files to the documentation Wiki.

The online reference is imho much more easily readable/browseable due to all the nice clickable cross-links and the much nicer formatting. The content however is almost the same as what you get from your Matlab console by typing help Psychtoolbox, help GetChar, Screen, Screen OpenWindow? etc. Unfortunately there don’t exist any scripts that automatically rewrite documentation :-(

The script is clever but it has the hard job of turning an organically grown mass of help texts, written by many different people and as many different opinions on formatting/writing style but without any formal markup language, into something half-way structured -) – iow. formatting glitches are to be expected. If you find small glitches, take them as beautful examples of modern art. If you are really annoyed by some formatting, feel free to find out what goes wrong and contribute fixes to the documentation or even better, volunteer to be the one with good taste and make it your personal mission to beautify our documentation.


Discussion

The current study aimed to replicate previously reported associations between higher EEG functional connectivity in 14-month-old infants, and later diagnosis of ASD and dimensional variation in restrictive and repetitive behaviours 10 . Our results partially replicate previous reports. Specifically, we did not replicate the observation of increased functional connectivity in the alpha range in the HR-ASD group for either global connectivity, or selected fronto-central connections. Nor did we replicate the trend level correlations between global functional connectivity and ASD symptoms measured by the ADI-R in the whole HR sample. However, we did replicate the significant correlation between higher functional connectivity over fronto-central regions, and the later severity of restricted and repetitive behaviours (RRBs) measured by the ADI-R within the group of infants with later ASD. Further, by combining the two samples we showed that functional connectivity across selected channels was specifically associated with circumscribed interests and not repetitive motor behaviour or insistence on sameness. Our findings are important both in terms of the failure to replicate effects in terms of categorical outcomes, and the replication of observed effects at a dimensional level. Conducting and full reporting of replication studies in independent cohorts sets a new standard for our field.

Brain connectivity and categorical outcome

We did not replicate previous observations of hyper-connectivity in the alpha range for infants later diagnosed with ASD at the group level 10 . Reports of altered EEG connectivity are highly inconsistent within the ASD literature, and several other studies report null effects in toddlers 29,48 and from infancy to adolescence 49 . Whilst heterogeneity in approach, population and analytic method could explain inconsistencies in previous work, our present failure to replicate findings at a group level using identical techniques including recruitment, and experimental and analyses methods is evidence that functional connectivity in the alpha band assessed using our present protocol is either not a strong candidate biomarker for categorical ASD, or is a only a feature of a sub-set of infants that go on to later diagnosis.

Our failure to observe altered connectivity using our present protocol does not rule out the possibility of atypicalities that could be detected through other methods. For instance, fNIRS methods provided evidence of atypical connectivity in 12-month-old infants at risk for ASD compared to infants with low risk 50 , and fMRI methods measuring functional connectivity in 6-month-old infants can predict later ASD diagnosis 51 . Nonetheless, the high temporal resolution of EEG connectivity provides an important measure of connectivity. Phase lagged measures such as the dbWPLI used here are also more likely to pick up on ‘true’ connectivity differences compared to other EEG measures of connectivity that are more influenced by volume conduction and the magnitude of the signal. However, the weighting we used removes the effect of small phase lags, thereby minimizing both volume conduction effects, and potential short-range connectivity with small phase lags. It is possible that differences between outcome groups exist for connectivity with small phase lags that were underestimated by the dbWPLI.

Other possible explanations to consider are intra- and interindividual variability. Intra-individual variability in connectivity within a session may constrain our ability to capture a stable marker of trait connectivity. Frequent short fluctuations in connectivity might be related to connectivity calculated over longer durations in EEG 52,53 , As for inter-individual variation, it is widely accepted that there is substantial heterogeneity in the genetic and environmental risk factors for ASD 54 . Analyses of large cohorts have indicated inter-individual variability in early cognitive and symptom trajectories 55� . Further, MEG and fMRI studies have shown that inter-individual variability in brain development and brain activation patterns in ASD is high 58� . The same likely applies to the current cohorts. Indeed, careful inspection of the individual data from Orekhova’s study and the current study reveals that inter-individual variation is also evident in HR-ASD infants. The sample in Orekhova’s study contained 6 infants with very high connectivity levels, whereas 4 other infants had lower connectivity levels. In contrast, the current sample contained 2 out of 13 infants displaying high levels of connectivity. Thus, the stratification of HR-ASD infants into subtypes of ASD based on functional connectivity in the alpha range at 14 months of age could be used to determine whether these represent distinct ‘subtypes’ of ASD. In relatively modest sample sizes there will be stochastic variation in the proportion of the sample showing elevated connectivity, creating difficulties in replicability at the group level.

Functional connectivity and the severity of restricted and repetitive behaviours

The heterogeneity poses a genuine challenge for research in ASD and some researchers even question the validity of ASD as a construct itself 61 . One potential solution to manage this is to take a dimensional approach in addition to a categorical classification and look at specific core ASD symptom domains and brain-behaviour associations as a diagnosis of ASD can be reached by multiple different combinations of specific symptoms 62,63 . To this end, we focussed on investigating associations between functional connectivity and ASD core symptoms measured by the ADI-R and ADOS-2. Replicating Orekhova and colleagues 10 , we found a significant relationship between functional connectivity averaged across selected connections from the previous study and the severity of RRBs within the HR-ASD group. Overall, our findings suggest that the heterogeneity in brain mechanisms is associated with specific heterogeneity in the behavioural phenotype. The absence of significant findings based on categories and the presence of significant brain-behaviour associations support the notion that a dimensional perspective should be taken when considering ASD, rather than solely a categorical approach 5 . To our knowledge, this result is the first replication of an infant neural predictor of dimensional variation in later ASD symptoms.

What mechanism underlies our replicated association between functional connectivity in fronto-central regions and later RRBs? The high alpha connectivity in fronto-central regions in HR-ASD infants in our study could contribute to the observed behavioural abnormalities and/or reflect common pathological factors affecting both EEG and behaviour, such as e.g. functional/neurochemical abnormalities 64� , and structural changes observed in frontal and related subcortical areas in ASD ( 69,70 , also see 71� ). Of note, it has been found that severity of RRBs in older children and adults with ASD correlated with frontal and/or striatal neurochemical abnormalities 64,66 , and structural changes in frontal cortex and related subcortical areas (such as the cerebellum, and caudate nuclei) 69 .

Studies of structural connectivity in infants at high familial risk for ASD show a somewhat converging pattern of findings in relation to RRBs. Specifically, cortical area and cortical thickness of the corpus callosum during the first year of life were positively associated with later RRBs in HR-ASD infants 75 . Further, higher structural connectivity between the genu of corpus callosum and the cerebellum at 6 months of age was related to more severe higher order RRBs (such as rituals, compulsions, insistence on sameness, and circumscribed interests) at later age 76 . The genus of the corpus callosum plays an important role in the frontal-striatal circuits. It has been suggested that fronto-striatal circuits might be implicated in the underlying mechanisms of RRBs 39,77,78 , occurring in ASD but also in for example obsessive compulsive disorder.

Further, analyses conducted on the combined high-risk sample suggest these findings arise from associations between functional connectivity in the alpha frequency EEG and circumscribed interests that can be detected at the trait level in the high-risk group as a whole. This observation seems consistent with the idea that elevated alpha connectivity reflects an over-focused attentional style, which is more closely related to circumscribed interests than the other subtypes of RRBs. Possibly, we did not observe an association between alpha connectivity and RRBs in the high-risk group because the circumscribed interests are the driving factor, and might not be strong enough to show an association with alpha connectivity when combined with the other RRB subtypes. Langen and colleagues 39 have proposed that circumscribed interests are mediated by a limbic loop consisting of the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), ventral striatum, ventral palladium, and medial dorsal thalamic nucleus. Our observation of functional hyperconnectivity over frontal and central scalp regions would at least be consistent with functional changes in the cortical part of this loop, though source analysis combining MRI and EEG techniques would be required to confirm this.

Finally, the association between functional connectivity for selected connections with RRBs should be taken with consideration of the measurements of RRBs. The association between functional connectivity and RRBs reached significance when measured with the ADI-R, but not with the ADOS-2, and only in the HR-ASD group, not the HR-TD or HR-Atyp group. This is consistent with the previous paper that did not find any associations reaching significance in HR infants who did not develop ASD at later age. The ADI-R has been designed to measure atypicalities in RRBs, and might therefore not pick up typical variation within smaller ranges in the HR-no ASD groups. Furthermore, RRBs in infants are relatively low-frequency behaviours which will more likely be reported during the ADI-R where parents report on their child’s behaviour over time, rather than being observed during the brief behavioural capture by the ADOS-2.

This study is the first to replicate a neuroimaging predictor of dimensional variation in ASD symptoms in young infants. Findings from structural and functional MRI studies show converging evidence in support of associations between abnormalities in fronto-striatal circuits and circumscribed interests in ASD. Future directions would be to combine EEG with methods with higher spatial resolution (like MRI or the more infant-friendly NIRS) to unravel the brain systems that underlie functional overconnectivity and circumscribed interests. Further, it will be important to trace whether we can identify early cognitive manifestations of circumscribed interests in infants with ASD that may relate to concurrent hyperconnectivity, for example atypical visual exploration during play 79 , or with eye-tracking methods 23 . Lastly, increased sample sizes would allow for specificity and sensitivity calculations needed for clinical application of this infant neural predictor of phenotypic variation.


Prefrontal Neuromodulation Using rTMS Improves Error Monitoring and Correction Function in Autism

One important executive function known to be compromised in autism spectrum disorder (ASD) is related to response error monitoring and post-error response correction. Several reports indicate that children with ASD show reduced error processing and deficient behavioral correction after an error is committed. Error sensitivity can be readily examined by measuring event-related potentials (ERP) associated with responses to errors, the fronto-central error-related negativity (ERN), and the error-related positivity (Pe). The goal of our study was to investigate whether reaction time (RT), error rate, post-error RT change, ERN, and Pe will show positive changes following 12-week long slow frequency repetitive TMS (rTMS) over dorsolateral prefrontal cortex (DLPFC) in high functioning children with ASD. We hypothesized that 12 sessions of 1 Hz rTMS bilaterally applied over the DLPFC will result in improvements reflected in both behavioral and ERP measures. Participants were randomly assigned to either active rTMS treatment or wait-list (WTL) groups. Baseline and post-TMS/or WTL EEG was collected using 128 channel EEG system. The task involved the recognition of a specific illusory shape, in this case a square or triangle, created by three or four inducer disks. ERN in TMS treatment group became significantly more negative. The number of omission errors decreased post-TMS. The RT did not change, but post-error RT became slower. There were no changes in RT, error rate, post-error RT slowing, nor in ERN/Pe measures in the wait-list group. Our results show significant post-TMS differences in the response-locked ERP such as ERN, as well as behavioral response monitoring measures indicative of improved error monitoring and correction function. The ERN and Pe, along with behavioral performance measures, can be used as functional outcome measures to assess the effectiveness of neuromodulation (e.g., rTMS) in children with autism and thus may have important practical implications.

This is a preview of subscription content, access via your institution.


IGT-Open: A freely available version of the Iowa Gambling Task

We have developed an open source version of the Iowa Gambling Task that is freely available for download, and use or modification. Footnote 1 This version of the IGT (IGT-Open) can run in both Matlab and Octave (an open-source version of the Matlab interpreter) it uses Psychtoolbox Footnote 2 libraries for Matlab (Brainard, 1997 Kleiner, 2010 Kleiner et al., 2007) that allow precise visual presentation timing so that the software may be used in conjunction with instruments that collect physiological data. IGT-Open has been used to collect human behavioral, eye-tracking, and electrodermal activity data. Computational process models that are built to use the JSON Network Interface (JNI) module in ACT-R (Hope et al., 2014) can also have ACT-R models complete the IGT task with the IGT-Open system.

Using the software with human subjects

IGT-Open was developed to be used for a decision-making study to examine the effects of subliminal emotional stimuli on task-related and physiological behavior over the course of the task (Dancy, 2014). This software uses the Psychtoolbox libraries that can be used with both Octave and Matlab so that the visual presentation within the task was precise enough to rapidly display images (i.e., subliminal presentation) and relate these visual events to physiological changes. Using these libraries also allowed behavioral data to be correctly aligned with eye-tracking data that was input into the system via a serial connection. A manual that includes details on setup of the libraries that the software uses is included with the software distribution.

Using the IGT-Open software is fairly simple as the program can be started by calling the GUI or directly calling the igt_open function from within either the Matlab or Octave interpreters. A user can call the start_up graphic user interface (from within the interpreter) that provides a visual interface for the user to see the parameters they are changing to confirm their task settings. Figure 1 shows a screenshot of the GUI with default parameters.

The GUI that can be used to specify parameters and start the software

The function (igt_open) can be called with 11 parameters (see Table 1 for an explanation of these parameters) that allow the user to customize the experiment.

Figure 2 displays an example functional diagram of the software called with the 11 possible arguments (the last function parameter is used for model-based simulations). IGT-Open has been designed to partition each trial into four phases: a deck selection phase, a card display phase, a reward/loss phase (in which an optional treatment-based image can be shown), and an intertrial break phase. Partitioning the trials into these phases allows for event-related analysis of some physiological-based measures (e.g., event-related fMRI analysis or EDA analysis).

A functional diagram of the IGT-Open software and its parameters

The IGT software has been used on both a machine running Windows 7 that uses a Matlab interpreter and a machine running Ubuntu 12.04 LTS (a Linux distribution) that uses an Octave interpreter for a study examining the effects of rapid visual stimuli presentation on physiological and decision-making behavior during the IGT (Dancy, 2014) the software has also been tested on a Mac machine (OSX 10.10) using the Matlab interpreter and with Ubuntu 14.04 LTS using the Octave interpreter. Those data were processed and analyzed using a combination of functions included with the IGT software and R software libraries. Descriptive statistics of behavioral data recorded using the software can be output using the AnalyzeIGT function. Figure 3 gives an example set of commands that can be run within Octave or Matlab to output descriptive statistics useful in data analysis.

An example of the descriptive data output function included with the IGT-Open software

Table 2 gives a list of the comma separated metrics output by the functions shown in Fig. 3. The file contains both overall metrics on the task (e.g., score per block), and metrics based on the context of the task when a participant makes a deck selection (e.g., number of cards selected from deck A when current amount of money is less than the amount a participant had at the beginning of the task). Because all three operating systems and both interpreters are officially supported by the Psychtoolbox libraries, there are no functional differences between running the IGT-Open software on the different system-interpreter combinations.

Visual stimulus presentation timing

When presenting visual stimuli rapidly (e.g., with ms interstimuli intervals), it is important that stimulus presentation timing is accurate so that the experiment can be replicated. Accurate and precise timing also allows an experimenter to correlate rapid stimulus events with any event-related physiological changes (e.g., event-related potentials or galvanic skin response). For studies involving subliminal or preconscious presentation (e.g., Dehaene et al., 1998), presentation timing as low as, or lower than, 16 ms may be desired.

Though the Psychtoolbox libraries have been used previously for accurate and precise visual stimulus presentation, it is useful to test the library functions independently within the software using them to ensure there are not any issues or incompatibilities that compromise the function precision. To ensure the timing accuracy of the stimulus presentation in the IGT-Open software, stimulus onset times of an image rapidly (i.e., 16.6 ms) presented were recorded over 1,000 trials during a simulated run of a normal IGT experiment. We also ran a similar test on the stimulus presentation timing of a version of the IGT that comes with the PEBL 0.14 software. Figure 4 shows a histogram of the image flip onset timing exhibited while running the IGT-Open software.

A histogram of image flip times recorded in IGT-Open over 1,000 trials

In the bimodal distribution shown in Fig. 4, the second peak contains the majority of flip times, as the images take between 16.65 ms and 16.70 ms for the majority of the 1,000 trials. The distribution of times is predominantly due to variance in refresh-rate timing of the monitor used for testing. As also indicated in Table 3, though there is a variance in timing, overall standard error of the mean (SEM) is still at a reasonable level.

As expected, the IGT-Open software reported visual stimulus onset timing that was within 0.1 ms of the minimum onset time allowed by the hardware used (a monitor with a 60-Hz refresh rate). More specifically, IGT-Open had an average recorded onset time of 16.67 ms (with a standard error of the mean of 0.002). We also recorded PEBL timing using the native timing functions provided by the software. Interestingly, the PEBL timing test resulted in an impossible average onset time of 12.07 ms (with a standard error of the mean of 0.17). There are two possible reasons for this timing result: (a) the timing mechanism used in the software malfunctioned (b) the visual stimulus experienced “tearing,” that is the image began to be presented after a monitor refresh has already begun, causing only part of the image to be presented during that particular cycle.

It is difficult to diagnose the first explanation without a clearer picture on the timing mechanisms used in PEBL, but the second possible explanation can be understood of as an artifact of using legacy Simple Direct Media Layer (SDL) 1.2 C libraries for visual stimulus presentation. In addition to adding an extra layer between the software and the graphics hardware (SDL can act as a middle layer between OpenGL and DirectX APIs), SDL 1.2 does not natively support hardware acceleration (Mueller & Piper, 2014), making visual presentation timing less precise and reliable. The Psychtoolbox libraries that are used by IGT-Open bypass this limitation by directly using the OpenGL API as an interface with graphics hardware. Thus, software interpreted by Matlab or Octave that uses the Psychtoolbox libraries (as IGT-Open does) has hardware acceleration available and enabled by default.

Using the software with computational process and models and statistical models

The IGT-Open software can be used to communicate with certain computational process models so that modelers can complete and get information from the same task environment that is presented to human participants (see Ritter et al., 2000 and Hope et al., 2014, for related discussions). To allow computational models to use the software, we developed functionality that can communicate with the JSON Network interface (JNI Hope et al., 2014). The JNI allows a computational process model to communicate with other software via a TCP connection.

The communication system is meant to be used with the JNI ACT-R module, though the software could be used with any software following the general JNI communication protocol the protocol essentially specifies a standard communication language between two systems (in this case, IGT-Open and an ACT-R model). We have used the IGT software and JNI communication module to have computational process models use simulated eyes and hands to perceive the visual stimuli in the task environment, use these stimuli to make a deck choice, and respond by using a virtual keyboard (Dancy, 2014). Because the source code is freely available, other computational modelers could also modify the communication system to work within their respective frameworks.

Statistical models can also be used with the software in a fairly straightforward manner. An expectancy valence model function is included with the software that will simulate the model on the given number of decks and penalty distribution using the decks variable that is output from a user’s penalty distribution function (specified by the penaltyFunStr parameter in the main igt_open function detailed in a previous section). Users can enter optional parameters for maximum number of cards in each deck, expectancy-valence model parameters (α, weight, and c from Busemeyer & Stout, 2002), number of trials for the model to run, a custom output file name, and the total number of simulation runs (which may be useful if using parameters that result in more random-like behavior, for example a c of 0). The function file that is included will also give users an idea of how they may create their own statistical model to run with their chosen penalty distribution function.


Functional EEG connectivity in infants associates with later restricted and repetitive behaviours in autism a replication study

We conducted a replication study of our prior report that increased alpha EEG connectivity at 14-months associates with later autism spectrum disorder (ASD) diagnosis, and dimensional variation in restricted interests/repetitive behaviours. 143 infants at high and low familial risk for ASD watched dynamic videos of spinning toys and women singing nursery rhymes while high-density EEG was recorded. Alpha functional connectivity (7–8 Hz) was calculated using the debiased weighted phase lag index. The final sample with clean data included low-risk infants (N = 20), and high-risk infants who at 36 months showed either typical development (N = 47), atypical development (N = 21), or met criteria for ASD (N = 13). While we did not replicate the finding that global EEG connectivity associated with ASD diagnosis, we did replicate the association between higher functional connectivity at 14 months and greater severity of restricted and repetitive behaviours at 36 months in infants who met criteria for ASD. We further showed that this association is strongest for the circumscribed interests subdomain. We propose that structural and/or functional abnormalities in frontal-striatal circuits underlie the observed association. This is the first replicated infant neural predictor of dimensional variation in later ASD symptoms.


Introduction

Accurate identification and analysis of facial expressions is critical for understanding others’ internal states 1 , and thus for regulating social relationships. This is particularly true for the preverbal infant, whose social world is comprised predominantly of face-to-face interactions with a primary caregiver 2 , and for whom communication is achieved largely via the ‘reading’ of faces 3 . Clearly then, it is highly advantageous for infants to detect and rapidly learn about faces very soon after birth 4 , and indeed, facial processing abilities appear at an early age. For example, even neonates demonstrate a bias towards looking at face-like stimuli 5,6,7 , and the ability to discriminate between different facial expressions emerges within the first few months of life 8,9,10 . By the end of their first year, infants can exploit information afforded by others’ expressions to guide their own behaviour in ambiguous situations 11,12 , and early difficulties in facial expression understanding have been linked to a number of adverse outcomes in later childhood 13,14 .

Processing facial expressions involves a widespread network of brain regions comprising both cortical and subcortical structures 15,16 . Essential components for socioemotional processing, including the amygdala and frontal cortex, are functional soon after birth 3,17,18 , and face-sensitive cortical areas such as the fusiform gyrus and superior temporal sulcus show some degree of facial tuning in the early months 19,20,21 . An extensive body of research with adults and nonhuman primates also suggests that sensorimotor brain regions, including parietal and premotor cortices, could support facial expression processing (e.g., refs 22,23,24 ), but whether this is the case in human infants has not been investigated. Recruitment of these parietal-premotor regions while observing others’ actions is widely thought to implement a mapping from the visual representation of an action to its corresponding motor representation 25 . This ‘mirror’ or ‘action-perception matching’ mechanism is believed to play a key role in the visual processing of others’ behaviour and in regulating social interactions 26,27 . Such a mechanism may be especially important for facial expressions in the early postnatal period, allowing infants to tune their own behaviour with that of their mother during complex face-to-face exchanges 28,29,30,31 , and serving as a basis for the development of more advanced socio–cognitive skills 32 . In macaque monkeys, evidence suggests that a mechanism matching own and other facial gestures is present in the very first days of life 18,33 , but in humans, the earliest evidence comes from 30-month-old children 34 .

Our study aimed to address two important and outstanding questions concerning a facial action-perception network: i) is a mechanism coupling own and other facial expressions present in the human infant and ii) if so, how does it develop (in particular, what is the role of the early social environment)? To answer the first question, we used electroencephalography (EEG) to measure event related desynchronization (ERD) in the mu frequency band during observation/execution of facial expressions, in a group of nine-month-old infants. One non-emotional condition (mouth opening) and two emotional conditions (happy, sad) were included, all of which are commonly occurring expressions in the infant repertoire. A scrambled control condition was also included to control for observation of any moving face-like stimulus (as in ref 34 ). Mu ERD in central electrodes is a commonly used index of motor system activity, and hence of an action-perception network if seen during both action observation and performance 35,36,37 (see the Supplementary Information file for more details). Note, we chose to look at mu ERD at nine months because other EEG research has already found evidence of motor system recruitment during observation of manual actions by this age (e.g., refs 38,39,40 ).

To address the second question, we identified specific behaviours during early mother-infant interactions that could support the development of a mechanism matching own and other facial gestures. Critically, unlike manual actions, where self-observation during action execution could strengthen a mapping between visual and motor representations 30,41,42 (a hypothesis supported by evidence from both infants 40 and adults 43,44 ), facial expressions are ‘opaque’ i.e. one cannot normally observe one’s own face while performing facial movements. Accordingly, self-observation could not facilitate the development of a facial action-perception network. Instead, development of this system may rely on maternal imitation of infant facial gestures, with caregivers acting as ‘biological mirrors’ for infants during very early interactions 29,31,41,42 . In other words, through maternal imitation (or ‘mirroring’), infants could observe the visual consequences of their own facial movements, providing the sensorimotor experience necessary to strengthen a link between motor and visual representations of facial gestures 30,41,45 .

During early mother-infant interactions, mothers regularly attempt to shape the exchange to include episodes of facial and vocal mirroring 46,47,48 , with the great majority of mirrors performed by the mother themselves 49,50 . This is a particularly enriching and preferred form of maternal response 51,52 , with maternal mirroring over the first nine of weeks of life found to predict the degree to which infants produce the same behaviours during subsequent social exchanges 31 . However, no previous research has investigated whether maternal mirroring guides the development of an action-perception network. Therefore, in addition to examining infant EEG responses to execution/observation of facial expressions at nine months, we also filmed the same infants interacting with their mothers at two months postpartum. These videos were coded to identify instances where mothers mirrored their infant’s facial expressions, including equivalents (smile, mouth opening, and negative) to those expressions observed during EEG acquisition later on. We chose to look at mother-infant interactions at two months because this is a privileged time in terms of face-to-face interaction and maternal mirroring of expressions, with infants showing the most interest in ‘pure’ face-to-face exchanges at this age 31,53 . We predicted that mothers’ tendency to imitate particular expressions during early interactions would relate to the strength of infant mu ERD during observation of the same expression, supporting the hypothesis that visuomotor experience provided by maternal mirroring supports the development of a facial action-perception matching mechanism.


2. Methods

2.1. Participant information

Infants in the High-Likelihood group (HL) had an older sibling with a community clinical diagnosis of Autism Spectrum Disorder (ASD) as confirmed by clinician judgment. Infants in the Low-Likelihood (LL) group had at least one older sibling with typical development (as reported by the parent) and no first-degree relatives with ASD. Further information about clinical ascertainment can be found in S1.0. Infants were primarily enrolled at 5 or 10 months a few infants were enrolled at 14 months. At each age point, the preferred testing window was +/-1 month from the relevant birthday if this was not possible, we allowed testing up to +2 months to minimise data loss. In the present manuscript, we report available data metrics from a subset of infants tested at 5, 10 and 14 months. Some sites did not collect EEG or eyetracking data in infancy and hence are not included in the present report. Other sites only collected particular data streams at particular time-points (e.g. Site E did not collect infant EEG from Eurosibs tasks site D only collected EEG at 5 months). Because data collection is ongoing, we have included all data uploaded to the central database and subjected to quality control assessment before February 2018 in this preliminary report. This data is thus intended to illustrate our protocol and procedures, and not to represent a finalised report on the cohort. Thus, all metrics are presented collapsed across likelihood group to avoid compromising future analysis plans, which are preregistered and embargoed until our defined data freeze points (see Discussion). Measures analysed, sample sizes and likelihood, gender and age profiles of samples at each site are shown in Tables 1a and b. Of note, there were significant differences between sites in the proportion of high and low likelihood infants in the sample (X 2  =�.6, p =  0.009), with site E having a relatively even balance (due to study design) whilst most sites had a majority of high likelihood infants. There were also significant age differences between sites (max. ηp 2  =𠂐.19), although numerically mean differences were small ( Table 1b ).

Table 1a

Participant gender balance and likelihood group for each site (%). HL = High Likelihood LL = Low Likelihood.


IGT-Open: A freely available version of the Iowa Gambling Task

We have developed an open source version of the Iowa Gambling Task that is freely available for download, and use or modification. Footnote 1 This version of the IGT (IGT-Open) can run in both Matlab and Octave (an open-source version of the Matlab interpreter) it uses Psychtoolbox Footnote 2 libraries for Matlab (Brainard, 1997 Kleiner, 2010 Kleiner et al., 2007) that allow precise visual presentation timing so that the software may be used in conjunction with instruments that collect physiological data. IGT-Open has been used to collect human behavioral, eye-tracking, and electrodermal activity data. Computational process models that are built to use the JSON Network Interface (JNI) module in ACT-R (Hope et al., 2014) can also have ACT-R models complete the IGT task with the IGT-Open system.

Using the software with human subjects

IGT-Open was developed to be used for a decision-making study to examine the effects of subliminal emotional stimuli on task-related and physiological behavior over the course of the task (Dancy, 2014). This software uses the Psychtoolbox libraries that can be used with both Octave and Matlab so that the visual presentation within the task was precise enough to rapidly display images (i.e., subliminal presentation) and relate these visual events to physiological changes. Using these libraries also allowed behavioral data to be correctly aligned with eye-tracking data that was input into the system via a serial connection. A manual that includes details on setup of the libraries that the software uses is included with the software distribution.

Using the IGT-Open software is fairly simple as the program can be started by calling the GUI or directly calling the igt_open function from within either the Matlab or Octave interpreters. A user can call the start_up graphic user interface (from within the interpreter) that provides a visual interface for the user to see the parameters they are changing to confirm their task settings. Figure 1 shows a screenshot of the GUI with default parameters.

The GUI that can be used to specify parameters and start the software

The function (igt_open) can be called with 11 parameters (see Table 1 for an explanation of these parameters) that allow the user to customize the experiment.

Figure 2 displays an example functional diagram of the software called with the 11 possible arguments (the last function parameter is used for model-based simulations). IGT-Open has been designed to partition each trial into four phases: a deck selection phase, a card display phase, a reward/loss phase (in which an optional treatment-based image can be shown), and an intertrial break phase. Partitioning the trials into these phases allows for event-related analysis of some physiological-based measures (e.g., event-related fMRI analysis or EDA analysis).

A functional diagram of the IGT-Open software and its parameters

The IGT software has been used on both a machine running Windows 7 that uses a Matlab interpreter and a machine running Ubuntu 12.04 LTS (a Linux distribution) that uses an Octave interpreter for a study examining the effects of rapid visual stimuli presentation on physiological and decision-making behavior during the IGT (Dancy, 2014) the software has also been tested on a Mac machine (OSX 10.10) using the Matlab interpreter and with Ubuntu 14.04 LTS using the Octave interpreter. Those data were processed and analyzed using a combination of functions included with the IGT software and R software libraries. Descriptive statistics of behavioral data recorded using the software can be output using the AnalyzeIGT function. Figure 3 gives an example set of commands that can be run within Octave or Matlab to output descriptive statistics useful in data analysis.

An example of the descriptive data output function included with the IGT-Open software

Table 2 gives a list of the comma separated metrics output by the functions shown in Fig. 3. The file contains both overall metrics on the task (e.g., score per block), and metrics based on the context of the task when a participant makes a deck selection (e.g., number of cards selected from deck A when current amount of money is less than the amount a participant had at the beginning of the task). Because all three operating systems and both interpreters are officially supported by the Psychtoolbox libraries, there are no functional differences between running the IGT-Open software on the different system-interpreter combinations.

Visual stimulus presentation timing

When presenting visual stimuli rapidly (e.g., with ms interstimuli intervals), it is important that stimulus presentation timing is accurate so that the experiment can be replicated. Accurate and precise timing also allows an experimenter to correlate rapid stimulus events with any event-related physiological changes (e.g., event-related potentials or galvanic skin response). For studies involving subliminal or preconscious presentation (e.g., Dehaene et al., 1998), presentation timing as low as, or lower than, 16 ms may be desired.

Though the Psychtoolbox libraries have been used previously for accurate and precise visual stimulus presentation, it is useful to test the library functions independently within the software using them to ensure there are not any issues or incompatibilities that compromise the function precision. To ensure the timing accuracy of the stimulus presentation in the IGT-Open software, stimulus onset times of an image rapidly (i.e., 16.6 ms) presented were recorded over 1,000 trials during a simulated run of a normal IGT experiment. We also ran a similar test on the stimulus presentation timing of a version of the IGT that comes with the PEBL 0.14 software. Figure 4 shows a histogram of the image flip onset timing exhibited while running the IGT-Open software.

A histogram of image flip times recorded in IGT-Open over 1,000 trials

In the bimodal distribution shown in Fig. 4, the second peak contains the majority of flip times, as the images take between 16.65 ms and 16.70 ms for the majority of the 1,000 trials. The distribution of times is predominantly due to variance in refresh-rate timing of the monitor used for testing. As also indicated in Table 3, though there is a variance in timing, overall standard error of the mean (SEM) is still at a reasonable level.

As expected, the IGT-Open software reported visual stimulus onset timing that was within 0.1 ms of the minimum onset time allowed by the hardware used (a monitor with a 60-Hz refresh rate). More specifically, IGT-Open had an average recorded onset time of 16.67 ms (with a standard error of the mean of 0.002). We also recorded PEBL timing using the native timing functions provided by the software. Interestingly, the PEBL timing test resulted in an impossible average onset time of 12.07 ms (with a standard error of the mean of 0.17). There are two possible reasons for this timing result: (a) the timing mechanism used in the software malfunctioned (b) the visual stimulus experienced “tearing,” that is the image began to be presented after a monitor refresh has already begun, causing only part of the image to be presented during that particular cycle.

It is difficult to diagnose the first explanation without a clearer picture on the timing mechanisms used in PEBL, but the second possible explanation can be understood of as an artifact of using legacy Simple Direct Media Layer (SDL) 1.2 C libraries for visual stimulus presentation. In addition to adding an extra layer between the software and the graphics hardware (SDL can act as a middle layer between OpenGL and DirectX APIs), SDL 1.2 does not natively support hardware acceleration (Mueller & Piper, 2014), making visual presentation timing less precise and reliable. The Psychtoolbox libraries that are used by IGT-Open bypass this limitation by directly using the OpenGL API as an interface with graphics hardware. Thus, software interpreted by Matlab or Octave that uses the Psychtoolbox libraries (as IGT-Open does) has hardware acceleration available and enabled by default.

Using the software with computational process and models and statistical models

The IGT-Open software can be used to communicate with certain computational process models so that modelers can complete and get information from the same task environment that is presented to human participants (see Ritter et al., 2000 and Hope et al., 2014, for related discussions). To allow computational models to use the software, we developed functionality that can communicate with the JSON Network interface (JNI Hope et al., 2014). The JNI allows a computational process model to communicate with other software via a TCP connection.

The communication system is meant to be used with the JNI ACT-R module, though the software could be used with any software following the general JNI communication protocol the protocol essentially specifies a standard communication language between two systems (in this case, IGT-Open and an ACT-R model). We have used the IGT software and JNI communication module to have computational process models use simulated eyes and hands to perceive the visual stimuli in the task environment, use these stimuli to make a deck choice, and respond by using a virtual keyboard (Dancy, 2014). Because the source code is freely available, other computational modelers could also modify the communication system to work within their respective frameworks.

Statistical models can also be used with the software in a fairly straightforward manner. An expectancy valence model function is included with the software that will simulate the model on the given number of decks and penalty distribution using the decks variable that is output from a user’s penalty distribution function (specified by the penaltyFunStr parameter in the main igt_open function detailed in a previous section). Users can enter optional parameters for maximum number of cards in each deck, expectancy-valence model parameters (α, weight, and c from Busemeyer & Stout, 2002), number of trials for the model to run, a custom output file name, and the total number of simulation runs (which may be useful if using parameters that result in more random-like behavior, for example a c of 0). The function file that is included will also give users an idea of how they may create their own statistical model to run with their chosen penalty distribution function.


Prefrontal Neuromodulation Using rTMS Improves Error Monitoring and Correction Function in Autism

One important executive function known to be compromised in autism spectrum disorder (ASD) is related to response error monitoring and post-error response correction. Several reports indicate that children with ASD show reduced error processing and deficient behavioral correction after an error is committed. Error sensitivity can be readily examined by measuring event-related potentials (ERP) associated with responses to errors, the fronto-central error-related negativity (ERN), and the error-related positivity (Pe). The goal of our study was to investigate whether reaction time (RT), error rate, post-error RT change, ERN, and Pe will show positive changes following 12-week long slow frequency repetitive TMS (rTMS) over dorsolateral prefrontal cortex (DLPFC) in high functioning children with ASD. We hypothesized that 12 sessions of 1 Hz rTMS bilaterally applied over the DLPFC will result in improvements reflected in both behavioral and ERP measures. Participants were randomly assigned to either active rTMS treatment or wait-list (WTL) groups. Baseline and post-TMS/or WTL EEG was collected using 128 channel EEG system. The task involved the recognition of a specific illusory shape, in this case a square or triangle, created by three or four inducer disks. ERN in TMS treatment group became significantly more negative. The number of omission errors decreased post-TMS. The RT did not change, but post-error RT became slower. There were no changes in RT, error rate, post-error RT slowing, nor in ERN/Pe measures in the wait-list group. Our results show significant post-TMS differences in the response-locked ERP such as ERN, as well as behavioral response monitoring measures indicative of improved error monitoring and correction function. The ERN and Pe, along with behavioral performance measures, can be used as functional outcome measures to assess the effectiveness of neuromodulation (e.g., rTMS) in children with autism and thus may have important practical implications.

This is a preview of subscription content, access via your institution.


Online function reference available

Thanks to the clever and enthusiastic work of Tobias Wolf, the help texts of Psychtoolbox are now also available online:

The same link is available from the top page of the Wiki as “Function reference”.

The reference is mostly automatically generated by a clever Python script that we will occassionally run over the whole ‘beta’ distribution. The script automatically extracts, formats, hyperlinks and uploads the help content from all Psychtoolbox M-Files and the online help of most MEX files to the documentation Wiki.

The online reference is imho much more easily readable/browseable due to all the nice clickable cross-links and the much nicer formatting. The content however is almost the same as what you get from your Matlab console by typing help Psychtoolbox, help GetChar, Screen, Screen OpenWindow? etc. Unfortunately there don’t exist any scripts that automatically rewrite documentation :-(

The script is clever but it has the hard job of turning an organically grown mass of help texts, written by many different people and as many different opinions on formatting/writing style but without any formal markup language, into something half-way structured -) – iow. formatting glitches are to be expected. If you find small glitches, take them as beautful examples of modern art. If you are really annoyed by some formatting, feel free to find out what goes wrong and contribute fixes to the documentation or even better, volunteer to be the one with good taste and make it your personal mission to beautify our documentation.


Discussion

This study was conducted to investigate brain responses to high speeds of simulated forward motion from optic flow. The stimulus was a road simulated by poles moving from near the center of the screen and out toward the edges of the screen, creating a realistic simulation of an optic flow field. Scalp potentials in two parietal channels of interest were investigated for the three different driving speeds (low, medium, and high) of forward motion and a time-frequency analysis was performed.

We found significant differences between the three forward motion speeds in parietal channels P3 and P4 in the VEP analyses. Peak latency significantly increased as speed increased, while amplitude decreased as speed increased. The time-frequency analysis showed alpha de-synchronizations in response to forward motion, while the static condition showed alpha synchronizations. Alpha de-synchronizations were not significantly different between the three forward motion speeds. The parietal midline (PM) source showed significant differences between alpha de-synchronizations in response to forward motion and alpha synchronizations in response to the static control condition.

Our results, showing that peak latency increased with the speed of forward motion but that peak amplitude was inversely related to the speed, are different from the findings in a study using considerably lower speeds (Maruyama et al., 2002), and are the opposite of what Heinrich (2007) in a review article concluded to be the most common finding. However, for high speeds of motion there are a number of studies that corroborate the present findings. An MEG study measuring neural responses to light spot motion onset with a wide range of motion speeds, reported a decrease and subsequent increase in latencies as a function of speed of motion (Kawakami et al., 2002). In another study, Maunsell and Van Essen (1983) reported that in the macaque monkey, most speed sensitive neurons in the MT area have a preference for relatively low motion speeds. Thus, with fewer neurons preferring higher speeds, peak amplitude is likely to decrease with motion speed at the high end of the scale. Amplitude reflects the number of synchronously active neurons (Elul, 1972 Pfurtscheller and Lopes da Silva, 1999). Low amplitude indicates fewer neurons firing in synchrony, and therefore fewer neurons attuned to the particular condition. Kawakami et al. (2002) argued that amplitude change is related to the size of the neuronal population responding to the stimuli. The present findings showed that amplitude decreased with increasing speed of forward motion, suggesting that most of the neurons in the motion sensitive area were attuned to the lowest speed. Indeed, Liu and Newsome (2003) found that neurons are clustered according to preference of speed. Thus, these studies together with the present findings provide evidence that motion speeds over a certain magnitude give rise to increasing latencies and decreasing amplitudes.

Earlier studies on visual motion perception with adults have generally reported N2 latencies of approximately 150� ms (Kuba and Kubová, 1992 van der Meer et al., 2008), while this study reported latencies of up to 292 ms. These longer latencies could indicate that the optic flow pattern specifying forward motion at driving speeds used in the present study was more challenging than previously used.

From a life-span developmental perspective, it has been argued that increased latencies in response to visual motion reflect slower information processing (Langrová et al., 2006 van der Meer et al., 2008). The increased latencies in the current study might be due to the increased amount of information contained in the high speed condition and, as a result, the participants might have perceived it as more complex than the lower speeds. This could explain why high speeds result in slower information processing compared to lower speeds. The increased latencies in response to the higher speeds might also reflect that high speeds are less familiar than lower speeds, as humans do not encounter these speeds as often in the real world. A long response time has been argued to be a result of a lack of, or less specialized, neuronal networks (Howard et al., 1996 Johnson, 2000 Dubois et al., 2006) and it has been suggested that an increase in processing speed might be the result of an individual's experience with different environmental stimuli (Mukherjee et al., 2002 Agyei et al., 2015). White matter myelination increases with experience and increases from infancy to adulthood, and improved white matter tracts have been shown to be related to improved processing speed (Mukherjee et al., 2002 Fields, 2008). This might indicate that participants had more experience with the lower speeds which resulted in lower latencies. Demanding tasks are thought to put a greater workload on the system (Strayer and Drews, 2007 Allison and Polich, 2008) and these have been shown to decrease the magnitude of the response. Our findings showed this decrease in amplitude in response to the higher optic flow speeds as compared to the lower speed. This might indicate that the visual system when dealing with higher speeds of simulated forward visual motion is put under greater strain and, therefore, displays lower amplitudes. The higher optic flow density at the higher speeds of simulated forward visual motion might be more demanding for the visual system.

Latencies in channel P4 showed a significantly greater increase with motion speed compared to channel P3. This finding suggests that the right hemisphere is more involved in the processing of optic flow. Right lateralization for motion stimuli was also found in a study by de Jong et al. (1994), who found that the right latero-posterior precuneus (superior parietal lobe) was an important contributor to the processing of optic flow, fed by area V3.

The current study found latency and amplitude changes in the N2 component as a function of speed changes in simulated forward visual motion from optic flow. The N2 component has been shown to respond to visual motion stimuli that can induce vection in participants. These N2 components have also been found in response to vection-inducing stimuli as reported by Keshavarz and Berti (2014). They found the highest N2 amplitudes in O1 and O2 in response to a translational moving stimulus with a moving periphery and stationary center. However, the authors stated that the stimuli used were too short (2.5𠄳.5 s) to induce vection during EEG recordings, but argued that their N2 findings could indicate the initial steps in vection processing. Vection ratings were obtained using longer visual motion presentations (45 s) after the EEG recordings, and the results showed highest vection ratings for the stationary center and moving surround stimulus. A study by Thilo et al. (2003) found a decrease in the N70 component in occipital electrodes O1, Oz, and O2 in response to perceived rotational self-motion with participant reports of vection. These studies have not investigated speed changes in optic flow, but suggested that EEG research can help find objective markers of vection (Palmisano et al., 2015). The current study, however, used a stimulus presentation of 1 s, which is too short to induce any sensation of vection (Palmisano et al., 2015), and had a static scene of 3 s in between every motion condition to avoid motion adaptation (Heinrich, 2007). In addition, we did not ask whether or not the participants were experiencing vection. This is necessary since vection is a (conscious) subjective sensation of self-motion (Palmisano et al., 2015), and participant report is crucial for establishing whether perception of vection has occurred. Further studies should use longer stimulus presentations and gather subjective reports of vection during the EEG recordings. This could determine whether or not changes in motion speed have any effect on perceived vection, and how this affects the N2 component of visual motion.

In addition to VEP analysis, a time-frequency analysis was carried out to study changes in brain oscillations in response to visual motion. In the TSE analysis the static control condition was compared with the three different speeds in the motion condition. The present findings showed alpha-band de-synchronizations in several parietal and occipital sources in response to visual motion. However, significant differences were mainly found between alpha-band synchronizations and de-synchronizations in the PM source. There were no significant differences between the three speeds of visual motion in the TSE analysis. This is probably because the same optic flow pattern was shown, only moving at three different speeds. So it may be that the motion sensitive areas of the brain are responding to visual motion, irrespective of speed. Differences lie in the perceived speeds of motion, not in the environment the participants find themselves in, and the differences between the three speeds are only seen in the VEP analysis of the N2 component. If the alpha de-synchronizations reflect the general processing of visual motion (Pfurtscheller et al., 1994), then VEP latency and amplitude reflect the processing time and load, respectively.

Occipital and parietal de-synchronizations in alpha-band frequencies are thought to reflect an activated state (Pfurtscheller et al., 1994 Klimesch, 1999), and this fits well with the present findings. There was a long alpha de-synchronization in the PM source in response to visual motion, followed by synchronization in response to the static condition, which is related to a deactivated or resting period. Pfurtscheller et al. (1994) reported two different alpha bands: a lower one reflecting visual processing situated in occipital areas, and a higher one reflecting cognitive processes and attention situated in parietal areas. The present participants' alpha de-synchronizations were, however, too variable and the data not accurate enough to identify whether they were higher or lower alpha waves. Coupled with the findings of visually evoked N2 components, the induced oscillations very likely indicate visual motion processing. These findings seem to be in line with several other studies (Pfurtscheller et al., 1994 Schürmann et al., 1997 Klimesch, 1999) reporting alpha de-synchronizations in connection with visual stimulation.

In conclusion, we demonstrated that for high speeds of simulated forward motion, peak latency increases whereas peak amplitude decreases with speed. This suggests that for driving speeds, lower speeds are perceived as less demanding, or that more experience with lower speeds results in shorter latencies and higher amplitudes. With fewer neurons attuned to higher visual speeds, the motion sensitive areas of the adult brain appear to be less attuned to relatively high motion speeds of up to 75 km/h. These findings have implications for road traffic safety with adult perceivers taking longer to respond to, and having fewer neurons specialized for, higher driving speeds. In addition, significant differences between alpha de-synchronizations in response to visual motion and alpha synchronizations in the static condition were found in the parietal midline (PM) source. We suggest that the alpha de-synchronizations reflect an activated state related to the visual processing of simulated forward motion, whereas the alpha synchronizations in response to the static condition reflect a deactivated resting period.


Frequency Bands

As is apparent from visual inspection, EEG is a complex signal composed of multiple frequencies oscillating simultaneously. In the brain, lower frequencies show greater power than higher frequencies, with power decreasing as the frequency increases. The majority of brain activity occurs at frequencies under 100 Hz. The frequency spectrum of the brain is divided into different bands of activity based around a center frequency. These bands and their ranges are, in increasing frequency order: delta (<4 Hz), theta (4–7 Hz), alpha (8–13 Hz), beta (14–30 Hz), and gamma (>35 Hz). Alpha activity has more power than beta activity, which has more power than gamma activity. Of these, alpha is perhaps the easiest to see, appearing as large rhythmic waves visible to the naked eye in an EEG signal. It should be noted that while the center frequency of each band never changes (e.g., alpha always straddles 10 Hz), the boundaries of the bands are imprecise. For example, some may report alpha as 8–13 Hz or beta as 15–30 Hz.

These EEG frequency bands are associated with psychological phenomena. Delta is often associated with drowsiness, sleep, and states of altered consciousness. Theta appears to serve as a carrier wave for and modulator of the other oscillations and is associated with the cessation of pleasurable activity. Alpha activity is associated with attention and inhibitory control in the brain. Alpha activity is most prominent during relaxation and is inversely related to brain activity, as indicated by PET and fMRI studies (Cook, O'Hara, Uijtdehaage, Mandelkern, & Leuchter, 1998 Goldman, Stern, Engel, & Cohen, 2002 ). Interruption of alpha, known as alpha blocking, occurs during cognitive tasks. During alpha blocking, alpha waves are replaced with higher frequency, lower amplitude beta waves. Beta activity occurs when one is alert and is related to the regulation of processing states. Finally, gamma activity is associated with object maintenance, memory, and a variety of cognitive processes.

The EEG signal is collected in the time domain and must be converted to the frequency domain before these bands can be analyzed. This is done using the Fourier transform. Fourier posited that a time series may be represented by the sum of series of sine waves and a coefficient corresponding to how much of that sine wave is needed to reconstruct the original signal. In reverse, multiplying these sine waves by their associated coefficient and adding them together will reproduce the starting waveform. These principles form the basis of frequency analysis. In EEG research, a Fourier transform is used to decompose the EEG into a series of frequency coefficients that represent the amount of power at each frequency needed to reconstruct the original waveform. Practically, this labels the amount of power at each frequency and therefore allows examination of EEG frequency bands. Stern, Ray, and Quigley ( 2001 ) describe the Fourier transform as comparing a “template” of frequencies (the sine waves) to an existing EEG signal to see how closely it matches the template. A power spectrum allows researchers to visually represent all the frequencies present in the dataset, collapsed over time. More complex versions of this analysis are used to examine time-locked frequency variations in the waveform.

A related procedure is a coherence analysis, which provides information about how the signal from two given electrodes co-vary at a certain frequency. In other words, a coherence analysis describes how an EEG signal at “each of two electrodes is related to each electrode” (Stern et al., 2001, p. 91). A coherence analysis allows researchers to investigate the extent to which frequencies at different electrode sites are synchronous.


Discussion

The current study aimed to replicate previously reported associations between higher EEG functional connectivity in 14-month-old infants, and later diagnosis of ASD and dimensional variation in restrictive and repetitive behaviours 10 . Our results partially replicate previous reports. Specifically, we did not replicate the observation of increased functional connectivity in the alpha range in the HR-ASD group for either global connectivity, or selected fronto-central connections. Nor did we replicate the trend level correlations between global functional connectivity and ASD symptoms measured by the ADI-R in the whole HR sample. However, we did replicate the significant correlation between higher functional connectivity over fronto-central regions, and the later severity of restricted and repetitive behaviours (RRBs) measured by the ADI-R within the group of infants with later ASD. Further, by combining the two samples we showed that functional connectivity across selected channels was specifically associated with circumscribed interests and not repetitive motor behaviour or insistence on sameness. Our findings are important both in terms of the failure to replicate effects in terms of categorical outcomes, and the replication of observed effects at a dimensional level. Conducting and full reporting of replication studies in independent cohorts sets a new standard for our field.

Brain connectivity and categorical outcome

We did not replicate previous observations of hyper-connectivity in the alpha range for infants later diagnosed with ASD at the group level 10 . Reports of altered EEG connectivity are highly inconsistent within the ASD literature, and several other studies report null effects in toddlers 29,48 and from infancy to adolescence 49 . Whilst heterogeneity in approach, population and analytic method could explain inconsistencies in previous work, our present failure to replicate findings at a group level using identical techniques including recruitment, and experimental and analyses methods is evidence that functional connectivity in the alpha band assessed using our present protocol is either not a strong candidate biomarker for categorical ASD, or is a only a feature of a sub-set of infants that go on to later diagnosis.

Our failure to observe altered connectivity using our present protocol does not rule out the possibility of atypicalities that could be detected through other methods. For instance, fNIRS methods provided evidence of atypical connectivity in 12-month-old infants at risk for ASD compared to infants with low risk 50 , and fMRI methods measuring functional connectivity in 6-month-old infants can predict later ASD diagnosis 51 . Nonetheless, the high temporal resolution of EEG connectivity provides an important measure of connectivity. Phase lagged measures such as the dbWPLI used here are also more likely to pick up on ‘true’ connectivity differences compared to other EEG measures of connectivity that are more influenced by volume conduction and the magnitude of the signal. However, the weighting we used removes the effect of small phase lags, thereby minimizing both volume conduction effects, and potential short-range connectivity with small phase lags. It is possible that differences between outcome groups exist for connectivity with small phase lags that were underestimated by the dbWPLI.

Other possible explanations to consider are intra- and interindividual variability. Intra-individual variability in connectivity within a session may constrain our ability to capture a stable marker of trait connectivity. Frequent short fluctuations in connectivity might be related to connectivity calculated over longer durations in EEG 52,53 , As for inter-individual variation, it is widely accepted that there is substantial heterogeneity in the genetic and environmental risk factors for ASD 54 . Analyses of large cohorts have indicated inter-individual variability in early cognitive and symptom trajectories 55� . Further, MEG and fMRI studies have shown that inter-individual variability in brain development and brain activation patterns in ASD is high 58� . The same likely applies to the current cohorts. Indeed, careful inspection of the individual data from Orekhova’s study and the current study reveals that inter-individual variation is also evident in HR-ASD infants. The sample in Orekhova’s study contained 6 infants with very high connectivity levels, whereas 4 other infants had lower connectivity levels. In contrast, the current sample contained 2 out of 13 infants displaying high levels of connectivity. Thus, the stratification of HR-ASD infants into subtypes of ASD based on functional connectivity in the alpha range at 14 months of age could be used to determine whether these represent distinct ‘subtypes’ of ASD. In relatively modest sample sizes there will be stochastic variation in the proportion of the sample showing elevated connectivity, creating difficulties in replicability at the group level.

Functional connectivity and the severity of restricted and repetitive behaviours

The heterogeneity poses a genuine challenge for research in ASD and some researchers even question the validity of ASD as a construct itself 61 . One potential solution to manage this is to take a dimensional approach in addition to a categorical classification and look at specific core ASD symptom domains and brain-behaviour associations as a diagnosis of ASD can be reached by multiple different combinations of specific symptoms 62,63 . To this end, we focussed on investigating associations between functional connectivity and ASD core symptoms measured by the ADI-R and ADOS-2. Replicating Orekhova and colleagues 10 , we found a significant relationship between functional connectivity averaged across selected connections from the previous study and the severity of RRBs within the HR-ASD group. Overall, our findings suggest that the heterogeneity in brain mechanisms is associated with specific heterogeneity in the behavioural phenotype. The absence of significant findings based on categories and the presence of significant brain-behaviour associations support the notion that a dimensional perspective should be taken when considering ASD, rather than solely a categorical approach 5 . To our knowledge, this result is the first replication of an infant neural predictor of dimensional variation in later ASD symptoms.

What mechanism underlies our replicated association between functional connectivity in fronto-central regions and later RRBs? The high alpha connectivity in fronto-central regions in HR-ASD infants in our study could contribute to the observed behavioural abnormalities and/or reflect common pathological factors affecting both EEG and behaviour, such as e.g. functional/neurochemical abnormalities 64� , and structural changes observed in frontal and related subcortical areas in ASD ( 69,70 , also see 71� ). Of note, it has been found that severity of RRBs in older children and adults with ASD correlated with frontal and/or striatal neurochemical abnormalities 64,66 , and structural changes in frontal cortex and related subcortical areas (such as the cerebellum, and caudate nuclei) 69 .

Studies of structural connectivity in infants at high familial risk for ASD show a somewhat converging pattern of findings in relation to RRBs. Specifically, cortical area and cortical thickness of the corpus callosum during the first year of life were positively associated with later RRBs in HR-ASD infants 75 . Further, higher structural connectivity between the genu of corpus callosum and the cerebellum at 6 months of age was related to more severe higher order RRBs (such as rituals, compulsions, insistence on sameness, and circumscribed interests) at later age 76 . The genus of the corpus callosum plays an important role in the frontal-striatal circuits. It has been suggested that fronto-striatal circuits might be implicated in the underlying mechanisms of RRBs 39,77,78 , occurring in ASD but also in for example obsessive compulsive disorder.

Further, analyses conducted on the combined high-risk sample suggest these findings arise from associations between functional connectivity in the alpha frequency EEG and circumscribed interests that can be detected at the trait level in the high-risk group as a whole. This observation seems consistent with the idea that elevated alpha connectivity reflects an over-focused attentional style, which is more closely related to circumscribed interests than the other subtypes of RRBs. Possibly, we did not observe an association between alpha connectivity and RRBs in the high-risk group because the circumscribed interests are the driving factor, and might not be strong enough to show an association with alpha connectivity when combined with the other RRB subtypes. Langen and colleagues 39 have proposed that circumscribed interests are mediated by a limbic loop consisting of the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), ventral striatum, ventral palladium, and medial dorsal thalamic nucleus. Our observation of functional hyperconnectivity over frontal and central scalp regions would at least be consistent with functional changes in the cortical part of this loop, though source analysis combining MRI and EEG techniques would be required to confirm this.

Finally, the association between functional connectivity for selected connections with RRBs should be taken with consideration of the measurements of RRBs. The association between functional connectivity and RRBs reached significance when measured with the ADI-R, but not with the ADOS-2, and only in the HR-ASD group, not the HR-TD or HR-Atyp group. This is consistent with the previous paper that did not find any associations reaching significance in HR infants who did not develop ASD at later age. The ADI-R has been designed to measure atypicalities in RRBs, and might therefore not pick up typical variation within smaller ranges in the HR-no ASD groups. Furthermore, RRBs in infants are relatively low-frequency behaviours which will more likely be reported during the ADI-R where parents report on their child’s behaviour over time, rather than being observed during the brief behavioural capture by the ADOS-2.

This study is the first to replicate a neuroimaging predictor of dimensional variation in ASD symptoms in young infants. Findings from structural and functional MRI studies show converging evidence in support of associations between abnormalities in fronto-striatal circuits and circumscribed interests in ASD. Future directions would be to combine EEG with methods with higher spatial resolution (like MRI or the more infant-friendly NIRS) to unravel the brain systems that underlie functional overconnectivity and circumscribed interests. Further, it will be important to trace whether we can identify early cognitive manifestations of circumscribed interests in infants with ASD that may relate to concurrent hyperconnectivity, for example atypical visual exploration during play 79 , or with eye-tracking methods 23 . Lastly, increased sample sizes would allow for specificity and sensitivity calculations needed for clinical application of this infant neural predictor of phenotypic variation.


2. Methods

2.1. Participant information

Infants in the High-Likelihood group (HL) had an older sibling with a community clinical diagnosis of Autism Spectrum Disorder (ASD) as confirmed by clinician judgment. Infants in the Low-Likelihood (LL) group had at least one older sibling with typical development (as reported by the parent) and no first-degree relatives with ASD. Further information about clinical ascertainment can be found in S1.0. Infants were primarily enrolled at 5 or 10 months a few infants were enrolled at 14 months. At each age point, the preferred testing window was +/-1 month from the relevant birthday if this was not possible, we allowed testing up to +2 months to minimise data loss. In the present manuscript, we report available data metrics from a subset of infants tested at 5, 10 and 14 months. Some sites did not collect EEG or eyetracking data in infancy and hence are not included in the present report. Other sites only collected particular data streams at particular time-points (e.g. Site E did not collect infant EEG from Eurosibs tasks site D only collected EEG at 5 months). Because data collection is ongoing, we have included all data uploaded to the central database and subjected to quality control assessment before February 2018 in this preliminary report. This data is thus intended to illustrate our protocol and procedures, and not to represent a finalised report on the cohort. Thus, all metrics are presented collapsed across likelihood group to avoid compromising future analysis plans, which are preregistered and embargoed until our defined data freeze points (see Discussion). Measures analysed, sample sizes and likelihood, gender and age profiles of samples at each site are shown in Tables 1a and b. Of note, there were significant differences between sites in the proportion of high and low likelihood infants in the sample (X 2  =�.6, p =  0.009), with site E having a relatively even balance (due to study design) whilst most sites had a majority of high likelihood infants. There were also significant age differences between sites (max. ηp 2  =𠂐.19), although numerically mean differences were small ( Table 1b ).

Table 1a

Participant gender balance and likelihood group for each site (%). HL = High Likelihood LL = Low Likelihood.


Introduction

Accurate identification and analysis of facial expressions is critical for understanding others’ internal states 1 , and thus for regulating social relationships. This is particularly true for the preverbal infant, whose social world is comprised predominantly of face-to-face interactions with a primary caregiver 2 , and for whom communication is achieved largely via the ‘reading’ of faces 3 . Clearly then, it is highly advantageous for infants to detect and rapidly learn about faces very soon after birth 4 , and indeed, facial processing abilities appear at an early age. For example, even neonates demonstrate a bias towards looking at face-like stimuli 5,6,7 , and the ability to discriminate between different facial expressions emerges within the first few months of life 8,9,10 . By the end of their first year, infants can exploit information afforded by others’ expressions to guide their own behaviour in ambiguous situations 11,12 , and early difficulties in facial expression understanding have been linked to a number of adverse outcomes in later childhood 13,14 .

Processing facial expressions involves a widespread network of brain regions comprising both cortical and subcortical structures 15,16 . Essential components for socioemotional processing, including the amygdala and frontal cortex, are functional soon after birth 3,17,18 , and face-sensitive cortical areas such as the fusiform gyrus and superior temporal sulcus show some degree of facial tuning in the early months 19,20,21 . An extensive body of research with adults and nonhuman primates also suggests that sensorimotor brain regions, including parietal and premotor cortices, could support facial expression processing (e.g., refs 22,23,24 ), but whether this is the case in human infants has not been investigated. Recruitment of these parietal-premotor regions while observing others’ actions is widely thought to implement a mapping from the visual representation of an action to its corresponding motor representation 25 . This ‘mirror’ or ‘action-perception matching’ mechanism is believed to play a key role in the visual processing of others’ behaviour and in regulating social interactions 26,27 . Such a mechanism may be especially important for facial expressions in the early postnatal period, allowing infants to tune their own behaviour with that of their mother during complex face-to-face exchanges 28,29,30,31 , and serving as a basis for the development of more advanced socio–cognitive skills 32 . In macaque monkeys, evidence suggests that a mechanism matching own and other facial gestures is present in the very first days of life 18,33 , but in humans, the earliest evidence comes from 30-month-old children 34 .

Our study aimed to address two important and outstanding questions concerning a facial action-perception network: i) is a mechanism coupling own and other facial expressions present in the human infant and ii) if so, how does it develop (in particular, what is the role of the early social environment)? To answer the first question, we used electroencephalography (EEG) to measure event related desynchronization (ERD) in the mu frequency band during observation/execution of facial expressions, in a group of nine-month-old infants. One non-emotional condition (mouth opening) and two emotional conditions (happy, sad) were included, all of which are commonly occurring expressions in the infant repertoire. A scrambled control condition was also included to control for observation of any moving face-like stimulus (as in ref 34 ). Mu ERD in central electrodes is a commonly used index of motor system activity, and hence of an action-perception network if seen during both action observation and performance 35,36,37 (see the Supplementary Information file for more details). Note, we chose to look at mu ERD at nine months because other EEG research has already found evidence of motor system recruitment during observation of manual actions by this age (e.g., refs 38,39,40 ).

To address the second question, we identified specific behaviours during early mother-infant interactions that could support the development of a mechanism matching own and other facial gestures. Critically, unlike manual actions, where self-observation during action execution could strengthen a mapping between visual and motor representations 30,41,42 (a hypothesis supported by evidence from both infants 40 and adults 43,44 ), facial expressions are ‘opaque’ i.e. one cannot normally observe one’s own face while performing facial movements. Accordingly, self-observation could not facilitate the development of a facial action-perception network. Instead, development of this system may rely on maternal imitation of infant facial gestures, with caregivers acting as ‘biological mirrors’ for infants during very early interactions 29,31,41,42 . In other words, through maternal imitation (or ‘mirroring’), infants could observe the visual consequences of their own facial movements, providing the sensorimotor experience necessary to strengthen a link between motor and visual representations of facial gestures 30,41,45 .

During early mother-infant interactions, mothers regularly attempt to shape the exchange to include episodes of facial and vocal mirroring 46,47,48 , with the great majority of mirrors performed by the mother themselves 49,50 . This is a particularly enriching and preferred form of maternal response 51,52 , with maternal mirroring over the first nine of weeks of life found to predict the degree to which infants produce the same behaviours during subsequent social exchanges 31 . However, no previous research has investigated whether maternal mirroring guides the development of an action-perception network. Therefore, in addition to examining infant EEG responses to execution/observation of facial expressions at nine months, we also filmed the same infants interacting with their mothers at two months postpartum. These videos were coded to identify instances where mothers mirrored their infant’s facial expressions, including equivalents (smile, mouth opening, and negative) to those expressions observed during EEG acquisition later on. We chose to look at mother-infant interactions at two months because this is a privileged time in terms of face-to-face interaction and maternal mirroring of expressions, with infants showing the most interest in ‘pure’ face-to-face exchanges at this age 31,53 . We predicted that mothers’ tendency to imitate particular expressions during early interactions would relate to the strength of infant mu ERD during observation of the same expression, supporting the hypothesis that visuomotor experience provided by maternal mirroring supports the development of a facial action-perception matching mechanism.


Functional EEG connectivity in infants associates with later restricted and repetitive behaviours in autism a replication study

We conducted a replication study of our prior report that increased alpha EEG connectivity at 14-months associates with later autism spectrum disorder (ASD) diagnosis, and dimensional variation in restricted interests/repetitive behaviours. 143 infants at high and low familial risk for ASD watched dynamic videos of spinning toys and women singing nursery rhymes while high-density EEG was recorded. Alpha functional connectivity (7–8 Hz) was calculated using the debiased weighted phase lag index. The final sample with clean data included low-risk infants (N = 20), and high-risk infants who at 36 months showed either typical development (N = 47), atypical development (N = 21), or met criteria for ASD (N = 13). While we did not replicate the finding that global EEG connectivity associated with ASD diagnosis, we did replicate the association between higher functional connectivity at 14 months and greater severity of restricted and repetitive behaviours at 36 months in infants who met criteria for ASD. We further showed that this association is strongest for the circumscribed interests subdomain. We propose that structural and/or functional abnormalities in frontal-striatal circuits underlie the observed association. This is the first replicated infant neural predictor of dimensional variation in later ASD symptoms.


Watch the video: ANT Neuro Webinar - An Introduction to EEGEye-Tracking Integration (January 2022).