The resting state EEG data analyzed here were recorded as part of EU-AIMS LEAP. A complete description of the study design and clinical characterization of the participants can be found elsewhere [21, 22]. Briefly, participants with ASD were recruited based on an existing clinical diagnosis of ASD according to DSM-IV, DSM-IV-TR, DSM-5 or ICD-10 criteria and all participants were between 6 and 32 years. ASD symptomatology was assessed with various instruments including the Autism Diagnostic Observation Schedule-2 (ADOS-2) and the Autism Diagnostic Interview-Revised (ADI-R), but participants were not excluded based on ADOS-2 and ADI-R scores. In this work, we only include autistic individuals with average to high intellectual capacities (IQ > 75) who successfully completed the resting state EEG recording (211 NT and 242 ASD).

Four minutes of resting state EEG were recorded per participant (2 min with eyes open, 2 min with eyes closed). To optimize participant compliance, resting state was acquired in 30 s blocks, alternating eyes open (fixating a physical hourglass) and eyes closed. Data were acquired at five sites: Central Institute of Mental Health (CIMH, Mannheim, Germany), King’s College London (KCL, United Kingdom), University Nijmegen Medical Centre (RUNMC, Netherlands), University Campus BioMedico (UCBM, Rome, Italy) and University Medical Centre Utrecht (UMCU, Netherlands). The following EEG systems were employed: Brainvision (CIMH, KCL, RUNMC), Biosemi (UMCU) and Micromed (UCBM), with sampling frequencies of 5000 Hz (KCL, RUNMC), 2048 Hz (UMCU), 2000 Hz (CIMH) and 256–1000 Hz (UCBM). All sites used 10–20 layout caps, with 60–70 electrodes.

The statistical analyses reported in subsequent sections were performed using 212 ASD and 199 NT participants, after discarding 42 participants because of unsuccessful preprocessing, lack of clear alpha peak or no available head model (26, 7 and 9 participants, respectively, as detailed in previous sections). The groups of included and excluded autistic participants differed significantly, at the uncorrected group level, on Vineland (Adaptive Behaviors subscale scores) and ADOS-2 (Restricted and Repetitive Behaviours scores), with higher symptomatology in the excluded group (see Additional file 1 Table S1 for more details). This might be expected since participants with higher symptom expression are less likely to be able to comply with instructions, and more likely to move or generate artifacts during the EEG and MRI acquisition sessions. Table 1 summarizes the main demographics and clinical characteristics of the included participants. Age and sex did not differ between groups (p = 0.71 and p = 0.24, respectively). Similarly, the amount of clean data did not differ significantly (p > 0.15) between groups for eyes open (ASD: mean 102 s, SD 13 s, range 55–120 s; NT: mean 104 s, SD 14 s, range 46–120 s) or eyes closed (ASD: mean 104 s, SD 13 s, range 60–125 s, NT: mean 106 s, SD 13 s, range 58–120 s) conditions.Table 1

Overview of clinical and demographic characteristics of the participants included in the statistical analyses

We opted for a hypothesis-free statistical analysis rather than a targeted approach testing concrete hypotheses given that the ASD resting state literature is contradictory and a variety of effects across brain rhythms and brain regions have been reported. We assessed potential power spectra and FC alterations across frequencies and brain regions. Furthermore, we evaluated the generalizability of our outcomes by evaluating the impact of several analyses choices such as FC metric, statistical models, or ASD definition. In order to do so, while maintaining high statistical flexibility, we followed a train/validation approach. That is, we separated the dataset into training (70% of the participants: 147 ASD and 140 NT) and validation datasets (30% of the participants: 65 ASD and 59 NT). The 30%–70% split was performed using stratified randomization by site, age group (children, adolescents, adults) and diagnosis. The univariate and multivariate statistics described in the following sections were performed using the training dataset exclusively. Clear hypotheses were derived from the analyses in the training dataset, which were then tested in the validation dataset. To assess the consistency between results in the training and validation datasets, prediction intervals were calculated for the univariate analysis. They estimate the range of values that can be expected in a replication due to chance and were calculated following [38] for comparing effect sizes. An overview of the statistical approach is shown in Fig. 1. Clinical and demographic characteristics of the participants in the training and validation datasets can be found in Additional file 1: Tables S2 and S3.Fig. 1

Overview of the statistical analysis approach. Univariate and multivariate statistics were performed in the training dataset, as well as control comparisons to evaluate the sensitivity of the results to pipeline choices. From this, concrete hypotheses are generated and tested in the validation dataset

First, we studied properties of the EEG alpha peak, the most prominent feature of the awake human EEG. As expected from the literature, both the alpha peak frequency and absolute power significantly changed with age, with increasing peak frequency and decreasing power (Fig. 2, p values for age coefficient: alpha peak frequency p = 7.0·10⁻¹¹, absolute power eyes open p = 3.2·10⁻¹⁷, absolute power eyes closed p = 1.9·10⁻⁹). We next investigated whether the groups differed in mean or variance. (A higher variance in the ASD group could be indicative of increased variability or heterogeneity.) The mean reactivity to eye opening differed between the ASD and NT groups (log-likelihood test p = 0.042): There was a significant interaction between group and age (F-test, p = 0.048): The age-related increases in reactivity were stronger in the NT than in the ASD cohort. We found no significant differences between ASD and NT for alpha peak frequency or absolute power in the training sample and no difference between variances for any measure (p > 0.12, Table 2).Fig. 2

Alpha peak measures. For each measure, the scatter plot of the raw values as a function of age is shown in the left side and the residuals of the linear mixed effects model y ~ 1 + age + sex + IQ + (1|site) on the right side. All plots derived from the training dataset (147 ASD and 140 NT)

Next, we investigated source space spectral power and functional connectivity. As expected from the literature, both PS and FC were strongly modulated by age. Cluster-based permutations tests showed two clusters of PS modulations with age (p < 0.001, log-likelihood tests of models with and without age coefficients): with PS decreasing with age for lower frequencies (< 7.7 Hz) and increasing for higher frequencies (> 10.6 Hz). Note that opposite trends in both frequency ranges are expected given the use of relative power. Comparing ASD to NT, we did not find any significant group effects for PS or FC, neither for mean nor for variance (p > 0.05, correcting for multiple comparisons with cluster-based permutation tests within PS and FC, respectively; Table 2).

We next investigated whether a multivariate combination of PS and FC features across space and frequency could separate ASD from NT using three machine learning approaches: linear SVC, elastic net logistic regression and radial basis function SVC with a nested Boruta feature selection (see Methods). Classification performance was overall poor (accuracy 47–57%, sensitivity 50–54%, and specificity 44–61%). For PS and wPLI, the classification performance with the elastic net classifier was greater than expected by chance (p = 0.025 and p = 0.009, respectively, Table 3).Table 3

ASD vs NT cross-validation classification performance of PS and FC features (training dataset)

Our results above depend on specific analysis choices including accounting for age as a covariate, criteria for inclusion in the ASD group and choice of specific PS and FC metrics. We next tested whether our results were robust to these specific choices. First, to account for possible nonlinear age dependencies, we built separate ASD vs. NT classification models for children (6–11 years), adolescents (12–17 years) and adults (18–32 years). Such models could uncover potential age-group specific ASD patterns (e.g., alterations only in adults, or only in children and adults but not adolescents) without committing to a pre-specified age-dependency model. Secondly, we used an alternate and “narrower” ASD definition and restricted the ASD sample to individuals who in addition to the ASD clinical diagnosis also met the ASD threshold on the ADOS-2 and on the ADI-R, following [50]. 51.8% of the ASD participants met this additional criterion (n = 77 ASD participants in the training dataset, n = 110 ASD participants overall, see Additional file 1: Table S5 for a full description of their clinical and demographic characteristics). Thirdly, the effect of EEG processing pipeline choices was assessed by evaluating absolute rather than relative power and alternate FC algorithms with different underlying assumptions on the nature of the coupling between regions’ activities (PowCorr, COH, iCOH and PLV, as described in Materials and Methods). The results are listed in Table 3. Of note, given the amplitude bias in beamforming source reconstructions, absolute power was estimated in sensor space. Classification performance was in general poor and in line with the main analysis described in the previous section, and only significantly better than expected by chance for the linear SVC and the elastic net model with wPLI in children (p = 0.02 and p = 0.013, respectively).

Data of 30% of the participants (65 ASD and 59 NT) were hold out from the previous analyses, in order to enable validation of the effects identified in the training dataset (significant group effects for reactivity to eye opening and classification performance for PS and wPLI):

Reactivity to eye opening was not significantly modulated by group in mean or variance in the validation dataset (p > 0.5). Some trends similar to those present in the training dataset can be found in the validation dataset: stronger increase with age in the NT than in the ASD group, and reduced values in adults in ASD compared to NT, see Fig. 3. However, these effects were weak and not significant. Although group effects were not significant, the effect sizes of the case–control alterations across age groups in the validation dataset fell within the 95% prediction intervals of the effects seen in the training dataset, indicating that random variability within the ASD and NT groups alone could account for the deviation of values observed between training and validation dataset. See Fig. 3 for a full overview of effect sizes and prediction intervals across age groups.

In summary, all the ASD vs NT alterations found in the training dataset were not significant in the validation dataset (ASD group effects p > 0.05), despite showing similar trends for reactivity to eye opening and PS elastic net model. This could be due to sampling error and to limited statistical power for detecting weak effects in the validation dataset. This is illustrated by the following power considerations. The statistical power to detect an effect size (standardized mean difference) of 0.5 in the validation dataset is 72%, but if this effect is for example restricted to adults only (which could be the case for reactivity to eye opening training dataset), the statistical power is only 18%. This means that there is a high chance of not finding significant effects in the validation dataset even if the true ASD phenotype was close to that found in the training dataset and there was a weak interaction between age and group. More details can be found in Additional file 1.

In line with previous studies of NT developmental trajectories, we observed the expected developmental changes of EEG features, such as age-related increases in alpha peak frequency, relative upper alpha/beta power and average FC [51–54]. This can be seen as a plausibility check for this dataset.

Directly comparing our results with previous publications is complicated by heterogeneity in the methods (e.g., specific EEG parameter extracted, FC metric, regions of interest to estimate pairwise connectivity) and the ASD subpopulation (e.g., age range, IQ, sex). A careful comparison with prior work is nonetheless critical to inform future work with EEG in ASD. Of particular relevance to this effort is O’Reilly et al.’s prior systematic analysis of EEG/MEG FC literature [17]. They evaluated the support for two popular hypotheses in the ASD field—long-range hypoconnectivity and local hyperconnectivity—and concluded that there was relatively strong support for long-range hypoconnectivity in ASD. However, heterogeneity in methods precluded a quantitative meta-analytic confirmation of this phenomenon or of which EEG parameter would best quantify it. Additionally, publications from the largest reviewed dataset [55, 56] enrolling 430 ASD and 554 NT found a mixture of long-range hypo- and hyperconnectivity, suggesting that the long-range hyperconnectivity hypothesis may not fully reflect the complexity of ASD. The authors found an impressive 86% ASD vs. NT classification accuracy, but we cannot test whether their results replicate in the LEAP dataset because their findings are based on a multivariate combination of thousands of features, and the exact combination of features and model weights is not available for download. Although the original study excluded high functioning autism and Asperger [56], the authors found that their original multivariate model also classified NT vs. Asperger with high accuracy [55], so the IQ range of the cohorts do not seem to be causing the discrepancy between their results and ours.

FC in ASD has also been extensively studied in fMRI. Some reviews indicate a consistent pattern of reduced fMRI cortico-cortical connectivity in ASD in adolescents and adults, with an associated and perhaps compensatory increase in short range connectivity [57–59]. In fact, FMRI FC alterations have also been found in the LEAP sample: with both hyperconnectivity in prefrontal and parietal cortices and hypoconnectivity in sensory-motor regions in ASD compared to NT [26]. This pattern of alterations was reproduced in two additional datasets, thereby confirming the robustness of the effects. While both fMRI- and EEG-derived FCs measure aspects of long-distance communication in the brain, the resulting FC values are different, as shown in [60]. Both modalities measure a different type of brain activity (synchronized neuronal activity for EEG, blood oxygenation for fMRI) and have distinct spatial and temporal resolution [61]. Future multimodal analyses comparing directly EEG- and fMRI-derived FC at the single subject level in ASD and NT would be best suited to understand whether the fMRI-derived ASD patterns have an EEG correlate.

Regarding PS, Edgar et al. published the largest study to date [62], with 183 ASD and 121 NT, and provided a testable hypothesis: an interaction between group and age in alpha peak frequency, with strong age-related increases in the NT group and no significant age effects in the ASD group. Within the LEAP dataset, we found age-related increases for both groups. Both datasets have a similar design (cross-sectional, similar age range, no lower IQ participants), so any difference in results is more likely to reflect other factors, such as differences in recruitment, and other more nonspecific effects (e.g., site effects or sampling variability). However, LEAP results for the ASD group are not far from the prediction interval of Edgar et al.: They found r = 0.10 in the ASD group, which according to [38] results in a prediction interval for LEAP of [− 0.10, 0.30] and that is close to the value we recovered (r = 0.33). For the NT group, our results converge with Edgar and colleagues: They found r = 0.57, which leads to a prediction interval [0.41 0.72] which in turn comprises the value obtained in LEAP (r = 0.44).

It is important to highlight that a lack of significant effects is different from demonstrating that there are no group differences. Although we had access to a large cohort and would have consequently greater sensitivity to detect core ASD patterns than most previous studies, it is possible and indeed even plausible that there are differences between ASD and NT with smaller effect sizes that we may not have detected. That said, our work suggests that possible differences in resting state EEG features between ASD (IQ > 75) and NT have relatively low effects sizes, which may limit their utility.

We used statistical power simulations to illustrate and quantify the impact of sample size. Accordingly, our statistical power to detect a true alteration of effect size of 0.3 would be 62% in the training dataset, which means that there is a 38% chance of having missed such an effect. In the training dataset, we found a significant interaction between age and group in the reactivity to eye opening, with stronger age-related increases in NT than ASD and strongest different between both groups for the adult cohort (ES =  − 0.57). The ES in the training was − 0.21, which was in the 95% prediction interval from the training dataset, and there is therefore no evidence for discrepancy between both effects. In fact, our statistical power to detect ASD alterations of effect size 0.5 in adults only is 18%, indicating that the validation dataset is not well suited for testing such small age-group specific effects and the need for a bigger cohort to fully assess the reproducibility of these findings.

For multivariate statistics, we found weak classification performances in the training dataset (cross-validation accuracies 0.56–0.64), and in the validation dataset the PS classifier had an accuracy of 0.57 along with a trend for statistical significance (p = 0.073). Although this classification pattern may reach significance in a bigger validation dataset, the performance in the training and validation dataset suggests that this is a rather weak classifier which may not have very useful clinical applications. Additionally, machine learning techniques often require a large sample size to achieve good predictive performance [63, 64]. A minimum of 10–20 observations (our in our case, subjects) per variable has been suggested for reliable performance of machine learning methods [65, 66]. Our EEG PS and FC matrices have a high dimensionality, but these variables are correlated and most of their variance is captured in a much smaller number of components (160–389 for PS and FC). Accordingly, future studies with at least 1500 individuals could be therefore better suited to boost the performance of the classifiers. Additionally, although we chose three different and popular state of the art machine learning approaches, it is conceivable that other approaches would be more suited to the problem and to a potential true ASD pattern of alterations. We therefore cannot exclude that with more data and/or other algorithms a better ASD vs. NT separation can be achieved.

The heterogeneity of ASD could underlie the lack of significant ASD vs NT differences. ASD has diverse clinical presentations and is in turn thought to be driven by diverse biological substrates [2, 67, 68]. In fact, ASD is common in genetic neurodevelopmental disorders that are characterized by specific and different, sometimes even “opposite” neurobiological and circuitry alterations. It may well be that idiopathic ASD could be split into subgroups of homogeneous neurobiology that substantially differ [3, 68] and possibly even deviate from NT in opposite directions. This could mean for EEG features that the subgroups would change in various directions and the mean might not differ from NT unless appropriate subgrouping is performed. If this were true, it may manifest as increased variance in the ASD group compared to the NT group. We investigated this possibility in the univariate analysis, but could not find any support for the idea that heterogeneous subgroups may drive differential variance in the ASD population as a whole, as compared to NT. Of note, we did not perform clustering in the ASD EEG PS and FC or attempt to find ASD subgroups, but rather focused on the variance of PS and FC measures in ASD compared to NT. Previous studies have used unsupervised statistical learning to identify clusters of ASD subjects based on clinical, behavioral or biological measures [69–71]. Additionally, some studies suggest that differences in brain structure and function in people with ASD are idiosyncratic [72–75]: They are peculiar to an individual person with ASD rather than following a group trend. In this view, each ASD individual could deviate from NT in a different set of features, and no substantial increase in the ASD group variance would be expected for individual PS or FC measures.

Paradigms and brain activity measurements other than resting state EEG may be more sensitive to capture ASD-specific brain circuitry alterations. Indeed, significant case–control effects have been found in previous analyses of resting state fMRI and task EEG in the baseline LEAP recordings. First, as mentioned earlier in the discussion, reproducible fMRI FC alterations have been found in the LEAP cohort [26]. Second, [76] showed an increase in the latency of the N170 component of the event-related potential to faces in ASD compared to NT in the LEAP cohort. The modulation of brain activity during tasks that challenge functions that are often altered in ASD may shed more light onto the ASD-specific circuitry than resting state activity. Interestingly, here we found trends for an effect in the reactivity to eye opening, which is a very basic sensory response measure. Other more elaborated EEG paradigms tapping into elements of sensory or social processing linked to the ASD symptomatology could show stronger effects [77–79].