Edited by: Alexandre Gramfort, INRIA, France
Reviewed by: Julien Lefèvre, Aix-Marseille Université, France; Rey Ramirez, Univesity of Washington, USA
*Correspondence: Cristina Campi, Department of Computer Science/HIIT, University of Helsinki, Gustaf Hällströmin katu 2b, FI-00014, Helsinki, Finland e-mail:
This article was submitted to Frontiers in Brain Imaging Methods, a specialty of Frontiers in Neuroscience.
This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.
Traditional stimulus-based analysis methods of magnetoencephalography (MEG) data are often dissatisfactory when applied to naturalistic experiments where two or more subjects are measured either simultaneously or sequentially. To uncover the commonalities in the brain activity of the two subjects, we propose a method that searches for linear transformations that output maximally correlated signals between the two brains. Our method is based on canonical correlation analysis (CCA), which provides linear transformations, one for each subject, such that the temporal correlation between the transformed MEG signals is maximized. Here, we present a non-linear version of CCA which measures the correlation of energies and allows for a variable delay between the time series to accommodate, e.g., leader–follower changes. We test the method with simulations and with MEG data from subjects who received the same naturalistic stimulus sequence. The method may help analyse future experiments where the two subjects are measured simultaneously while engaged in social interaction.
Magnetoencephalography (MEG) is a powerful functional neuroimaging method with a millisecond-scale temporal resolution (Hämäläinen et al.,
This paper introduces a novel method for the analysis of MEG signals measured from two brains. The data can come from two different scenarios. In the simpler case, we have measurements of brain activity elicited by the same naturalistic stimulation. In the second case, which requires unconventional instrumentation, we consider data simultaneously measured from two subjects during social interaction, here referred to as two-person data (Baess et al.,
In the case of two-person data, we anticipate a small, fluctuating time delay between the activations in the two brains. Such lags are neurophysiologically interesting; they may signify successful predictions on other person's actions as well as leadership in the dyadic interaction. Obviously, during interaction the leader and follower can switch their roles, and the analysis method should be able to take this into account by allowing the delay to vary.
In general, searching for neural sources of unaveraged MEG data is very difficult because of the low signal-to-noise ratio (SNR). Moreover, common activations are unlikely tightly phase locked. Therefore, it is more reasonable to look for correlations of energies (powers) of the activations, rather than the source time series themselves, for identifying common sources.
Let us denote the data measured from
A smoothed version of the energies can be obtained as a convolution with a temporal filter
The aim of this analysis is to find the spatial filters that maximize the correlation of the energies of the measurements from two subjects. We have to take into account that the best correlation can occur with some lag τ, which further depends on
Thus, for estimating the spatial filters
A general method for finding maximally-correlated features in datasets is the Canonical Correlation Analysis (CCA) (Hotelling,
We implemented a Non-Linear CCA (NLCCA) in the following way: we expanded the energies
In the space of binomials, the problem of finding
Here, we also see how smoothing of the energies is possible by a direct application of a temporal filter to the expanded data
For computational efficiency, we center and whiten
To perform the actual maximization, we first compute the derivative of this objective function with respect to the index
For estimating τ, we maximize the objective function with respect to τ, in alternation with the maximization with respect to the spatial filters. We start by splitting the datasets in to a sequence of
Once the pair of filters (
The method described in the previous section was applied both to synthetic and real MEG measurements. The synthetic data were designed to test in which conditions our algorithm can discriminate correlated and anti-correlated sources in two datasets from a two-person MEG experiment.
We simulated two sources in both of the two brains, one in the occipital lobe and the other in the left parietal lobe close to the midline. No delays were introduced in this simulation. For Subject 1, we simulated strong rhythmic activity in the occipital lobe, occuring at the same time as a weaker activity in the parietal lobe. After a period of no activity, the activations were “flipped”; strong rhythmic activity appeared in the parietal source and a weak one in the occipital source and so on. For Subject 2, timing of the envelopes of the oscillations was the same as for Subject 1, but the strengths in occipital vs. parietal lobe were reversed.
These sources produced the strongest signals in occipital and parietal areas slightly leftwards from the midline; see Figure
Once the source locations and envelopes were chosen, the two resulting magnetic field time courses were simulated using lead field matrices. These matrices were computed using a boundary-element method (BEM) with a single compartment (Hämäläinen and Sarvas,
We set the weak source to be either 50 or 30% of the strength of the strong source.
Next, we used previously recorded real MEG data (Ramkumar et al.,
The data were used in two different experiments:
First, all the possible couplings of the 11 subjects were considered for creating 55 pairs. In all these pairs, the data of one subject were artificially delayed with respect to the other subject by 0, 100, 250, 500, or 1000 ms. Second, the subjects were divided into two groups of five subjects and the data were concatenated in time to build two long datasets.
The rationale of the first setting was to test the ability of our method to detect lags between expected correlated activations in the pairs of measurements. The delay in the data was constant for the whole recording, but this does not reduce the generality of the experiment too much since the estimated lag was not constrained to be constant.
The second setting was used to test whether increasing the amount of data would improve the accuracy of estimating the spatial filters.
In both cases, all the data were filtered to the frequency band of 8–30 Hz and dowsampled to 150 Hz. Altogether 30 components, explaining more than 90% of the variance, were extracted by PCA for all the pairs with the five different artificially-introduced delays and for the concatenated datasets.
For the first setting, we set the interval of admissible lags to [−750, 750] ms for the case of 0, 100, 250, and 500 ms delays, while for the 1000-ms delay the interval of admissible lags was [−1500, 1500], with a step 100 ms. The length of the window used for estimating the spatial filters and the lags was 10 s in all cases.
In the first case (the weak source 50% of the strong one), the method estimated just one relevant component, while the other components represented noise, see Figure
In the second case (the weak source 30% of the strong one), the method was further able to properly separate the locations of the coupled sources: the first two estimated components (Figures
Figure
However, the results of the estimated spatial filters were less promising. In Figure
This negative result provided the motivation for considering data concatened over subjects. Concatenated datasets (with data from five subjects each) were thus analyzed to find out whether a larger amount of data would allow our algorithm to detect coupled activation between the subjects. In fact, as the first row of Figure
We proposed a data-driven method based on non-linear canonical correlation analysis for finding, on the basis of MEG recordings from two subjects, linear transformations of the data representing cerebral sources with maximally correlated energies, allowing a delay of 0–1500 ms. A simple algorithmic implementation was proposed by expanding the data to the space of all possible products of the channels. Application to synthetic and semi-real two-person data sets indicated that the method is promising.
The experiments with synthetic data pointed out that the method can recover the original, independent sources in the data provided that the relative differences between the amplitudes of the sources are large enough. Otherwise, this method may include all the sources in a single component, as a kind of a larger network. An important question is whether and how the separation capability of the method could be improved. On the other hand, estimating such coarse large networks may also be useful in some applications.
In experiments with real, although articifically delayed MEG recordings, NLCCA estimated correctly the delays even in single subjects. In contrast, the estimated spatial filters did not always represent reliable coupled activations for single subjects. The method seems to require more data for reliable estimation of spatial filters as was seen in the long datasets created by concatenating measurements from different subjects, for which the estimated spatial filters were similar enough in the two data sets. The concatenation, although useful for the estimation of the spatial filter, precludes real-time analysis of the data. A statistical optimization of our energy correlation measures to enable reliable estimation in single subjects is an important topic for future research. Essentially the same problem of finding maximally correlated energies was considered by Gutmann and Hyvärinen (
The search of sources with maximally correlated energies could be carried out on the source time series estimated by some inversion method but in this work we preferred to investigate the possibility to operate directly on the MEG data, without introducing the inversion step.
Further methodological developments are required for the analysis of data measured during real two-person interaction. The delay parameter τ assumes a crucial role in such analysis, and its estimation, due to its time dependency, induces a great computational cost. Moreover, it would be important to estimate τ, and consequently the often unpredictable changes in the leadership of the interaction, with a good temporal accuracy which would require shorter time windows than what we had in the current work. The consequent problems in NLCCA estimation form a further motivation for future research.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This study was financially supported by the Academy of Finland (LASTU program, the National Centres of Excellence Programme, and a separate grant) as well as by the ERC Advanced Grant ♯232946. We thank Dr. Pavan Ramkumar for data collection for the original MEG study.