Anees Abrol
Barnaly Rashid
Srinivas Rachakonda
Eswar Damaraju
Vince D. Calhoun- 1The Mind Research Network, Albuquerque, NM, United States
- 2Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, NM, United States
Studies featuring multimodal neuroimaging data fusion for understanding brain function and structure, or disease characterization, leverage the partial information available in each of the modalities to reveal data variations not exhibited through the independent analyses. Similar to other complex syndromes, the characteristic brain abnormalities in schizophrenia may be better understood with the help of the additional information conveyed by leveraging an advanced modeling method involving multiple modalities. In this study, we propose a novel framework to fuse feature spaces corresponding to functional magnetic resonance imaging (functional) and gray matter (structural) data from 151 schizophrenia patients and 163 healthy controls. In particular, the features for the functional and structural modalities include dynamic (i.e., time-varying) functional network connectivity (dFNC) maps and the intensities of the gray matter (GM) maps, respectively. The dFNC maps are estimated from group independent component analysis (ICA) network time-courses by first computing windowed functional correlations using a sliding window approach, and then estimating subject specific states from this windowed data using temporal ICA followed by spatio-temporal regression. For each subject, the functional data features are horizontally concatenated with the corresponding GM features to form a combined feature space that is subsequently decomposed through a symmetric multimodal fusion approach involving a combination of multiset canonical correlation analysis (mCCA) and joint ICA (jICA). Our novel combined analyses successfully linked changes in the two modalities and revealed significantly disrupted links between GM volumes and time-varying functional connectivity in schizophrenia. Consistent with prior research, we found significant group differences in GM comprising regions in the superior parietal lobule, precuneus, postcentral gyrus, medial/superior frontal gyrus, superior/middle temporal gyrus, insula and fusiform gyrus, and several significant aberrations in the inter-regional functional connectivity strength as well. Importantly, structural and dFNC measures have independently shown changes associated with schizophrenia, and in this work we begin the process of evaluating the links between the two, which could shed light on the illness beyond what we can learn from a single imaging modality. In future work, we plan to evaluate replication of the inferred structure-function relationships in independent partitions of larger multi-modal schizophrenia datasets.
Introduction
Structural neuroimaging modalities evaluate anatomical brain structure and tissue type (e.g., structural MRI or sMRI) or brain tissue microstructure (e.g., diffusion MRI or dMRI), whereas functional neuroimaging modalities indirectly estimate brain function/activity through respective characteristic “source signals” or “indicators” of the underlying neuronal (e.g., electroencephalography or EEG, magnetoencephalography or MEG), metabolic (e.g., positron emission tomography or PET) or hemodynamic (e.g., functional MRI or fMRI) activity. As the modalities are acquired at different spatial and temporal scales, the spatiotemporal precisions can be enhanced immensely, for example, by combining a modality with superior spatial resolution with another modality with superior temporal resolution. Furthermore, if the modalities are generically thought of as producing filtered sights of brain's organization or activity, working with multiple modalities would enable complementary sights into brain structure and/or function, thus intrinsically accomplishing a more comprehensive view (Calhoun et al., 2006; Calhoun and Adali, 2009; Schultz et al., 2012; Sui et al., 2012b; Uludag and Roebroeck, 2014; Calhoun and Sui, 2016). Given the above benefits, multimodal neuroimaging data acquisition and analysis has become much more widely utilized in recent years.
In a multimodal study, data corresponding to the different modalities might be acquired separately or simultaneously depending on the research question being addressed. Generically, separate data acquisition results in marginally higher signal to noise ratio (SNR) and lesser artifacts. However, simultaneous data acquisition is essential in studies where the objective is to study time-dependent responses to events, and inter-modality correlates (Uludag and Roebroeck, 2014). As an example, it would be important to simultaneously acquire EEG and fMRI data if the study goal is to identify potential correlates of time-varying functional connectivity measures in fMRI data to the EEG data. In this case, the acquired modalities could be analyzed through separate or collective pipelines using a variety of univariate or multivariate algorithm through a model-based or data-driven approach (Calhoun and Sui, 2016). Previous multimodal work has typically analyzed data from different modalities separately and correlated the independent results from the unimodal analyses, or used one of the modalities to constrain models corresponding to the other modality. The above mentioned types of multimodal studies have proven to be very useful, but make minimal or limited use of the cross-modality (i.e., joint) information, a resource that is now being increasingly availed by use of “symmetric” data fusion approaches (Calhoun and Sui, 2016). “Feature-based” symmetric data fusion approaches inherently first estimate useful features from the different modalities independently and then evaluate relationships between these features. This practice leverages the partial information available in each of the modalities to reveal data variations not exhibited through the independent analyses. To date, there have been several interesting demonstrations of the potential of utilizing such cross-modality or joint information in understanding the human brain and its disorders, disease characterization or biomarker identification, and uncovering disrupted links in complex mental illness (see Calhoun and Sui, 2016 for a detailed review).
Notably, multimodal studies with advanced modeling methods assume greater significance in diagnosis of a complex syndrome, for example schizophrenia, where striking pathological and etiological heterogeneity has been observed. Several previous studies (Olesen et al., 2003; Bassett et al., 2008; Hagmann et al., 2008; Rykhlevskaia et al., 2008; Honey et al., 2009; van den Heuvel et al., 2009; Camara et al., 2010; Michael et al., 2010; Skudlarski et al., 2010; Yu et al., 2011; Segall et al., 2012; Alexander-Bloch et al., 2013) clearly suggest interactions between structural and functional connectivity. Thus, it is reasonable to hypothesize covariation between “feature spaces” i.e., distilled (or lower dimensional or second/higher order) measures of brain structure and function in each modality. Importantly, reducing or projecting the very high dimensional data to feature spaces facilitates removal of redundant data while promoting identification of inter-modality relationships in a simpler, lower-dimensional space. Hence, in symmetric fusion approaches, it is the lower dimensional feature spaces that are fused to extract joint information. Some examples of lower dimensional feature spaces include contrast maps, amplitude of low frequency fluctuation maps (ALFF), etc. for fMRI data, segmented gray or white matter maps for sMRI data, fractional anisotropy (FA) or mean diffusivity (MD) maps for diffusion tensor imaging (DTI) data, single-nucleotide polymorphism (SNP) or methylation data for genetic data, etc.
Feature space projection is carried on by using a wide range of model-driven or data-driven approaches as reviewed in Calhoun and Sui (2016). Model-driven approaches have their own benefits in the form of enabling specific hypothesis testing of inter-regional interaction (provided there is enough prior information available on the problem being studied). On the other hand, data-driven approaches, in general, require specification of lesser assumptions about the data upfront. This makes them (the data-driven approaches) more suitable for studying complex problems, for instance a complex syndrome, such as schizophrenia, wherein little reliable prior knowledge is available. Data-driven approaches typically explore use of a blind or semi-blind multivariate approach so as to reveal hidden structure of inter-relationships between two (or more) data feature spaces. The use of multivariate approaches enables estimation of multiple variables jointly, and has some additional advantages over the use of univariate approaches. Multivariate approaches are relatively easy to interpret due to co-varying nature of the variables (i.e., regions of interest) and warrant additional robustness to noise as measures from patterns are explored rather than measures from paired relationships (Calhoun and Sui, 2016). Recently used blind multivariate decomposition methods include, but are not limited to, joint independent component analysis (jICA) (Calhoun et al., 2006), multiset canonical correlation analysis (mCCA) (Correa et al., 2007), partial least squares (PLS) (Martinez-Montes et al., 2004; Chen et al., 2009), and linked ICA (Groves et al., 2011), while adaptive (semi-blind) approaches, such as coefficient constrained ICA (CC-ICA) (Sui et al., 2009), and parallel ICA (Liu et al., 2009) also exist.
The above discussed multivariate approaches are all based on linear mixture models, but differ considerably in the optimization strategies/priorities they evolve the data sources through as well as in their basic limitations. Additionally, combining multiple multivariate algorithms has also been recently suggested to allow flexibility in the estimations by reducing the limiting effects of the individual approaches (Sui et al., 2011) as discussed next. The joint sources estimated by the jICA (or the linked ICA) algorithm are optimally maximally independent but share a common mixing matrix, thus assuming a very strong correlation between the joint sources. Contrarily, the mCCA algorithm jointly maximizes the inter-subject covariations thus allowing for varying levels of connectivity strengths between the joint sources. In this method, each dataset is decomposed into a set of sources with corresponding mixing profiles, also termed as canonical variates (CVs), and their corresponding correlation values, also called canonical correlation coefficients (CCCs). Despite allowing for varying activation levels of the joint sources, there remains the possibility that the spatial maps of the emergent joint sources in mCCA are highly similar in some cases where, for example, the CCC estimates are not sufficiently distinct (Correa et al., 2010; Sui et al., 2012a, 2013). Sui et al. (2011) used the mCCA+jICA algorithm for fusing the fMRI contrast maps and DTI FA maps to investigate group differences in healthy controls, schizophrenia patients and bipolar patients. Interestingly, this study concluded increased group classification accuracy with this algorithm as compared to its constituent algorithms tested alone. The combined mCCA+jICA model basically uses mCCA in the first step (Sui et al., 2011, 2013) followed by joint ICA (jICA) in the second step. In the first step, the different feature spaces are first linked with flexible linkages, thus adding to the investigator's confidence to perform jICA with an objective of identifying both highly and weakly correlated joint sources in the second step. A recent paper has proposed a unifying framework to link together a wide variety of multimodal approaches including the ones mentioned above (Silva et al., 2016).
Studies leveraging the above mentioned multivariate approaches have revealed significant information on clinical aspects of schizophrenia as discussed in several recent reviews on multimodal fusion (Biessmann et al., 2011; Schultz et al., 2012; Sui et al., 2012a; Lahat et al., 2015; Calhoun and Sui, 2016). Simultaneous analysis of anatomical and functional connectivity in Skudlarski et al. (2010) suggested that fusion allowed identification of deficits in white matter anatomy, and complex alterations in functional connectivity. In another multivariate, multimodal analysis, Michael et al. (2010) fused structural and functional brain images to reveal decreased overall structure-function linkage in schizophrenia as compared to healthy controls both in a working memory and an auditory sensorimotor task. Camchong et al. (2011) revealed convergent findings in multiple modalities (DTI and fMRI) consistent with the disconnection hypothesis in medial frontal regions in subjects with schizophrenia. jICA was used in Sugranyes et al. (2012) to characterize linked functional and white-matter changes related to working memory dysfunction, and in Stephen et al. (2013) to identify structure-function relationships using MEG and DTI modalities. The latter study concluded impairments in a posterior visual network in schizophrenia, with reduced FA and MEG amplitude, and overall weaker cognitive performance. Furthermore, Xu et al. (2009b) used joint source based morphometry (joint SBM) to identify linked white and gray matter (GM) differences in regions comprising temporal-corpus callosum, occipital/frontal-inferior fronto-occipital fasciculus, parietal/frontal-thalamus, and frontal/parietal/temporal-superior longitudinal fasciculus. Using the mCCA multivariate algorithm, Sui et al. (2015) identified linked structural and functional deficits in distributed cortico-striato-thalamic circuits and their association with cognitive impairments as measured through the Measurement And Treatment Research to Improve Cognition in Schizophrenia (MATRICS) consensus cognitive battery (MCCB). Finally, several classification studies have made use of multivariate, multimodal approaches to demonstrate improved classification with use of multiple modalities as compared to the use of a single modality in classifying patients from controls (Yang et al., 2010; Ulaş et al., 2011; Nieuwenhuis et al., 2012; Sui et al., 2013; Peruzzo et al., 2015; Cabral et al., 2016; Cetin et al., 2016).
Recent work assessing dynamic (i.e., time-varying) functional network connectivity (dFNC) suggests availability of substantial information beyond time-averaged functional connectivity (FC) estimates in both resting state and task-based fMRI data (Hutchison et al., 2013a; Calhoun et al., 2014; Preti et al., 2016). The term “chronnectome” was recently introduced to describe a focus on identifying whole brain transient and recurring patterns in temporal coupling of the human brain (Calhoun et al., 2014). The temporal dynamics of the time-courses are often characterized by the sliding window correlation (SWC) method (Sakoglu et al., 2010; Allen et al., 2012) to estimate the windowed functional network connectivity (wFNC) data followed by a rigorous FC “state” estimation process from the wFNC data (Allen et al., 2012; Miller et al., 2016). These FC states have been proven to be stably present in the data and reoccurring over time in a highly structured manner in our recent evaluations of replicability of time-varying FC states and state summary measures (Abrol et al., 2016, 2017). Several studies have shown the emergent states to be functionally and behaviorally relevant by demonstrating direct links with signatures of consciousness (Hutchison et al., 2013b; Amico et al., 2014; Hudson et al., 2014; Barttfeld et al., 2015; Wang et al., 2016), tracking day-dreaming/mind-wandering (Kucyi and Davis, 2014; Kucyi, in press), sleep and awake states (Tagliazucchi and Laufs, 2014), ongoing cognitive function and performance (Craddock et al., 2012; Schaefer et al., 2014; Gonzalez-Castillo et al., 2015; Madhyastha et al., 2015; Shine et al., 2016a,b). Furthermore, evidence of potential electrophysiological signatures of dynamic blood-oxygen-level dependent (BOLD) FC also hints the fluctuations in the BOLD FC (as captured in the states) to be interesting i.e., having a neurophysiological origin (Tagliazucchi et al., 2012; Chang et al., 2013; Allen et al., 2017), although further confirmation is still needed. Besides, several studies have also used time-varying connectivity measures to characterize pathophysiology i.e., identification of disease states (Damaraju et al., 2014; Rashid et al., 2014; Yu et al., 2015; Du et al., 2016; Miller et al., 2016).
In this work, we focus on feature based fusion analysis of brain structural (sMRI) and functional (fMRI) images hypothesizing correspondence between brain structure and function (or more specifically, correspondence between the feature spaces of the two studied modalities). We propose exploring where and how GM corresponds to time-varying functional connections will improve our understanding of both structural and functional connectivity. We estimate the feature space for the functional data as subject-specific states that are revealed from the wFNC data using a novel framework featuring temporal ICA and dual regression (Figure 1A). More specifically, aggregate states are estimated by decomposing the wFC data using temporal ICA in the first step, followed by a dual regression analysis to estimate the subject-specific states in the second step. This derived feature space from fMRI data, referred to as “functional data feature space” hereon, is simultaneously analyzed with the corresponding “structural data feature space,” i.e., GM maps from sMRI data, using the mCCA+jICA data fusion algorithm.
Figure 1. Estimation of the functional (fMRI) data feature space. Aggregate states were estimated by decomposing the windowed correlations by temporal ICA. Subject-specific states were next estimated through a spatio-temporal (dual) regression procedure wherein, for each subject, the aggregate states were regressed into the subject's windowed FNC data to estimate subject-specific component time-courses in the first regression step, and the estimated time-courses were regressed into the subject's windowed FNC to derive the subject-specific states in the second regression step; (B) Summary of the mCCA + jICA framework. For each subject, the functional data feature space as estimated in (A) was concatenated with the smoothed, modulated and warped gray matter maps (as the structural data feature space) and fused using the joint “mCCA+jICA” framework. This framework combines the mCCA and jICA algorithms to decompose the observed data into a linear combination of sources mixed through “effective” modality-specific mixing matrices as illustrated above.
The fusion analysis in this work could be explained in a four-stage process (Figure 1B). In the first stage, mCCA reveals links between the modalities by maximizing the correlations between their mixing matrices i.e., CVs. In the second step, the associated maps to the CVs, i.e., the CCCs, are concatenated and decomposed using jICA to estimate the joint sources. In the third step, the (effective) modality-specific mixing matrices are estimated for the combined framework and analyzed for group differences for each joint source. In the last step, we focus on qualitative analysis of the subset of joint sources that feature linked, highly correlated and significant group difference showing structural and functional component maps from the jICA decomposition. Our exploratory investigation on data from 151 schizophrenia patients and 163 healthy controls shows general correspondence between GM and time-varying FC and also reveals few missing links in schizophrenia.
Materials and Methods
Data Acquisition and Pre-processing
This study worked with T1-weighted structural and T2*-weighted resting state (subjects instructed to keep eyes closed but stay awake) functional images from 151 schizophrenia patients (114 males, 37 females; average age 37.8), and age and gender matched 163 healthy controls (117 males, 46 females; average age 36.9). This data were collected at seven different sites across the United States as a part of the function biomedical informatics research network (fBIRN) data repository (Keator et al., 2016). Informed consent was received from the participants as per institutional guidelines practiced at the seven collection sites. Six sites used the 3T Siemens Tim Trio System while one site used the 3T General Electric Discovery MR750 scanner. A total number of 162 volumes of standard gradient echo planar imaging (EPI) BOLD fMRI data were captured with a repetition time (TR) of 2 s, echo time (TE) of 30 s, field of view (FOV) of 220 × 220 mm2 (64 × 64 matrix), flip angle (FA) of 77° and 32 sequential ascending axial slices of 4 mm thickness and 1 mm skip.
The sMRI data were spatial normalized, bias corrected, and segmented using the SPM unified segmentation model in an automated analysis pipeline developed at the Mind Research Network (Bockholt et al., 2010) to obtain the smoothed (using a full width half maximum Gaussian kernel of 10 mm), modulated and warped GM images for all subjects. The GM maps were then used as the input feature space to the fusion algorithm with an objective of estimating the patterns of brain structure that exhibit co-variations across subjects. The fMRI data were pre-processed using the SPM, AFNI and GIFT toolboxes as well as custom code written in Matlab in a similar fashion as Damaraju et al. (2014). Briefly, rigid body motion correction was performed using the INRIAlign SPM toolbox for subject head motion correction. This was followed by slice-timing correction step in order to account for any timing differences in scan acquisition following which the data were de-spiked using AFNI's “3dDespike” algorithm so as to reduce the impact of outliers. Next, the data were warped to the Montreal Neurological Institute (MNI) template and resampled to 3 mm cubic isotropic voxels. Since the fBIRN data came from multiple sites, the site or scanner variability needed to be smoothed equivalently. This was done using AFNI's “BlurToFWHM” algorithm, an approach that has been shown to decrease scanner-specific variability in smoothness and provide “smoothness equivalence” to the multi-site data (Friedman et al., 2008). Finally, the voxel time-courses were variance normalized before running the group ICA.
Functional Data Feature Space
Group ICA, Resting State Network (RSN) Selection and Post-processing
The pre-processed fMRI data were decomposed using spatial group ICA to reveal spatially independent components each with a unique time-course profile (Calhoun et al., 2001; Calhoun and Adali, 2012). The pre-processed datasets were first reduced to 130 principal components in the time-point dimension at the subject level. Using a (relatively) higher number of principal components at the subject level has been shown to stabilize back-reconstruction and retain maximum variance in the data, and if this is the case the specific number does not substantially impact the results (Erhardt et al., 2011). Accordingly, the entire dataset was transformed into 130 principal components using standard principal component analysis (PCA) at the subject level in the first data reduction step of the group ICA analysis (similar to Damaraju et al. (2014) on the same fBIRN phase 3 dataset). Using a relatively high number of principal components in this step retained a very high percentage of subject level variance (>99.99%). In the second data reduction step, the PCA reduced subject data were then concatenated along the time dimension and further reduced to 100 components by implementing group level PCA. A higher model order for group ICA was chosen to enable a more refined (i.e., finer) RSN parcellation (Kiviniemi et al., 2009; Abou-Elseoud et al., 2010), thus allowing evaluation of sub-nodes within network domains (Allen et al., 2012; Sockeel et al., 2016; Abrol et al., 2017; Li et al., 2017; Fu et al., in press). The reliability of the estimated independent components from this step was evaluated using ICASSO (Himberg et al., 2004), and it was found that the estimates exhibited tight clustering and converged consistently amongst several (twenty) runs. Finally, subject-specific component spatial maps (SMs) and time-courses (TCs) were estimated using the GICA back-reconstruction methods as implemented in the GIFT toolbox (Erhardt et al., 2011).
The back-reconstructed subject-specific SMs and TCs for the 100 independent components were extensively analyzed to identify the physiological, non-artefactual, previously established RSNs. More specifically, 47 components whose SMs showed peak activations in GM and low overlap with any known imaging, physiological, or movement-related artifacts, and mean power spectra exhibited higher low frequency content were established as RSNs for further analysis. The RSNs were assessed and distributed into the sub-cortical (SC), auditory (AUD), visual (VIS), sensorimotor (SM), attention/cognitive control (CC), default-mode (DMN), and cerebellar (CB) network domains (Figure 2).
Figure 2. Resting State Networks (RSNs). Spatial maps of the 47 retained RSNs at the most activated sagittal, coronal and axial slices.
Subject-specific TCs corresponding to the retained RSNs underwent additional post-processing steps. The TCs were de-trended to remove any existing linear, quadratic or cubic low frequency trends originating from scanner drift, orthogonalized with respect to estimated subject motion and realignment parameters, and de-spiked using AFNI's 3dDespike function to replace outlier points with values estimated from third order spline fit to neighboring portions of the TCs.
FC Estimation and Temporal Variability
Similar to previous works (Allen et al., 2012; Damaraju et al., 2014), time-varying FC was estimated by sliding a window of length 22 TRs (44 s) in steps of 1 TR (2 s). This sliding window analysis used a tapered window generated by convolving a rectangular window of length 22 TRs (44 s) with a Gaussian window of standard deviation equal to 3 TRs. The window length parameter has a significant impact on the observed dynamics, however our choice of 44 s (similar to window duration as used in Damaraju et al., 2014 on the same fBIRN phase 3 dataset) falls within recommended ranges in multiple works. In background, Leonardi and Van De Ville (2015) proposed a lower limit for window length using the (inverse of minimum frequency) thumb rule, which Zalesky and Breakspear (2015) formally demonstrated to be overly conservative especially in moderate SNR conditions (i.e., relatively shorter windows than as suggested by the thumb rule can be used to capture the fluctuations in time-varying connectivity). Recent studies have reported peak maximum detection probability of time-varying fluctuations (Hindriks et al., 2016) and peaks of significance of window lengths (Liégeois et al., 2016) in a similar (40–60 s) range. Furthermore, there are several studies that corroborate that varying the window length parameter over a range beyond a certain “safety limit” did not change the overall observed dynamics (Allen et al., 2012; Li et al., 2014; Yaesoubi et al., 2015; Deng et al., 2016; Preti et al., 2016).
Functional Data Feature Space Estimation
The wFNC data were decomposed using temporal ICA to reveal a set of “n” aggregate connectivity patterns (or aggregate states) shared amongst subjects and a set of “n” temporally independent connectivity patterns. Notably, the “n” temporally independent connectivity patterns are a concatenation of “n” individual subject time-courses which are not independent subject-wise. We estimate the feature space for the functional data as subject-specific “versions” of the aggregate states through a modified form of spatio-temporal (dual) regression (Filippini et al., 2009; Erhardt et al., 2011). In this analysis, the aggregate states are regressed into each subject's wFNC data to obtain a set of subject-specific time-courses in the first regression step which are then regressed into each subject's wFNC data to get the subject-specific states in the second regression step. The estimated functional data feature space is next simultaneously analyzed with the GM maps estimated from structural data using the mCCA+jICA data fusion algorithm.
The mCCA+jICA Framework
As a framework to evaluate fusion of feature spaces from two imaging modalities, this method reveals flexible, i.e., both highly and weakly correlated, joint sources from both the modalities. The framework (Figure 1B) assumes the multimodal dataset (Xk) to be a linear mixture of a (M) number of sources (Sk) mixed with non-singular matrices (Ak), where k is the modality index. Following (Sui et al., 2011, 2013), we used the minimum description length (MDL) criterion to estimate the number of independent components to be nine. Hence, we evaluate the feature spaces for a total number of nine components (M = 9) for both the fMRI and sMRI modalities.
In the first phase of the joint framework, the mCCA algorithm commences with dimensionality reduction of the feature spaces of each of the modalities using principal component analysis. In this work, we reduce the input data to a high (number of subjects-one) number of principal components so as to capture maximum subject level variance. Next, the canonical variates (Dk) are estimated by maximizing the sum of squared correlations (SSQCOR) cost (Kettenring, 1971) in the “M” columns of canonical variates. In the final step of the first phase, the canonical correlation coefficients (CCCs) are estimated as associated maps (Ck) by inverting the Xk = DkCk model [i.e., Ck = pinv(Dk)Xk].
In the second phase of the joint framework, the estimated CCCs are concatenated [(C1..Ck)] and input to the jICA algorithm which enables transformation of these CCCs to an orthogonal space. This decomposition reveals “M” maximally independent joint sources (S) each of which can be interpreted as a stacked form of co-varying modality-specific components i.e., S = [S1 … Sk]. The stacked components for the different modalities share a common mixing matrix (W) with the jICA linear mixing model evaluated as [C1..Ck] = W [S1..Sk]. Hence, the effective mCCA+jICA can be summarized as Xk = (DkW−1)Sk, where the effective modality-specific mixing matrices are estimated as Ak = DkW−1. The combined framework is illustrated in Figure 1B and further details on the parametrical/methodological choices in the algorithm can be found in the referenced original works (Sui et al., 2011, 2013).
Results
The mCCA+jICA framework identified two sMRI-fMRI joint sources with (1) significant correlations between their constituent structural and functional components; and (2) significant group differences in each of these constituent structural and functional components. Figures 3, 4 show the spatial maps for the constituent structural component, the connectivity strengths for the co-varying functional component's inter-regional connections and other associated results for the first and the second joint source, respectively. It must be noted that the constituent structural components in the joint sources estimated here are patterns of brain structure (i.e., clusters of brain voxels) that exhibit co-variations across subjects. These could be interpreted analogous to sources as identified with source based morphometry (SBM) (Xu et al., 2009a; Caprihan et al., 2011; Turner et al., 2012; Castro et al., 2014; Gupta et al., 2015), an approach that can be essentially considered as a multivariate extension of a voxel based approach, for example, voxel based morphometry (VBM). In both figures, for display purposes, only the high (and low) activation regions for the structural component and only the edges or connections with high (and low) connectivity strengths for the functional component are shown. More specifically, the structural component maps are lower thresholded at 25% of the maximum absolute activation value, whereas for the functional component, the inter-regional connectivity strengths, after converting to z-scores are thresholded at |z| > 3. For the functional component, we will hereon refer to the (post-thresholding) retained inter-regional connections as “significant links.”
Figure 3. Joint Source 1. (A) Spatial maps of the most activated regions for the structural component in the first joint source; (B) A visualization of significant links (functional connections with highest connectivity strengths i.e., with z-scores of connectivity strengths: |z| > 3) and their connectivity strengths for the functional component in the first joint source; (C) Scatterplot of the functional data loadings with the structural data loadings revealed a significant correlation (r = −0.28, p = 1.08 × 10−6); and (D) The group mean for the loading parameters was significantly lower for participants with schizophrenia, thus suggesting significant reductions in gray matter volume for this structural component.
Figure 4. Joint Source 2. (A) Spatial maps of the most activated regions for the structural component in the second joint source; (B) A visualization of significant links (functional connections with highest connectivity strengths i.e., with z-scores of connectivity strengths: |z| > 3) and their connectivity strengths for the functional component in the second joint source; (C) Scatterplot of the functional data loadings with the structural data loadings revealed a significant correlation (r = −0.40, p = 3.91 × 10−13); and (D) The group mean for the loading parameters was significantly lower for participants with schizophrenia, thus suggesting significant reductions in gray matter volume for this structural component.
Joint Source 1
As illustrated in Figure 3A, the structural component for the first joint source consists of peak activations in the superior parietal lobule (major constituent), precuneus, postcentral gyrus, and inferior parietal lobule. The number of significant connections in the linked functional component were high for the default mode, cognitive control and visual network domains in state 2, whereas the other states had a lot fewer total number of significant connections (five in state 4, one each in states 1 and 5, and none in state 3) as seen in Figure 3B. For this joint source, these constituent co-varying structural and functional components were found to be significantly correlated (r = −0.28, p = 1.08 × 10−6) as also evident from the scatterplot of their loading parameters in Figure 3C. Since negative correlation was observed, participants showing lower GM loadings generally exhibited higher connectivity strength in the functional connections. Finally, the structural component showed significant group difference (p = 0.0032) with a significantly lower group mean of the loadings for patients with schizophrenia (Figure 3D), whereas its linked i.e., co-varying functional correlate also showed significant group difference (p = 0.0072) with a significantly lower group mean for controls.
Joint Source 2
The structural component for the second joint source depicted in Figure 4A consisted of two major positively activated regions. The first major activation comprised regions from the medial frontal gyrus and superior frontal gyrus, whereas the second major activation comprised regions from the superior temporal gyrus, inferior temporal gyrus, insula, fusiform gyrus, and middle temporal gyrus. The number of significant connections in the linked functional component were high particularly for the default mode, cognitive control and visual network domains in state 1 (an observation similar to state 2 of the functional component corresponding to the first joint source) and moderate for the cognitive control, sensorimotor, and visual domains in state 2, whereas the other states had a lot fewer total number of significant connections (one each in states 4 and 5, and none in state 3) as seen in Figure 3B. For this joint source, these constituent co-varying structural and functional components were found to be significantly correlated (r = −0.40, p = 3.91 × 10−13). The corresponding scatterplot of their loading parameters can be seen in Figure 4C. Similar to the first joint source, since negative correlation was observed, participants showing lower GM loadings had higher connectivity strength in the functional connections. Finally, the structural component showed significant group difference (p = 0.0022) with a significantly reduced group mean for the schizophrenia patients, whereas its linked i.e., co-varying functional correlative also showed significant group difference (p = 0.0438) reflecting a significantly lower group mean for controls.
Discussion
In this study, we investigated whether a relationship between GM and time-varying FC measures exists and if that relationship could be used to study characteristic brain aberrations in schizophrenia. Using a novel, unified framework, we first estimated distilled, i.e., (relatively) lower-dimensional feature spaces from the high-dimensional fMRI and sMRI data. Next, we performed joint analysis on the estimated feature spaces leveraging a symmetric fusion approach, mCCA+jICA, to extract jointly co-varying structural and functional components and characterize interactions between these components. In this specific section, we will discuss our results specifically addressing few important questions, such as how the co-variation in the inter-modality components could be interpreted and how the underlying associations are meaningful. We will conclude by highlighting some critical facets and limitations that could be explored in immediate future work.
Specifically, our results revealed two mCCA+jICA joint sources that featured significant correlation between their constituent modality-specific components and highlighted group differences in both of their modality-specific components. Both the joint sources showed significant negative correlations between their modality-specific constituent components (joint source 1: r = −0.28, p = 1.08 × 10−6; joint source 2: r = −0.40, p = 3.91 × 10−13), as seen in Figures 3C, 4C. This implies that from joint source 1, for a given subject, if the GM volumes in the positively activated regions in the structural component (superior parietal lobule, precuneus, postcentral gyrus and inferior parietal lobule) are estimated to be higher, it will exhibit significantly decreased connectivity strength in the inter-regional links in the functional component (i.e., the absolute magnitude of inter-regional links with positive connectivity strengths in the functional component will show significant decrease, and the absolute magnitude of inter-regional links with negative connectivity strengths in the functional component will show significant increase). Alternatively, decreased observed GM volumes in the positively activated regions would imply higher connectivity strengths in the significant functional links for that given subject. Similar inferences can be deduced for source 2 wherein changes in GM volumes in both of the positively activated distinct regions in the structural component (i.e., medial frontal gyrus and superior frontal gyrus; and superior temporal gyrus, inferior temporal gyrus, insula, fusiform gyrus, and middle temporal gyrus) would drive the estimated significant inter-regional links accordingly.
An introspection of the modality-specific components of the joint sources revealed several lines of evidence of conformance with previously reported findings in the literature as discussed next. To begin with, the structural components in both joint sources showed significant group differences in the loading parameters (joint source 1: p = 0.0032; joint source 2: p = 0.0022), with significantly lower group mean for the schizophrenia group. This suggests a significant decrease in GM volume in the brain regions depicted by these components in participants with schizophrenia. Our results are consistent with several previous studies (as discussed in detail next), where reduced GM volume in schizophrenia has been reported in the similar brain regions as identified in our structural components. The first joint source highlighted peak activations in the superior parietal lobule (major constituent), precuneus, postcentral gyrus, and inferior parietal lobule brain regions as the structural modality component (as illustrated in Figure 3A). Interestingly, a recent study on SBM and VBM evaluating GM abnormalities in schizophrenia patients also found a similar structural component showing positive activation patterns and that captured group differences between schizophrenia and healthy controls (Gupta et al., 2015). Besides, previous studies have also concluded reduced GM volume in superior parietal regions (Buchanan et al., 2004), precuneus (Hulshoff Pol et al., 2001) and postcentral gyrus (Glahn et al., 2008); thus, our work adds further evidence that abnormal patterns of GM volume in these regions play an important role as schizophrenia biomarkers. Similar evidence could be established for the significant structural component in the second joint source captured by our framework (as illustrated in Figure 4A). This structural component included two major positively activated brain regions, where one of them consisted of regions from the medial frontal gyrus, anterior cingulate and superior frontal gyrus, and the other included regions from the superior temporal gyrus, inferior temporal gyrus, insula, fusiform gyrus and middle temporal gyrus. Particularly, the anterior cingulate has been recognized as a vital structure for social cognitive processing and has been previously identified as one of the major sources of social dysfunction in schizophrenia patients (Fujiwara et al., 2007). Additionally, very-similar fronto-temporal GM changes capturing group difference between schizophrenia and healthy participants were also found in Gupta et al. (2015). In fact, there are several other studies/reviews on GM differences in schizophrenia patients that have suggested significant reduction in GM volume in the temporal and frontal cortices (Shenton et al., 2001; Thompson et al., 2001; Giuliani et al., 2005).
Significant correspondence with previously reported studies in literature could also be drawn for few evaluated significant inter-regional (i.e., inter-RSN) links in the estimated functional components. Firstly, the functional components for both of the retained joint sources showed significant connectivity links in time-varying connectivity states 1, 2, 4, and 5. For the first functional component (joint source 1), most of the significant inter-RSN links are captured in state 2, where both positive and negative connectivity strengths across various network domains can be observed (Figure 3B). In this state, RSNs in the DMN domain showed significant connectivity within themselves and with RSNs from CC, SM, and VIS domains as well. Interestingly, one of the DMN RSNs, IC95, highlighted by the brain regions in left angular gyrus, showed positive connectivity weight with a RSN from the CC domain, IC35, left precuneus. This is in line with a previous study that has shown aberrant connectivity patterns between angular gyrus and precuneus in schizophrenia patients (Rashid et al., 2014). Indeed, studies have widely reported the involvement of angular gyrus in language processing, memory and social cognition (Hall et al., 2005; Binder et al., 2009; Price, 2010; Clos et al., 2014), and abnormal connectivity patterns in schizophrenia in the precuneus, which is involved in episodic memory (Rugg and Henson, 2002), mental imagery recall (Fletcher et al., 1996), and self-processing operations (Cavanna and Trimble, 2006). Furthermore, several studies have shown strong evidence of disrupted DMN connectivity in schizophrenia patients (Garrity et al., 2007; Ongur et al., 2010), and so it would be interesting to explore significant links involving the DMN RSNs. As an example, in state 2 of this functional component (joint source 1), we observed negative connectivity strength between another DMN component (IC61: left middle temporal gyrus) and a VIS RSN (IC43: right calcarine gyrus), while the same DMN RSN (IC61) showed negative connectivity strength with a SM RSN (IC5; bi-lateral precentral gyrus) as well. For this functional component (joint source 1), we also note that the other states in this functional component (i.e., states 1, 4, and 5) showed significant inter-RSN links between VIS and CC domains (state 1), between DMN and CC, between AUD and CB domains, within DMN, CC, and VIS domains (state 4), and between DMN and VIS domains (state 5). Furthermore, an examination of the functional component from joint source 2 revealed some interesting significant links in state 1, the most densely connected state (Figure 4B). In this state, a DMN RSN (IC95; left angular gyrus) showed significant positive connectivity strength with a CC RSN (IC35; left precuneus), an observation also found in state 2 emergent in the first joint source. Again, similar to state 2 from the first joint source, another DMN RSN (IC61; left middle temporal gyrus) showed significantly positive connectivity strength with the same CC RSN (IC35; left precuneus). In fact, dysfunctional temporal lobe connectivity has been reported in several schizophrenia connectivity studies (Shenton et al., 1992; Ford et al., 2002), suggesting that networks from the temporal regions play a significant role in schizophrenia etiology. Finally, for this functional component (joint source 2) the other states (i.e., states 2, 4 and 5) showed significant inter-RSN links from the DMN, CC and SM and CB domains.
While we closely evaluate a few interesting connections in scope of this work, there is much more that could be done to evaluate these results to further enhance our understanding of the structure-function relationships and further contribute to characterizing schizophrenia. In the specific context of findings from our mCCA+jICA based framework as studied in this paper, it would be most appropriate to first extensively validate the significant findings in a future analysis evaluating multiple multimodal datasets featuring schizophrenia participants. We also note that while different combinations of cost functions and model orders can yield similar results, they can also introduce decompositions different to a degree; hence, comparing performance of the mCCA+jICA approach for a range of these parameters would be another interesting future work. Further investigations could also benefit from evaluating associations of sleep indices and schizophrenia risk factors with the structural and functional component patterns. We anticipate that similar methods could be easily extended to the study of other brain conditions; likewise, different feature spaces, combinations of neuroimaging modalities and algorithms could be evaluated in the proposed fashion.
Conclusion
Multimodal data fusion through symmetric approaches provides an opportunity to understand brain complexities. Using a multivariate symmetric fusion approach, we were able to identify co-varying GM and time-varying FC components that revealed disrupted links in schizophrenia. We suggest that studying such interactions can provide a useful way of evaluating structure-function relationships and characterizing schizophrenia or other brain conditions.
Ethics Statement
This study used data collected in accordance with the local institutional review boards at the University of California Irvine (UCI), Los Angeles (UCLA); University of New Mexico (NM), University of Iowa (IA), University of Minnesota (MN), Duke University/University of North Carolina, University of California San Diego (UCSD; healthy subjects only), and the University of California San Francisco (UCSF). All subjects gave written informed consent in accordance with the Declaration of Helsinki before participation. The study protocol was approved by the institutional review board at the University of New Mexico. All studies were collected in accordance with the local (site-specific) institutional review board guidelines with written informed consent from all subjects.
Author Contributions
AA and VC designed the research. AA, SR, and ED performed the research. AA, BR, and VC analyzed the results. AA, BR, and VC wrote the paper.
Funding
This work was supported by National Institute of Health grants P20GM103472/5P20RR021938, R01EB005846, 1R01EB006841, 1R01DA040487, R01EB020407, and the National Science Foundation (NSF) grant #1539067.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Abou-Elseoud, A., Starck, T., Remes, J., Nikkinen, J., Tervonen, O., and Kiviniemi, V. (2010). The effect of model order selection in group PICA. Hum. Brain Mapp. 31, 1207–1216. doi: 10.1002/hbm.20929
Abrol, A., Chaze, C., Damaraju, E., and Calhoun, V. D. (2016). The chronnectome: evaluating replicability of dynamic connectivity patterns in 7500 resting fMRI datasets. Conf. Proc. IEEE. Eng. Med. Biol. Soc. 2016, 5571–5574. doi: 10.1109/EMBC.2016.7591989
Abrol, A., Damaraju, E., Miller, R. L., Stephen, J. M., Claus, E. D., Mayer, A. R., et al. (2017). Replicability of time-varying connectivity patterns in large resting state fMRI samples. Neuroimage 163, 160–176 doi: 10.1016/j.neuroimage.2017.09.020
Alexander-Bloch, A., Giedd, J. N., and Bullmore, E. (2013). Imaging structural co-variance between human brain regions. Nat. Rev. Neurosci. 14, 322–336. doi: 10.1038/nrn3465
Allen, E. A., Damaraju, E., Eichele, T., Wu, L., and Calhoun, V. D. (2017). EEG Signatures of dynamic functional network connectivity states. Brain Topogr. doi: 10.1007/s10548-017-0546-2. [Epub ahead of print].
Allen, E. A., Damaraju, E., Plis, S. M., Erhardt, E. B., Eichele, T., and Calhoun, V. D. (2012). Tracking whole-brain connectivity dynamics in the resting state. Cereb. Cortex 24, 663–676. doi: 10.1093/cercor/bhs352
Amico, E., Gomez, F., Di Perri, C., Vanhaudenhuyse, A., Lesenfants, D., Boveroux, P., Laureys, S., et al. (2014). Posterior cingulate cortex-related co-activation patterns: a resting state fMRI study in propofol-induced loss of consciousness. PLoS ONE 9:e100012. doi: 10.1371/journal.pone.0100012
Barttfeld, P., Uhrig, L., Sitt, J. D., Sigman, M., Jarraya, B., and Dehaene, S. (2015). Signature of consciousness in the dynamics of resting-state brain activity. Proc. Nat. Acad. Sci. U.S.A. 112, 887–892. doi: 10.1073/pnas.1418031112
Bassett, D. S., Bullmore, E., Verchinski, B. A., Mattay, V. S., Weinberger, D. R., and Meyer-Lindenberg, A. (2008). Hierarchical organization of human cortical networks in health and schizophrenia. J. Neurosci. 28, 9239–9248. doi: 10.1523/JNEUROSCI.1929-08.2008
Biessmann, F., Plis, S., Meinecke, F. C., Eichele, T., and Muller, K.-R. (2011). Analysis of multimodal neuroimaging data. IEEE Rev. Biomed. Eng. 4, 26–58. doi: 10.1109/RBME.2011.2170675
Binder, J. R., Desai, R. H., Graves, W. W., and Conant, L. L. (2009). Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb. Cortex 19, 2767–2796. doi: 10.1093/cercor/bhp055
Bockholt, H., Scully, M., Courtney, W., Rachakonda, S., Scott, A., Caprihan, A., et al. (2010). Mining the mind research network: a novel framework for exploring large scale, heterogeneous translational neuroscience research data sources. Front. Neuroinform. 3:36. doi: 10.3389/neuro.11.036.2009
Buchanan, R. W., Francis, A., Arango, C., Miller, K., Lefkowitz, D. M., and Pearlson, G. D. (2004). Morphometric assessment of the heteromodal association cortex in schizophrenia. Am. J. Psychiatry 161, 322–331. doi: 10.1176/appi.ajp.161.2.322
Cabral, C., Kambeitz-Ilankovic, L., Kambeitz, J., Calhoun, V. D., Dwyer, D. B., and Koutsouleris, N. (2016). Classifying schizophrenia using multimodal multivariate pattern recognition analysis: evaluating the impact of individual clinical profiles on the neurodiagnostic performance. Schizophr. Bull. 42(Suppl. 1), S110–S117. doi: 10.1093/schbul/sbw053
Calhoun, V. D., and Adali, T. (2009). Feature-based fusion of medical imaging data. IEEE Trans. Inf. Technol. Biomed. 13, 711–720. doi: 10.1109/TITB.2008.923773
Calhoun, V. D., and Adali, T. (2012). Multi-subject independent component analysis of fMRI: a decade of intrinsic networks, default mode, and neurodiagnostic discovery. IEEE Rev. Biomed. Eng. 5, 60–73. doi: 10.1109/RBME.2012.2211076
Calhoun, V. D., Adali, T., Giuliani, N. R., Pekar, J. J., Kiehl, K. A., and Pearlson, G. D. (2006). Method for multimodal analysis of independent source differences in schizophrenia: combining gray matter structural and auditory oddball functional data. Hum. Brain Mapp. 27, 47–62. doi: 10.1002/hbm.20166
Calhoun, V. D., Adali, T., Pearlson, G. D., and Pekar, J. J. (2001). A method for making group inferences from functional MRI data using independent component analysis. Hum. Brain Mapp. 14, 140–151. doi: 10.1002/hbm.1048
Calhoun, V. D., Miller, R., Pearlson, G., and Adali, T. (2014). The chronnectome: time-varying connectivity networks as the next frontier in fMRI data discovery. Neuron 84, 262–274. doi: 10.1016/j.neuron.2014.10.015
Calhoun, V. D., and Sui, J. (2016). Multimodal fusion of brain imaging data: a key to finding the missing link(s) in complex mental illness. Biol. Psychiatry Cogn. Neurosci. Neuroimag 1, 230–244. doi: 10.1016/j.bpsc.2015.12.005
Camara, E., Rodriguez-Fornells, A., and Münte, T. F. (2010). Microstructural brain differences predict functional hemodynamic responses in a reward processing task. J. Neurosci. 30, 11398–11402. doi: 10.1523/JNEUROSCI.0111-10.2010
Camchong, J., MacDonald, A. W. III., Bell, C., Mueller, B. A., and Lim, K. O. (2011). Altered functional and anatomical connectivity in schizophrenia. Schizophr. Bull. 37, 640–650. doi: 10.1093/schbul/sbp131
Caprihan, A., Abbott, C., Yamamoto, J., Pearlson, G., Perrone-Bizzozero, N., Sui, J., et al. (2011). Source-Based morphometry analysis of group differences in fractional anisotropy in schizophrenia. Brain Connect. 1, 133–145. doi: 10.1089/brain.2011.0015
Castro, E., Gupta, C. N., Martínez-Ramón, M., Calhoun, V. D., Arbabshirani, M. R., and Turner, J. (2014). “Identification of Patterns of Gray Matter Abnormalities in Schizophrenia using Source-Based Morphometry and Bagging,” Conference proceedings : Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, Vol. 2014 (Chicago, IL), 1513–1516. Available online at: http://ieeexplore.ieee.org/document/6943889/
Cavanna, A. E., and Trimble, M. R. (2006). The precuneus: a review of its functional anatomy and behavioural correlates. Brain 129(Pt 3), 564–583. doi: 10.1093/brain/awl004
Cetin, M. S., Houck, J. M., Rashid, B., Agacoglu, O., Stephen, J. M., Calhoun, V. D., et al. (2016). Multimodal classification of schizophrenia patients with MEG and fMRI data using static and dynamic connectivity measures. Front. Neurosci. 10:466. doi: 10.3389/fnins.2016.00466
Chang, C., Liu, Z., Chen, M. C., Liu, X., and Duyn, J. H. (2013). EEG correlates of time-varying BOLD functional connectivity. Neuroimage 72, 227–236. doi: 10.1016/j.neuroimage.2013.01.049
Chen, K., Reiman, E. M., Huan, Z., Caselli, R. J., Bandy, D., Alexander, G. E., et al. (2009). Linking functional and structural brain images with multivariate network analyses: a novel application of the partial least square method. Neuroimage 47, 602–610. doi: 10.1016/j.neuroimage.2009.04.053
Clos, M., Langner, R., Meyer, M., Oechslin, M. S., Zilles, K., and Eickhoff, S. B. (2014). Effects of prior information on decoding degraded speech: an fMRI study. Hum. Brain Mapp. 35, 61–74. doi: 10.1002/hbm.22151
Correa, N., Adali, T., and Calhoun, V. D. (2007). Performance of blind source separation algorithms for fMRI analysis using a group ICA method. Magn. Reson. Imag. 25, 684–694. doi: 10.1016/j.mri.2006.10.017
Correa, N. M., Adali, T. Y., Li, O., and Calhoun, V. D. (2010). Canonical correlation analysis for data fusion and group inferences: examining applications of medical imaging data. IEEE Signal Process. Mag. 27, 39–50. doi: 10.1109/MSP.2010.936725
Craddock, R. C., James, G. A., Holtzheimer, P. E., Hu, X. P., and Mayberg, H. S. (2012). A whole brain fMRI atlas generated via spatially constrained spectral clustering. Hum. Brain Mapp. 33, 1914–1928. doi: 10.1002/hbm.21333
Damaraju, E., Allen, E. A., Belger, A., Ford, J. M., McEwen, S., Calhoun, V. D., et al. (2014). Dynamic functional connectivity analysis reveals transient states of dysconnectivity in schizophrenia. Neuroimage Clin. 5, 298–308. doi: 10.1016/j.nicl.2014.07.003
Deng, L., Sun, J., Cheng, L., and Tong, S. (2016). Characterizing dynamic local functional connectivity in the human brain. Sci. Rep. 6:26976. doi: 10.1038/srep26976
Du, Y., Pearlson, G. D., Yu, Q., He, H., Lin, D., Calhoun, V. D., et al. (2016). Interaction among subsystems within default mode network diminished in schizophrenia patients: a dynamic connectivity approach. Schizophr. Res. 170, 55–65. doi: 10.1016/j.schres.2015.11.021
Erhardt, E. B., Rachakonda, S., Bedrick, E. J., Allen, E. A., Adali, T., and Calhoun, V. D. (2011). Comparison of multi-subject ICA methods for analysis of fMRI data. Hum. Brain Mapp. 32, 2075–2095. doi: 10.1002/hbm.21170
Filippini, N., MacIntosh, B. J., Hough, M. G., Goodwin, G. M., Frisoni, G. B., Mackay, C. E., et al. (2009). Distinct patterns of brain activity in young carriers of the APOE-epsilon4 allele. Proc. Natl. Acad. Sci. U.S.A. 106, 7209–7214. doi: 10.1073/pnas.0811879106
Fletcher, P. C., Shallice, T., Frith, C. D., Frackowiak, R. S., and Dolan, R. J. (1996). Brain activity during memory retrieval. The influence of imagery and semantic cueing. Brain 119(Pt 5), 1587–1596. doi: 10.1093/brain/119.5.1587
Ford, J. M., Mathalon, D. H., Whitfield, S., Faustman, W. O., and Roth, W. T. (2002). Reduced communication between frontal and temporal lobes during talking in schizophrenia. Biol. Psychiatry 51, 485–492. doi: 10.1016/S0006-3223(01)01335-X
Friedman, J., Hastie, T., and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9, 432–441. doi: 10.1093/biostatistics/kxm045
Fu, Z., Tu, Y., Di, X., Du, Y., Pearlson, G. D., Calhoun, V. D., et al. (in press). Characterizing dynamic amplitude of low-frequency fluctuation its relationship with dynamic functional connectivity: an application to schizophrenia. Neuroimage. doi: 10.1016/j.neuroimage.2017.09.035
Fujiwara, H., Hirao, K., Namiki, C., Yamada, M., Shimizu, M., Fukuyama, H., et al. (2007). Anterior cingulate pathology and social cognition in schizophrenia: a study of gray matter, white matter and sulcal morphometry. Neuroimage 36, 1236–1245. doi: 10.1016/j.neuroimage.2007.03.068
Garrity, A. G., Pearlson, G. D., McKiernan, K., Lloyd, D., Kiehl, K. A., and Calhoun, V. D. (2007). Aberrant default mode functional connectivity in schizophrenia. Am. J. Psychiatry 164, 450–457. doi: 10.1176/ajp.2007.164.3.450
Giuliani, N. R., Calhoun, V. D., Pearlson, G. D., Francis, A., and Buchanan, R. W. (2005). Voxel-based morphometry versus region of interest: a comparison of two methods for analyzing gray matter differences in schizophrenia. Schizophr. Res. 74, 135–147. doi: 10.1016/j.schres.2004.08.019
Glahn, D. C., Laird, A. R., Ellison-Wright, I., Thelen, S. M., Robinson, J. L., Fox, P. T., et al. (2008). Meta-analysis of gray matter anomalies in schizophrenia: application of anatomic likelihood estimation and network analysis. Biol. Psychiatry 64, 774–781. doi: 10.1016/j.biopsych.2008.03.031
Gonzalez-Castillo, J., Hoy, C. W., Handwerker, D. A., Robinson, M. E., Buchanan, L. C., Bandettini, P. A., et al. (2015). Tracking ongoing cognition in individuals using brief, whole-brain functional connectivity patterns. Proc. Nat. Acad. Sci. U.S.A. 112, 8762–8767. doi: 10.1073/pnas.1501242112
Groves, A. R., Beckmann, C. F., Smith, S. M., and Woolrich, M. W. (2011). Linked independent component analysis for multimodal data fusion. Neuroimage 54, 2198–2217. doi: 10.1016/j.neuroimage.2010.09.073
Gupta, C. N., Calhoun, V. D., Rachakonda, S., Chen, J., Patel, V., Turner, J. A., et al. (2015). Patterns of gray matter abnormalities in schizophrenia based on an international mega-analysis. Schizophr. Bull. 41, 1133–1142. doi: 10.1093/schbul/sbu177
Hagmann, P., Cammoun, L., Gigandet, X., Meuli, R., Honey, C. J., Sporns, O., et al. (2008). Mapping the structural core of human cerebral cortex. PLoS Biol. 6:e159. doi: 10.1371/journal.pbio.0060159
Hall, D. A., Fussell, C., and Summerfield, A. Q. (2005). Reading fluent speech from talking faces: typical brain networks and individual differences. J. Cogn. Neurosci. 17, 939–953. doi: 10.1162/0898929054021175
Himberg, J., Hyvarinen, A., and Esposito, F. (2004). Validating the independent components of neuroimaging time series via clustering and visualization. Neuroimage 22, 1214–1222. doi: 10.1016/j.neuroimage.2004.03.027
Hindriks, R., Adhikari, M. H., Murayama, Y., Ganzetti, M., Mantini, D., Deco, G., et al. (2016). Can sliding-window correlations reveal dynamic functional connectivity in resting-state fMRI? Neuroimage 127, 242–256. doi: 10.1016/j.neuroimage.2015.11.055
Honey, C. J., Sporns, O., Cammoun, L., Gigandet, X., Thiran, J. P., Hagmann, P., et al. (2009). Predicting human resting-state functional connectivity from structural connectivity. Proc. Nat. Acad. Sci. U.S.A. 106, 2035–2040. doi: 10.1073/pnas.0811168106
Hudson, A. E., Calderon, D. P., Pfaff, D. W., and Proekt, A. (2014). Recovery of consciousness is mediated by a network of discrete metastable activity states. Proc. Nat. Acad. Sci. U.S.A. 111, 9283–9288. doi: 10.1073/pnas.1408296111
Hulshoff Pol, H. E., Schnack, H. G., Mandl, R. C., van Haren, N. E., Koning, H., Kahn, R. S., et al. (2001). Focal gray matter density changes in schizophrenia. Arch. Gen. Psychiatry 58, 1118–1125. doi: 10.1001/archpsyc.58.12.1118
Hutchison, R. M., Womelsdorf, T., Allen, E. A., Bandettini, P. A., Calhoun, V. D., Chang, C., et al. (2013a). Dynamic functional connectivity: promise, issues, and interpretations. Neuroimage 80, 360–378. doi: 10.1016/j.neuroimage.2013.05.079
Hutchison, R. M., Womelsdorf, T., Gati, J. S., Everling, S., and Menon, R. S. (2013b). Resting-state networks show dynamic functional connectivity in awake humans and anesthetized macaques. Hum. Brain Mapp. 34, 2154–2177. doi: 10.1002/hbm.22058
Keator, D. B., van Erp, T. G., Turner, J. A., Glover, G. H., Mueller, B. A., Potkin, S. G., et al. (2016). The function biomedical informatics research network data repository. Neuroimage 124(Pt B), 1074–1079. doi: 10.1016/j.neuroimage.2015.09.003
Kettenring, J. R. (1971). Canonical analysis of several sets of variables. Biometrika 58, 433–451. doi: 10.1093/biomet/58.3.433
Kiviniemi, V., Starck, T., Remes, J., Long, X., Nikkinen, J., Haapea, M., Tervonen, O., et al. (2009). Functional segmentation of the brain cortex using high model order group PICA. Hum. Brain Mapp. 30, 3865–3886. doi: 10.1002/hbm.20813
Kucyi, A. (in press). Just a thought: how mind-wandering is represented in dynamic brain connectivity. Neuroimage. doi: 10.1016/j.neuroimage.2017.07.001
Kucyi, A., and Davis, K. D. (2014). Dynamic functional connectivity of the default mode network tracks daydreaming. Neuroimage 100, 471–480. doi: 10.1016/j.neuroimage.2014.06.044
Lahat, D., Adali, T., and Jutten, C. (2015). Multimodal data fusion: an overview of methods, challenges, and prospects. Proc. IEEE 103, 1449–1477. doi: 10.1109/JPROC.2015.2460697
Leonardi, N., and Van De Ville, D. (2015). On spurious and real fluctuations of dynamic functional connectivity during rest. Neuroimage 104, 430–436. doi: 10.1016/j.neuroimage.2014.09.007
Li, J., Zhang, D., Liang, A., Liang, B., Wang, Z., Cai, Y., et al. (2017). High transition frequencies of dynamic functional connectivity states in the creative Brain 7:46072. doi: 10.1038/srep46072
Li, X., Zhu, D., Jiang, X., Jin, C., Zhang, X., Guo, L., et al. (2014). Dynamic functional connectomics signatures for characterization and differentiation of PTSD patients. Hum. Brain Mapp. 35, 1761–1778. doi: 10.1002/hbm.22290
Liégeois, R., Ziegler, E., Phillips, C., Geurts, P., Gómez, F., Bahri, M., et al. (2016). Cerebral functional connectivity periodically (de)synchronizes with anatomical constraints. Brain Struc. Funct. 221, 2985–2997. doi: 10.1007/s00429-015-1083-y
Liu, J., Pearlson, G., Windemuth, A., Ruano, G., Perrone-Bizzozero, N. I., and Calhoun, V. (2009). Combining fMRI and SNP data to investigate connections between brain function and genetics using parallel ICA. Hum. Brain Mapp. 30, 241–255. doi: 10.1002/hbm.20508
Madhyastha, T. M., Askren, M. K., Boord, P., and Grabowski, T. J. (2015). Dynamic connectivity at rest predicts attention task performance. Brain Connect. 5, 45–59. doi: 10.1089/brain.2014.0248
Martinez-Montes, E., Valdes-Sosa, P. A., Miwakeichi, F., Goldman, R. I., and Cohen, M. S. (2004). Concurrent EEG/fMRI analysis by multiway Partial Least Squares. Neuroimage 22, 1023–1034. doi: 10.1016/j.neuroimage.2004.03.038
Michael, A. M., Baum, S. A., White, T., Demirci, O., Andreasen, N. C., Calhoun, V. D., et al. (2010). Does function follow form?: methods to fuse structural and functional brain images show decreased linkage in schizophrenia. Neuroimage 49, 2626–2637. doi: 10.1016/j.neuroimage.2009.08.056
Miller, R. L., Yaesoubi, M., Turner, J. A., Mathalon, D., Preda, A., Calhoun, V. D., et al. (2016). Higher dimensional meta-state analysis reveals reduced resting fmri connectivity dynamism in schizophrenia patients. PLoS ONE 11:e0149849. doi: 10.1371/journal.pone.0149849
Nieuwenhuis, M., van Haren, N. E., Hulshoff Pol, H. E., Cahn, W., Kahn, R. S., and Schnack, H. G. (2012). Classification of schizophrenia patients and healthy controls from structural MRI scans in two large independent samples. Neuroimage 61, 606–612. doi: 10.1016/j.neuroimage.2012.03.079
Olesen, P. J., Nagy, Z., Westerberg, H., and Klingberg, T. (2003). Combined analysis of DTI and fMRI data reveals a joint maturation of white and grey matter in a fronto-parietal network. Brain Res. Cogn. Brain Res. 18, 48–57. doi: 10.1016/j.cogbrainres.2003.09.003
Ongur, D., Lundy, M., Greenhouse, I., Shinn, A. K., Menon, V., Renshaw, P. F., et al. (2010). Default mode network abnormalities in bipolar disorder and schizophrenia. Psychiatry Res. 183, 59–68. doi: 10.1016/j.pscychresns.2010.04.008
Peruzzo, D., Castellani, U., Perlini, C., Bellani, M., Marinelli, V., Rambaldelli, G., et al. (2015). Classification of first-episode psychosis: a multi-modal multi-feature approach integrating structural and diffusion imaging. J. Neural Transm. 122, 897–905. doi: 10.1007/s00702-014-1324-x
Preti, M. G., Bolton, T., and Van De Ville, D. (2016). The dynamic functional connectome: state-of-the-art and perspectives. Neuroimage S1053–S8119, 30788-1. doi: 10.1016/j.neuroimage.2016.12.061
Price, C. J. (2010). The anatomy of language: a review of 100 fMRI studies published in 2009. Ann. N. Y. Acad. Sci. 1191, 62–88. doi: 10.1111/j.1749-6632.2010.05444.x
Rashid, B., Damaraju, E., Pearlson, G. D., and Calhoun, V. D. (2014). Dynamic connectivity states estimated from resting fMRI Identify differences among Schizophrenia, bipolar disorder, and healthy control subjects. Front. Hum. Neurosci. 8:897. doi: 10.3389/fnhum.2014.00897
Rugg, M., and Henson, R. (2002). Episodic memory retrieval: an (event-related) functional neuroimaging perspective. The Cognitive Neuroscience of Memory Encoding and Retrieval. New York, NY: Psychology Press. Available online at: http://trove.nla.gov.au/work/32304905?selectedversion=NBD23745272
Rykhlevskaia, E., Gratton, G., and Fabiani, M. (2008). Combining structural and functional neuroimaging data for studying brain connectivity: a review. Psychophysiology 45, 173–187. doi: 10.1111/j.1469-8986.2007.00621.x
Sakoglu, U., Pearlson, G. D., Kiehl, K. A., Wang, Y. M., Michael, A. M., and Calhoun, V. D. (2010). A method for evaluating dynamic functional network connectivity and task-modulation: application to schizophrenia. MAGMA 23, 351–366. doi: 10.1007/s10334-010-0197-8
Schaefer, A., Margulies, D. S., Lohmann, G., Gorgolewski, K. J., Smallwood, J., Villringer, A., et al. (2014). Dynamic network participation of functional connectivity hubs assessed by resting-state fMRI. Front. Hum. Neurosci. 8:195. doi: 10.3389/fnhum.2014.00195
Schultz, C. C., Fusar-Poli, P., Wagner, G., Koch, K., Schachtzabel, C., Gruber, O., et al. (2012). Multimodal functional and structural imaging investigations in psychosis research. Eur. Arch. Psychiatry Clin. Neurosci. 262, 97–106. doi: 10.1007/s00406-012-0360-5
Segall, J. M., Allen, E. A., Jung, R. E., Erhardt, E. B., Arja, S. K., Calhoun, V. D., et al. (2012). Correspondence between structure and function in the human brain at rest. Front. Neuroinform. 6:10. doi: 10.3389/fninf.2012.00010
Shenton, M. E., Dickey, C. C., Frumin, M., and McCarley, R. W. (2001). A review of MRI findings in schizophrenia. Schizophr. Res. 49, 1–52. doi: 10.1016/S0920-9964(01)00163-3
Shenton, M. E., Kikinis, R., Jolesz, F. A., Pollak, S. D., LeMay, M., Wible, C. G., et al. (1992). Abnormalities of the left temporal lobe and thought disorder in schizophrenia. A quantitative magnetic resonance imaging study. N. Engl. J. Med. 327, 604–612. doi: 10.1056/NEJM199208273270905
Shine, J. M., Bissett, P. G., Bell, P. T., Koyejo, O., Balsters, J. H., Gorgolewski, K. J., et al. (2016a). The dynamics of functional brain networks: integrated network states during cognitive task performance. Neuron 92, 544–554. doi: 10.1016/j.neuron.2016.09.018
Shine, J. M., Koyejo, O., and Poldrack, R. A. (2016b). Temporal metastates are associated with differential patterns of time-resolved connectivity, network topology, and attention. Proc. Nat. Acad. Sci. U.S.A. 113, 9888–9891. doi: 10.1073/pnas.1604898113
Silva, R. F., Plis, S. M., Sui, J., Pattichis, M. S., Adali, T., and Calhoun, V. D. (2016). Blind Source separation for unimodal and multimodal brain networks: a unifying framework for subspace modeling. IEEE J. Sel. Top. Sig. Process. 10, 1134–1149. doi: 10.1109/JSTSP.2016.2594945
Skudlarski, P., Jagannathan, K., Anderson, K., Stevens, M. C., Calhoun, V. D., Pearlson, G., et al. (2010). Brain connectivity is not only lower but different in schizophrenia: a combined anatomical and functional approach. Biol. Psychiatry 68, 61–69. doi: 10.1016/j.biopsych.2010.03.035
Sockeel, S., Schwartz, D., Pélégrini-Issac, M., and Benali, H. (2016). Large-scale functional networks identified from resting-state EEG using spatial ICA. PLoS ONE 11:e0146845. doi: 10.1371/journal.pone.0146845
Stephen, J. M., Coffman, B. A., Jung, R. E., Bustillo, J. R., Aine, C. J., and Calhoun, V. D. (2013). Using joint ICA to link function and structure using MEG and DTI in schizophrenia. NeuroImage 83(Suppl. C), 418–430. doi: 10.1016/j.neuroimage.2013.06.038
Sugranyes, G., Kyriakopoulos, M., Dima, D., O'Muircheartaigh, J., Corrigall, R., Pendelbury, G., Frangou, S., et al. (2012). Multimodal analyses identify linked functional and white matter abnormalities within the working memory network in schizophrenia. Schizophr. Res. 138, 136–142. doi: 10.1016/j.schres.2012.03.011
Sui, J., Adali, T., Pearlson, G. D., and Calhoun, V. D. (2009). An ICA-based method for the identification of optimal FMRI features and components using combined group-discriminative techniques. Neuroimage 46, 73–86. doi: 10.1016/j.neuroimage.2009.01.026
Sui, J., Adali, T., Yu, Q., Chen, J., and Calhoun, V. D. (2012a). A review of multivariate methods for multimodal fusion of brain imaging data. J. Neurosci. Methods 204, 68–81. doi: 10.1016/j.jneumeth.2011.10.031
Sui, J., He, H., Pearlson, G. D., Adali, T., Kiehl, K. A., Calhoun, V. D., et al. (2013). Three-way (N-way) fusion of brain imaging data based on mCCA+jICA and its application to discriminating schizophrenia. Neuroimage 66, 119–132. doi: 10.1016/j.neuroimage.2012.10.051
Sui, J., Pearlson, G., Caprihan, A., Adali, T., Kiehl, K. A., Calhoun, V. D., et al. (2011). Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model. Neuroimage 57, 839–855. doi: 10.1016/j.neuroimage.2011.05.055
Sui, J., Pearlson, G. D., Du, Y., Yu, Q., Jones, T. R., Calhoun, V. D., et al. (2015). In search of multimodal neuroimaging biomarkers of cognitive deficits in schizophrenia. Biol. Psychiatry 78, 794–804. doi: 10.1016/j.biopsych.2015.02.017
Sui, J., Yu, Q., He, H., Pearlson, G. D., and Calhoun, V. D. (2012b). A selective review of multimodal fusion methods in schizophrenia. Front. Hum. Neurosci. 6:27. doi: 10.3389/fnhum.2012.00027
Tagliazucchi, E., and Laufs, H. (2014). Decoding wakefulness levels from typical fMRI resting-state data reveals reliable drifts between wakefulness and sleep. Neuron 82, 695–708. doi: 10.1016/j.neuron.2014.03.020
Tagliazucchi, E., von Wegner, F., Morzelewski, A., Brodbeck, V., and Laufs, H. (2012). Dynamic BOLD functional connectivity in humans and its electrophysiological correlates. Front. Hum. Neurosci. 6:339. doi: 10.3389/fnhum.2012.00339
Thompson, P. M., Vidal, C., Giedd, J. N., Gochman, P., Blumenthal, J., Rapoport, J. L., et al. (2001). Mapping adolescent brain change reveals dynamic wave of accelerated gray matter loss in very early-onset schizophrenia. Proc. Natl. Acad. Sci. U.S.A. 98, 11650–11655. doi: 10.1073/pnas.201243998
Turner, J. A., Calhoun, V. D., Michael, A., van Erp, T. G., Ehrlich, S., et al. (2012). Heritability of multivariate gray matter measures in schizophrenia. Twin Res. Hum. Genet. 15, 324–335. doi: 10.1017/thg.2012.1
Ulaş, A., Castellani, U., Mirtuono, P., Bicego, M., Murino, V., Cerruti, S., et al. (2011). “Multimodal Schizophrenia Detection by Multiclassification Analysis,” in Proceedings of the Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 16th Iberoamerican Congress, CIARP 2011, Pucón, eds C. San Martin and S.-W. Kim (Berlin; Heidelberg: Springer), 491–498.
Uludag, K., and Roebroeck, A. (2014). General overview on the merits of multimodal neuroimaging data fusion. Neuroimage 102(Pt 1), 3–10. doi: 10.1016/j.neuroimage.2014.05.018
van den Heuvel, M. P., Mandl, R. C., Kahn, R. S., and Hulshoff Pol, H. E. (2009). Functionally linked resting-state networks reflect the underlying structural connectivity architecture of the human brain. Hum. Brain Mapp. 30, 3127–3141. doi: 10.1002/hbm.20737
Wang, C., Ong, J. L., Patanaik, A., Zhou, J., and Chee, M. W. L. (2016). Spontaneous eyelid closures link vigilance fluctuation with fMRI dynamic connectivity states. Proc Nat. Acad. Sci. U.S.A. 113, 9653–9658. doi: 10.1073/pnas.1523980113
Xu, L., Groth, K. M., Pearlson, G., Schretlen, D. J., and Calhoun, V. D. (2009a). Source-based morphometry: the use of independent component analysis to identify gray matter differences with application to schizophrenia. Hum. Brain Mapp. 30, 711–724. doi: 10.1002/hbm.20540
Xu, L., Pearlson, G., and Calhoun, V. D. (2009b). Joint source based morphometry identifies linked gray and white matter group differences. Neuroimage 44, 777–789. doi: 10.1016/j.neuroimage.2008.09.051
Yaesoubi, M., Miller, R. L., and Calhoun, V. D. (2015). Mutually temporally independent connectivity patterns: a new framework to study the dynamics of brain connectivity at rest with application to explain group difference based on gender. Neuroimage 107, 85–94. doi: 10.1016/j.neuroimage.2014.11.054
Yang, H., Liu, J., Sui, J., Pearlson, G., and Calhoun, V. D. (2010). A hybrid machine learning method for fusing fmri and genetic data: combining both improves classification of schizophrenia. Front. Hum. Neurosci. 4:192. doi: 10.3389/fnhum.2010.00192
Yu, Q., Erhardt, E. B., Sui, J., Du, Y., He, H., Calhoun, V. D., et al. (2015). Assessing dynamic brain graphs of time-varying connectivity in fMRI data: application to healthy controls and patients with schizophrenia. Neuroimage 107, 345–355. doi: 10.1016/j.neuroimage.2014.12.020
Yu, Q., Sui, J., Rachakonda, S., He, H., Pearlson, G., and Calhoun, V. D. (2011). Altered small-world brain networks in temporal lobe in patients with schizophrenia performing an auditory oddball task. Front. Syst. Neurosci. 5:7. doi: 10.3389/fnsys.2011.00007
Keywords: multimodal fusion, structure-function relationship, schizophrenia, gray matter, dynamic functional connectivity, mCCA, joint ICA
Citation: Abrol A, Rashid B, Rachakonda S, Damaraju E and Calhoun VD (2017) Schizophrenia Shows Disrupted Links between Brain Volume and Dynamic Functional Connectivity. Front. Neurosci. 11:624. doi: 10.3389/fnins.2017.00624
Received: 24 August 2017; Accepted: 26 October 2017;
Published: 07 November 2017.
Edited by:
Sung Chan Jun, Gwangju Institute of Science and Technology, South KoreaReviewed by:
Mingrui Xia, Beijing Normal University, ChinaFeng Liu, Tianjin Medical University General Hospital, China
Copyright © 2017 Abrol, Rashid, Rachakonda, Damaraju and Calhoun. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Anees Abrol, aabrol@mrn.org