Feasibility and Diagnostic Accuracy of Ischemic Stroke Territory Recognition Based on Two-Dimensional Projections of Three-Dimensional Diffusion MRI Data

This study was conducted to assess the feasibility and diagnostic accuracy of brain artery territory recognition based on geoprojected two-dimensional maps of diffusion MRI data in stroke patients. In this retrospective study, multiplanar diffusion MRI data of ischemic stroke patients was used to create a two-dimensional map of the entire brain. To guarantee correct representation of the stroke, a computer-aided brain artery territory diagnosis was developed and tested for its diagnostic accuracy. The test recognized the stroke-affected brain artery territory based on the position of the stroke in the map. The performance of the test was evaluated by comparing it to the reference standard of each patient’s diagnosed stroke territory on record. This study was designed and conducted according to Standards for Reporting of Diagnostic Accuracy (STARD). The statistical analysis included diagnostic accuracy parameters, cross-validation, and Youden Index optimization. After cross-validation on a cohort of 91 patients, the sensitivity of this territory diagnosis was 81% with a specificity of 87%. With this, the projection of strokes onto a two-dimensional map is accurate for representing the affected stroke territory and can be used to provide a static and printable overview of the diffusion MRI data. The projected map is compatible with other two-dimensional data such as EEG and will serve as a useful visualization tool.

Introduction

Worldwide, ischemic stroke is the third-most common cause of years of live lost (1). In developed countries, stroke even ranks second (1). Routine patients presenting with stroke symptoms are typically evaluated by assessing diffusion MRI data, which is the most accurate method of illustrating the affected brain area in the acute phase of a stroke (2). In diffusion MRI, the degree to which water molecules can diffuse freely in the tissue is measured. In brain regions affected by stroke, the diffusion of water is impaired and can be detected as a reduced darkening of the MRI image over a recording time of (typically) 20 ms per layer (3). To evaluate the results, the two-dimensional images are stacked to form a three-dimensional dataset. Through this method of representing the imaging data, an observer can only assess the full information through visual inspection and steady rotation or direct manipulation of the images (e.g., scrolling through the layers). This approach is not always possible and limits the applicability of the method in printed reports.

Computer-aided diagnosis of stroke has gained in importance over the last decade. There is a multitude of approaches (4–7). Most groups use CT images to make early developments of stroke visible and detectable (8–10). Tyan et al. (11) considered diffusion MRI data as well as CT images for unsupervised computer-aided diagnosis of stroke-affected brain volume. The focus of all these approaches lies in enhancing the imaging data so that search algorithms can detect stroke symptoms. With this work, we aim to add to the visualization possibilities, which are not addressed by most computer-aided diagnosis approaches. The dimensionality reduction to a two-dimensional depiction of the entire brain provides advantages in print, in compatibility with brain surface measurements such as EEG and in tracking changes of the stroke lesion over time. We here present a new approach of converting the three-dimensional imaging data set to a two-dimensional map by geoprojection. To ensure the correct representation of strokes in the two-dimensional map, we developed a computer-aided artery territory recognition algorithm, using the projected maps as input.

The projection of three-dimensional data into a two-dimensional map via Mollweide geoprojection is a common method in other fields (12–14). We applied this mathematical method to diffusion MRI data to create a static, two-dimensional representation of the entire brain providing accurate information on stroke position and size.

The purpose of this retrospective study was to develop an intuitive two-dimensional representation of three-dimensional brain imaging data and to probe it for its diagnostic validity. To validate the projection outcome, we assessed the diagnostic accuracy of computer-aided stroke territory recognition based on geoprojected two-dimensional maps of diffusion MRI data.

Materials and Methods

Study Population and Design

This retrospective study was designed and conducted according to Standards for Reporting of Diagnostic Accuracy (STARD) (15). The data for this study were collected using the institutional database of the Neurological Department at the Friedrich–Alexander University of Erlangen-Nuremberg. The patient’s demographic information was obtained from medical records. Patients of any sex and age were accepted if diffusion-weighted MR imaging data with apparent ischemic stroke was available. The patients’ stroke diagnoses on record were categorized according to the affected brain artery territory into the following groups: the anterior cerebral artery (ACA) territory, superior or inferior division of the middle cerebral artery (MCA) territory, the posterior cerebral artery (PCA) territory or the posterior inferior cerebellar artery (PICA) territory. This division of the brain follows the definition of vascular territories used in other studies (16–18). Superior cerebellar artery infarctions were rare in our search group; thus, this type of infarction was not included in the study.

Test Methods

To report the diagnostic accuracy of the projected representation of stroke territories in two-dimensional maps, we used each of the diagnoses on record for the 125 patients in the study, which were approved by an experienced neurologist and neuroradiologist as a reference standard. The clinical diagnoses were based on the evaluation of the same diffusion MRI datasets used for our test method.

Diffusion-Weighted MR Imaging

Image acquisition was performed on a 1.5-T MR-Scanner (Aera; Siemens, Erlangen, Germany) using routine protocols for stroke imaging, including an axial diffusion-weighted echo planar sequence (b value = 1000 s/mm²; repetition time = 6800 ms; echo time = 89 ms; slices = 25; slice thickness = 5 mm).

Projection Method

The projection of MRI data into two-dimensional maps, as well as statistical analysis, was performed with MATLAB software (Mathworks). The scripts are available at https://github.com/janawrosch/mrt_projection.

To create the two-dimensional maps of the entire brain, we first arranged all diffusion MRI data points to form a three-dimensional data stack, which was similar to the stacked image layers commonly created for MRI evaluation.

In the first step, the voxels depicting the brain were distinguished from the ones showing the background by background determination-based detection (19). For that, we plotted the values of all diffusion MRI data points from all layers into a histogram (Figure 1). This plot resulted in a large peak at low intensity values, which represents the background, and a smaller peak at higher intensities, which represents the brain. For the cut-off value between the background and brain, the minimum of the dip between those peaks was calculated (left line in Figure 1). This method is based on the assumption of a Gaussian distribution of the values in the histogram. Since it has been shown that MRI data follows a Rician distribution (20–22), we ensured applicability of the background determination-based method with the signal-to-noise levels at hand by testing the distributions in the histogram via Kolmogorov–Smirnov test for Gaussian distribution and found that it is applicable in each of the 125 cases (Figure S1 in Supplementary Material).

FIGURE 1

Figure 1. Histogram of diffusion MRI values of one exemplary dataset. The intense peak at the diffusion level around 20 a.u. represents the background. The broad peak around 180 a.u. representing the healthy brain values is accompanied by a smaller second “peak” around 300 a.u., which is representing the stroke-affected tissue. The left vertical line shows the calculated cut-off value between the background and the brain. The right vertical line shows the cut-off value between the healthy brain and stroke-affected tissue.

To evaluate the quality of the two-dimensional projection, the proportion of stroke volume per total brain volume in the diffusion MRI data was compared to the proportion of stroke area per total brain area in the two-dimensional map. For the separation of MRI voxels representing healthy brain from stroke voxels representing stroke-affected tissue, we also used the intensity histogram in which the broad peak representing the brain is positively skewed by the scarce high intensity stroke values. Based on a normal, and therefore symmetrical, distribution of healthy brain values, the cut-off point between the healthy brain and stroke is the minimum value from the left side of the curve reflected across the maximum of the healthy brain values, which is indicated by the right line in Figure 1. These cut-off values were calculated for each patient’s individual histogram.

In the next step, all diffusion MRI data points representing the brain (healthy and stroke-affected tissue) were projected onto a sphere around the surface of the brain (see Figure 2A). For the center of this sphere, we chose the ventral surface of the lower mesencephalon between the cerebral peduncles, as shown in Figure 2B. We chose this location due to its centric and distinct position within the brain. The voxels within the sphere are projected straight out onto their nearest neighbor on the sphere. The generated surface contains all the information included in the three-dimensional MRI data volume. In a final step, this surface was transformed into a plane.

FIGURE 2

Figure 2. Projection of three-dimensional diffusion MRI data onto a two-dimensional map. (A) All data points are projected onto a sphere around the surface of the brain, which is illustrated by the dashed circles; (B) center of the projection sphere at the ventral surface of the lower mesencephalon between both cerebral peduncles; (C) plane transformation of the projection sphere via Mollweide geoprojection.

We used the mathematical method of Mollweide geoprojection (23) to convert the spherical coordinates λ and θ of each point on the sphere with radius r to Cartesian coordinates x and y using the following formulae:

$x=\left(\sqrt{8}\cdot r\cdot \mathrm{\lambda }\cdot \cos \mathrm{\theta }\right)∕\mathrm{\pi };y=\sqrt{2}\cdot r\cdot \sin \mathrm{\theta }$

With this geoprojection, the position of each data point on the sphere is transformed to a point on a two-dimensional map of the brain (see Figure 2C). In about 40% of the pixels on the map, it is the case that several data points fall into the range of the same pixel. In those pixels, the maximum diffusion value was plotted onto the map to ensure that no stroke data (higher values) gets lost or obstructed by healthy brain data (lower values).

To determine which diffusion levels indicate a stroke at each point on the map, we created a diffusion level reference map from the healthy brain hemispheres of 50 patients. For each single pixel of the map, we averaged the diffusion values of healthy brains and calculated the SD of the diffusion values in that point. A patient’s projected data are compared, pixel-wise, to this reference map. If the data differ by more than 2 SDs from the averaged healthy diffusion value at that point, the deviation is considered indicative of stroke. We determined the threshold of the mean plus 2 SDs by minimizing the difference in the proportion of the stroke-affected volume per total brain volume and the proportion of the stroke-affected area per total brain area in the two-dimensional projection, as shown in Figure 3.

FIGURE 3

Figure 3. Diffusion level reference. (A–C) Projections with strokes indicated for the pixels of values above the mean plus different numbers of SDs from the healthy diffusion levels; (A) projection referenced at one sigma level, which corresponds to a mean difference in the proportion of stroke per total brain in the original MRI data and in the projected maps (SPB-ratio) of 16.5% (SD: 31.9%); (B) projection referenced at two sigma levels, which corresponds to a SPB-ratio of 5.4% (SD: 8.1%); (C) projection referenced at three sigma levels, which corresponds to a SPB-ratio of 9.2% (SD: 6.9%); (D) representative layer of the diffusion MRI data of the projected brain in (A) through (C), showing the stroke-affected region.

With this software, the three-dimensionally stacked MRI data were automatically converted into a two-dimensional map showing the entire brain at once and indicating the stroke area with above threshold values.

To be able to assign stroke coordinates in the two-dimensional map to a defined region in the patients’ brains, we created a stroke territory reference map referencing the pixels according to brain artery territory. We created this map by merging the stroke-covered regions of the patients with a distinct diagnosis of stroke in only the one artery territory. For good coverage of the complete artery territory, we chose patients who had a widespread lesion within that territory (see Figure 4). The patients used for this stroke territory assessment (14 patients) were excluded from further analysis to prevent biased test results.

FIGURE 4

Figure 4. Stroke territory reference map. (A) Anterior cerebral artery territory; (B) superior division of the middle cerebral artery territory; (C) inferior division of the middle cerebral artery territory; (D) posterior cerebral artery territory; (E) posterior inferior cerebellar artery territory; left side of the map also shows the left side of the brain.

To ensure accurate representation of strokes in the two-dimensional maps, we developed an algorithm for computer-aided stroke territory diagnosis that is using projected maps as input and assessed this test’s diagnostic accuracy.

The algorithm tests each territory for stroke by applying two criteria: (1) If a large percentage of the territories’ area is covered by stroke-indicating pixels, that territory is positive for stroke. The cut-off value for this criterion decides to what extend each region needs to be covered to be counted as positive. (2) If a large bulk of all stroke-indicating pixels falls together into the same territory, that territory is positive for stroke. The cut-off value for this criterion decides how many of all the stroke-indicating pixels need to be convened in a territory for this to be counted as positive. The values for each criterion and each round of outer cross-validation ore listed in Table 1.

TABLE 1

Table 1. Cut-off values for computer-aided brain artery territory diagnosis.

The resulting stroke territory diagnosis was compared to the patient’s recorded clinical diagnosis (reference standard).

Statistical Methods

To assess sensitivity and specificity (24) as well as the positive predictive value (PPV) and negative predictive value (NPV) and positive likelihood ratio (PLR) and negative likelihood ratio (NLR), we used leave-p-out cross-validation (25–27). The study population of 91 patients was randomly divided into 10 subgroups. In 10 rounds of parameter optimization and subsequent test validation one subgroup at a time was excluded from the parameter optimization. The following validation was applied to the excluded data set respectively. The test performance results of all 10 validation rounds were averaged. Parameter optimization as well as test performance evaluation was based on the counts of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) results.

Test parameters were optimized to find the best cut-off values for the brain artery territory recognition criteria. Optimization was achieved by maximizing the Youden Index Y (28), which is calculated by the formula:

$Y=\frac{\text{TP}}{\text{TP}+\text{FN}}+\frac{\text{TN}}{\text{FP}+\text{TN}}-1$

The associated receiver operating diagrams (29) are provided in Figure S2 in Supplementary Material.

To assess the test performance, the following formulae were used to calculate sensitivity and specificity (24), PPV and NPV (30), and PLR and NLR (31) from the result counts:

$\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{TN}}$

$\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FN}}$

$\text{PPV}=\frac{\text{TP}}{\text{TP}+\text{FP}}$

$\text{NPV}=\frac{\text{TN}}{\text{TN}+\text{FN}}$

$\text{PLR}=\frac{\text{Sensitivity}}{1-\text{Specificity}}$

$\text{NLR}=\frac{1-\text{Sensitivity}}{\text{Specificity}}$

Results

Participants

The study was performed in 2015 with archived patient data that ranged from February 2012 until August 2015. We included 125 patients in this study with a mean age of 65 years (range 23–93 years). The study population included 47 female patients (38%). Twenty-four patients had an ACA territory stroke, 48 patients had a stroke in the superior division of the MCA territory, and 51 in the inferior division. The PCA territory stroke was diagnosed in 33 patients, and PICA territory infarctions occurred in 32 patients. The patient demographics are presented in Table 2.

TABLE 2

Table 2. Demographic characteristics of the study population (n = 125).

Of the total study cohort of 125 patients, 14 data sets were used to create the stroke territory reference map and were excluded from the test evaluation. Therefore, the analysis of our method’s diagnostic accuracy is based on 101 cases. These 101 cases were split into two groups of sizes 91 and 20. For cross-validation, the 91 cases were used and randomly divided into 10 subgroups on which 10 rounds of outer cross-validation were performed. In each round, the test parameters were optimized using the data of nine subgroups. Subsequently, the test performance was validated on the tenth subgroup each round. The performance results of all 10 rounds of cross-validation were then averaged. For an independent validation, the test parameters were optimized using the data of all 91 patients and then validated on the 20 independent data sets, which were excluded from cross-validation. Of those, 20 data sets were excluded from the cross-validation, to be later used for independent validation. The remaining 91 cases were randomly split up into 10 groups of 9 cases each (one group contained 10 cases). The study procedure is illustrated as a flow chart in Figure S3 in Supplementary Material.

The neurological diagnoses were approved by an experienced neurologist and neuroradiologist and were used as a reference standard. This reference and the stroke projection were both based on the same diffusion MRI dataset. Discrepancies in the test results were not ascribed to any change in the disease pattern.

Our study population had a median National Institute of Health Stroke Scale (NIHSS) Score (32–34) of 4 with an interquartile range of 9. As a measure of quantitative severity, we calculated the stroke per brain ratio from the diffusion MRI pixels and found that the study population had a mean of 4.3% (SD: 5.8%) of brain volume affected by stroke. The median of stroke per brain ratio was 2.3% with an interquartile range of 3.7%. For patients with strokes in the PCA territory, the quantitative severity even reached a median of 2.8% with an interquartile range of 5.4%. The most severely affected group of patients according to NIHSS was the one with infarctions in the superior division of the MCA territory. Here, the median score was 7 with an interquartile range of 13. A detailed list of the NIHSS Scores and quantitative severity for all brain artery territories is shown in Table 3.

TABLE 3

Table 3. Severity of stroke within the study population (n = 125).

Diagnostic Accuracy of Territory Recognition Based on Stroke Projections

The detection of stroke-affected brain artery territories based on geoprojected diffusion MRI data with our method reached a sensitivity of 81% (SD: 7.8%) and a specificity of 87% (SD: 6.8%) after cross-validation on 91 cases. In independent validation on another 20 cases, the test had a sensitivity of 75% with specificity of 86%. A detailed cross tabulation of the results for each round of cross-validation is given in Table S1 in Supplementary Material. The statistical analysis results are presented in Table 4.

TABLE 4

Table 4. Statistical analysis of stroke territory recognition based on the geoprojected, two-dimensional map compared to the clinical diagnoses.

In our study, we can rule out diagnostic review bias (35) because the diagnoses used as reference standards were already set before the study was conducted. Because the test software does not have any information about the reference standard results, there is also no test review bias (35).

The reference standard is affected by observer variability (36) because there is no standardized method for physicians to definitively diagnose the affected stroke area, whereas the projection method is fixed software that has no analytical noise.

Discussion

Two-dimensional projections of three-dimensional data are a benefit because of their “at a glance” usability. Projection methods are commonly used to transform geographical data (spheres, planet surfaces) to maps that allow direct and two-dimensional analysis. In contrast to three-dimensional data, two-dimensional maps are immediately distinct without the need for dynamic presentation (e.g., continuous rotation of an object) or interaction (such as scrolling through stack image layers). These maps can be printed out in clinical reports without the loss of information on stroke position and size to provide a complete overview of the brain.

To assess projection quality, we compared the ratio of brain pixels covered by stroke in the original diffusion MRI data and in the two-dimensional projection. The projection of three-dimensional values onto a sphere is more accurate for near-surface values. Diffusion deficits that lie too deep in the brain cannot be projected properly and result in an oversized and scattered stroke across the two-dimensional map, which leads to a large difference in the stroke per brain ratios of the diffusion MRI and the projection as well as a loss of specificity. A possible way of addressing this problem would be to first project the inner MRI voxels onto a small sphere just around the center. Then in further steps, the voxels lying further outwards could be projected onto bigger spheres. Another weakness of the method is the step of mapping all projected MRI data points into the limited pixels of the two-dimensional maps. To ensure that the data lost during this process is information about healthy brain rather than the stroke position and size, each pixel of the map shows the maximum diffusion value of all data points that it represents. The histogram in Figure S4 in Supplementary Material shows, that this step is concerning about 40% of the map pixels. In many of them the maximum value is taken from a group of only a few data points. In some pixels though, the number of data points being represented by only one pixel on the map reaches up to about 35. Such losses are an inherent problem of data reduction methods and represent the cost of implementing a user-friendly and easily accessible two-dimensional map.

Since there still always is a loss of information and small infarctions could be missed, this method alone is not a reliable basis for clinical diagnosis. It can only add a way of illustrating the MR imaging results after careful inspection of the original imaging data by a physician. This step of visual inspection of the imaging data is inevitable to reach a reliable and safe diagnosis for the patients. The projection of strokes with our method will still serve a useful function as an easily accessible overview for reference for example in patient records and in research.

The projection may be weaker in representing inner and too small infarctions, yet it proves to be a useful tool for visualizing cortical damage and can be powerful in monitoring the progression of lesions over time. Moreover, it is compatible with other brain surface methods such as EEG. There is a plethora of different mathematical methods to project surface data to maps. Among these methods, the Mollweide projection is widely used in visible studies (37, 38). As an equal area projection, it has the advantage of representing projected areas in their correct size. The correct size is attained at the cost of distortions and users will need to adapt to the new topography. However, with the equal area method, the extent of cortical damage can be measured quantitatively.

When applying the technique, we found that the different brain artery territories are represented as intuitively comprehensible areas on the map. User friendliness is, of course, not enough to justify a new method. We therefore chose to apply the STARD criteria for reporting diagnostic accuracy as well as leave-p-out cross-validation and independent validation to substantiate the parameter thresholds. We find that with a sensitivity of 81% and a specificity of 87% in cross-validation and with sensitivity of 75% and specificity of 86% in independent validation, the detection of strokes in geoprojected brain maps can compete with diagnostic methods from other disciplines (39–42). The specificity could be improved in the future by using a more accurate and complete stroke territory reference map based on a larger number of patients. These small error rates in the brain artery detection show that strokes are presented sufficiently correct in the geoprojected two-dimensional maps for them to be a useful visualization tool.

Diffusion MR imaging is the gold standard for illustrating stroke lesions, especially in the acute phase. We have reported a new method for presenting these imaging data. There is no additional data acquisition needed, and on a standard current office computer, the projection of one dataset takes approximately 3 min. Our method only adds to the visualization possibilities of diffusion MRI data without imposing any additional risks or inconveniences on patients or physicians.

In summary, our analytical method complements the classical approach in terms of intuitiveness, time and repeatability compared to pure subjective visual inspection. Whereas the possibility of presenting and printing a comprehensive overview of the brain all at once is a big advantage, one can also imagine this method could be useful in the future for studies comparing and overlaying diffusion MRI results with other brain imaging methods as well as surface methods such as EEG. Altogether, this work contributes to furthering the applicability of diffusion MRI data in ischemic stroke territory recognition.

Author Contributions

First and corresponding author JKW carried out the main part of the work including programing the processing scripts, all statistical analyses and wrote the manuscript. BV supported this work by approving clinical diagnoses taken from patient records. He contributed to the study design and critically revised the manuscript. PG and DFG kindly revised the manuscript and provided helpful comments to improve the study design. AD and SS provided the necessary MRI data and gave helpful feedback on the manuscript. JK critically revised the manuscript and provided helpful comments; also he provided the lab’s research environment in which this work developed. TWG provided the idea for this work. He supervised the working process and revised the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

This study was supported by the Else-Kröner-Fresenius Stiftung under Grant 2012_A35.

Supplementary Material

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fneur.2015.00239

References

1. Global burden of Disease MaCoDC. Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990-2013: a systematic analysis for the global burden of disease study 2013. Lancet (2015) 385(9963):117–71. doi: 10.1016/S0140-6736(14)61682-2

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Simonsen CZ, Madsen MH, Schmitz ML, Mikkelsen IK, Fisher M, Andersen G. Sensitivity of diffusion- and perfusion-weighted imaging for diagnosing acute ischemic stroke is 97.5%. Stroke (2015) 46(1):98–101. doi:10.1161/STROKEAHA.114.007107

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Baird AE, Warach S. Magnetic resonance imaging of acute stroke. J Cereb Blood Flow Metab (1998) 18(6):583–609. doi:10.1097/00004647-199806000-00001

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Takahashi N, Lee Y, Tsai DY, Kinoshita T, Ouchi N, Ishii K. Computer-aided detection scheme for identification of hypoattenuation of acute stroke in unenhanced CT. Radiol Phys Technol (2012) 5(1):98–104. doi:10.1007/s12194-011-0143-0

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Lee JD, Chang TC, Huang CH, Wu SJ, Chen CJ. Computer-aided diagnosis system for acute stroke using diffusion weighted images with volume calculation. Conf Proc IEEE Eng Med Biol Soc (2004) 2:1529–32. doi:10.1109/IEMBS.2004.1403468

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Przelaskowski A, Sklinda K, Bargieł P, Walecki J, Biesiadko-Matuszewska M, Kazubek M. Improved early stroke detection: wavelet- based perception enhancement of computerized tomography exams. Comput Biol Med (2007) 37(4):524–33. doi:10.1016/j.compbiomed.2006.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Shieh Y, Chang CH, Shieh M, Lee TH, Chang YJ, Wong HF, et al. Computer-aided diagnosis of hyperacute stroke with thrombolysis decision support using a contralateral comparative method of CT image analysis. J Digit Imaging (2014) 27(3):392–406. doi:10.1007/s10278-013-9672-x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Maldjian JA, Chalela J, Kasner SE, Liebeskind D, Detre JA. Automated CT segmentation and analysis for acute middle cerebral artery stroke. AJNR Am J Neuroradiol (2001) 22(6):1050–5.

PubMed Abstract | Google Scholar

9. Hema Rajini N, Bhavani R. Computer aided detection of ischemic stroke using segmentation and texture features. Measurement (2013) 46(6):1865–74. doi:10.1016/j.measurement.2013.01.010

CrossRef Full Text | Google Scholar

10. Tang FH, Ng DK, Chow DH. An image feature approach for computer-aided detection of ischemic stroke. Comput Biol Med (2011) 41(7):529–36. doi:10.1016/j.compbiomed.2011.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Tyan YS, Wu MC, Chin CL, Kuo YL, Lee MS, Chang HY. Ischemic stroke detection system with a computer-aided diagnostic ability using an unsupervised feature perception enhancement method. Int J Biomed Imaging (2014) 2014:947539. doi:10.1155/2014/947539

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Monnereau M, Calvet M, Margerin L, Souriau A. Lopsided growth of earth’s inner core. Science (2010) 328(5981):1014–7. doi:10.1126/science.1186212

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Hemingway D, Nimmo F, Zebker H, Iess L. A rigid and weathered ice shell on Titan. Nature (2013) 500(7464):550–2. doi:10.1038/nature12400

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Marchi S, Bottke WF, Elkins-Tanton LT, Bierhaus M, Wuennemann K, Morbidelli A, et al. Widespread mixing and burial of earth’s hadean crust by asteroid impacts. Nature (2014) 511(7511):578–82. doi:10.1038/nature13539

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med (2003) 138(1):W1–12. doi:10.7326/0003-4819-138-1-200301070-00012-w1

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Paciaroni M, Silvestrelli G, Caso V, Corea F, Venti M, Milia P, et al. Neurovascular territory involved in different etiological subtypes of ischemic stroke in the Perugia stroke registry. Eur J Neurol (2003) 10(4):361–5. doi:10.1046/j.1468-1331.2003.00646.x

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Rovira A, Grive E, Rovira A, Alvarez-Sabin J. Distribution territories and causative mechanisms of ischemic stroke. Eur Radiol (2005) 15(3):416–26. doi:10.1007/s00330-004-2633-5

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Chatterjee NR, Ansari SA, Vakil P, Prabhakaran S, Carroll TJ, Hurley MC. Automated analysis of perfusion weighted MRI using asymmetry in vascular territories. Magn Reson Imaging (2015) 33(5):618–23. doi:10.1016/j.mri.2015.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Stroebel A, Welzel O, Kornhuber J, Groemer TW. Background determination-based detection of scattered peaks. Microsc Res Tech (2010) 73(12):1115–22. doi:10.1002/jemt.20858

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Rajan J, den Dekker AJ, Sijbers J. A new non-local maximum likelihood estimation method for rician noise reduction in magnetic resonance images using the Kolmogorov-Smirnov test. Signal Processing (2014) 103:16–23. doi:10.1016/j.sigpro.2013.12.018

CrossRef Full Text | Google Scholar

21. Pizurica A, Philips W, Lemahieu I, Acheroy M. A versatile wavelet domain noise filtration technique for medical imaging. IEEE Trans Med Imaging (2003) 22(3):323–31. doi:10.1109/TMI.2003.809588

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Manjon JV, Coupe P, Buades A. MRI noise estimation and denoising using non-local PCA. Med Image Anal (2015) 22(1):35–47. doi:10.1016/j.media.2015.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Grafarend E, Heidenreich A. The generalized mollweide projection of the biaxial ellipsoid. J Geod (1995) 69(3):164–72. doi:10.1007/BF00815485

CrossRef Full Text | Google Scholar

24. Altman DG, Bland JM. Diagnostic tests. 1: sensitivity and specificity. BMJ (1994) 308(6943):1552. doi:10.1136/bmj.308.6943.1552

CrossRef Full Text | Google Scholar

25. Celisse A, Robin S. Nonparametric density estimation by exact leave – out cross-validation. Comput Stat Data Anal (2008) 52(5):2350–68. doi:10.1016/j.csda.2007.10.002

CrossRef Full Text | Google Scholar

26. Zhang P. Model selection via multifold cross validation. Ann Stat (1993) 21(1):299–313. doi:10.1214/aos/1176349027

CrossRef Full Text | Google Scholar

27. Shao J. Linear model selection by cross-validation. J Am Stat Assoc (1993) 88(422):486–94. doi:10.1080/01621459.1993.10476299

CrossRef Full Text | Google Scholar

28. Youden WJ. Index for rating diagnostic tests. Cancer (1950) 3(1):32–5. doi:10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3

CrossRef Full Text | Google Scholar

29. Altman DG, Bland JM. Diagnostic tests 3: receiver operating characteristic plots. BMJ (1994) 309(6948):188. doi:10.1136/bmj.309.6948.188

CrossRef Full Text | Google Scholar

30. Altman DG, Bland JM. Diagnostic tests 2: predictive values. BMJ (1994) 309(6947):102. doi:10.1136/bmj.309.6947.102

CrossRef Full Text | Google Scholar

31. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ (2004) 329(7458):168–9. doi:10.1136/bmj.329.7458.168

CrossRef Full Text | Google Scholar

32. Brott T, Adams HP Jr, Olinger CP, Marler JR, Barsan WG, Biller J, et al. Measurements of acute cerebral infarction: a clinical examination scale. Stroke (1989) 20(7):864–70. doi:10.1161/01.STR.20.7.871

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Brott T, Marler JR, Olinger CP, Adams HP Jr, Tomsick T, Barsan WG, et al. Measurements of acute cerebral infarction: lesion size by computed tomography. Stroke (1989) 20(7):871–5. doi:10.1161/01.STR.20.7.871

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Lyden P, Brott T, Tilley B, Welch KM, Mascha EJ, Levine S, et al. Improved reliability of the NIH stroke scale using video training. NINDS TPA stroke study group. Stroke (1994) 25(11):2220–6. doi:10.1161/01.STR.25.11.2220

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Philbrick JT, Horwitz RI, Feinstein AR. Methodologic problems of exercise testing for coronary artery disease: groups, analysis and bias. Am J Cardiol (1980) 46(5):807–12. doi:10.1016/0002-9149(80)90432-4

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Elmore JG, Feinstein AR. A bibliography of publications on observer variability (final installment). J Clin Epidemiol (1992) 45(6):567–80. doi:10.1016/0895-4356(92)90128-A

CrossRef Full Text | Google Scholar

37. Solomon SC, Aharonson O, Aurnou JM, Banerdt WB, Carr MH, Dombard AJ, et al. New perspectives on ancient Mars. Science (2005) 307(5713):1214–20. doi:10.1126/science.1101812

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Russo CJ, Passmore LA. Controlling protein adsorption on graphene for cryo-EM using low-energy hydrogen plasmas. Nat Methods (2014) 11(6):649–52. doi:10.1038/nmeth.2931

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Chapin KC, Dickenson RA, Wu F, Andrea SB. Comparison of five assays for detection of Clostridium difficile toxin. J Mol Diagn (2011) 13(4):395–400. doi:10.1016/j.jmoldx.2011.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Williams JR, Hirsch ES, Anderson K, Bush AL, Goldstein SR, Grill S, et al. A comparison of nine scales to detect depression in Parkinson disease: which scale to use? Neurology (2012) 78(13):998–1006. doi:10.1212/WNL.0b013e31824d587f

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Trepte CJ, Eichhorn V, Haas SA, Stahl K, Schmid F, Nitzschke R, et al. Comparison of an automated respiratory systolic variation test with dynamic preload indicators to predict fluid responsiveness after major surgery. Br J Anaesth (2013) 111(5):736–42. doi:10.1093/bja/aet204

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Jiang Y, Zhang C, Wang W, Su T, Zhou W, Jiang L, et al. Diagnostic value of ACTH stimulation test in determining the subtypes of primary aldosteronism. J Clin Endocrinol Metab (2015) 100(5):1837–44. doi:10.1210/jc.2014-3551

PubMed Abstract | CrossRef Full Text | Google Scholar