Edited by: Xi-Nian Zuo, Institute of Psychology (CAS), China
Reviewed by: Daniel Haehn, Harvard University, United States; Graham J. Galloway, Translational Research Institute, Australia
*Correspondence: Louis-Philippe Ledoux
Maxime Descoteaux
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Data visualization is one of the most important tool to explore the brain as we know it. In this work, we introduce a novel browser-based solution for medical imaging data visualization and interaction with diffusion-weighted magnetic resonance imaging (dMRI) and tractography data: Fiberweb. It uses a recent technology, WebGL, that has yet to be fully explored for medical imaging purposes. There are currently very few software tools that allow medical imaging data visualization in the browser, and none of these tools support efficient data interaction and processing, such as streamlines selection and real-time deterministic and probabilistic tractography (RTT). With Fiberweb allowing these types of interaction, it is no longer the case. We show results of the visualization of medical imaging data, and demonstrate that our new RTT probabilistic algorithm can compare to a state of the art offline algorithm. Overall, Fiberweb pushes the boundary of interaction combined with scientific visualization, which opens great perspectives for quality control and neurosurgical navigation on browser-based mobile and static devices.
In recent years, the world has seen a huge increase in the use of mobile devices and web technologies. It is becoming more and more important that all resources are always available at anytime, anywhere (e.g., Dropbox, GoogleDrive, iCloud, OneDrive, and many other such providers). Currently, in the field of medical imaging, there are a lot of software tools to view and interact with data. However, only a few exist for the browser, even with all the recent possibilities that the Web Graphics Library (WebGL) enables. Having the possibility to simply load a web page to start interacting with brain imaging data would open possibilities for the whole field.
There are a few existing software tools that are currently available and offer visualization of medical imaging data in the web browser. First, there is Slice:Drop (Haehn,
In this work, we propose
Full-brain tractogram (up to 75,000 “streamlines”) loading and smooth interaction using a selection object. This enables virtual dissection manipulations directly in Fiberweb.
Real-time web-based fiber tractography.
A novel probabilistic algorithm for real-time tracking, which is the main contribution.
These features allow the user to interact and process his data in ways that were not previously possible in the browser. This means no installation is required for the user, and no server is required for backend computation. This opens great perspectives for quality control and neurosurgical navigation on browser-based mobile and static devices.
WebGL is the driving technology that enabled the creation of Fiberweb. WebGL's creation was made possible by the arrival of HTML5, and more specifically one of its new features, the canvas element. This element allows the creation of a drawable region in the browser, which can then be modified using JavaScript. WebGL is a low-level graphics library for 3D rendering in the browser and is functional with all the major players, such as Chrome, Firefox, Edge and Safari.
To simplify the development of Fiberweb, a high-level library called Three.js (Cabello,
Fiberweb can currently display three types of medical imaging data: anatomies (peaks) and streamlines. These datasets can be visualized both from a 3D perspective as well as a 2D one, as seen in Figure
Anatomy visualization in 3D and 2D views inside Fiberweb.
“Anatomies” are any 3D volume datasets, such as T1, T2, fractional anisotropy (FA), mean diffusivity (MD), and colored FA (4D image). They are represented by plane buffer geometries. These are simple textured planes, both in 3D and 2D. These textures are computed during the initialization. They can be navigated through with the help of different sliders for each view. When a slice change is made, planes are simply translated, and a simple texture swap is applied since they are already loaded in memory.
Next are the “peaks,” which are 4D volumes. The first three represent the coordinates of a peak (X, Y, Z). The fourth one contains the directions extracted from the fiber ODFs (X, Y, Z, N where
Anatomy and peaks visualization in 3D and 2D views inside Fiberweb.
Finally, there are the “streamlines”. A streamline is a set of 3D points (
Anatomy and streamlines visualization inside Fiberweb, with the intersected streamlines in the 2D views.
Fiberweb supports various interaction mechanisms: streamlines selection and real-time tractography.
Fiberweb provides a streamlines selection mechanism based on a points inclusion test. All streamlines having at least one point inside the selection box will be displayed.
To reduce complexity and increase performance (trying to ensure that feedback is provided in real time), an octree is used to classify streamlines points. An octree is a data structure that recursively subdivides space in eight smaller regions until a certain level of simplification is obtained. Once the space is divided, each point is classified into the octree. This results in a much smaller search space and fewer points to search through.
Even though an octree was already available in Three.js, a new implementation was done in Fiberweb. The main reason was that the octree of Three.js dynamically transforms the octree every time a new element is added to it. Since a full dataset of streamlines contains a high number of points, it was unnecessarily slow to initialize, especially since the position of the points is static and known a priori.
From the data drop-down menu (see Figure
where f is the FA (or GFA, AFD, or any other underlying scalar map or mask) at the current voxel, g is the puncture value that was mentionned previously and
Fiber bundle (part of the corpus callosum), result of the real-time tractography, with the initialization box (red) in the mid-sagittal slice of the CC and the following parameters: angle threshold of 50°, fiber length minimum of 10 mm and maximum of 200 mm, FA threshold of 0.1, a step size of 0.5 mm, 10 seeds per axis and a puncture value (g) of 0.2.
Additionally, the user can save the computed/extracted streamlines, from both the selection and the real-time tractography. The parameters are also saved, making it easier for a user to recreate its selection or tracking.
Chamberland et al. (
FiberWeb also introduces a new probabilistic real-time tracking technique, building on the already existing deterministic RTT technique. Instead of loading the standard peaks file containing only the directions of the maximas, the user needs to load a peaks file also containing the uncertainty α
Illustration of the uncertainty alpha parameter at 95% (red), 75% (yellow), and 35% (green) of the maxima. The script to compute these alpha values for each fODF peak is given in the Appendix.
The first part of the tracking algorithm is the same as for the deterministic version. Seeds are chosen randomly in the selection box. Then, a random peak associated to the voxel at the current position is chosen. Next, instead of simply following that tracking direction, the uncertainty angle associated with that peak is fetched. Then, a random direction in the “cone of uncertainty” formed by the uncertainty angle and the peak is chosen. Then, the algorithm continues executing normally, with the extra step of modifying the direction by an angle of uncertainty every time. Equation 1 then becomes:
where θ() represents the function modifying the direction
The dataset used for the evaluation of the probabilistic tracking algorithm is found on the SCIL website (
To determine the validity of our new real-time tracking algorithm, the evaluation was done in two ways, as similarly done in Chamberland et al. (
In this work, it is considered that the baseline performance for the visualization part of the tool is to display an anatomical dataset. Displaying a peaks dataset does not affect the framerate in any noticeable way. In Figure
The main bottleneck of Fiberweb is the display of streamlines in general. The framerate and the maximal streamlines capacity are directly affected by the number of streamlines and the number of points per streamline. To avoid any potential bias, all the streamlines datasets that were used are raw and have roughly the same average number points per streamline (100 points). In Figure
Mean framerate [in frames per second (fps)] vs. number of streamlines.
Every interaction in Fiberweb works in real time. The framerate is not impacted whether or not the intersected streamlines are shown. This is because of the use of a specific shader to show only the portion intersecting the anatomy. This means that rendering the intersected streamlines is just like rendering standard streamlines.
The selection also has little impact on the framerate due to the use of an octree, and this data structure does not impact the loading time of the streamlines. However, when loading bigger streamline datasets, the framerate can drop when in areas of high streamline density during the selection. The framerate is only affected when moving the selection box, going back to normal once it is stopped.
Real-time tractography can have a bigger impact on the framerate of Fiberweb. A noticeable reduction in framerate is obtained when getting close to 1,000 seeds (10 seeds per axis). This drop is more present when tracking longer bundles. In general, the number of seeds is the limiting factor. The rest of the parameters do not negatively impact the framerate. However, using stricter parameters can speed up the algorithm since the streamlines will be shorter in general. Furthermore, just like the selection, the problems with the framerate only exist when moving the box around, since this is when the tractography is computed.
The proposed probabilistic real-time tractography algorithm is based on a field of vectors (cartesian coordinates) of directions and their associated uncertainty angle. This data is extracted from fiber ODFs and is compared to probabilistic tracking based on fODF (Girard et al.,
To determine whether or not this proposed tracking was valid, both probabilistic algorithms were used to generate 1,000 streamlines from the same regions (Figure
Various bundles resulting from probabilistic tractography (real-time in the Fiberweb vs offline). Dice coefficients are: CST (0.865), CC (0.928), and AF (0.811). The yellow arrows show the location of seeding regions.
Bundles similarities were also quantitatively scored using a weighted Dice ratio. Table
Comparison between offline and online algorithms using the Dice coefficient.
CST | 0.865 |
CC | 0.928 |
AF | 0.811 |
Medical imaging software tools are often hard to install and hard to use for non-technical users. Fiberweb bypasses that problem by being built for the web. This makes the application cross-platform, and ready to use without the need for the installation of anything outside of having a web browser that is up-to-date. Furthermore, out of the few visualization web applications that already exist, none allow for real-time interaction and processing such as selection or real-time tractography.
In this article, a modified version of an existing real-time tractography algorithm was introduced, making it possible to use it in a probabilistic way. This allows for a new way to visualize data in real time. It was validated qualitatively and quantitatively against a state of the art offline tractography algorithm. When compared visually, one can see that the offline version of the bundles recreated tend to look more probabilistic (more fanning and more spurious streamlines). This is due to multiple factors. It is also important to remember that both algorithms use a different tracking equation and that the real-time FiberWeb version only uses peaks and an uncertainty as opposed to the full fODF.
The uncertainty angle has the biggest impact on how probabilistic the algorithm will be. Using a 100% threshold, the tractography would give results identical to the deterministic version of the algorithm. Inversely, using a 0% threshold, the tracking would take any random direction part of the fiber ODF. In our case, 35% was used because it yielded the best Dice quantitative scores, while still looking probabilistic. In Figure
CST bundles resulting from probabilistic tractography, with a varying uncertainty percentage threshold.
The puncture parameter of the real-time tractography algorithm can modify the effect of the uncertainty angle since it differently weighs the incoming and next directions. A higher puncture value will make it look more probabilistic, but also generate more noisy spurious streamlines. Results in this paper were generated using a value of 0.2 since it is the suggested value for the deterministic real-time tractography algorithm (Chamberland et al.,
Probabilistic RTT of the CST for different puncture values.
The resolution of the sphere on which the fiber ODFs are mapped affects the accuracy of the uncertainty angle, even though it is the factor that has the least impact on the final result. Results presented in this paper were generated using a sphere with 724 symmetrical vertices. A sphere with a lower resolution would give us uncertainty angles that are less representative of the user-inputted percentage value of a maxima, and not even an accurate maxima. The difference between two different sphere resolutions to display the same fODF can be seen in Figure
fODF represented by spheres of different resolutions.
One of the applications of FiberWeb is quality assurance. With the possibility to access a medical visualization tool anywhere or on any platform, without the need for anything else, it will make it easier to validate and check smaller datasets. For instance, suppose one has 1,000 subjects in a tract-based white matter study. One can imagine using the FiberWeb to rapidly inspect tractograms through a web quality control interface such a CBRAIN (Sherif et al.,
Fiberweb could also be used as a teaching tool. For example, one could imagine a scenario where a teacher in medical school would want to illustrate a few fiber bundles to his class without having prepared for it. This could be achieved simply by opening a web page, loading his medical brain data and using the visualization, selection or real-time tracking features of Fiberweb to display the bundles. Then, by just sharing the data he used for the demonstration, every student could be able to review and explore it on their own device without the need to install an external software.
The new probabilistic algorithm allows for a new way to explore data without the need of having huge streamlines datasets, especially in the case where a specific region of interest is the sole object of the exploration. For example, in neurosurgical planning, it could be of great interest to use Fiberweb. Around regions of oedema, necrosis and mass effect due to the presence of a tumor, standard tracking parameters no longer hold. It is important to be able to reduce the FA threshold, change the step size and explore the uncertainty of the fODF peaks to present a more realistic representation of the data to the surgeon. Having both deterministic and probabilistic RTT could help more easily assess the effect of the tumor mass effect and oedema on the peaks, diffusion measures and corresponding reconstructed streamlines. It could be the first step towards showing the surgeon the most probable and improbable streamlines around and inside the tumor region of interest.
The first goal of Fiberweb is to support larger datasets. Unfortunately, it is mostly tied with the evolution of the varying browsers. Another main goal is making Fiberweb work just as well on tablets as in the browser. With the recent release of WebGL 2.0, one of the first task that will need to be done is to upgrade Fiberweb to use this version, mainly through the Three.js library. WebGL 2.0 will bring performance upgrades compared to the 1.0 version. For example, it will enable the use of 3D textures instead of only 2D, which is great considering anatomies are just like 3D textures. Currently, WebGL 2.0 was only announced, and it will probably be a while before it is widely available. Another major focus of Fiberweb will be to add new features. Currently, only selection is available to interact with streamlines datasets. It would be interesting to add the possibility to use more than one selection box to make the streamlines selection possibly more precise. It would also be interesting to add exclusion boxes for the same reason. Additionally, it would be great to add segment based filtering (Houde et al.,
In this article, Fiberweb, a new browser-based medical dMRI tractography visualization tool, was presented. This web application, unlike others of its kind, allows the user to interact with his data through virtual dissection with the help of a selection box, and through streamlines reconstruction in real time. Furthermore, it introduces a new real-time probabilistic tractography algorithm. It was shown to perform very closely to a state of the art probabilistic algorithm, both qualitatively and quantitatively. This makes data exploration efficient, simple and fast. This is especially appropriate when you want to look at a smaller region in the brain, or know the effect of anomalies, like a tumor, lesion or oedema. This new tool will be particularly important to do quick quality validation imaging datasets, without the need to install any external software, which are sometimes restricted to a specific platform. Fiberweb aims to be a free to use quality assurance tool. You can see Fiberweb in action at:
LL has implemented both Fiberweb and the new RTT probabilistic algorithm presented by the paper. FM has participated in the development of Fiberweb. MC has helped generating the results regarding the new RTT algorithm. JH has participated in the development of Fiberweb. KW and MD supervised the project, helping design Fiberweb and its goals, as well as the new algorithm. All authors mentionned previously drafted or revised the article.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
We would like to thank Matthieu Dumont and Nil Goyette for their help with the logic behind the implementation of Fiberweb, and Jean-René Bélanger for his constant support. We would also like to thank the funding sources that made this work possible: Imeka, CRSNG, and FRQNT.
The Supplementary Material for this article can be found online at: