Human brain functional MRI and DTI visualization with virtual reality
Cite this article as: Chen B, Moreland J, Zhang J. Human brain functional MRI and DTI visualization with virtual reality. Quant Imaging Med Surg 2011;1:11-16. DOI: 10.3978/j.issn.2223-4292.2011.11.01
Original Article
Human brain functional MRI and DTI visualization with virtual reality
Bin Chen1, John Moreland2, Jingyu Zhang1
1Department of Engineering, Purdue University Calumet, Hammond, IN 46323; 2CIVS, Purdue University Calumet, Hammond, IN 46323
Corresponding to: Bin Chen,PhD. Department of Engineering, Purdue University Calumet, Hammond, IN 46323. Tel/Fax:219-989-2486. Email:chen121@purduecal.edu.
Abstract
Magnetic resonance diffusion tensor imaging (DTI) and functional MRI (fMRI) are two active research areas in neuroimaging. DTI is sensitive to the anisotropic diffusion of water exerted by its macromolecular environment and has been shown useful in characterizing structures of ordered tissues such as the brain white matter, myocardium, and cartilage. The diffusion tensor provides two new types of information of water diffusion: the magnitude and the spatial orientation of water diffusivity inside the tissue. This information has been used for white matter fiber tracking to review physical neuronal pathways inside the brain. Functional MRI measures brain activations using the hemodynamic response. The statistically derived activation map corresponds to human brain functional activities caused by neuronal activities. The combination of these two methods provides a new way to understand human brain from the anatomical neuronal fiber connectivity to functional activities between different brain regions. In this study, virtual reality (VR) based MR DTI and fMRI visualization with high resolution anatomical image segmentation and registration, ROI definition and neuronal white matter fiber tractography visualization and fMRI activation map integration is proposed. Rationale and methods for producing and distributing stereoscopic videos are also discussed.
Key words
Human brain visualization; stereoscopic 3D visualization
Quant Imaging Med Surg 2011;1:11-16. DOI: 10.3978/j.issn.2223-4292.2011.11.01
Introduction
Human Brain DWI and DTI Visualization
Methods for visualizing the human brain have evolved in recent years. Traditional methods of visualizing volume brain data include functionality for segmenting volumetric head data into brain, skull, etc., viewing 2-D slices and rendering and manipulating surface models (1). The other dominant visualization method has been volume rendering, including volume texture mapping, splatting, and other techniques (2). More recent work has explored the use of streamlines and streamtubes for visualizing brain fibers (3,4).
The principles of diffusion tensor imaging (DTI) (5-8) involve the acquisition of diffusion-weighted images sensitized in various gradient directions, with one or more encoding levels in each direction, followed by pixel-by-pixel calculation and diagonalization of the diffusion tensor (9,10). The diffusion tensor provides two new types of information of water diffusion: the magnitude and the spatial orientation of water diffusivity inside the tissue. Whereas brain gray matter has weak anisotropy in tensor orientations, myelinated axons have shown strong tensor orientation (11-13). The anisotropic information, therefore, is exploited by a new technique called DTI fiber tracking or tractography to determine neuronal fiber bundles in a noninvasive manner which opens a new way to assess white matter development and pathology (14).
Functional BOLD Signal and Brain Activation Maps
During brain activation, metabolic activities in certain regions increase and consequently require more oxygen and glucose (15,16). The cerebral blood volume (CBV) and cerebral blood flow (CBF) increase correspondingly in order to remedy the deficit of oxygen and glucose. Because the deoxyhemoglobin has paramagnetic susceptibility, it induces magnetic field changes in the intravascular and extravascular spaces and increases T2*. The susceptibility-related intravoxeldephasing decreases and the spin coherence increases, resulting in increased MRI signal intensity (17-19). The functional contrast on the blood oxygen leveldependent (BOLD) mechanism can be statistically calculated with a linear model by multiple linear regression algorithms. The regions with higher significance scores will be treated as activated regions. The calculation of brain activation can be parallelized to dramatically reduce the computing time (20,21).
Human Brain Diffusion Tensor and Activation Map Visualization
Methods for visualizing the human brain have evolved in recent years (22-26). Traditional methods of visualizing volume brain data include functionality for segmenting volumetric head data into brain, skull, etc., viewing 2-D slices and rendering and manipulating surface models. The other dominant visualization method have been volume rendering, including volume texture mapping, splatting, and other techniques. More recent work has explored the use of tensor visualization instead of traditional streamlines and streamtubes for brain fiber visualization (27-31).
In addition to visualization techniques, display technology and virtual reality has also evolved on multiple fronts including stereoscopic displays, head mounted displays, and immersive projection systems like the CAVE and its derivatives (31). Some attempts have been made to use these types of displays in presurgical planning (32,33). However, many of these efforts are highly specialized and there are still gaps in research involving brain visualization on virtual reality systems for basic research, education and presurgical planning. In this study, virtual reality (VR) based MR DTI and fMRI visualization with high resolution anatomical image segmentation and registration, ROI definition and neuronal white matter fiber tractography visualization and fMRI activation map integration is proposed as well as a new function for remote stereoscopic 3D visualization. The main purpose of the application is for basic research, clinical applications and education. This work builds on previously proposed developments in DTI with VR (34).
Methods
DTI Data Acquisition
The diffusion tensor imaging data were acquired using a singleshot diffusion-weighted SE EPI sequence with repetition time TR=2000 ms, time of echo TE=61.6 ms, field-of-view (FOV)=25.6 cm, matrix size=128×128, slice thickness=2 mm, an axial slice orientation and left/right as the frequency encoding direction. To reduce the acquisition time and signal to noise ratio, a SENSE reduction factor of 2, ramp sampling, and 5/8 partial Fourier encoding were used to minimize the readout window and TE, and hence the geometric distortions and signal loss due to T2 relaxation. Diffusion-weighting was applied along 15 non-coplanar directions evenly distributed over a sphere with a b-factor of 800 s/mm2, and 10 averages were used to increase the SNR.
The B0 field map data were collected for geometric distortion correction using identical TR, TE, FOV, slice thickness, and diffusion-weighting scheme with lower resolution and multiple echoes. All B0 maps were acquired on a 20 cm diameter spherical uniform phantom, whereas only a non-diffusion-weighted B0 map was acquired in vivo.
The diffusion tensor was then computed back from the corrected diffusion weighted images with theLevenberg- Marquardt nonlinear fitting algorithm. The tensors were decomposed into eigenvectors and eigenvalues for DTI fiber tracking.
The DTI and fMRI experiments were conducted on a 3T Tesla General Electric SignaHDx MRI scanner (GE Healthcare, Milwaukee, WI) at Purdue University Research Park. The Institutional Review Board (IRB) was obtained through Purdue University for the research involving human subjects. The datasets were crosschecked with the datasets acquired from other MRI scanners (Philips or Siemens scanners) in DTI, stimulus fMRI experiments with identical image acquisition parameters.
Functional MRI data acquisition and activation map computation
Total acquisition time was approximately 2 hours including the preparation time with 6 runs in a session.
(i)     Acquire a high-resolution whole brain image using a T1 weighted MPRAGE sequence with 1 × 1 × 1 mm voxel size.
(ii)    Acquire images using a single shot EPI sequence with TR=2 s, TE=30 ms, matrix size 64 × 64, voxel size: 3mm × 3mm × 5mm, and slice thickness=5 mm (no gap). The number of slice is approximately 28 for the whole brain coverage.
(iii)   Monitor the physiological parameters, respiratory and cardiac signals as well as head.
(iv)   movement during functional data collections. Block design of rotating checkboxes with 6 runs for fMRI experiments with stimulus.
Block design of rotating checkboxes with 6 runs for fMRI experiments with stimulus.
The functional MRI data were processed with the following steps:
(i)     Image re-alignment and head motion correction
(ii)    Spatial normalization and smoothing
(iii)   Statistical model specification
(iv)   Activation map generation and mapping
Activation maps were calculated with Statistical Parametric Mapping (SPM).
Virtual Reality and Human Brain Visualization
Several different software packages were used in the initial development of the various brain models and the VR application. Initially, surface models were exported as VRML and converted to OpenSceneGraph for integration with the VR application (35,36). The surface models were then registered to fit within a volume rendered brain and skull. The volume rendering was produced with osgVolume, using DICOM data.The virtual reality application was developed using OpenSceneGraph for both surface models of the segmented brain data and volume rendering of composite brain. The application was originally developed for a two-screen immersive projection system as seen in Figure 1. The system included 7'6"x10' screens, using passive projection and infitec glasses for stereoscopic 3-D effect. The system also utilized an Intersense IS-900 6-DOF tracking system with head-tracker and wand (37).
In this iteration of the software, using the wanda user could select individual segments of the brain and display both the appropriate surface model (enclosed within the partially opaque volume rendering of the composite brain) along with the correct brain segment name. Selection was done using raycasting along with some OpenSceneGraph functions (38).
Eventually, Avizosoftwarewas used to develop additional functions in this project so that Purdue Calumet and Purdue West Lafayette share the same framework for visualization (39). Instead of rendering pre-calculated fibers in low accuracy, the request was passed to a Linux cluster for on-demand neuronal tracking and functional connectivity for high accuracy and flexibility.The quality of calculated neuronal fiber tracks were evaluated using the Fréchet distance against the standard MRI templates. Well studied neuronal fiber bundles such as uncinate fasciculus, superior longitudinal fasciculus and corpus callosum were used for validation.
Figure 1. The configuration of the VR system for the initial developments of the software.
Results
Figure 2A shows the transparent and volume rendered human brain with the color coded corpus callosum with the “see through” effect as well as the intersected 2D sagittal slice. Figure 2B shows the sliding slice inside the brain and can be served as a reference image. These visualization technique can be displayed at the same time.
Figure 3A is surface rendered deskull brain with color coded segmented regions. Figure 3B and 3C show views of the brain after “taking off ” the superior frontal gyrus (SFG), superior temporal gyrun (STG), middle temporal gyrun (MTG) and postcentral gyrus in the parietal lobe.
The remote stereoscopic 3D visualization is demonstrated with video samples on YouTube 3D. Video is initially captured from Avizo in a Side-by-Side format (using Avizo's built-in stereoscopic viewing functionality). The aspect ratio of the video needs to be corrected to accommodate Youtube's 3D format requirements.
Figure 2. (A) Human brain volume rendering and color coded region visualization. The 2D sagittal slice provides additional reference views. (B) The slice can slide inside of the brain and can be turned on or off by users.
Figure 3. A: The surface rendered deskull brain with color coded segmented regions; B/C: the views of the brain after “taking off” the superior frontal gyrus (SFG), superior temporal gyrun (STG), middle temporal gyrun (MTG) and postcentral gyrus in the parietal lobe.
Discussion
The benefit of stereoscopic viewing for medical applications has been discussed in recent publications (40,41). However a widespread adoption of stereoscopic viewing of 3-D data across medical domains is still lacking. We have adopted a workflow that provides the option of stereoscopic viewing to the user regardless of their display capabilities. It is then at the user's discretion whether to view monoscopically or stereoscopically.
While most 3D visualization packages, such as Avizo, provide functionality that allows the display to be output in stereoscopic 3D, this functionality still depends on the end-user possessing a stereoscopic display. Additionally, if one wishes to share visualizations with someone who doesn't have the Avizo software, there are limited options for stereoscopic content sharing. Generating anaglyph for colored glasses provides a quick way to distribute 3D contents on traditional displays. With the rise of 3DTV and increased supply of stereoscopic content in popular culture (42), an additional method of stereoscopic deployment now available through Youtube's 3D content capabilities (43). Due to the in-progress nature of Youtube's 3D functionality however, the methods used to create and distribute videos from Avizo has changed and may change again as the stereoscopic community provides additional feedback to the Youtube 3D developer.
In conclusion, an immersive visualization application for MR DTI, neuronal white matter fiber tractography and fMRI activation map visualization have been proposed and implemented. Visualization techniques include surface modeling, volume rendering, and streamlines or streamtubes. The application will have potential applications in basic research, education, and surgical planning. Continued development is needed to implement the tensor visualization, 3D stereoscopy and interactive visualization.
Acknowledgment
The authors would like to thank the support from the Center for Innovation through Visualization and Simulation (CIVS) at Purdue University Calumet during the course of this work.
References
  • Ranken D, George J. MRIVIEW: An interactive computational tool for investigation of brain structure and function. Proceedings of IEEE Visualization 1993. San Jose, CA.
  • Valentino DJ, Mazziotta JC, Huang HK. Volume rendering of multimodal images: application to MRI and PET imaging of the human brain. IEEE Trans Med Imaging 1991;10:554-62.[LinkOut]
  • Jianu R, Demiralp C, Laidlaw DH. Exploring 3D DTI fiber tracts with linked 2D representations. IEEE Trans Vis Comput Graph 2009;15:1449- 56.[LinkOut]
  • Mori S, van Zijl PC. Fiber tracking: principles and strategies - a technical review. NMR Biomed 2002;15:468-80.[LinkOut]
  • Chenevert TL, Brunberg JA, Pipe JG. Anisotropic diffusion in human white matter: demonstration with MR techniques in vivo. Radiology 1990;177:401-5.[LinkOut]
  • Moseley ME, Cohen Y, Kucharczyk J, et al. Diffusion-weighted MR imaging of anisotropic water diffusion in cat central nervous system. Radiology 1990;176:439-45.[LinkOut]
  • Basser PJ, Jones DK. Diffusion-tensor MRI: theory, experimental design and data analysis - a technical review. NMR Biomed 2002;15:456-67.[LinkOut]
  • Mori S, Zhang J. Principles of diffusion tensor imaging and its applications to basic neuroscience research. Neuron 2006;51:527-39.[LinkOut]
  • Jones DK, Basser PJ. "Squashing peanuts and smashing pumpkins": how noise distorts diffusion-weighted MR data. Magn Reson Med 2004;52:979- 93.[LinkOut]
  • Anderson AW. Theoretical analysis of the effects of noise on diffusion tensor imaging. Magn Reson Med 2001;46:1174-88.[LinkOut]
  • Poonawalla AH, Zhou XJ. Analytical error propagation in diffusion anisotropy calculations. J Magn Reson Imaging 2004;19:489-98.[LinkOut]
  • Skare S, Li T, Nordell B, et al. Noise considerations in the determination of diffusion tensor anisotropy. Magn Reson Imaging 2000;18:659-69.[LinkOut]
  • Chen B, Hsu EW. Noise removal in magnetic resonance diffusion tensor imaging. Magn Reson Med 2005;54:393-401.[LinkOut]
  • Basser PJ, Pajevic S, Pierpaoli C, et al. In vivo fiber tractography using DTMRI data. Magn Reson Med 2000;44:625-32.[LinkOut]
  • Detre JA, Wang J. Technical aspects and utility of fMRI using BOLD and ASL. ClinNeurophysiol 2002;113:621-34.[LinkOut]
  • Lee SP, Silva AC, Kim SG. Comparison of diffusion-weighted highresolution CBF and spin-echo BOLD fMRI at 9.4 T. MagnReson Med 2002;47:736-41.[LinkOut]
  • Michelich CR, Song AW, Macfall JR. Dependence of gradient-echo and spin-echo BOLD fMRI at 4 T on diffusion weighting. NMR Biomed 2006;19:566-72.[LinkOut]
  • Lu H, van Zijl PC, Hendrikse J, et al. Multiple acquisitions with global inversion cycling (MAGIC): a multislice technique for vascular-spaceoccupancy dependent fMRI. MagnReson Med 2004;51:9-15.[LinkOut]
  • Krüger G, Kastrup A, Glover GH. Neuroimaging at 1.5 T and 3.0 T: comparison of oxygenation-sensitive magnetic resonance imaging. MagnReson Med 2001;45:595-604.[LinkOut]
  • Gangstead SL, Song AW. On the timing characteristics of the apparent diffusion coefficient contrast in fMRI. MagnReson Med 2002;48:385-8.[LinkOut]
  • Detre JA, Wang J. Technical aspects and utility of fMRI using BOLD and ASL. ClinNeurophysiol 2002;113:621-34.[LinkOut]
  • Ranken D, George J. MRIVIEW: An interactive computational tool for investigation of brain structure and function. Proceedings of IEEE Visualization 1993. San Jose, CA.
  • Valentino DJ, Mazziotta JC, Huang HK. Volume rendering of multimodal images: application to MRI and PET imaging of the human brain. IEEE Trans Med Imaging 1991;10:554-62.[LinkOut]
  • Jianu R, Demiralp C, Laidlaw DH. Exploring 3D DTI fiber tracts with linked 2D representations. IEEE Trans Vis Comput Graph 2009;15:1449- 56.[LinkOut]
  • Mori S, van Zijl PC. Fiber tracking: principles and strategies - a technical review. NMR Biomed 2002;15:468-80.[LinkOut]
  • Kitamura Y, Nakashima T, Tanaka K, et al. The IllusionHole for Medical Applications. Proceedings of IEEE VR 2007. Charlotte, NC.
  • Nowinski, WL. Virtual reality in brain intervention: models and applications. 27th Annual International Conference on Engineering in Medicine and Biology 2005, Shanghai.[LinkOut]
  • Reitinger B, Schmalstiegm D, Bornik A, et al. Spatial analysis tools for virtual reality-based surgical planning. Proceedings of IEEE 3DUI 2006.
  • Smith SM. Fast robust automated brain extraction. Hum Brain Mapp 2002;17:143-55.[LinkOut]
  • Tu Z, Narr KL, Dollar P, et al. Brain anatomical structure segmentation by hybrid discriminative/generative models. IEEE Trans Med Imaging 2008;27:495-508.[LinkOut]
  • Cruz-Neira, C. Projection{based Virtual Reality: The CAVE and its Applicationsto Computational Science, PhD thesis, University of Illinois at Chicago,1995.
  • vanBeurden MHPH, van Hoey G, Hatzakis H, et al. Stereoscopic Displays in Medical Domains: A Review of Perception and Performance Effects. In Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 7240:8.
  • Chan HP, Goodsitt MM, Helvie MA, et al. ROC study of the effect of stereoscopic imaging on assessment of breast lesions. Med Phys 2005;32:1001-9.[LinkOut]
  • Chen B, Moreland J. Human Brain Diffusion Tensor Imaging Visualization With Virtual Reality. ASME WINVR 2010 Conference Proceedings 2010;p137-43.
  • Available online:http://www.openscenegraph.org[LinkOut]
  • Bierbaum A, Just C, Hartling P, et al. VR Juggler: a virtual platform for virtual reality application development. Proceedings of IEEE VR 2001;17:89-96.
  • Available online: http://www.intersense.com[LinkOut]
  • Poupyrev I, Weghorst S, Billinghurst M, et al. A framework and testbed for studying manipulation techniques for immersive VR. In Proceedings of the ACM symposium on Virtual reality software and technology 1997;p21–8.
  • Available online: http://www.avizo.com[LinkOut]
  • Birkfellner W, Figl M, Matula C, et al. Computer-enhanced stereoscopic vision in a head-mounted operating binocular. Phys Med Biol 2003;48:N49-57.
  • Ilgner J, Park JJH, Labbé D, et al. Using a high-definition stereoscopic video system to teach microscopic surgery.Stereoscopic Displays and Virtual Reality Systems XIV, 6490:649008.
  • Seel PB. Digital TV in 3D: a passing fad or the next step in the evolution of digital television? International Journal of Digital Television2010;1:309-25.[LinkOut]
  • 3D Content - Youtube Help (2011). Retrieved March 1, 2011.Available online: http://www.google.com/support/youtube/bin/answer. py?answer=157640[LinkOut]

Refbacks

  • There are currently no refbacks.