Immersive and interactive virtual reality to improve learning and retention of neuroanatomy in medical students: a randomized controlled study
==============================================================================================================================================

* Chelsea Ekstrand
* Ali Jamal
* Ron Nguyen
* Annalise Kudryk
* Jennifer Mann
* Ivar Mendez

## Abstract

**Background:** Spatial 3-dimensional understanding of the brain is essential to learning neuroanatomy, and 3-dimensional learning techniques have been proposed as tools to enhance neuroanatomy training. The aim of this study was to examine the impact of immersive virtual-reality neuroanatomy training and compare it to traditional paper-based methods.

**Methods:** In this randomized controlled study, participants consisted of first- or second-year medical students from the University of Saskatchewan recruited via email and posters displayed throughout the medical school. Participants were randomly assigned to the virtual-reality group or the paper-based group and studied the spatial relations between neural structures for 12 minutes after performing a neuroanatomy baseline test, with both test and control questions. A postintervention test was administered immediately after the study period and 5-9 days later. Satisfaction measures were obtained.

**Results:** Of the 66 participants randomly assigned to the study groups, 64 were included in the final analysis, 31 in the virtual-reality group and 33 in the paper-based group. The 2 groups performed comparably on the baseline questions and showed significant performance improvement on the test questions following study. There were no significant differences between groups for the control questions, the postintervention test questions or the 7-day postintervention test questions. Satisfaction survey results indicated that neurophobia was decreased.

**Interpretation:** Results from this study provide evidence that training in neuroanatomy in an immersive and interactive virtual-reality environment may be an effective neuroanatomy learning tool that warrants further study. They also suggest that integration of virtual-reality into neuroanatomy training may improve knowledge retention, increase study motivation and decrease neurophobia.

Spatial understanding of neuroanatomy is essential to neurologic-based medical and surgical specialties, including neurology, neurosurgery and psychiatry.1 A comprehensive understanding of neuroanatomy is reliant on thorough knowledge of the intricate relations among multiple 3-dimensional structures.2 Traditionally, methods of learning neuroanatomy have included extended teaching time and supplementation through anatomic dissection and histological slides to examine the spatial 3-dimensional relations between structures.3 Time constraints and desire for cost-effectiveness have prompted medical schools to reduce anatomy and laboratory teaching time with human cadaveric specimens,3-6 the main supplementation material being textbooks and 2-dimensional images.3,7 These restrictions may influence a person's ability to translate 2-dimensional to 3-dimensional spatial relations, as it requires the student to perform complex cognitive reconstructions,8,9 potentially impairing neuroanatomical learning. A previous study showed a dramatic decrease in neuroanatomy recall among graduating medical students,10 and other studies indicated that medical graduates' overall anatomic competence did not meet safe practising level.11-14 One potential factor contributing to this is a phenomenon coined "neurophobia," a perceived reluctance of medical students to learn or relearn neuroanatomy.15-19 In an effort to diminish neurophobia and improve spatial and 3-dimensional neuroanatomy learning, 3-dimensional strategies have been proposed.3,20-22 Several studies have shown that 3-dimensional neuroanatomical learning is an effective strategy for increasing neuroanatomical knowledge, motivation and retention of neuroanatomy material.2,22-26 Furthermore, participants improved in their knowledge of spatial relations when they were exposed to both physical11 and virtual25-27 3-dimensional brain models.

Virtual-reality technology may be a logical next step for enhanced 3-dimensional and interactive learning. Few studies have investigated the efficacy of immersive virtual-reality environments on neuroanatomy training. Kockro and colleagues26 found significantly better performance at test for a virtual-reality study group than for a group that used a 2-dimensional PowerPoint presentation; however, their study did not include a pretest or retention test. Armstrong and colleagues28 developed an immersive, interactive virtual-reality environment and found that this system was qualitatively effective and user-friendly, but they did not quantitatively examine participants' performance. In the current study, we sought to explore the efficacy and limitations of immersive and interactive virtual reality on neuroanatomy learning. Furthermore, we aimed to examine neuroanatomical knowledge retention after virtual-reality learning. We hypothesized that study in the virtual-reality environment would lead to learning and knowledge retention outcomes regarding the spatial relations of neural structures at least equivalent, or potentially superior, to those acquired via paper-based study.

## Methods

### Design and participants

Data collection for this study was conducted between Mar. 11 and Mar. 25, 2017. We used a randomized controlled design to compare virtual-reality learning to traditional paper-based learning. Using an independent samples 2-tailed *t* test (5% significance level), we calculated a sample size of 64 participants (32 in either group) to provide a statistical power of 0.90. A 10% (standard deviation 12%) higher score on the postintervention test for the virtual-reality group was the primary outcome, which constitutes an increase in learning outcome equivalent to 1 letter grade.

Participants consisted of first- or second-year medical students from the University of Saskatchewan. Potential participants were recruited via emails sent to all first- and second-year medical students and posters displayed throughout the medical school inviting interested students to contact an experimenter.

### Materials

Both the virtual-reality and paper-based study materials contained the following labelled brain structures: the caudate, putamen, globus pallidus, thalamus, ventricles, amygdala, hippocampus, lateral corticospinal tract and spinothalamic tract.

For the virtual-reality brain, we created structures from T1-weighted magnetic resonance images of a healthy person's brain using FreeSurfer software ([http://surfer.nmr.mgh.harvard.edu/](http://surfer.nmr.mgh.harvard.edu/)) and DSI Studio ([http://dsi-studio.labsolver.org](http://dsi-studio.labsolver.org)) for the white matter tracts and compiled them for the virtual-reality environment in Blender ([https://www.blender.org](https://www.blender.org)). In the virtual-reality environment, visualization of the different structures and their labels was controlled by the participant, which allowed him or her to navigate and examine the 3-dimensional relations between the structures (see Appendix 1, available at [www.cmajopen.ca/content/6/1/E103/suppl/DC1](http://www.cmajopen.ca/content/6/1/E103/suppl/DC1), for an example of the user interface and virtual-reality environment). We used Vive (HTC Corporation) as the virtual-reality system. The virtual-reality neuroanatomy interface was developed by Sprockety Ventures ([https://www.sprockety.com/](https://www.sprockety.com/)).

The paper-based group was provided with a booklet containing 15 colour figures adapted to display labels for the relevant structures to be studied from various views and orientations from the second edition of Blumenfeld's *Neuroanatomy Through Clini**cal Cases*,29 a textbook commonly used to teach neuroanatomy.

We developed 22 pretest and 22 posttest multiple-choice questions designed to assess the participants' ability to visualize relations of structures in 3 dimensions (Appendix 1). Both the baseline and postintervention test contained 13 test questions (i.e., related to the study materials) and 9 control questions (containing neuroanatomy content not available from the study materials), resulting in a total of 22 questions. We included control questions were to ensure that the 2 groups did not differ in prior neuroanatomy knowledge. Two neuroanatomy experts at the University of Saskatchewan reviewed the questions before implementation; all were considered valid, accurate and curriculum relevant.

We also developed a satisfaction survey containing 25 questions to assess participant satisfaction with the learning experience (Appendix 1).

### Procedure

Participants were randomly assigned to the virtual-reality or paper-based group in blocks of 4 with the use of an online randomization tool (Figure 1). Both groups were given 10 minutes to complete the baseline test to assess their knowledge of neuroanatomy. Participants in the virtual-reality group were then given a 5-minute tutorial by the experimenter to help them navigate the virtual-reality system. The tutorial contained a simple, unlabelled model of a human brain, consisting of an outer shell and 2 inner shells to simulate the cortex and putamen. Participants were instructed on how to turn on and off the brain structures to ensure sufficient proficiency in virtual-reality navigation.

![Figure 1](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/1/E103/F1.medium.gif)

[Figure 1](http://www.cmajopen.ca/content/6/1/E103/F1)

Figure 1 
Flow chart showing experimental procedure.

Participants in the virtual-reality group were then presented with the labelled study brain and given 12 minutes to memorize the spatial relations between the different structures. In a separate room, those in the paper-based group were provided with the booklet of adapted figures and were also given 12 minutes of study time. Individual study time was not recorded; however, participants were encouraged by the experimenter to study for the duration of the study period.

All participants were then given 10 minutes to complete a postintervention test to assess immediate information retention.

Demographic information was then obtained (age, sex, neuroanatomy experience), and the satisfaction survey was administered.

A further postintervention test (7-d postintervention) was administered 5-9 days after the original intervention for both the baseline and postintervention questions (44 in total); participants had 20 minutes to respond. Participants were asked to refrain from studying neuroanatomy between the 2 postintervention tests.

Before analysis, a researcher blind to participant responses assessed the test questions to ensure that each question could be answered from the study material. Two questions were identified as problematic and were removed. The resolution of the T1-weighted magnetic resonance image from which the virtual-reality structures were developed limited segmentation of the entire caudate tail, and therefore the question "The caudate tail is \_\_\_|\\_\_\_| to the thalamus" yielded ambiguous results. In addition, the label for the subthalamic nucleus in the virtual-reality environment malfunctioned at test, requiring removal of "In advanced Parkinson's disease, the subthalamic nucleus can be stimulated by deep brain stimulation; this structure lies \\_\_\_|\\_\__| to the putamen." Thus, 12 questions were included in both the baseline and postintervention tests.

### Statistical analysis

Participants' scores were converted to percentage correct by a researcher blind to experimental group. We used IBM SPSS Statistics version 24 to manage, aggregate and analyze the data.

As this study used both within- and between-subject factors, the primary analysis conducted was a 4 (Test; Baseline, Postintervention, Baseline Questions 7 Days Postintervention, Postintervention Questions 7 Days Postintervention) × 2 (Question type; Test, Control) × 2 (Group; Virtual-reality, Paper-based) mixed-measures analysis of variance (ANOVA) on Percent Correct, with Test and Question type as within-subject factors and Group as a between-subject factor.

To address our primary research question (i.e., whether virtual reality would lead to equivalent or superior learning outcomes compared to paper-based study methods), we conducted two 4 (Test; Baseline, Postintervention, Baseline Questions 7 Days Postintervention, Postintervention Questions 7 Days Postintervention) × 2 (Group; Virtual reality, Paper-based) mixed-measures ANOVAs for the Test and Control questions. We used independent samples *t* tests to further examine these effects.

### Ethics approval

This study was approved by the Research Ethics Board at the University of Saskatchewan.

## Results

Thirty-three participants were randomly assigned to either group. One participant in the virtual-reality group was excluded from analysis because of a methodological error in administering the tests, and 1 participant, also in the virtual-reality group, dropped out of the study, resulting in 64 participants in the final analysis, 31 in the virtual-reality group and 33 in the paper-based group.

The demographic characteristics of the participants are given in Table 1. The 2 groups did not differ significantly in age or number of semesters of neuroanatomy (independent *t* tests) or in medical school year, sex, previous neuroanatomy experience or study strategy (χ2 tests). The first- and second-year students did not differ significantly on baseline neuroanatomy knowledge (*F*(1, 62) = 1.095, mean squared error = 257.1, *p* = 0.3).

View this table:
[Table 1:](http://www.cmajopen.ca/content/6/1/E103/T1)

Table 1:   Demographic characteristics, previous neuroanatomy experience and study preferences of participants, by group*

For the primary analysis, we found significant main effects of Test and Question type and a significant Test × Question type interaction (Table 2). The main effect for Group was not significant, nor were the Test × Group and Question type × Group or Test × Question type × Group interactions.

View this table:
[Table 2:](http://www.cmajopen.ca/content/6/1/E103/T2)

Table 2:   Results of omnibus analysis of variance

The results for the mixed-measures ANOVAs as a function of question type are as follows. For the Test questions, we found significant main effects of Test and Group (Table 3). The Test × Group interaction was not significant. Independent samples *t* tests showed no significant difference between groups for baseline questions, postintervention questions or 7-day postintervention questions (Table 4). The difference between groups for baseline questions 7 days postintervention was significant (see Figure 2 for means and 95% confidence intervals30).

![Figure 2](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/1/E103/F2.medium.gif)

[Figure 2](http://www.cmajopen.ca/content/6/1/E103/F2)

Figure 2 
Test question Percent Correct for the paper-based (95% confidence interval [CI] ± 5.112) and virtual-reality (95% CI ± 4.849) groups at each testing point. Error bars represent CIs. See Appendix 1 for mean scores.

View this table:
[Table 3:](http://www.cmajopen.ca/content/6/1/E103/T3)

Table 3:   Results of mixed-measures analysis of variance as a function of question type

View this table:
[Table 4:](http://www.cmajopen.ca/content/6/1/E103/T4)

Table 4:   Results of independent samples *t* tests as a function of question type comparing the accuracy of the paper-based and virtual-reality groups for each condition

For the Control questions, there were no significant main effects of Test or Group, which suggests that participants in the 2 groups performed equally well (Table 3). The Test × Group interaction was not significant. Independent samples *t* tests showed no significant differences between groups for any condition (Table 4, Figure 3).

![Figure 3](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/1/E103/F3.medium.gif)

[Figure 3](http://www.cmajopen.ca/content/6/1/E103/F3)

Figure 3 
Control question Percent Correct for the paper-based (95% confidence interval [CI] ± 5.314) and virtual-reality (95% CI ± 6.599) groups at each testing point. Error bars represent CIs. See Appendix 1 for mean scores.

### Satisfaction

Of particular interest, when asked to rate on a scale of 1 to 5 (where 1 = strongly disagree and 5 = strongly agree) the statement "This method should be used in the curriculum," 29 participants (94%) in the virtual-reality group strongly agreed or agreed, compared to 11 participants (33%) in the paper-based group. For "I feel less afraid with the complexity of neuroanatomy," the corresponding values were 25 (81%) and 4 (12%), which suggests decreased neurophobia following virtual-reality learning.

## Interpretation

Our results provide evidence that learning neuroanatomy in an immersive and interactive virtual-reality environment may be an effective learning tool that warrants further study. In contrast to the control questions, for which no significant differences were found between the tests or the groups, both groups showed significant improvement in scores on the test questions after the intervention that persisted at 7 days. This provides evidence that both methods were successful learning techniques. The fact that a main effect of group was found for the test questions, with the virtual-reality group having significantly greater accuracy than the paper-based group, suggests that virtual reality is an effective learning tool for understanding the complex spatial relations between different structures of the brain. These findings are relevant when evaluating immersive and interactive 3-dimensional virtual reality as a learning tool, since the primary goal of neuroanatomy learning is to obtain and maintain knowledge that is essential to medical practice.1

Based on both the quantitative results and the satisfaction survey results, there is evidence that virtual-reality technology may provide an effective supplemental tool for learning neuroanatomy by decreasing neurophobia and increasing knowledge retention. Our results are in concordance with those of studies using virtual 2- and 3-dimensional models25-27 and suggest that learning complex, 3-dimensional relations between neural structures is facilitated by training in a 3-dimensional virtual-reality environment. Our findings corroborate those of Armstrong and colleagues28 showing qualitative benefits of immersive and interactive virtual-reality compared to other methods and extends them into the quantitative domain. Practically, integration of virtual-reality technology into medical education may be beneficial to "self-taught" models of learning in that students may experience increased motivation to study and decreased neurophobia without learning outcomes' being compromised. This is supported by recent evidence from Stepan and colleagues31 that lecture-style virtual-reality neuroanatomy training increases motivation without compromising learning outcomes.

### Limitations

Our follow-up period was relatively short, and we used a limited number of questions to investigate knowledge retention. Future research should examine retention outcomes from virtual-reality neuroanatomy training over a longer time course with a greater amount of information to be learned. Furthermore, there is an inherent learning curve when learning a new technology, and the relatively short training and study period on the virtual-reality system may have decreased study time compared to the paper-based group, even with the training module. It is also possible that the virtual-reality group enjoyed a different level of motivation for the study materials owing to exposure to a novel environment, which may have contributed to their performance. However, the novelty and potential for increased motivation to learn is an asset for virtual-reality technology in the medical curriculum that may enhance students' incentive to study. Our results are generalizable only to a particular population. Future studies should examine other student populations, including undergraduate students and neurology and neurosurgery trainees. Furthermore, it may be interesting to examine how virtual-reality technology may aid in quantification of the actual spatial distance between structures, rather than just relative positions. It is also important to expand the number of structures and to integrate training of functional relations between structures into the virtual-reality environment, to examine whether the training benefits of virtual reality extend beyond an understanding of 3-dimensional relations of structures to grasping functional aspects of complex neural pathways that are difficult to visualize with 2-dimensional methods.32 Finally, future studies should perform a cost-benefit analysis of immersive and interactive virtual reality to further justify its integration into the medical curriculum.

### Conclusion

The results of this study suggest that integration of immersive and interactive virtual reality into neuroanatomy training may help to improve knowledge attainment and retention, increase motivation to study and decrease neurophobia. As immersive virtual-reality technology evolves, the feasibility of integrating this technology into medical curricula could be vastly improved. The potential applications and benefits of this technology may extend beyond undergraduate medical education into specialized neurologic-based fields, such as neurosurgery, making learners more prepared to navigate the complexities of the human brain in clinical practice. Virtual-reality technology is in its infancy in terms of its potential applications as a tool for facilitating medical education and clinical practice, and this potential and limitations warrant further evaluation and study.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/6/1/E103/suppl/DC1.](http://www.cmajopen.ca/content/6/1/E103/suppl/DC1.)

## Acknowledgements

Acknowledgement: The authors acknowledge Dr. Carl Gutwin (Professor of Computer Science, and Director of the Human-Computer Interaction Lab at the University of Saskatchewan) for his contributions to developing the structure of the virtual reality brain study.

## Footnotes

*   **Competing interests:** None declared.

*   **Contributors:** Ali Jamal, Ron Nguyen, Annalise Kudryk and Jennifer Mann acquired the data, and Chelsea Ekstrand analyzed and interpreted the data. Chelsea Ekstrand, Ali Jamal and Ron Nguyen drafted the manuscript. Ivar Mendez was the senior investigator for this study. All of the authors contributed to the study conception and design, critically revised the manuscript for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

*   **Data sharing:** Access to data from this study is available by contacting Chelsea Ekstrand at [chelsea.ekstrand](http://chelsea.ekstrand)@[usask.ca](http://usask.ca) and is subject to the University of Saskatchewan Data Sharing Policies.

*   **Funding:** This research was supported by an Alexander Graham Bell Canada Graduate Scholarship from the Natural Sciences and Engineering Research Council of Canada to Chelsea Ekstrand.

## References

1.  Hazelton L (2011) Changing concepts of neuroanatomy teaching in medical education. Teach Learn Med 23:359–64.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22004321&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

2.  Allen LK, Bhattacharyya S, Wilson TD (2015) Development of an interactive anatomical three‐dimensional eye model. Anat Sci Educ 8:275–82.
    
    

3.  Chang BS, Molnar Z (2015) Practical neuroanatomy teaching in the 21st century. Ann Neurol 77:911–6.
    
    

4.  Drake RL, McBride JM, Lachman N, et al. (2009) Medical education in the anatomical sciences: The winds of change continue to blow. Anat Sci Educ 2:253–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/ase.117&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19890982&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

5.  McLachlan JC, Bligh J, Bradley P, et al. (2004) Teaching anatomy without cadavers. Med Educ 38:418–24.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1046/j.1365-2923.2004.01795.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15025643&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000220272400011&link_type=ISI) 

6.  Gartner LP (2003) Anatomical sciences in the allopathic medical school curriculum in the United States between 1967-2001. Clin Anat 16:434–9.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12903066&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

7.  Allen LK, Eagleson R, de Ribaupierre S (2016) Evaluation of an online three‐dimensional interactive resource for undergraduate neuroanatomy education. Anat Sci Educ 9:431–9.
    
    

8.  Langlois J, Wells GA, Lecourtois M, et al. (2013) Sex differences in spatial abilities of medical graduates entering residency programs. Anat Sci Educ 6:368–75.
    
    

9.  Lufler RS, Zumwalt AC, Romney CA, et al. (2012) Effect of visual-spatial ability on medical students' performance in a gross anatomy course. Anat Sci Educ 5:3–9.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22127919&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

10. Mateen, D'Eon MF (2008) Neuroanatomy: a single institution study of knowledge loss. Med Tech 30:FJ, 537–9.
    
    

11. Estevez ME, Lindgren KA, Bergethon PR (2010) A novel three-dimensional tool for teaching human neuroanatomy. Anat Sci Educ 3:309–17.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20939033&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

12. Waterston SW, Stewart IJ (2005) Survey of clinicians' attitudes to the anatomical teaching and knowledge of medical students. Clin Anat 18:380–4.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/ca.20101&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15971223&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000230217000014&link_type=ISI) 

13. Older J (2004) Anatomy: a must for teaching the next generation. Surgeon 2:79–90.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S1479-666X(04)80050-7&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15568432&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000221047100003&link_type=ISI) 

14. Cottam WW (1999) Adequacy of medical school gross anatomy education as perceived by certain postgraduate residency programs and anatomy course directors. Clin Anat 12:55–65.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/(SICI)1098-2353(1999)12:1<55::AID-CA8>3.0.CO;2-O&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=9890730&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000077789000008&link_type=ISI) 

15. Zinchuk AV, Flanagan EP, Tubridy NJ, et al. (2010) Attitudes of US medical trainees towards neurology education: "neurophobia" - a global issue. BMC Med Educ 10:49.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1472-6920-10-49&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20573257&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

16. Flanagan E, Walsh C, Tubridy N (2007) "Neurophobia" - attitudes of medical students and doctors in Ireland to neurological teaching. Eur J Neurol 14:1109–12.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1468-1331.2007.01911.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17880566&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000249660900014&link_type=ISI) 

17. Abulaban AA, Obeid TH, Algahtani HA, et al. (2015) Neurophobia among medical students. Neurosciences 20:37–40.
    
    

18. McCarron MO, Stevenson M, Loftus AM, et al. (2014) Neurophobia among general practice trainees: the evidence, perceived causes and solutions. Clin Neurol Neurosurg 122:124–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.clineuro.2014.03.021&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24794168&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

19. Fantaneanu TA, Moreau K, Eady K, et al. (2014) Neurophobia inception: a study of trainees' perceptions of neurology education. Can J Neurol Sci 41:421–9.
    
    

20. Manson A, Poyade M, Rea P (2015) A recommended workflow methodology in the creation of an educational and training application incorporating a digital reconstruction of the cerebral ventricular system and cerebrospinal fluid circulation to aid anatomical understanding. BMC Med Imaging 15:44.
    
    

21. Panchaphongsaphak B, Burgkart R, Riener R (2007) Three-dimensional touch interface for medical education. IEEE Trans Inf Technol Biomed 11:251–63.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17521075&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

22. Ferdig R, Blank J, Kratcoski A, et al. (2015) Using stereoscopy to teach complex biological concepts. Adv Physiol Educ 39:205–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1152/advan.00034.2014&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26330039&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

23. Küçük S, Kapakin S, Göktas Y (2016) Learning anatomy via mobile augmented reality: effects on achievement and cognitive load. Anat Sci Educ 9:411–21.
    
    

24. Gould DJ, Terrell MA, Fleming J (2008) A usability study of users' perceptions toward a multimedia computer-assisted learning tool for neuroanatomy. Anat Sci Educ 1:175–83.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/ase.36&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19177405&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000207578600006&link_type=ISI) 

25. Brewer DN, Wilson TD, Eagleson R, et al. Evaluation of neuroanatomical training using a 3D visual reality model. In: Westwood JD, Westwood SW, Fellander-Tsai L, et al., editors. Medicine meets virtual reality 19. Amsterdam: IOS Press; 2012:85-91
    
    

26. Kockro RA, Amaxopoulou C, Killeen T, et al. (2015) Stereoscopic neuroanatomy lectures using a three-dimensional virtual reality environment. Ann Anat 201:91–8.
    
    

27. Allen LK, Eagleson R, Ribaupierre S (2016) Evaluation of an online three-dimensional interactive resource for undergraduate neuroanatomy education. Anat Sci Educ 9:431–9.
    
    

28. Armstrong R, de Ribaupierre S, Eagleson R (2014) A software system for evaluation and training of spatial reasoning and neuroanatomical knowledge in a virtual environment. Comput Methods Programs Biomed 114:29–37.
    
    

29. Blumenfeld H. Neuroanatomy through clinical cases. 2nd ed. Sunderland (MA): Sinauer Associates; 2010
    
    

30. Loftus GR, Masson ME (1994) Using confidence intervals in within-subject designs. Psychon Bull Rev 1:476–90.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.3758/BF03210951&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24203555&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1994PW15800008&link_type=ISI) 

31. Stepan K, Zeiger J, Hanchuk S, et al. (2017) Immersive virtual reality as a teaching tool for neuroanatomy. Int Forum Allergy Rhinol 7:1006–13.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/alr.21986&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

32. Calabresi P, Picconi B, Tozzi A, et al. (2014) Direct and indirect pathways of the basal ganglia: a critical reappraisal. Nat Neurosci 17:1022–30.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1038/nn.3743&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25065439&link_type=MED&atom=%2Fcmajo%2F6%2F1%2FE103.atom) 

*   Copyright 2018, Joule Inc. or its licensors