Publication Types:

Sort by year:

Ultra-high-fidelity virtual reality mastoidectomy simulation training—a randomized, controlled trial.

ArticleImmersive simulationMastoidectomyTemporal bone surgeryVR simulation
Frithioff A, Frendø M, Sørensen MS, Andersen SA.
Eur Arch Otorhinolaryngol. 2020 Feb 17 [Epub ahead of print].
Publication year: 2020

PURPOSE: Ultra-high-fidelity (UHF) graphics in virtual reality (VR) simulation might improve surgical skill acquisition in temporal bone training. This study aims to compare UHF VR simulation training with conventional, screen-based VR simulation training (cVR) with respect to performance and cognitive load (CL).

METHODS: In a randomized trial with a cross-over design, 24 students completed a total of four mastoidectomies in a VR temporal bone surgical simulator: two performances under UHF conditions using a digital microscope and two performances under conventional conditions using screen-based VR simulation. Performances were assessed by two blinded raters using an established assessment tool. In addition, CL was estimated as the relative change in secondary-task reaction time during simulation when compared with individual baseline measurements. Data were analyzed using linear mixed model analysis for repeated measurements.

RESULTS: The mean final-product performance score was significantly lower in UHF VR simulation compared to cVR simulation [mean difference 1.0 points out of 17 points, 95% CI (0.2–1.7), p = 0.02]. The most important factor for performance during UHF simulation was the ability to achieve stereovision (mean difference = 3.4 points, p < 0.001). Under the UHF VR condition, CL was significantly higher than during cVR (28% vs. 18%, respectively, p < 0.001).

CONCLUSION: UHF graphics in VR simulation training reduced performance and induced a higher CL in novices than conventional, screen-based VR simulation training. Consequently, UHF VR simulation training should be preceded by cVR training and might be better suited for the training of intermediates or experienced surgeons.

The effect of simulator-integrated tutoring for guidance in virtual reality simulation training

ArticleAssessmentDirected self-regulated learningInstructional designLearning supportsSurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Mikkelsen PT, Sørensen MS
Simul Healthc. 10 Jan 2020 [Epub ahead of print].
Publication year: 2020

INTRODUCTION: Simulation-integrated tutoring in virtual reality (VR) simulation training by green-lighting is a common learning support in simulation-based temporal bone surgical training. However, tutoring overreliance can negatively affect learning. We therefore wanted to investigate the effects of simulator-integrated tutoring on performance and learning.

METHODS: A prospective, educational cohort study of a learning intervention (simulator-integrated tutoring) during repeated and distributed VR simulation training for directed, self-regulated learning of the mastoidectomy procedure. Two cohorts of novices (medical students) were recruited: 16 participants were trained using the intervention program (intermittent simulator-integrated tutoring) and 14 participants constituted a non-tutored reference cohort. Outcomes were final-product performance assessed by two blinded raters, and simulator-recorded metrics.

RESULTS: Simulator-integrated tutoring had a large and positive effect on the final-product performance while turned on (mean difference 3.8 points, p<0.0001). However, this did not translate to a better final-product performance in subsequent non-tutored procedures. The tutored cohort had a better metrics-based score, reflecting higher efficiency of drilling (mean difference 3.6 %, p=0.001). For the individual metrics, simulator-integrated tutoring had mixed effects both during procedures and on the tutored cohort in general (learning effect).

CONCLUSIONS: Simulator-integrated tutoring by green-lighting did not induce a better final-product performance but increased efficiency. The mixed effects on learning could be caused by tutoring overreliance, resulting from a lack of cognitive engagement when the tutor-function is on. Further learning strategies such as feedback should be explored to support novice learning and cognitive engagement.

Understanding the effects of structured self-assessment in directed, self-regulated simulation-based training of mastoidectomy: a mixed methods study

ArticleCognitive loadInstructional designLearning supportsMastoidectomyOtologySurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Frendø M, Guldager M, Sørensen MS.
J Otol 2019 23 Dec [Epub ahead of print].
Publication year: 2019

OBJECTIVE: Self-directed training represents a challenge in simulation-based training as low cognitive effort can occur when learners overrate their own level of performance. This study aims to explore the mechanisms underlying the positive effects of a structured self-assessment intervention during simulation-based training of mastoidectomy.

METHODS: A prospective, educational cohort study of a novice training program consisting of directed, self-regulated learning with distributed practice (5×3 procedures) in a virtual reality temporal bone simulator. The intervention consisted of structured self-assessment after each procedure using a rating form supported by small videos. Semi-structured telephone interviews upon completion of training were conducted with 13 out of 15 participants. Interviews were analysed using directed content analysis and triangulated with quantitative data on secondary task reaction time for cognitive load estimation and participants’ self-assessment scores.

RESULTS: Six major themes were identified in the interviews: goal-directed behaviour, use of learning supports for scaffolding of the training, cognitive engagement, motivation from self-assessment, self-assessment bias, and feedback on self-assessment (validation). Participants seemed to self-regulate their learning by forming individual sub-goals and strategies within the overall goal of the procedure. They scaffolded their learning through the available learning supports. Finally, structured self-assessment was reported to increase the participants’ cognitive engagement, which was further supported by a quantitative increase in cognitive load.

CONCLUSIONS: Structured self-assessment in simulation-based surgical training of mastoidectomy seems to promote cognitive engagement and motivation in the learning task and to facilitate self-regulated learning.

The effect of structured self-assessment in virtual reality simulation training of mastoidectomy.

ArticleAssessmentDirected self-regulated learningInstructional designLearning supportsSurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Guldager M, Mikkelsen PT, Sørensen MS
Eur Arch Otorhinolaryngol. 2019;276(12):3345–3352.
Publication year: 2019

PURPOSE: Virtual reality (VR) simulation surgical skills training is well established, but self-directed practice is often associated with a learning curve plateau. In this study, we investigate the effects of structured self-assessment as a means to improve performance in mastoidectomy training.

METHODS: The study was a prospective, educational study. Two cohorts of novices (medical students) were recruited for practice of anatomical mastoidectomy in a training program with five distributed training blocks. Fifteen participants performed structured self-assessment after each procedure (intervention cohort). A reference cohort of another 14 participants served as controls. Performances were assessed by two blinded raters using a modified Welling Scale and simulator-recorded metrics.

RESULTS: The self-assessment cohort performed superiorly to the reference cohort (mean difference of final product score 0.87 points, p = 0.001) and substantially reduced the number of repetitions needed. The self-assessment cohort also had more passing performances for the combined metrics-based score reflecting increased efficiency. Finally, the self-assessment cohort made fewer collisions compared with the reference cohort especially with the chorda tympani, the facial nerve, the incus, and the malleus.

CONCLUSIONS: VR simulation training of surgical skills benefits from having learners perform structured self-assessment following each procedure as this increases performance, accelerates the learning curve thereby reducing time needed for training, and induces a safer performance with fewer collisions with critical structures. Structured self-assessment was in itself not sufficient to counter the learning curve plateau and for continued skills development additional supports for deliberate practice are needed.

Expert sampling of VR simulator metrics for automated assessment of mastoidectomy performance.

ArticleAssessmentMastoidectomyOtologyTemporal bone surgeryVR simulation
Andersen SA, Mikkelsen PT, Sørensen MS.
Laryngoscope. 2019;129(9):2170–2177
Publication year: 2019

OBJECTIVE: Often the assessment of mastoidectomy performance requires time-consuming manual rating. Virtual reality (VR) simulators offer potentially useful automated assessment and feedback but should be supported by validity evidence. We aimed to investigate simulator metrics for automated assessment based on the expert performance approach, comparison with an established assessment tool, and the consequences of standard setting.

METHODS: The performances of 11 experienced otosurgeons and 37 otorhinolaryngology residents. Participants performed three mastoidectomies in the Visible Ear Simulator. Nine residents contributed additional data on repeated practice in the simulator. One hundred and twenty-nine different performance metrics were collected by the simulator and final-product files were saved. These final products were analyzed using a modified Welling Scale by two blinded raters.

RESULTS: Seventeen metrics could discriminate between resident and experienced surgeons’ performances. These metrics mainly expressed various aspects of efficiency: Experts demonstrated more goal-directed behavior and less hesitancy, used less time, and selected large and sharp burrs more often. The combined metrics-based score (MBS) demonstrated significant discriminative ability between experienced surgeons and residents with a mean difference of 16.4% (95% confidence interval [12.6-20.2], P << 0.001). A pass/fail score of 83.6% was established. The MBS correlated poorly with the final-product score but excellently with the final-product score per time.

CONCLUSION: The MBS mainly reflected efficiency components of the mastoidectomy procedure, and although it could have some uses in self-directed training, it fails to measure and encourage safe routines. Supplemental approaches and feedback are therefore required in VR simulation training of mastoidectomy.

Decentralized Virtual Reality Training of Mastoidectomy Improves Cadaver Dissection Performance: A Prospective, Controlled Cohort Study

ArticleDirected self-regulated learningInstructional designMastoidectomySurgical trainingTemporal bone surgeryVR simulation
Frendø M, Cayé-Thomasen P, Konge L, Sørensen MS, Andersen SA.
Otol Neurotol. 2019 20 Dec [Epub ahead of print].
Publication year: 2019

OBJECTIVE: Virtual reality (VR) simulation training can improve temporal bone (TB) cadaver dissection skills and distributed, self-regulated practice is optimal for skills consolidation. Decentralized training (DT) at the trainees’ own department or home offers more convenient access compared with centralized VR simulation training where the simulators are localized at one facility. The effect of DT in TB surgical training is unknown. We investigated the effect of decentralized VR simulation training of TB surgery on subsequent cadaver dissection performance.

STUDY DESIGN: Prospective, controlled cohort study.

SETTING: Otorhinolaryngology (ORL) teaching hospitals and the Danish national TB course.

PARTICIPANTS: Thirty-eight ORL residents: 20 in the intervention cohort (decentralized training) and 18 in the control cohort (standard training during course).

INTERVENTION: Three months of access to decentralized VR simulation training at the local ORL department or the trainee’s home. A freeware VR simulator (the visible ear simulator [VES]) was used, supplemented by a range of learning supports for directed, self-regulated learning.

MAIN OUTCOME MEASURE: Mastoidectomy final-product scores from the VR simulations and cadaver dissection were rated using a modified Welling Scale by blinded expert raters.

RESULTS: Participants in the intervention cohort trained decentrally a median of 3.5 hours and performed significantly better than the control cohort during VR simulation (p < 0.01), which importantly also transferred to a 76% higher performance score during subsequent cadaver training (mean scores: 8.8 versus 5.0 points; p < 0.001).

CONCLUSIONS: Decentralized VR simulation training of mastoidectomy improves subsequent cadaver dissection performance and can potentially improve implementation of VR simulation surgical training.

Decentralized Virtual Reality Mastoidectomy Simulation Training: A Prospective, Mixed-Methods Study

ArticleDirected self-regulated learningInstructional designLearning supportsSurgical trainingTemporal bone surgeryVR simulation
Frendø M, Thinggaard E, Konge L, Sørensen MS, Andersen SA.
Eur Arch Otorhinolaryngol. 2019;276(10):2783–2789.
Publication year: 2019

PURPOSE: Virtual reality (VR) training of mastoidectomy is effective in surgical training-particularly if organized as distributed practice. However, centralization of practice facilities is a barrier to implementation of distributed simulation training. Decentralized training could be a potential solution. Here, we aim to assess the feasibility, use, and barriers to decentralized VR mastoidectomy training using a freeware, high-fidelity temporal bone simulator.

METHODS: In a prospective, mixed-methods study, 20 otorhinolaryngology residents were given three months of local access to a VR mastoidectomy simulator. Additionally, trainees were provided a range of learning supports for directed, self-regulated learning. Questionnaire data were collected and focus group interviews conducted. The interviews were analyzed using thematic analysis and compared with quantitative findings.

RESULTS: Participants trained 48.5 h combined and mainly towards the end of the trial. Most participants used between two and four different learning supports. Qualitative analysis revealed five main themes regarding implementation of decentralized simulation training: convenience, time for training, ease of use, evidence for training, and testing.

CONCLUSIONS: Decentralized VR training using a freeware, high-fidelity mastoidectomy simulator is feasible but did not lead to a high training volume or truly distributed practice. Evidence for training was found motivational. Access to training, educational designs, and the role of testing are important for participant motivation and require further evaluation.

Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial

ArticleCognitive loadDirected self-regulated learningImmersive simulationInstructional designSurgical trainingVR simulation
Frederiksen JG, Sørensen SM, Konge L, Svendsen MB, Nobel-Jørgensen M, Bjerrum F, Andersen SA.
Surg Endosc. 2019 Jun 6 [Epub ahead of print].
Publication year: 2019

BACKGROUND: Virtual reality simulators combined with head-mounted displays enable highly immersive virtual reality (VR) for surgical skills training, potentially bridging the gap between the simulation environment and real-life operating room conditions. However, the increased complexity of the learning situation in immersive VR could potentially induce high cognitive load thereby inhibiting performance and learning. This study aims to compare cognitive load and performance in immersive VR and conventional VR simulation training.

METHODS: A randomized controlled trial of residents (n = 31) performing laparoscopic salpingectomies with an ectopic pregnancy in either immersive VR or conventional VR simulation. Cognitive load was estimated by secondary-task reaction time at baseline, and during nonstressor and stressor phases of the procedure. Simulator metrics were used to evaluate performance.

RESULTS: Cognitive load was increased by 66% and 58% during immersive VR and conventional VR simulation, respectively (p < 0.001), compared to baseline. A light stressor induced a further increase in cognitive load by 15.2% and a severe stressor by 43.1% in the immersive VR group compared to 23% (severe stressor) in the conventional VR group. Immersive VR also caused a significantly worse performance on most simulator metrics.

CONCLUSION: Immersive VR simulation training induces a higher cognitive load and results in a poorer performance than conventional VR simulation training in laparoscopy. High extraneous load and element interactivity in the immersive VR are suggested as mechanisms explaining this finding. However, immersive VR offers some potential advantages over conventional VR such as more real-life conditions but we only recommend introducing immersive VR in surgical skills training after initial training in conventional VR.

Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure

ArticleRobotic surgerySurgical trainingVR simulation
Hovgaard LH, Andersen SAW, Konge L, Dalsgaard T, Larsen CR.
Surg Endosc. 2018;32(10):4200–4208.
Publication year: 2018

BACKGROUND: The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study’s objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum.

METHODS: Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module ‘Guided Vaginal Cuff Closure’ six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach’s alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups’ method.

RESULTS: The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach’s alpha). The experienced surgeons’ mean composite score for all six repetitions were significantly better than the novice surgeons’ (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives).

CONCLUSION: Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.

The effect of distributed virtual reality simulation training on cognitive load during subsequent dissection training

ArticleCognitive loadDirected self-regulated learningInstructional designSurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Konge L, Sørensen MS.
Med Teach. 2018;40(7):684–689.
Publication year: 2018

BACKGROUND: Complex tasks such as surgical procedures can induce excessive cognitive load (CL), which can have a negative effect on learning, especially for novices.

AIM: To investigate if repeated and distributed virtual reality (VR) simulation practice induces a lower CL and higher performance in subsequent cadaveric dissection training.

METHODS: In a prospective, controlled cohort study, 37 residents in otorhinolaryngology received VR simulation training either as additional distributed practice prior to course participation (intervention) (9 participants) or as standard practice during the course (control) (28 participants). Cognitive load was estimated as the relative change in secondary-task reaction time during VR simulation and cadaveric procedures.

RESULTS: Structured distributed VR simulation practice resulted in lower mean reaction times (32% vs. 47% for the intervention and control group, respectively, p < 0.01) as well as a superior final-product performance during subsequent cadaveric dissection training.

CONCLUSIONS: Repeated and distributed VR simulation causes a lower CL to be induced when the learning situation is increased in complexity. A suggested mechanism is the formation of mental schemas and reduction of the intrinsic CL. This has potential implications for surgical skills training and suggests that structured, distributed training be systematically implemented in surgical training curricula.

The Effect of a Distributed Virtual Reality Simulation Training Program on Dissection Mastoidectomy Performance

ArticleDirected self-regulated learningInstructional designMastoidectomySurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Foghsgaard S, Cayé-Thomasen P, Sørensen MS.
Otol. Neurotol. 2018;39(10):1277–1284.
Publication year: 2018

OBJECTIVE: To investigate the effect on final-product performance of a distributed, virtual reality (VR) simulation training program on cadaveric dissection performance and learning curves compared with standard VR simulation training during a temporal bone course.

STUDY DESIGN: Educational interventional cohort study.

SETTING: The national Danish temporal bone courses of 2016 and 2017.

SUBJECTS: Postgraduate year 2 to 5 residents in otorhinolaryngology.

INTERVENTION: Nine participants volunteered for additional VR simulation training (intervention) before the temporal bone course, with training blocks distributed (i.e., separated). The remaining 28 participants received standard VR simulation training during the temporal bone course (control).

MAIN OUTCOME MEASURE: VR simulation and cadaveric dissection final-product performances were analyzed by blinded raters using a 26-item modified Welling Scale.

RESULTS: Distributed VR simulation training before the temporal bone course (intervention) significantly increased dissection final-product performance by 25% compared with standard VR simulation training during the course (control) (mean scores 12.8 points versus 10.3 points, p < 0.01). Distributed and repeated VR simulation practice markedly decreased drilling time. Guidance by the simulator-integrated tutor-function significantly increased final-product performance by 2.3 points compared with nontutored procedures but at the cost of increased drilling time.

CONCLUSION: Skills acquired in a VR simulation environment translate to cadaveric dissection skills and repeated and distributed VR simulation can be used to further increase performance compared with standard VR simulation training during a temporal bone course. Further dissemination of inexpensive VR simulators would allow all future temporal bone course participants to train locally before attending future centralized courses.

Mapping the plateau of novices in virtual reality simulation training of mastoidectomy

ArticleDirected self-regulated learningLearning supportsMastoidectomyOtologySurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Konge L, Mikkelsen PT, Cayé-Thomasen P, Sørensen MS.
Laryngoscope. 2017;127(4):907–914.
Publication year: 2017

OBJECTIVES/HYPOTHESIS: To explore why novices’ performance plateau in directed, self-regulated virtual reality (VR) simulation training and how performance can be improved.

STUDY DESIGN: Prospective study.

METHODS: Data on the performances of 40 novices who had completed repeated, directed, self-regulated VR simulation training of mastoidectomy were included. Data were analyzed to identify key areas of difficulty as well as the procedures terminated without using all the time allowed.

RESULTS: Novices had difficulty in avoiding drilling holes in the outer anatomical boundaries of the mastoidectomy and frequently made injuries to vital structures such as the lateral semicircular canal, the ossicles, and the facial nerve. The simulator-integrated tutor function improved performance on many of these items, but overreliance on tutoring was observed. Novices also demonstrated poor self-assessment skills and often did not make use of the allowed time, lacking knowledge on when to stop or how to excel.

CONCLUSION: Directed, self-regulated VR simulation training of mastoidectomy needs a strong instructional design with specific process goals to support deliberate practice because cognitive effort is needed for novices to improve beyond an initial plateau.

Virtual reality simulation training of mastoidectomy - studies on novice performance

Virtual reality (VR) simulation-based training is increasingly used in surgical technical skills training including in temporal bone surgery. The potential of VR simulation in enabling high-quality surgical training is great and VR simulation allows high-stakes and complex procedures such as mastoidectomy to be trained repeatedly, independent of patients and surgical tutors, outside traditional learning environments such as the OR or the temporal bone lab, and with fewer of the constraints of traditional training. This thesis aims to increase the evidence-base of VR simulation training of mastoidectomy and, by studying the final-product performances of novices, investigates the transfer of skills to the current gold-standard training modality of cadaveric dissection, the effect of different practice conditions and simulator-integrated tutoring on performance and retention of skills, and the role of directed, self-regulated learning. Technical skills in mastoidectomy were transferable from the VR simulation environment to cadaveric dissection with significant improvement in performance after directed, self-regulated training in the VR temporal bone simulator. Distributed practice led to a better learning outcome and more consolidated skills than massed practice and also resulted in a more consistent performance after three months of non-practice. Simulator-integrated tutoring accelerated the initial learning curve but also caused over-reliance on tutoring, which resulted in a drop in performance when the simulator-integrated tutor-function was discontinued. The learning curves were highly individual but often plateaued early and at an inadequate level, which related to issues concerning both the procedure and the VR simulator, over-reliance on the tutor function and poor self-assessment skills. Future simulator-integrated automated assessment could potentially resolve some of these issues and provide trainees with both feedback during the procedure and immediate assessment following each procedure. Standard setting by establishing a proficiency level that can be used for mastery learning with deliberate practice could also further sophisticate directed, self-regulated learning in VR simulation-based training. VR simulation-based training should be embedded in a systematic and competency-based training curriculum for high-quality surgical skills training, ultimately leading to improved safety and patient care.

The effect of self-directed virtual reality simulation on dissection training performance in mastoidectomy

ArticleDirected self-regulated learningInstructional designMastoidectomyOtologyTemporal bone surgeryVR simulation
Andersen SA, Foghsgaard S, Konge L, Cayé-Thomasen P, Sørensen MS.
Laryngoscope. 2016;126(8):1883–1888.
Publication year: 2016

OBJECTIVES/HYPOTHESIS: To establish the effect of self-directed virtual reality (VR) simulation training on cadaveric dissection training performance in mastoidectomy and the transferability of skills acquired in VR simulation training to the cadaveric dissection training setting.

STUDY DESIGN: Prospective study.

METHODS: Two cohorts of 20 novice otorhinolaryngology residents received either self-directed VR simulation training before cadaveric dissection training or vice versa. Cadaveric and VR simulation performances were assessed using final-product analysis with three blinded expert raters.

RESULTS: The group receiving VR simulation training before cadaveric dissection had a mean final-product score of 14.9 (95 % confidence interval [CI] [12.9-16.9]) compared with 9.8 (95% CI [8.4-11.1]) in the group not receiving VR simulation training before cadaveric dissection. This 52% increase in performance was statistically significantly (P < 0.0001). A single dissection mastoidectomy did not increase VR simulation performance (P = 0.22).

CONCLUSIONS: Two hours of self-directed VR simulation training was effective in increasing cadaveric dissection mastoidectomy performance and suggests that mastoidectomy skills are transferable from VR simulation to the traditional dissection setting. Virtual reality simulation training can therefore be employed to optimize training, and can spare the use of donated material and instructional resources for more advanced training after basic competencies have been acquired in the VR simulation environment.

LEVEL OF EVIDENCE: NA.

The effect of implementing cognitive load theory-based design principles in virtual reality simulation training of surgical skills: a randomized controlled trial

ArticleCognitive loadDirected self-regulated learningInstructional designLearning supportsMastoidectomyOtologySurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Mikkelsen PT, Konge L, Cayé-Thomasen P, Sørensen MS.
Adv Simul (Lond). 2016;1: 20.
Publication year: 2016

BACKGROUND: Cognitive overload can inhibit learning, and cognitive load theory-based instructional design principles can be used to optimize learning situations. This study aims to investigate the effect of implementing cognitive load theory-based design principles in virtual reality simulation training of mastoidectomy.

METHODS: Eighteen novice medical students received 1 h of self-directed virtual reality simulation training of the mastoidectomy procedure randomized for standard instructions (control) or cognitive load theory-based instructions with a worked example followed by a problem completion exercise (intervention). Participants then completed two post-training virtual procedures for assessment and comparison. Cognitive load during the post-training procedures was estimated by reaction time testing on an integrated secondary task. Final-product analysis by two blinded expert raters was used to assess the virtual mastoidectomy performances.

RESULTS: Participants in the intervention group had a significantly increased cognitive load during the post-training procedures compared with the control group (52 vs. 41 %, p = 0.02). This was also reflected in the final-product performance: the intervention group had a significantly lower final-product score than the control group (13.0 vs. 15.4, p < 0.005).

CONCLUSIONS: Initial instruction using worked examples followed by a problem completion exercise did not reduce the cognitive load or improve the performance of the following procedures in novices. Increased cognitive load when part tasks needed to be integrated in the post-training procedures could be a possible explanation for this. Other instructional designs and methods are needed to lower the cognitive load and improve the performance in virtual reality surgical simulation training of novices.

Retention of Mastoidectomy Skills After Virtual Reality Simulation Training

ArticleDirected self-regulated learningLearning supportsMastoidectomyOtologySurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Konge L, Cayé-Thomasen P, Sørensen MS.
JAMA Otolaryngol Head Neck Surg. 2016;142(7):635–640.
Publication year: 2016

IMPORTANCE: The ultimate goal of surgical training is consolidated skills with a consistently high performance. However, surgical skills are heterogeneously retained and depend on a variety of factors, including the task, cognitive demands, and organization of practice. Virtual reality (VR) simulation is increasingly being used in surgical skills training, including temporal bone surgery, but there is a gap in knowledge on the retention of mastoidectomy skills after VR simulation training.

OBJECTIVES: To determine the retention of mastoidectomy skills after VR simulation training with distributed and massed practice and to investigate participants’ cognitive load during retention procedures.

DESIGN, SETTING, AND PARTICIPANTS: A prospective 3-month follow-up study of a VR simulation trial was conducted from February 6 to September 19, 2014, at an academic teaching hospital among 36 medical students: 19 from a cohort trained with distributed practice and 17 from a cohort trained with massed practice.

INTERVENTIONS: Participants performed 2 virtual mastoidectomies in a VR simulator a mean of 3.2 months (range, 2.4-5.0 months) after completing initial training with 12 repeated procedures. Practice blocks were spaced apart in time (distributed), or all procedures were performed in 1 day (massed).

MAIN OUTCOMES AND MEASURES: Performance of the virtual mastoidectomy as assessed by 2 masked senior otologists using a modified Welling scale, as well as cognitive load as estimated by reaction time to perform a secondary task.

RESULTS: Among 36 participants, mastoidectomy final-product skills were largely retained at 3 months (mean change in score, 0.1 points; P = .89) regardless of practice schedule, but the group trained with massed practice took more time to complete the task. The performance of the massed practice group increased significantly from the first to the second retention procedure (mean change, 1.8 points; P = .001), reflecting that skills were less consolidated. For both groups, increases in reaction times in the secondary task (distributed practice group: mean pretraining relative reaction time, 1.42 [95% CI, 1.37-1.47]; mean end of training relative reaction time, 1.24 [95% CI, 1.16-1.32]; and mean retention relative reaction time, 1.36 [95% CI, 1.30-1.42]; massed practice group: mean pretraining relative reaction time, 1.34 [95% CI, 1.28-1.40]; mean end of training relative reaction time, 1.31 [95% CI, 1.21-1.42]; and mean retention relative reaction time, 1.39 [95% CI, 1.31-1.46]) indicated that cognitive load during the virtual procedures had returned to the pretraining level.

CONCLUSIONS AND RELEVANCE: Mastoidectomy skills acquired under time-distributed practice conditions were retained better than skills acquired under massed practice conditions. Complex psychomotor skills should be regularly reinforced to consolidate both motor and cognitive aspects. Virtual reality simulation training provides the opportunity for such repeated training and should be integrated into training curricula.

Notes From the Field: Secondary Task Precision for Cognitive Load Estimation During Virtual Reality Surgical Simulation Training

ArticleCognitive loadInstructional designVR simulation
Rasmussen SR, Konge L, Mikkelsen PT, Sørensen MS, Andersen SA.
Eval Health Prof. 2016;39(1):114–120.
Publication year: 2016

Cognitive load (CL) theory suggests that working memory can be overloaded in complex learning tasks such as surgical technical skills training, which can impair learning. Valid and feasible methods for estimating the CL in specific learning contexts are necessary before the efficacy of CL-lowering instructional interventions can be established. This study aims to explore secondary task precision for the estimation of CL in virtual reality (VR) surgical simulation and also investigate the effects of CL-modifying factors such as simulator-integrated tutoring and repeated practice. Twenty-four participants were randomized for visual assistance by a simulator-integrated tutor function during the first 5 of 12 repeated mastoidectomy procedures on a VR temporal bone simulator. Secondary task precision was found to be significantly lower during simulation compared with nonsimulation baseline, p < .001. Contrary to expectations, simulator-integrated tutoring and repeated practice did not have an impact on secondary task precision. This finding suggests that even though considerable changes in CL are reflected in secondary task precision, it lacks sensitivity. In contrast, secondary task reaction time could be more sensitive, but requires substantial postprocessing of data. Therefore, future studies on the effect of CL modifying interventions should weigh the pros and cons of the various secondary task measurements.

Cognitive Load in Mastoidectomy Skills Training: Virtual Reality Simulation and Traditional Dissection Compared

ArticleCognitive loadDirected self-regulated learningSurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Mikkelsen PT, Konge L, Cayé-Thomasen P, Sørensen MS.
J Surg Educ. 2016;73(1):45–50.
Publication year: 2016

OBJECTIVE: The cognitive load (CL) theoretical framework suggests that working memory is limited, which has implications for learning and skills acquisition. Complex learning situations such as surgical skills training can potentially induce a cognitive overload, inhibiting learning. This study aims to compare CL in traditional cadaveric dissection training and virtual reality (VR) simulation training of mastoidectomy.

DESIGN: A prospective, crossover study. Participants performed cadaveric dissection before VR simulation of the procedure or vice versa. CL was estimated by secondary-task reaction time testing at baseline and during the procedure in both training modalities.

SETTING: The national Danish temporal bone course.

PARTICIPANTS: A total of 40 novice otorhinolaryngology residents.

RESULTS: Reaction time was increased by 20% in VR simulation training and 55% in cadaveric dissection training of mastoidectomy compared with baseline measurements. Traditional dissection training increased CL significantly more than VR simulation training (p < 0.001).

CONCLUSIONS: VR simulation training imposed a lower CL than traditional cadaveric dissection training of mastoidectomy. Learning complex surgical skills can be a challenge for the novice and mastoidectomy skills training could potentially be optimized by employing VR simulation training first because of the lower CL. Traditional dissection training could then be used to supplement skills training after basic competencies have been acquired in the VR simulation.

Cognitive load in distributed and massed practice in virtual reality mastoidectomy simulation

ArticleCognitive loadInstructional designMastoidectomyTemporal bone surgeryVR simulation
Andersen SA, Mikkelsen PT, Konge L, Cayé-Thomasen P, Sørensen MS.
Laryngoscope. 2016;126(2):74–79.
Publication year: 2016

OBJECTIVES/HYPOTHESIS: Cognitive load theory states that working memory is limited. This has implications for learning and suggests that reducing cognitive load (CL) could promote learning and skills acquisition. This study aims to explore the effect of repeated practice and simulator-integrated tutoring on CL in virtual reality (VR) mastoidectomy simulation.

STUDY DESIGN: Prospective trial.

METHODS: Forty novice medical students performed 12 repeated virtual mastoidectomy procedures in the Visible Ear Simulator: 21 completed distributed practice with practice blocks spaced in time and 19 participants completed massed practice (all practices performed in 1 day). Participants were randomized for tutoring with the simulator-integrated tutor function. Cognitive load was estimated by measuring reaction time in a secondary task. Data were analyzed using linear mixed models for repeated measurements.

RESULTS: The mean reaction time increased by 37% during the procedure compared with baseline, demonstrating that the procedure placed substantial cognitive demands. Repeated practice significantly lowered CL in the distributed practice group but not in massed practice group. In addition, CL was found to be further increased by 10.3% in the later and more complex stages of the procedure. The simulator-integrated tutor function did not have an impact on CL.

CONCLUSION: Distributed practice decreased CL in repeated VR mastoidectomy training more consistently than was seen in massed practice. This suggests a possible effect of skills and memory consolidation occurring over time. To optimize technical skills learning, training should be organized as time-distributed practice rather than as a massed block of practice, which is common in skills-training courses.

Peak and ceiling effects in final-product analysis of mastoidectomy performance

ArticleAssessmentMastoidectomyTemporal bone surgeryVR simulation
West N, Konge L, Cayé-Thomasen P, Sørensen MS, Andersen, SA.
J Laryngol Otol. 2015;129(11):1091–1096.
Publication year: 2015

BACKGROUND: Virtual reality surgical simulation of mastoidectomy is a promising training tool for novices. Final-product analysis for assessing novice mastoidectomy performance could be limited by a peak or ceiling effect. These may be countered by simulator-integrated tutoring.

METHODS: Twenty-two participants completed a single session of self-directed practice of the mastoidectomy procedure in a virtual reality simulator. Participants were randomised for additional simulator-integrated tutoring. Performances were assessed at 10-minute intervals using final-product analysis.

RESULTS: In all, 45.5 per cent of participants peaked before the 60-minute time limit. None of the participants achieved the maximum score, suggesting a ceiling effect. The tutored group performed better than the non-tutored group but tutoring did not eliminate the peak or ceiling effects.

CONCLUSION: Timing and adequate instruction is important when using final-product analysis to assess novice mastoidectomy performance. Improved real-time feedback and tutoring could address the limitations of final product based assessment.

Novices perform better in virtual reality simulation than in traditional cadaveric dissection training of mastoidectomy

ArticleAssessmentInstructional designSurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Cayé-Thomasen P, Sørensen MS.
Journal of Surgical Simulation. 2015; 2:68–75.
Publication year: 2015

BACKGROUND: Temporal bone surgery requires integration of complex knowledge and technical skills. This can be difficult to accomplish with traditional cadaveric dissection training, which is often organized as single-instance participation in a temporal bone course. Simulator-integrated tutoring in virtual reality (VR) surgical simulators can visually guide the procedure and facilitate self-directed surgical skills acquisition. This study aims to explore the performances of novice otorhinolaryngology residents in a freeware VR simulator and in cadaveric dissection training of mastoidectomy.

METHODS: Thirty-four novice otorhinolaryngology residents performed a single and self-directed mastoidectomy procedure in a freeware VR temporal bone simulator before performing a similar procedure on a cadaveric temporal bone. VR simulation and cadaveric dissection performances were assessed by two blinded expert raters using final product analysis.

RESULTS: Participants achieved a higher mean final product score in VR simulation compared with cadaveric dissection (14.9 and 13.2, respectively; P = 0.02). Significantly more of the participants had their best performance in VR simulation (P = 0.04). No differences in computer experience and interest were found between the group that performed better in VR simulation and the group that performed better in cadaveric dissection.

CONCLUSIONS: Novice performance in a freeware VR temporal bone simulator was significantly better than in cadaveric dissection. The simulator-integrated tutor function and reduced complexity of the procedure in VR simulation could be possible explanations for this finding. VR simulation training could be used in the initial training of novices, reserving dissection training for more advanced training after basic competencies have been acquired with VR simulation.

Mastoidectomy performance assessment of virtual simulation training using final-product analysis

ArticleAssessmentMastoidectomySurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Cayé-Thomasen P, Sørensen MS.
Laryngoscope. 2015;125(2):431–435.
Publication year: 2015

OBJECTIVES/HYPOTHESIS: The future development of integrated automatic assessment in temporal bone virtual surgical simulators calls for validation against currently established assessment tools. This study aimed to explore the relationship between mastoidectomy final-product performance assessment in virtual simulation and traditional dissection training.

STUDY DESIGN: Prospective trial with blinding.

METHODS: A total of 34 novice residents performed a mastoidectomy on the Visible Ear Simulator and on a cadaveric temporal bone. Two blinded senior otologists assessed the final-product performance using a modified Welling scale. The simulator gathered basic metrics on time, steps, and volumes in relation to the on-screen tutorial and collisions with vital structures.

RESULTS: Substantial inter-rater reliability (kappa = 0.77) for virtual simulation and moderate inter-rater reliability (kappa = 0.59) for dissection final-product assessment was found. The simulation and dissection performance scores had significant correlation (P = .014). None of the basic simulator metrics correlated significantly with the final-product score except for number of steps completed in the simulator.

CONCLUSIONS: A modified version of a validated final-product performance assessment tool can be used to assess mastoidectomy on virtual temporal bones. Performance assessment of virtual mastoidectomy could potentially save the use of cadaveric temporal bones for more advanced training when a basic level of competency in simulation has been achieved.

Learning Curves of Virtual Mastoidectomy in Distributed and Massed Practice

ArticleAssessmentDirected self-regulated learningInstructional designLearning supportsMastoidectomyTemporal bone surgeryVR simulation
Andersen SA, Konge L, Cayé-Thomasen P, Sørensen MS.
JAMA Otolaryngol Head Neck Surg, 2015;141(10):913–918.
Publication year: 2015

IMPORTANCE: Repeated and deliberate practice is crucial in surgical skills training, and virtual reality (VR) simulation can provide self-directed training of basic surgical skills to meet the individual needs of the trainee. Assessment of the learning curves of surgical procedures is pivotal in understanding skills acquisition and best-practice implementation and organization of training.

OBJECTIVE: To explore the learning curves of VR simulation training of mastoidectomy and the effects of different practice sequences with the aim of proposing the optimal organization of training.

DESIGN, SETTING, AND PARTICIPANTS: A prospective trial with a 2 × 2 design was conducted at an academic teaching hospital. Participants included 43 novice medical students. Of these, 21 students completed time-distributed practice from October 14 to November 29, 2013, and a separate group of 19 students completed massed practice on May 16, 17, or 18, 2014. Data analysis was performed from June 6, 2014, to March 3, 2015.

INTERVENTIONS: Participants performed 12 repeated virtual mastoidectomies using a temporal bone surgical simulator in either a distributed (practice blocks spaced in time) or massed (all practice in 1 day) training program with randomization for simulator-integrated tutoring during the first 5 sessions.

MAIN OUTCOMES AND MEASURES: Performance was assessed using a modified Welling Scale for final product analysis by 2 blinded senior otologists.

RESULTS: Compared with the 19 students in the massed practice group, the 21 students in the distributed practice group were older (mean age, 25.1 years), more often male (15 [62%]), and had slightly higher mean gaming frequency (2.3 on a 1-5 Likert scale). Learning curves were established and distributed practice was found to be superior to massed practice, reported as mean end score (95% CI) of 15.7 (14.4-17.0) in distributed practice vs. 13.0 (11.9-14.1) with massed practice (P = .002). Simulator-integrated tutoring accelerated the initial performance, with mean score for tutored sessions of 14.6 (13.9-15.2) vs. 13.4 (12.8-14.0) for corresponding nontutored sessions (P < .01) but at the cost of a drop in performance once tutoring ceased. The performance drop was less with distributed practice, suggesting a protective effect when acquired skills were consolidated over time. The mean performance of the nontutored participants in the distributed practice group plateaued on a score of 16.0 (15.3-16.7) at approximately the ninth repetition, but the individual learning curves were highly variable.

CONCLUSIONS AND RELEVANCE: Novices can acquire basic mastoidectomy competencies with self-directed VR simulation training. Training should be organized with distributed practice, and simulator-integrated tutoring can be useful to accelerate the initial learning curve. Practice should be deliberate and toward a standard set level of proficiency that remains to be defined rather than toward the mean learning curve plateau.

[Good experiences with interactive temporal bone surgical simulator]

ArticleMastoidectomySurgical trainingTemporal bone surgeryVR simulation
Andersen SA, Mikkelsen PT, Noe KØ, Sørensen MS.
Ugeskr Laeger. 2014;176(5):444–6.
Publication year: 2014

The Visible Ear Simulator (VES) is a freeware temporal bone surgical simulator utilizing a high-fidelity haptic and graphical voxel model compiled from segmented digital images of fresh frozen sections. A haptic device provides the 3-dimensional handling and drilling with force-feedback in real time. In a multilingual user interface the integrated tutor function provides stepwise instructions during drilling through an intuitive, volumetric approach. A censor function draws on metrics derived from the simulator to provide instant and summary feedback for the user. The VES can be downloaded from http://ves.cg.alexandra.dk.