Steven Andersen
MD, PhD
Publications
Steven Andersen has published 66 papers in peer-reviewed journals (29 as first author and 33 as last author), primarily within the field of otology, surgical simulation, technical skills training, virtual reality simulation, cognitive load, immersive simulation, and image processing of clinical cone-beam CT of the temporal bone. In addition, several commentaries, an editorial, book chapter, and dissection manuals.
Current list of publications as indexed by PubMed / ORCID record.
PDF version of the publication list (as of 20 May 2022).
Google Scholar: H-index 18, total citations 987 (as of 29 September 2022).
Filter by type:
The cutting edge of customized surgery: 3D-printed models for patient-specific interventions in otology and auricular management-a systematic review
Purpose: 3D-printing (three-dimensional printing) is an emerging technology with promising applications for patient-specific interventions. Nonetheless, knowledge on the clinical applicability of 3D-printing in otology and research on its use remains scattered. Understanding these new treatment options is a prerequisite for clinical implementation, which could improve patient outcomes. This review aims to explore current applications of 3D-printed patient-specific otologic interventions, including state of the evidence, strengths, limitations, and future possibilities.
Methods: Following the PRISMA statement, relevant studies were identified through Pubmed, EMBASE, the Cochrane Library, and Web of Science. Data on the manufacturing process and interventions were extracted by two reviewers. Study quality was assessed using Joanna Briggs Institute’s critical appraisal tools.
Results: Screening yielded 590 studies; 63 were found eligible and included for analysis. 3D-printed models were used as guides, templates, implants, and devices. Outer ear interventions comprised 73% of the studies. Overall, optimistic sentiments on 3D-printed models were reported, including increased surgical precision/confidence, faster manufacturing/operation time, and reduced costs/complications. Nevertheless, study quality was low as most studies failed to use relevant objective outcomes, compare new interventions with conventional treatment, and sufficiently describe manufacturing.
Conclusion: Several clinical interventions using patient-specific 3D-printing in otology are considered promising. However, it remains unclear whether these interventions actually improve patient outcomes due to lack of comparison with conventional methods and low levels of evidence. Further, the reproducibility of the 3D-printed interventions is compromised by insufficient reporting. Future efforts should focus on objective, comparative outcomes evaluated in large-scale studies.
Keywords: 3D-printing; Additive manufacturing; Ear surgery; Otology; Patient-specific.74
Simulation-based surgical training needs in otorhinolaryngology
Introduction: This study aimed to conduct a targeted needs assessment to identify and prioritise technical skills and procedures suited for simulation-based training (SBT) in private otorhinolaryngology (ORL) practice in Denmark, including mapping the learning environment related to implementation of SBT.
Methods: A panel of trainers and trainees in private ORL practice was recruited. Using the Delphi method, three rounds of surveys were conducted. Round one consisted of a survey of the learning environment and a brainstorming phase. Round two quantified the frequency of procedures, ranked the importance of procedural competency, impact on patient safety and feasibility for SBT. In round three, panelists eliminated and ranked procedures for final prioritisation.
Results: A total of 26 of 57 invited trainers and trainees accepted participation. The educational environment was described and 136 skills were suggested in the brainstorming phase. “Non-technical” skills were removed, and the remaining 46 technical skills were grouped for appraisal in round two. In round three, panelists reduced these to eight technical skills and procedures which were maintained for final prioritisation for SBT with myringotomy with ventilation tube insertion ranking highest. Trainees and trainers indicated that close supervision and dedicated time for training were major strengths of the learning environment.
Conclusions: Our findings extend the results obtained in a previous general needs assessment and may inform curricular implementation of SBT in private ORL practice. A structured “package” with SBT and assessment for the identified procedures are desired by trainers. This work is already in progress and implementation is facilitated by a positive attitude towards SBT among trainers and trainees alike.
Mastoidectomy training—is anatomical variation needed? A randomized, controlled trial on performance and skills transfer from Virtual Reality to a 3D-printed model
Effect of 3D-Printed Models on Cadaveric Dissection in Temporal Bone Training
Objective: Mastoidectomy is a cornerstone in the surgical management of middle and inner ear diseases. Unfortunately, training is challenged by insufficient access to human cadavers. Three-dimensional (3D) printing of temporal bones could alleviate this problem, but evidence on their educational effectiveness is lacking. It is largely unknown whether training on 3D-printed temporal bones improves mastoidectomy performance, including on cadavers, and how this training compares with virtual reality (VR) simulation. To address this knowledge gap, this study investigated whether training on 3D-printed temporal bones improves cadaveric dissection performance, and it compared this training with the already-established VR simulation.
Study design: Prospective cohort study of an educational intervention.
Setting: Tertiary university hospital, cadaver dissection laboratory, and simulation center in Copenhagen, Denmark.
Methods: Eighteen otorhinolaryngology residents (intervention) attending the national temporal bone dissection course received 3 hours of mastoidectomy training on 3D-printed temporal bones. Posttraining cadaver mastoidectomy performances were rated by 3 experts using a validated assessment tool and compared with those of 66 previous course participants (control) who had received time-equivalent VR training prior to dissection.
Results: The intervention cohort outperformed the controls during cadaver dissection by 29% (P < .001); their performances were largely similar across training modalities but remained at a modest level (~50% of the maximum score).
Conclusion: Mastoidectomy skills improved from training on 3D-printed temporal bone and seemingly more so than on time-equivalent VR simulation. Importantly, these skills transferred to cadaveric dissection. Training on 3D-printed temporal bones can effectively supplement cadaver training when learning mastoidectomy.
Keywords: 3D printing; additive manufacturing; education; mastoidectomy; neurotology; otology; rapid prototyping; surgical simulation; temporal bone; training.
Cochlear implantation: Exploring the effects of 3D stereovision in a digital microscope for virtual reality simulation training - A randomized controlled trial
Background: In cochlear implantation (CI), excellent surgical technique is critical for hearing outcomes. Recent advances in temporal bone Virtual Reality (VR) training allow for specific training of CI and through introduction of new digital microscopes with ultra-high-fidelity (UHF) graphics. This study aims to investigate whether UHF increases performance in VR simulation training of CI electrode insertion compared with conventional, screen-based VR (cVR).
Methods: Twenty-four medical students completed a randomized, controlled trial of an educational intervention. They performed a total of eight CI electrode insertions each in blocks of four using either UHF-VR or cVR, in randomized order. CI electrode insertion performances were rated by two blinded expert raters using a structured assessment tool supported by validity evidence.
Results: Performance scores in cVR were higher than in the UHF-VR simulation although this was not significant (19.8 points, 95% CI [19.3-20.3] vs. 18.8 points, 95% CI [18.2-19.4]; P = 0.09). The decisive factor for performance was participants’ ability to achieve stereovision (mean difference = 1.1 points, 95% CI [0.15-2.08], P = 0.02).
Discussion: No additional benefit was found from UHF-VR over cVR training of CI electrode insertion for novices. Consequently, standard cVR simulation should be used for novices’ basic skills acquisition in CI surgery. Future studies should instead explore the effects of other improvements in CI surgery training and if the lacking benefit of UHF-VR also applies for more experienced learners.
Conclusion: The increased graphical perception and the superior lifelikeness of UHF-VR does not improve early skills acquisition of CI insertion for novices.
Keywords: Cochlear implantation; Simulation-based surgical training; Virtual reality simulation.
Automated calculation of cochlear implant electrode insertion parameters in clinical cone-beam CT
Hypothesis: Automated processing of postoperative clinical cone-beam CT (CBCT) of cochlear implant (CI) patients can be used to accurately determine electrode contacts and integrated with an atlas-based mapping of cochlear microstructures to calculate modiolar distance, angular insertion distance, and scalar location of electrode contacts.
Background: Hearing outcomes after CI surgery are dependent on electrode placement. CBCT is increasingly used for in-office temporal bone imaging and might be routinely used for pre- and post-surgical evaluation.
Methods: Thirty-six matched pairs of pre- and postimplant CBCT scans were obtained. These were registered with an atlas to model cochlear microstructures in each dataset. Electrode contact center points were automatically determined using thresholding and electrode insertion parameters were calculated. Automated localization and calculation were compared with manual segmentation of contact center points as well as manufacturer specifications.
Results: Automated electrode contact detection aligned with manufacturer specifications of spacing and our algorithms worked for both distantly- and closely spaced arrays. The average difference between the manual and the automated selection was 0.15 mm, corresponding to a 1.875 voxel difference in each plane at the scan resolution. For each case, we determined modiolar distance, angular insertion depth, and scalar location. These calculations also resulted in similar insertion values using manual and automated contact points as well as aligning with electrode properties.
Conclusion: Automated processing of implanted high-resolution CBCT images can provide the clinician with key information on electrode placement. This is one step toward routine use of clinical CBCT after CI surgery to inform and guide postoperative treatment.
Am I doing this right? Structured self-assessment during simulation training of mastoidectomy improves cadaver dissection performance: a prospective educational study
Purpose: Temporal bone surgery requires excellent surgical skills and simulation-based training can aid novices’ skills acquisition. However, simulation-based training is challenged by early stagnation of the learning after relatively few performances. Structured self-assessment during practice might enhance learning by inducing reflection and engagement in the learning task. In this study, structured self-assessment was introduced during virtual reality (VR) simulation of mastoidectomy to investigate the effects on subsequent performance during cadaveric dissection.
Methods: This was a prospective educational study with comparison with historical controls (reference cohort). At a temporal bone dissection course, eighteen participants performed structured self-assessment during three hours of VR simulation training of mastoidectomy before proceeding to cadaver dissection/surgery (intervention cohort). At a previous course, eighteen participants received similar VR simulation training but without the structured self-assessment (reference cohort). Final products from VR simulation and cadaveric dissection were video-recorded and assessed by two blinded raters using a 19-point modified Welling Scale.
Results: The intervention cohort completed fewer procedures (average 4.2) during VR simulation training than the reference cohort (average 5.7). Nevertheless, the intervention cohort achieved a significantly higher average dissection score both in VR simulation (11.1 points, 95% CI [10.6–11.5]) and subsequent cadaveric dissection (11.8 points, 95% CI [10.7–12.8]) compared with the reference cohort who scored 9.1 points (95% CI [8.7–9.5]) during VR simulation and 5.8 points (95% CI [4.8–6.8]) during cadaveric dissection.
Conclusion: Structured self-assessment is a valuable learning support during self-directed VR simulation training of mastoidectomy and the positive effect on performance transfers to subsequent cadaveric dissection performance.
Use of simulation-based training of surgical technical skills among ENTs: an international YO-IFOS survey
Purpose: The aim of this study was to investigate and compare the use of simulation-based technical skills training (SBTST) in the otolaryngology curriculum in different countries, and to explore the needs and opinions about the use of simulation among young otolaryngologists.
Methods: An e-survey conducted among Young Otolaryngologists of the International Federation of Oto-rhino-laryngological Societies (Yo-IFOS) members.
Results: 139 Yo-IFOS members from 51 countries completed the survey. During residency training, 82.7% of respondents have used SBTST on cadavers, 51.8% on physical simulators, and 43.8% on virtual reality simulators. High costs (65.5%), lack of availability (49.2%) and lack of time (25.5%) were the main barriers limiting the practice of SBTST. These barriers also limited teaching using simulation. Sinonasal surgery (72.7%), temporal bone surgery (67.6%), and head/neck surgery (44.6%) were significantly more frequently taught using SBTST than suspension microlaryngoscopy (25.9%) and pediatric surgery (22.3%) (p < 0.001). The procedures rated as the most important to learn through SBTST were tracheotomy (50.4%), emergency cricothyroidotomy (48.9%), and rigid bronchoscopy (47.5%). On an analogic visual scale (0-100) for the question “how important will simulation be for future ENTs in surgical training?”, the mean score was 79.5/100 (± 23.3), highlighting the positive attitude toward the use of SBTST in otorhinolaryngology training.
Conclusion: SBTST is an attractive learning and teaching method in otorhinolaryngology, but associated costs, lack of access, and lack of time are the main barriers limiting its use. Emergency procedures are key technical skills to be learned using simulation but in some cases, lack relevant simulators for training.
Training and education of healthcare workers during viral epidemics: A systematic review
Background: It is necessary to train a large number of healthcare workers (HCW) within a limited time to ensure adequate human resources during an epidemic. There remains an urgent need for best practices on development and implementation of training programmes.
Objective: To explore published literature in relation to training and education for viral epidemics as well as the effect of these interventions to inform training of HCW.
Data sources: Systematic searches in five databases performed between 1 January 2000 and 24 April 2020 for studies reporting on educational interventions in response to major viral epidemics.
Study eligibility criteria: All studies on educational interventions developed, implemented and evaluated in response to major global viral outbreaks from 2000 to 2020.
Participants: Healthcare workers.
Interventions: Educational or training interventions.
Study appraisal and synthesis methods: Descriptive information were extracted and synthesised according to content, competency category, educational methodology, educational effects and level of educational outcome. Quality appraisal was performed using a criterion-based checklist.
Results: A total of 15 676 records were identified and 46 studies were included. Most studies were motivated by the Ebola virus outbreak with doctors and nurses as primary learners. Traditional didactic methods were commonly used to teach theoretical knowledge. Simulation-based training was used mainly for training of technical skills, such as donning and doffing of personal protective equipment. Evaluation of the interventions consisted mostly of surveys on learner satisfaction and confidence or tests of knowledge and skills. Only three studies investigated transfer to the clinical setting or effect on patient outcomes.
Conclusions and implications of findings: The included studies describe important educational experiences from past epidemics with a variety of educational content, design and modes of delivery. High-level educational evidence is limited. Evidence-based and standardised training programmes that are easily adapted locally are recommended in preparation for future outbreaks.
Ten tips for performing your first peer review: the next step for the aspiring academic plastic surgeon
Standard setting in simulation-based training of surgical procedures: a systematic review
Objective: This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures.
Summary of background: Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods.
Methods: Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE.
Results: Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence.
Conclusion: Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.
Segmentation of temporal bone anatomy for patient-specific virtual reality simulation
Objectives: Virtual reality (VR) simulation for patient-specific pre-surgical planning and rehearsal requires accurate segmentation of key surgical landmark structures such as the facial nerve, ossicles, and cochlea. The aim of this study was to explore different approaches to segmentation of temporal bone surgical anatomy for patient-specific VR simulation.
Methods: De-identified, clinical computed tomography imaging of 9 pediatric patients aged 3 months to 12 years were obtained retrospectively. The patients represented normal anatomy and key structures were manually segmented using open source software. The OTOPLAN (CAScination AG, Bern, Switzerland) otological planning software was used for guided segmentation. An atlas-based algorithm was used for computerized, automated segmentation. Experience with the different approaches as well as time and resulting models were compared.
Results: Manual segmentation was time consuming but also the most flexible. The OTOPLAN software is not designed specifically for our purpose and therefore the number of structures that can be segmented is limited, there was some user-to-user variation as well as volume differences compared with manual segmentation. The atlas-based automated segmentation potentially allows a full range of structures to be segmented and produces segmentations comparable to those of manual segmentation with a processing time that is acceptable because of the minimal user interaction.
Conclusion: Segmentation is fundamental for patient-specific VR simulation for pre-surgical planning and rehearsal in temporal bone surgery. The automated segmentation algorithm currently offers the most flexible and feasible approach and should be implemented. Further research is needed in relation to cases of abnormal anatomy.
Patient-specific virtual temporal bone simulation based on clinical cone-beam computed tomography
Objectives: Patient-specific surgical simulation allows presurgical planning through three-dimensional (3D) visualization and virtual rehearsal. Virtual reality simulation for otologic surgery can be based on high-resolution cone-beam computed tomography (CBCT). This study aimed to evaluate clinicians’ experience with patient-specific simulation of mastoid surgery.
Methods: Prospective, multi-institutional study. Preoperative temporal bone CBCT scans of patients undergoing cochlear implantation (CI) were retrospectively obtained. Automated processing and segmentation routines were used. Otologic surgeons performed a complete mastoidectomy with facial recess approach on the patient-specific virtual cases in the institution’s temporal bone simulator. Participants completed surveys regarding the perceived accuracy and utility of the simulation.
Results: Twenty-two clinical CBCTs were obtained. Four attending otologic surgeons and 5 otolaryngology trainees enrolled in the study. The mean number of simulations completed by each participant was 16.5 (range 3-22). “Overall experience” and “usefulness for presurgical planning” were rated as “good,” “very good,” or “excellent” in 84.6% and 71.6% of the simulations, respectively. In 10.7% of simulations, the surgeon reported to have gained a significantly greater understanding of the patient’s anatomy compared to standard imaging. Participants were able to better appreciate subtle anatomic findings after using the simulator for 60.4% of cases. Variable CBCT acquisition quality was the most reported limitation.
Conclusion: Patient-specific simulation using preoperative CBCT is feasible and may provide valuable insights prior to otologic surgery. Establishing a CBCT acquisition protocol that allows for consistent segmentation will be essential for reliable surgical simulation.
OpenEar image data enables case variation in high fidelity virtual reality ear surgery
Background: Virtual reality (VR) simulation is an established option for temporal bone surgical training. Most VR simulators are based on computed tomography imaging, whereas the Visible Ear Simulator (VES) is based on high-fidelity cryosections of a single temporal bone specimen. Recently published OpenEar datasets combine cone-beam computed tomography (CBCT) and micro-slicing to achieve similar model quality. This study explores integration of OpenEar datasets into VES to enable case variation in simulation with implications for patient-specific modeling based on CBCT.
Methods: The OpenEar dataset consists of segmented, coregistered, multimodal imaging sets of human temporal bones. We derived drillable bone segments from the dataset as well as triangulated surface models of critical structures such as facial nerve or dura. Realistic visualization was achieved using coloring from micro-slicing, custom tinting, and texture maps. Resulting models were validated by clinical experts.
Results: Six of the eight OpenEar datasets could be integrated in VES complete with instructional guides for various temporal bone surgical procedures. Resulting models were of high quality because of postprocessing steps taken to increase realism including colorization and imaging artifact removal. Bone artifacts were common in CBCT, resulting in dehiscences that most often could not be found in the ground truth micro-slicing data.
Conclusion: New anatomy models are included in VES version 3.5 freeware and provide case variation for training which could help trainees to learn more quickly and transferably under variable practice conditions. The use of CBCT for VR simulation models without postprocessing results in bone artifacts, which should be considered when using clinical imaging for patient-specific simulation, surgical rehearsal, and planning.
Current status of handheld otoscopy training: a systematic review
Objective: Otoscopy is a frequently performed procedure and competency in this skill is important across many specialties. We aim to systematically review current medical educational evidence for training of handheld otoscopy skills.
Methods: Following the PRISMA guideline, studies reporting on training and/or assessment of handheld otoscopy were identified searching the following databases: PubMed, Embase, OVID, the Cochrane Library, PloS Medicine, Directory of Open Access Journal (DOAJ), and Web of Science. Two reviewers extracted data on study design, training intervention, educational outcomes, and results. Quality of educational evidence was assessed along with classification according to Kirkpatrick’s model of educational outcomes.
Results: The searches yielded a total of 6064 studies with a final inclusion of 33 studies for the qualitative synthesis. Handheld otoscopy training could be divided into workshops, physical simulators, web-based training/e-learning, and smartphone-enabled otoscopy. Workshops were the most commonly described educational intervention and typically consisted of lectures, hands-on demonstrations, and training on peers. Almost all studies reported a favorable effect on either learner attitude, knowledge, or skills. The educational quality of the studies was reasonable but the educational outcomes were mostly evaluated on the lower Kirkpatrick levels with only a single study determining the effects of training on actual change in the learner behavior.
Conclusion: Overall, it seems that any systematic approach to training of handheld otoscopy is beneficial in training regardless of learner level, but the heterogeneity of the studies makes comparisons between studies difficult and the relative effect sizes of the interventions could not be determined.
Current evidence for simulation-based training and assessment of myringotomy and ventilation tube insertion: A systematic review
Objective: Myringotomy and ventilation tube insertion (MT) is a key procedure in otorhinolaryngology and can be trained using simulation models. We aimed to systematically review the literature on models for simulation-based training and assessment of MT and supporting educational evidence.
Databases reviewed: PubMed, Embase, Cochrane Library, Web of Science, Directory of Open Access Journals.
Methods: Inclusion criteria were MT training and/or skills assessment using all types of training modalities and learners. Studies were divided into 1) descriptive and 2) educational interventional/observational in the analysis. For descriptive studies, we provide an overview of available models including materials and cost. Educational studies were appraised using Kirkpatrick’s level of educational outcomes, Messick’s framework of validity, and a structured quality assessment tool.
Results: Forty-six studies were included consisting of 21 descriptive studies and 25 educational studies. Thirty-one unique physical and three virtual reality simulation models were identified. The studies report moderate to high realism of the different simulators and trainees and educators perceive them beneficial in training MT skills. Overall, simulation-based training is found to reduce procedure time and errors, and increase performance as measured using different assessment tools. None of the studies used a contemporary validity framework and the current educational evidence is limited.
Conclusion: Numerous simulation models and assessment tools have been described in the literature but educational evidence and systematic implementation into training curricula is scarce. There is especially a need to establish the effect of simulation-based training of MT in transfer to the operating room and on patient outcomes.
Content validity evidence for a simulation-based test of handheld otoscopy skills
Purpose: At graduation from medical school, competency in otoscopy is often insufficient. Simulation-based training can be used to improve technical skills, but the suitability of the training model and assessment must be supported by validity evidence. The purpose of this study was to collect content validity evidence for a simulation-based test of handheld otoscopy skills.
Methods: First, a three-round Delphi study was conducted with a panel of nine clinical teachers in otorhinolaryngology (ORL) to determine the content requirements in our educational context. Next, the authenticity of relevant cases in a commercially available technology-enhanced simulator (Earsi, VR Magic, Germany) was evaluated by specialists in ORL. Finally, an integrated course was developed for the simulator based on these results.
Results: The Delphi study resulted in nine essential diagnoses of normal variations and pathologies that all junior doctors should be able to diagnose with a handheld otoscope. Twelve out of 15 tested simulator cases were correctly recognized by at least one ORL specialist. Fifteen cases from the simulator case library matched the essential diagnoses determined by the Delphi study and were integrated into the course.
Conclusion: Content validity evidence for a simulation-based test of handheld otoscopy skills was collected. This informed a simulation-based course that can be used for undergraduate training. The course needs to be further investigated in relation to other aspects of validity and for future self-directed training.
Cochlear implant surgery: Virtual reality simulation training and transfer of skills to cadaver dissection—a randomized, controlled trial
BACKGROUND: Cochlear implantation requires excellent surgical skills; virtual reality simulation training is an effective method for acquiring basic competency in temporal bone surgery before progression to cadaver dissection. However, cochlear implantation virtual reality simulation training remains largely unexplored and only one simulator currently supports the training of the cochlear implantation electrode insertion. Here, we aim to evaluate the effect of cochlear implantation virtual reality simulation training on subsequent cadaver dissection performance and self-directedness.
METHODS: This was a randomized, controlled trial. Eighteen otolaryngology residents were randomized to either mastoidectomy including cochlear implantation virtual reality simulation training (intervention) or mastoidectomy virtual reality simulation training alone (controls) before cadaver cochlear implantation surgery. Surgical performance was evaluated by two blinded expert raters using a validated, structured assess- ment tool. The need for supervision (reflecting self-directedness) was assessed via post-dissection questionnaires.
RESULTS: The intervention group achieved a mean score of 22.9 points of a maximum of 44 points, which was 5.4% higher than the control group’s 21.8 points (P = .51). On average, the intervention group required assistance 1.3 times during cadaver drilling; this was 41% more frequent in the control group who received assistance 1.9 times (P = .21).
CONCLUSION: Cochlear implantation virtual reality simulation training is feasible in the context of a cadaver dissection course. The addition of cochlear implantation virtual reality training to basic mastoidectomy virtual reality simulation training did not lead to a significant improvement of performance or self-directedness in this study. Our findings suggest that learning an advanced temporal bone procedure such as cochlear implantation surgery requires much more training than learning mastoidectomy.
Cochlear Implant Surgery: Learning Curve in Virtual Reality Simulation Training and Transfer of Skills to a 3D-printed Temporal Bone—a prospective Trial.
Objective: Mastering Cochlear Implant (CI) surgery requires repeated practice, preferably initiated in a safe – i.e. simulated – environment. Mastoidectomy Virtual Reality (VR) simulation-based training (SBT) is effective, but SBT of CI surgery largely uninvestigated. The learning curve is imperative for understanding surgical skills acquisition and developing competency-based training. Here, we explore learning curves in VR SBT of CI surgery and transfer of skills to a 3D-printed model.
Methods: Prospective, single-arm trial. Twenty-four novice medical students completed a pre-training CI inserting test on a commercially available pre-drilled 3D-printed temporal bone. A training program of 18 VR simulation CI procedures was completed in the Visual Ear Simulator over four sessions. Finally, a post-training test similar to the pre-training test was completed. Two blinded experts rated performances using the validated Cochlear Implant Surgery Assessment Tool (CISAT). Performance scores were analyzed using linear mixed models.
Results: Learning curves were highly individual with primary performance improvement initially, and small but steady improvements throughout the 18 procedures. CI VR simulation performance improved 33% (p < 0.001). Insertion performance on a 3D-printed temporal bone improved 21% (p < 0.001), demonstrating skills transfer.
Discussion: VR SBT of CI surgery improves novices’ performance. It is useful for introducing the procedure and acquiring basic skills. CI surgery training should pivot on objective performance assessment for reaching pre-defined competency before cadaver – or real-life surgery. Simulation-based training provides a structured and safe learning environment for initial training.
Conclusion: CI surgery skills improve from VR SBT, which can be used to learn the fundamentals of CI surgery.
Automated summative feedback improves performance and retention in simulation training of mastoidectomy: A randomised, controlled trial
Objective: This study aimed to investigate the effects of automated metrics-based summative feedback on performance, retention and cognitive load in distributed virtual reality simulation training of mastoidectomy.
Method: Twenty-four medical students were randomised in two groups and performed 15 mastoidectomies on a distributed virtual reality simulator as practice. The intervention group received additional summative metrics-based feedback; the control group followed standard instructions. Two to three months after training, participants performed a retention test without learning supports.
Results: The intervention group had a better final-product score (mean difference = 1.0 points; p = 0.001) and metrics-based score (mean difference = 12.7; p < 0.001). At retention, the metrics-based score for the intervention group remained superior (mean difference = 6.9 per cent; p = 0.02). Also at the retention, cognitive load was higher in the intervention group (mean difference = 10.0 per cent; p < 0.001).
Conclusion: Summative metrics-based feedback improved performance and lead to a safer and faster performance compared with standard instructions and seems a valuable educational tool in the early acquisition of temporal bone skills.
Atlas-Based Segmentation of Cochlear MicroStructures in Cone Beam CT
Purpose: To develop an automated segmentation approach for cochlear microstructures [scala tympani (ST), scala vestibuli (SV), modiolus (Mod), mid-modiolus (Mid-Mod), and round window membrane (RW)] in clinical cone beam computed tomography (CBCT) images of the temporal bone for use in surgical simulation software and for preoperative surgical evaluation.
Methods: This approach was developed using the publicly available OpenEar (OE) Library that includes temporal bone specimens with spatially registered CBCT and 3D micro-slicing images. Five of these datasets were spatially aligned to our internal OSU atlas. An atlas of cochlear microstructures was created from one of the OE datasets. An affine registration of this atlas to the remaining OE CBCT images was used for automatically segmenting the cochlear microstructures. Quantitative metrics and visual review were used for validating the automatic segmentations.
Results: The average DICE metrics were 0.77 and 0.74 for the ST and SV, respectively. The average Hausdorff distance (AVG HD) was 0.11 mm and 0.12 mm for both scalae. The mean distance between the centroids for the round window was 0.32 mm, and the mean AVG HD was 0.09 mm. The mean distance and angular rotation between the mid-modiolar axes were 0.11 mm and 9.8 degrees, respectively. Visually, the segmented structures were accurate and similar to that manually traced by an expert observer.
Conclusions: An atlas-based approach using 3D micro-slicing data and affine spatial registration in the cochlear region was successful in segmenting cochlear microstructures of temporal bone anatomy for use in simulation software and potentially for pre-surgical planning and rehearsal.
Assessing competence in cochlear implant surgery using the newly developed Cochlear Implant Surgery Assessment Tool
Purpose: To develop and gather validity evidence for a novel tool for assessment of cochlear implant (CI) surgery, including virtual reality CI surgery training.
Methods: Prospective study gathering validity evidence according to Messick’s framework. Four experts developed the CI Surgery Assessment Tool (CISAT). A total of 35 true novices (medical students), trained novices (residents) and CI surgeons performed two CI-procedures each in the Visible Ear Simulator, which were rated by three blinded experts. Classical test theory and generalizability theory were used for reliability analysis.
Results: The CISAT significantly discriminated between the three groups (p < 0.001). The generalizability coefficient was 0.76 and most of the score variance (53.3%) was attributable to the participant and only 6.8% to the raters. When exploring a standard setting for CI surgery, the contrasting groups method suggested a pass/fail score of 36.0 points (out of 55), but since the trained novices performed above this, we propose using the mean CI surgeon performance score (45.3 points).
Conclusion: Validity evidence for simulation-based assessment of CI performance supports the CISAT. Together with the standard setting, the CISAT might be used to monitor progress in competency-based training of CI surgery and to determine when the trainee can advance to further training.
3D-printed models for temporal bone surgical training: A systematic review.
Objective: 3D-printed models hold great potential for temporal bone surgical training as a supplement to cadaveric dissection. Nevertheless, critical knowledge on manufacturing remains scattered, and little is known about whether use of these models improves surgical performance. This systematic review aims to explore (1) methods used for manufacturing and (2) how educational evidence supports using 3D-printed temporal bone models.
Data sources: PubMed, Embase, the Cochrane Library, and Web of Science.
Review methods: Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, relevant studies were identified and data on manufacturing and validation and/or training extracted by 2 reviewers. Quality assessment was performed using the Medical Education Research Study Quality Instrument tool; educational outcomes were determined according to Kirkpatrick’s model.
Results: The search yielded 595 studies; 36 studies were found eligible and included for analysis. The described 3D-printed models were based on computed tomography scans from patients or cadavers. Processing included manual segmentation of key structures such as the facial nerve; postprocessing, for example, consisted of removal of print material inside the model. Overall, educational quality was low, and most studies evaluated their models using only expert and/or trainee opinion (ie, Kirkpatrick level 1). Most studies reported positive attitudes toward the models and their potential for training.
Conclusion: Manufacturing and use of 3D-printed temporal bones for surgical training are widely reported in the literature. However, evidence to support their use and knowledge about both manufacturing and the effects on subsequent surgical performance are currently lacking. Therefore, stronger educational evidence and manufacturing knowhow are needed for widespread implementation of 3D-printed temporal bones in surgical curricula.
Ultra-high-fidelity virtual reality mastoidectomy simulation training—a randomized, controlled trial.
PURPOSE: Ultra-high-fidelity (UHF) graphics in virtual reality (VR) simulation might improve surgical skill acquisition in temporal bone training. This study aims to compare UHF VR simulation training with conventional, screen-based VR simulation training (cVR) with respect to performance and cognitive load (CL).
METHODS: In a randomized trial with a cross-over design, 24 students completed a total of four mastoidectomies in a VR temporal bone surgical simulator: two performances under UHF conditions using a digital microscope and two performances under conventional conditions using screen-based VR simulation. Performances were assessed by two blinded raters using an established assessment tool. In addition, CL was estimated as the relative change in secondary-task reaction time during simulation when compared with individual baseline measurements. Data were analyzed using linear mixed model analysis for repeated measurements.
RESULTS: The mean final-product performance score was significantly lower in UHF VR simulation compared to cVR simulation [mean difference 1.0 points out of 17 points, 95% CI (0.2–1.7), p = 0.02]. The most important factor for performance during UHF simulation was the ability to achieve stereovision (mean difference = 3.4 points, p < 0.001). Under the UHF VR condition, CL was significantly higher than during cVR (28% vs. 18%, respectively, p < 0.001).
CONCLUSION: UHF graphics in VR simulation training reduced performance and induced a higher CL in novices than conventional, screen-based VR simulation training. Consequently, UHF VR simulation training should be preceded by cVR training and might be better suited for the training of intermediates or experienced surgeons.
The Use of Generalizability Theory for Exploring Reliability of Assessment of Technical Skills and Sources of Variance: A Systematic Review and Meta-Analysis
Purpose: Competency-based education relies on the validity and reliability of assessment scores. Generalizability (G) theory is well suited to explore the reliability of assessment tools in medical education but has only been applied to a limited extent. This study aimed to systematically review the literature using G-theory to explore the reliability of structured assessment of medical and surgical technical skills and to assess the relative contributions of different factors to variance.
Method: In June 2020, 11 databases, including PubMed, were searched from inception through May 31, 2020. Eligible studies included the use of G-theory to explore reliability in the context of assessment of medical and surgical technical skills. Descriptive information on study, assessment context, assessment protocol, participants being assessed, and G-analyses were extracted. Data were used to map G-theory and explore variance components analyses. A meta-analyses was conducted to synthesize the extracted data on the sources of variance and reliability.
Results: Forty-four studies were included; of these, 39 had sufficient data for meta-analysis. The total pool included 35,284 unique assessments of 31,496 unique performances of 4,154 participants. Person variance had a pooled effect of 44.2% (95% confidence interval [CI] [36.8%-51.5%]). Only assessment tool type (Objective Structured Assessment of Technical Skills-type vs task-based checklist-type) had a significant effect on person variance. The pooled reliability (G-coefficient) was .65 (95% CI [.59-.70]). Most studies included D-studies (39, 89%) and generally seemed to have higher ratios of performances to assessors to achieve a sufficiently reliable assessment.
Conclusions: G-theory is increasingly being used to examine reliability of technical skills assessment in medical education but more rigor in reporting is warranted. Contextual factors can potentially affect variance components and thereby reliability estimates and should be considered, especially in high-stakes assessment. Reliability analysis should be a best practice when developing assessment of technical skills.
The effect of simulator-integrated tutoring for guidance in virtual reality simulation training
INTRODUCTION: Simulation-integrated tutoring in virtual reality (VR) simulation training by green-lighting is a common learning support in simulation-based temporal bone surgical training. However, tutoring overreliance can negatively affect learning. We therefore wanted to investigate the effects of simulator-integrated tutoring on performance and learning.
METHODS: A prospective, educational cohort study of a learning intervention (simulator-integrated tutoring) during repeated and distributed VR simulation training for directed, self-regulated learning of the mastoidectomy procedure. Two cohorts of novices (medical students) were recruited: 16 participants were trained using the intervention program (intermittent simulator-integrated tutoring) and 14 participants constituted a non-tutored reference cohort. Outcomes were final-product performance assessed by two blinded raters, and simulator-recorded metrics.
RESULTS: Simulator-integrated tutoring had a large and positive effect on the final-product performance while turned on (mean difference 3.8 points, p<0.0001). However, this did not translate to a better final-product performance in subsequent non-tutored procedures. The tutored cohort had a better metrics-based score, reflecting higher efficiency of drilling (mean difference 3.6 %, p=0.001). For the individual metrics, simulator-integrated tutoring had mixed effects both during procedures and on the tutored cohort in general (learning effect).
CONCLUSIONS: Simulator-integrated tutoring by green-lighting did not induce a better final-product performance but increased efficiency. The mixed effects on learning could be caused by tutoring overreliance, resulting from a lack of cognitive engagement when the tutor-function is on. Further learning strategies such as feedback should be explored to support novice learning and cognitive engagement.
Reliable assessment of surgical technical skills is dependent on context: an exploration of different variables using Generalizability theory
Purpose: Reliable assessment of surgical skills is vital for competency-based medical training. Several factors influence not only the reliability of judgements but also the number of observations needed for making judgments of competency that are both consistent and reproducible. The aim of this study was to explore the role of various conditions-through the analysis of data from large-scale, simulation-based assessments of surgical technical skills-by examining the effects of those conditions on reliability using Generalizability theory.
Method: Assessment data from large-scale, simulation-based temporal bone surgical training research studies in 2012-2018 were pooled, yielding collectively 3,574 assessments of 1,723 performances. The authors conducted generalizability analyses using an unbalanced random-effects design, and they performed decision studies to explore the effect of the different variables on projections of reliability.
Results: Overall, five observations were needed to achieve a Generalizability coefficient > 0.8. Several variables modified the projections of reliability: increased learner experience necessitated more observations (5 for medical students, 7 for residents, and 8 for experienced surgeons); the more complex cadaveric dissection required fewer observations than virtual reality simulation (2 vs. 5 observations); and increased fidelity simulation graphics reduced the number of observations needed from 7 to 4. The training structure (either massed or distributed practice) and simulator-integrated tutoring had little effect on reliability. Finally, more observations were needed during initial training when the learning curve was steepest (6 observations) compared with the plateau phase (4 observations).
Conclusions: Reliability in surgical skills assessment seems less stable than it is often reported to be. Training context and conditions influence reliability. The findings from this study highlight that medical educators should exercise caution when using a specific simulation-based assessment in other contexts.
Letter on “3D printed patient individualized models versus cadaveric models in an undergraduate Oral and Maxillofacial Surgery Curriculum: Comparison of student’s perceptions"
Effects on cognitive load of tutoring in virtual reality simulation training
Aims: According to the guidance hypothesis, tutoring during technical skills training can result in tutoring over-reliance, reflected in a negative effect on performance when tutoring is discontinued. In this study, we wanted to explore if similar effects would be found for cognitive load.
Methods: Two cohorts of novice medical students were recruited for distributed virtual simulation training (five practice blocks of three procedures): 16 participants received intermittent simulator-integrated tutoring and 14 participants served as a reference cohort and did not receive simulator-integrated tutoring. Cognitive load during simulation was estimated using secondary task reaction time. Linear mixed models were used to account for repeated measurements.
Results: Overall, the tutored cohort had a significantly higher cognitive load than the reference cohort (mean difference = 7 %, p=0.006). Simulator-integrated tutoring did seem to lower cognitive load when active but also caused the tutored cohort to have a substantially higher cognitive load in subsequent performances where it was turned off (mean difference = 7 %, respectively, p<<0.001).
Conclusions: Concurrent feedback by simulator-integrated tutoring causes tutoring over-reliance and modifies cognitive load. This suggests that tutoring, in addition to degrading motor skills learning also affects the cognitive processes involved.
Evidence of Mobile Apps in ORL targeted at patients
Development and validation of an assessment tool for technical skills in handheld otoscopy
OBJECTIVE: Handheld otoscopy requires both technical and diagnostic skills, and is often reported to be insufficient after medical training. We aimed to develop and gather validity evidence for an assessment tool for handheld otoscopy using contemporary medical educational standards.
STUDY DESIGN: Educational study.
SETTING: University/teaching hospital.
SUBJECTS AND METHODS: A structured Delphi methodology was used to develop the assessment tool: nine key opinion leaders (otologists) in undergraduate training of otoscopy iteratively achieved consensus on the content. Next, validity evidence was gathered by the video-taped assessment of two handheld otoscopy performances of 15 medical students (novices) and 11 specialists in otorhinolaryngology using two raters. Standard setting (pass/fail criteria) was explored using the contrasting groups and Angoff methods.
RESULTS: The developed Copenhagen Assessment Tool of Handheld Otoscopy Skills (CATHOS) consists 10 items rated using a 5-point Likert scale with descriptive anchors. Validity evidence was collected and structured according to Messick’s framework: for example the CATHOS had excellent discriminative validity (mean difference in performance between novices and experts 20.4 out of 50 points, p<0.001); and high internal consistency (Cronbach’s alpha=0.94). Finally, a pass/fail score was established at 30 points for medical students and 42 points for specialists in ORL.
CONCLUSION: We have developed and gathered validity evidence for an assessment tool of technical skills of handheld otoscopy and set standards of performance. Standardized assessment allows for individualized learning to the level of proficiency and could be implemented in under- and postgraduate handheld otoscopy training curricula, and is also useful in evaluating training interventions.
Developing a national e-learning course in Otorhinolaryngology: the Danish experience
PURPOSE: E-learning provides a flexible and effective approach to learning and is increasingly used in otorhinolaryngology (ORL). We developed a national theoretical e-learning course for ORL trainees and describe our experiences with implementation as well as piloting the e-learning course.
METHODS: E-learning course content was developed as structured multiple-choice quizzes for the European core curriculum textbook. An open source learning management system was adapted for a self-paced e-learning course. We piloted the e-learning course as a non-mandatory option for the 15 residents participating in the Danish four-day national training course in rhinology in February 2019. A post-course survey was sent out to the participants and used in the evaluation along with activity data from the learning management system.
RESULTS: Fourteen out of 15 trainees participated in the e-learning course. Nine participants completed >95 % of the course. The activity data demonstrated that participants with the highest completion rate typically began well in advance of the course (>2 months). Overall the e-learning course was rated positively in relation to learning and as preparation for the national training course. Participants responded that the level of the e-learning course was higher than and also at times in slight incongruity with the content of the national curriculum. Participants would like protected study time for e-learning activities in their residency program. All participants responded that they would use e-learning in relation to future national training courses.
CONCLUSIONS: Developing a national e-learning course is feasible and is well-received by trainees as well as other educational stakeholders.
Design and validation of a cross-specialty simulation-based training course in basic robotic surgical skills
Background: The aim of this study was to design and validate a cross-specialty basic robotic surgical skills training program on the RobotiX Mentor virtual reality simulator.
Methods: A Delphi panel reached consensus on six modules to include in the training program. Validity evidence was collected according to Messick’s framework with three performances in each simulator module by 11 experienced robotic surgeons and 11 residents without robotic surgical experience.
Results: For five of the six modules, a compound metrics-based score could significantly discriminate between the performances of novices and experienced robotic surgeons. Pass/fail levels were established, resulting in very few novices passing in their first attempt.
Conclusions: This validated course can be used for structured simulation-based basic robotic surgical skills training within a mastery learning framework where the individual trainee can practice each module until they achieve proficiency and can continue training on other modalities and more specific to their specialty.
Understanding the effects of structured self-assessment in directed, self-regulated simulation-based training of mastoidectomy: a mixed methods study
OBJECTIVE: Self-directed training represents a challenge in simulation-based training as low cognitive effort can occur when learners overrate their own level of performance. This study aims to explore the mechanisms underlying the positive effects of a structured self-assessment intervention during simulation-based training of mastoidectomy.
METHODS: A prospective, educational cohort study of a novice training program consisting of directed, self-regulated learning with distributed practice (5×3 procedures) in a virtual reality temporal bone simulator. The intervention consisted of structured self-assessment after each procedure using a rating form supported by small videos. Semi-structured telephone interviews upon completion of training were conducted with 13 out of 15 participants. Interviews were analysed using directed content analysis and triangulated with quantitative data on secondary task reaction time for cognitive load estimation and participants’ self-assessment scores.
RESULTS: Six major themes were identified in the interviews: goal-directed behaviour, use of learning supports for scaffolding of the training, cognitive engagement, motivation from self-assessment, self-assessment bias, and feedback on self-assessment (validation). Participants seemed to self-regulate their learning by forming individual sub-goals and strategies within the overall goal of the procedure. They scaffolded their learning through the available learning supports. Finally, structured self-assessment was reported to increase the participants’ cognitive engagement, which was further supported by a quantitative increase in cognitive load.
CONCLUSIONS: Structured self-assessment in simulation-based surgical training of mastoidectomy seems to promote cognitive engagement and motivation in the learning task and to facilitate self-regulated learning.
The effect of structured self-assessment in virtual reality simulation training of mastoidectomy.
PURPOSE: Virtual reality (VR) simulation surgical skills training is well established, but self-directed practice is often associated with a learning curve plateau. In this study, we investigate the effects of structured self-assessment as a means to improve performance in mastoidectomy training.
METHODS: The study was a prospective, educational study. Two cohorts of novices (medical students) were recruited for practice of anatomical mastoidectomy in a training program with five distributed training blocks. Fifteen participants performed structured self-assessment after each procedure (intervention cohort). A reference cohort of another 14 participants served as controls. Performances were assessed by two blinded raters using a modified Welling Scale and simulator-recorded metrics.
RESULTS: The self-assessment cohort performed superiorly to the reference cohort (mean difference of final product score 0.87 points, p = 0.001) and substantially reduced the number of repetitions needed. The self-assessment cohort also had more passing performances for the combined metrics-based score reflecting increased efficiency. Finally, the self-assessment cohort made fewer collisions compared with the reference cohort especially with the chorda tympani, the facial nerve, the incus, and the malleus.
CONCLUSIONS: VR simulation training of surgical skills benefits from having learners perform structured self-assessment following each procedure as this increases performance, accelerates the learning curve thereby reducing time needed for training, and induces a safer performance with fewer collisions with critical structures. Structured self-assessment was in itself not sufficient to counter the learning curve plateau and for continued skills development additional supports for deliberate practice are needed.
Standard setting of competency in mastoidectomy for the Cross-Institutional Mastoidectomy Assessment Tool
OBJECTIVE: Competency-based surgical training involves progressive autonomy given to the trainee. This requires systematic and evidence-based assessment with well-defined standards of proficiency. The objective of this study is to develop standards for the cross-institutional mastoidectomy assessment tool to inform decisions regarding whether a resident demonstrates sufficient skill to perform a mastoidectomy with or without supervision.
METHODS: A panel of fellowship-trained content experts in mastoidectomy was surveyed in relation to the 16 items of the assessment tool to determine the skills needed for supervised and unsupervised surgery. We examined the consensus score to investigate the degree of agreement among respondents for each survey item as well as additional analyses to determine whether the reported skill level required for each survey item was significantly different for the supervised versus unsupervised level.
RESULTS: Ten panelists representing different US training programs responded. There was considerable consensus on cut-off scores for each item and trainee level between panelists, with moderate (0.62) to very high (0.95) consensus scores depending on assessment item. Further analyses demonstrated that the difference between supervised and unsupervised skill levels was significantly meaningful for all items. Finally, minimum-passing scores for each item was established.
CONCLUSION: We defined performance standards for the cross-institutional mastoidectomy assessment tool using the Angoff method. These cut-off scores that can be used to determine when trainees can progress from performance under supervision to performance without supervision. This can be used to guide training in a competency-based training curriculum.
Performance metrics in mastoidectomy training: A systematic review
OBJECTIVE: To investigate validity evidence, and strengths and limitations of performance metrics in mastoidectomy training.
METHODS: A systematic review following the PRISMA guidelines. Studies reporting performance metrics in mastoidectomy/temporal bone surgery were included. Data on design, outcomes, and results were extracted by two reviewers. Validity evidence according to Messick’s framework and level of evidence were assessed.
RESULTS: The search yielded a total of 1085 studies from the years 1947-2018 and 35 studies were included for full data extraction after abstract and full-text screening. 33 different metrics on mastoidectomy performance were identified and ranked according to the number of reports. Most of the 33 metrics identified had some amount of validity evidence. The metrics with most validity evidence were related to drilling time, volume drilled per time, force applied near vital structures, and volume removed.
CONCLUSIONS: This review provides an overview of current metrics of mastoidectomy performance, their validity, strengths and limitations, and identifies the gap in validity evidence of some metrics. Evidence-based metrics can be used for performance assessment in temporal bone surgery and for providing integrated and automated feedback in virtual reality simulation training. The use of such metrics in simulation-based mastoidectomy training can potentially address some of the limitations in current temporal bone skill assessment and ease assessment in repeated practice. However, at present, an automated feedback based on metrics in VR simulation does not have sufficient empirical basis and has not been generally accepted for use in training and certification.
Letter to the Editor: Design and fabrication of a generic 3D-printed silicone unilateral cleft lip and palate model
Identifying and prioritizing technical procedures in otorhinolaryngology for simulation-based training: a national needs assessment in Denmark
PURPOSE: To conduct a national needs assessment using a structured approach to identify and prioritize technical skills and procedures in otorhinolaryngology (ORL) for simulation-based training.
METHODS: The study was designed as a national Danish survey of key educational stakeholders in ORL. A Delphi methodology with three rounds was used: the first round constituted a brainstorming phase to identify relevant procedures; the second round was a survey of importance, frequency, number of physicians needed to train, and patient safety/discomfort of the procedures, and feasibility of simulation-based training; and a final third round for prioritization.
RESULTS: A total of 62 key opinion leaders were identified and 50 responded in the first round, constituting our panel. Fifty technical skills and procedures were identified in the brainstorming phase and were sent out for assessment, with responses from 56.5% of still eligible panellists. Thirty-six procedures were found important in ORL residency training by the panel. After final prioritization by the panel (response rate 43.4%), there was broad consensus (> 75%) on the need for simulation-based training of 13 technical skills and procedures, with the most highly ranking procedures being emergency cricothyroidotomy, flexible fibre pharyngo-laryngoscopy, and basic surgical skills.
CONCLUSIONS: As educational decisions are increasingly required to be evidence-based, this study represents a structured approach to identifying procedures for simulation-based training in ORL. This information can be valuable in the development and implementation of simulation-based training programmes in the ORL residency training curriculum.
Failure affects subjective estimates of cognitive load through a negative carry-over effect in virtual reality simulation of hip fracture surgery
BACKGROUND: Cognitive overload can impair learning, and different factors might affect cognitive load during simulation-based training. In this study, we investigate the role of failure in repeated practice of virtual reality (VR) simulation of hip fracture surgery on cognitive load (CL) estimated by secondary-task reaction time test and two questionnaires.
METHODS: The VR simulation training program consisted of three competency levels of increasing complexity starting with the placement of a Kirschner wire in a fractured hip of one patient, adding clinical variability at the intermediate level, and performing the entire dynamic hip screw procedure in 24 different patients at the highest level. Thirteen consecutive passed simulations were required to advance to the next level. Performance was measured as passing/failing a procedure and the number of failed procedures within the latest three and five simulations. CL was measured objectively using reaction time testing during simulation and subjectively using the NASA-TLX and PAAS questionnaires. The study was carried out at a simulation center from November 2016 to March 2019. Forty-two first-year orthopedic surgery residents from the Central Denmark Region and the North Denmark Region participated in the training program.
RESULTS: A failing performance in the simulated procedure was associated with a higher CL than passing a procedure. The subjective CL estimates were affected by the number of failures during last three and five procedures with a higher number of failures being associated with a higher CL. In contrast, relative reaction time estimates of CL were not affected by previous failures.
CONCLUSIONS: Questionnaires for estimation of CL seem to be affected by participant frustration after failure-a meta-cognitive “carry-over” effect. This could be a general limitation of the subjective questionnaire approach to estimate CL. Reducing CL through instructional design and handling of participant frustration might improve the learning outcome of simulation training programs.
Expert sampling of VR simulator metrics for automated assessment of mastoidectomy performance.
OBJECTIVE: Often the assessment of mastoidectomy performance requires time-consuming manual rating. Virtual reality (VR) simulators offer potentially useful automated assessment and feedback but should be supported by validity evidence. We aimed to investigate simulator metrics for automated assessment based on the expert performance approach, comparison with an established assessment tool, and the consequences of standard setting.
METHODS: The performances of 11 experienced otosurgeons and 37 otorhinolaryngology residents. Participants performed three mastoidectomies in the Visible Ear Simulator. Nine residents contributed additional data on repeated practice in the simulator. One hundred and twenty-nine different performance metrics were collected by the simulator and final-product files were saved. These final products were analyzed using a modified Welling Scale by two blinded raters.
RESULTS: Seventeen metrics could discriminate between resident and experienced surgeons’ performances. These metrics mainly expressed various aspects of efficiency: Experts demonstrated more goal-directed behavior and less hesitancy, used less time, and selected large and sharp burrs more often. The combined metrics-based score (MBS) demonstrated significant discriminative ability between experienced surgeons and residents with a mean difference of 16.4% (95% confidence interval [12.6-20.2], P << 0.001). A pass/fail score of 83.6% was established. The MBS correlated poorly with the final-product score but excellently with the final-product score per time.
CONCLUSION: The MBS mainly reflected efficiency components of the mastoidectomy procedure, and although it could have some uses in self-directed training, it fails to measure and encourage safe routines. Supplemental approaches and feedback are therefore required in VR simulation training of mastoidectomy.
Decentralized Virtual Reality Training of Mastoidectomy Improves Cadaver Dissection Performance: A Prospective, Controlled Cohort Study
OBJECTIVE: Virtual reality (VR) simulation training can improve temporal bone (TB) cadaver dissection skills and distributed, self-regulated practice is optimal for skills consolidation. Decentralized training (DT) at the trainees’ own department or home offers more convenient access compared with centralized VR simulation training where the simulators are localized at one facility. The effect of DT in TB surgical training is unknown. We investigated the effect of decentralized VR simulation training of TB surgery on subsequent cadaver dissection performance.
STUDY DESIGN: Prospective, controlled cohort study.
SETTING: Otorhinolaryngology (ORL) teaching hospitals and the Danish national TB course.
PARTICIPANTS: Thirty-eight ORL residents: 20 in the intervention cohort (decentralized training) and 18 in the control cohort (standard training during course).
INTERVENTION: Three months of access to decentralized VR simulation training at the local ORL department or the trainee’s home. A freeware VR simulator (the visible ear simulator [VES]) was used, supplemented by a range of learning supports for directed, self-regulated learning.
MAIN OUTCOME MEASURE: Mastoidectomy final-product scores from the VR simulations and cadaver dissection were rated using a modified Welling Scale by blinded expert raters.
RESULTS: Participants in the intervention cohort trained decentrally a median of 3.5 hours and performed significantly better than the control cohort during VR simulation (p < 0.01), which importantly also transferred to a 76% higher performance score during subsequent cadaver training (mean scores: 8.8 versus 5.0 points; p < 0.001).
CONCLUSIONS: Decentralized VR simulation training of mastoidectomy improves subsequent cadaver dissection performance and can potentially improve implementation of VR simulation surgical training.
Decentralized Virtual Reality Mastoidectomy Simulation Training: A Prospective, Mixed-Methods Study
PURPOSE: Virtual reality (VR) training of mastoidectomy is effective in surgical training-particularly if organized as distributed practice. However, centralization of practice facilities is a barrier to implementation of distributed simulation training. Decentralized training could be a potential solution. Here, we aim to assess the feasibility, use, and barriers to decentralized VR mastoidectomy training using a freeware, high-fidelity temporal bone simulator.
METHODS: In a prospective, mixed-methods study, 20 otorhinolaryngology residents were given three months of local access to a VR mastoidectomy simulator. Additionally, trainees were provided a range of learning supports for directed, self-regulated learning. Questionnaire data were collected and focus group interviews conducted. The interviews were analyzed using thematic analysis and compared with quantitative findings.
RESULTS: Participants trained 48.5 h combined and mainly towards the end of the trial. Most participants used between two and four different learning supports. Qualitative analysis revealed five main themes regarding implementation of decentralized simulation training: convenience, time for training, ease of use, evidence for training, and testing.
CONCLUSIONS: Decentralized VR training using a freeware, high-fidelity mastoidectomy simulator is feasible but did not lead to a high training volume or truly distributed practice. Evidence for training was found motivational. Access to training, educational designs, and the role of testing are important for participant motivation and require further evaluation.
Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial
BACKGROUND: Virtual reality simulators combined with head-mounted displays enable highly immersive virtual reality (VR) for surgical skills training, potentially bridging the gap between the simulation environment and real-life operating room conditions. However, the increased complexity of the learning situation in immersive VR could potentially induce high cognitive load thereby inhibiting performance and learning. This study aims to compare cognitive load and performance in immersive VR and conventional VR simulation training.
METHODS: A randomized controlled trial of residents (n = 31) performing laparoscopic salpingectomies with an ectopic pregnancy in either immersive VR or conventional VR simulation. Cognitive load was estimated by secondary-task reaction time at baseline, and during nonstressor and stressor phases of the procedure. Simulator metrics were used to evaluate performance.
RESULTS: Cognitive load was increased by 66% and 58% during immersive VR and conventional VR simulation, respectively (p < 0.001), compared to baseline. A light stressor induced a further increase in cognitive load by 15.2% and a severe stressor by 43.1% in the immersive VR group compared to 23% (severe stressor) in the conventional VR group. Immersive VR also caused a significantly worse performance on most simulator metrics.
CONCLUSION: Immersive VR simulation training induces a higher cognitive load and results in a poorer performance than conventional VR simulation training in laparoscopy. High extraneous load and element interactivity in the immersive VR are suggested as mechanisms explaining this finding. However, immersive VR offers some potential advantages over conventional VR such as more real-life conditions but we only recommend introducing immersive VR in surgical skills training after initial training in conventional VR.
Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure
BACKGROUND: The use of robotic surgery for minimally invasive procedures has increased considerably over the last decade. Robotic surgery has potential advantages compared to laparoscopic surgery but also requires new skills. Using virtual reality (VR) simulation to facilitate the acquisition of these new skills could potentially benefit training of robotic surgical skills and also be a crucial step in developing a robotic surgical training curriculum. The study’s objective was to establish validity evidence for a simulation-based test for procedural competency for the vaginal cuff closure procedure that can be used in a future simulation-based, mastery learning training curriculum.
METHODS: Eleven novice gynaecological surgeons without prior robotic experience and 11 experienced gynaecological robotic surgeons (> 30 robotic procedures) were recruited. After familiarization with the VR simulator, participants completed the module ‘Guided Vaginal Cuff Closure’ six times. Validity evidence was investigated for 18 preselected simulator metrics. The internal consistency was assessed using Cronbach’s alpha and a composite score was calculated based on metrics with significant discriminative ability between the two groups. Finally, a pass/fail standard was established using the contrasting groups’ method.
RESULTS: The experienced surgeons significantly outperformed the novice surgeons on 6 of the 18 metrics. The internal consistency was 0.58 (Cronbach’s alpha). The experienced surgeons’ mean composite score for all six repetitions were significantly better than the novice surgeons’ (76.1 vs. 63.0, respectively, p < 0.001). A pass/fail standard of 75/100 was established. Four novice surgeons passed this standard (false positives) and three experienced surgeons failed (false negatives).
CONCLUSION: Our study has gathered validity evidence for a simulation-based test for procedural robotic surgical competency in the vaginal cuff closure procedure and established a credible pass/fail standard for future proficiency-based training.
The effect of distributed virtual reality simulation training on cognitive load during subsequent dissection training
BACKGROUND: Complex tasks such as surgical procedures can induce excessive cognitive load (CL), which can have a negative effect on learning, especially for novices.
AIM: To investigate if repeated and distributed virtual reality (VR) simulation practice induces a lower CL and higher performance in subsequent cadaveric dissection training.
METHODS: In a prospective, controlled cohort study, 37 residents in otorhinolaryngology received VR simulation training either as additional distributed practice prior to course participation (intervention) (9 participants) or as standard practice during the course (control) (28 participants). Cognitive load was estimated as the relative change in secondary-task reaction time during VR simulation and cadaveric procedures.
RESULTS: Structured distributed VR simulation practice resulted in lower mean reaction times (32% vs. 47% for the intervention and control group, respectively, p < 0.01) as well as a superior final-product performance during subsequent cadaveric dissection training.
CONCLUSIONS: Repeated and distributed VR simulation causes a lower CL to be induced when the learning situation is increased in complexity. A suggested mechanism is the formation of mental schemas and reduction of the intrinsic CL. This has potential implications for surgical skills training and suggests that structured, distributed training be systematically implemented in surgical training curricula.
The Effect of a Distributed Virtual Reality Simulation Training Program on Dissection Mastoidectomy Performance
OBJECTIVE: To investigate the effect on final-product performance of a distributed, virtual reality (VR) simulation training program on cadaveric dissection performance and learning curves compared with standard VR simulation training during a temporal bone course.
STUDY DESIGN: Educational interventional cohort study.
SETTING: The national Danish temporal bone courses of 2016 and 2017.
SUBJECTS: Postgraduate year 2 to 5 residents in otorhinolaryngology.
INTERVENTION: Nine participants volunteered for additional VR simulation training (intervention) before the temporal bone course, with training blocks distributed (i.e., separated). The remaining 28 participants received standard VR simulation training during the temporal bone course (control).
MAIN OUTCOME MEASURE: VR simulation and cadaveric dissection final-product performances were analyzed by blinded raters using a 26-item modified Welling Scale.
RESULTS: Distributed VR simulation training before the temporal bone course (intervention) significantly increased dissection final-product performance by 25% compared with standard VR simulation training during the course (control) (mean scores 12.8 points versus 10.3 points, p < 0.01). Distributed and repeated VR simulation practice markedly decreased drilling time. Guidance by the simulator-integrated tutor-function significantly increased final-product performance by 2.3 points compared with nontutored procedures but at the cost of increased drilling time.
CONCLUSION: Skills acquired in a VR simulation environment translate to cadaveric dissection skills and repeated and distributed VR simulation can be used to further increase performance compared with standard VR simulation training during a temporal bone course. Further dissemination of inexpensive VR simulators would allow all future temporal bone course participants to train locally before attending future centralized courses.
European status on temporal bone training: a questionnaire study
PURPOSE: In otorhinolaryngology training, introduction to temporal bone surgery through hands-on practice on cadaveric human temporal bones is the gold-standard training method before commencing supervised surgery. During the recent decades, the availability of such specimens and the necessary laboratory facilities for training seems to be decreasing. Alternatives to traditional training can consist of drilling artificial models made of plaster or plastic but also virtual reality (VR) simulation. Nevertheless, the integration and availability of these alternatives into specialist training programs remain unknown.
METHODS: We conducted a questionnaire study mapping current status on temporal bone training and included responses from 113 departments from 23 countries throughout Europe.
RESULTS: In general, temporal bone training during residency in ORL is organized as in-house training, or as participation in national or international temporal bone courses or some combination hereof. There are considerable differences in the availability of training facilities for temporal bone surgery and the number of drillings each ORL trainee can perform. Cadaveric dissection is still the most commonly used training modality.
CONCLUSIONS: VR simulation and artificial models are reported to be used at many leading training departments already. Decreasing availability of cadavers, lower costs of VR simulation and artificial models, in addition to established evidence for a positive effect on the trainees’ competency, were reported as the main reasons. Most remaining departments expect to implement VR simulation and artificial models for temporal bone training into their residency programs in the near future.
Authors’ response to Commentary on European status on temporal bone training: a questionnaire study
Response to: Escada PA. Commentary on “European status on temporal bone training: a questionnaire study”. Eur Arch Otorhinolaryngol. 2018;275(5):1349-1350. doi: 10.1007/s00405-018-4916-5
Otologic Skills Training
This article presents a summary of the current simulation training for otologic skills. There is a wide variety of educational approaches, assessment tools, and simulators in use, including simple low-cost task trainers to complex computer-based virtual reality systems. A systematic approach to otologic skills training using adult learning theory concepts, such as repeated and distributed practice, self-directed learning, and mastery learning, is necessary for these educational interventions to be effective. Future directions include development of measures of performance to assess efficacy of simulation training interventions and, for complex procedures, improvement in fidelity based on educational goals.
Mapping the plateau of novices in virtual reality simulation training of mastoidectomy
OBJECTIVES/HYPOTHESIS: To explore why novices’ performance plateau in directed, self-regulated virtual reality (VR) simulation training and how performance can be improved.
STUDY DESIGN: Prospective study.
METHODS: Data on the performances of 40 novices who had completed repeated, directed, self-regulated VR simulation training of mastoidectomy were included. Data were analyzed to identify key areas of difficulty as well as the procedures terminated without using all the time allowed.
RESULTS: Novices had difficulty in avoiding drilling holes in the outer anatomical boundaries of the mastoidectomy and frequently made injuries to vital structures such as the lateral semicircular canal, the ossicles, and the facial nerve. The simulator-integrated tutor function improved performance on many of these items, but overreliance on tutoring was observed. Novices also demonstrated poor self-assessment skills and often did not make use of the allowed time, lacking knowledge on when to stop or how to excel.
CONCLUSION: Directed, self-regulated VR simulation training of mastoidectomy needs a strong instructional design with specific process goals to support deliberate practice because cognitive effort is needed for novices to improve beyond an initial plateau.
Virtual reality simulation training of mastoidectomy - studies on novice performance
Virtual reality (VR) simulation-based training is increasingly used in surgical technical skills training including in temporal bone surgery. The potential of VR simulation in enabling high-quality surgical training is great and VR simulation allows high-stakes and complex procedures such as mastoidectomy to be trained repeatedly, independent of patients and surgical tutors, outside traditional learning environments such as the OR or the temporal bone lab, and with fewer of the constraints of traditional training. This thesis aims to increase the evidence-base of VR simulation training of mastoidectomy and, by studying the final-product performances of novices, investigates the transfer of skills to the current gold-standard training modality of cadaveric dissection, the effect of different practice conditions and simulator-integrated tutoring on performance and retention of skills, and the role of directed, self-regulated learning. Technical skills in mastoidectomy were transferable from the VR simulation environment to cadaveric dissection with significant improvement in performance after directed, self-regulated training in the VR temporal bone simulator. Distributed practice led to a better learning outcome and more consolidated skills than massed practice and also resulted in a more consistent performance after three months of non-practice. Simulator-integrated tutoring accelerated the initial learning curve but also caused over-reliance on tutoring, which resulted in a drop in performance when the simulator-integrated tutor-function was discontinued. The learning curves were highly individual but often plateaued early and at an inadequate level, which related to issues concerning both the procedure and the VR simulator, over-reliance on the tutor function and poor self-assessment skills. Future simulator-integrated automated assessment could potentially resolve some of these issues and provide trainees with both feedback during the procedure and immediate assessment following each procedure. Standard setting by establishing a proficiency level that can be used for mastery learning with deliberate practice could also further sophisticate directed, self-regulated learning in VR simulation-based training. VR simulation-based training should be embedded in a systematic and competency-based training curriculum for high-quality surgical skills training, ultimately leading to improved safety and patient care.
The effect of self-directed virtual reality simulation on dissection training performance in mastoidectomy
OBJECTIVES/HYPOTHESIS: To establish the effect of self-directed virtual reality (VR) simulation training on cadaveric dissection training performance in mastoidectomy and the transferability of skills acquired in VR simulation training to the cadaveric dissection training setting.
STUDY DESIGN: Prospective study.
METHODS: Two cohorts of 20 novice otorhinolaryngology residents received either self-directed VR simulation training before cadaveric dissection training or vice versa. Cadaveric and VR simulation performances were assessed using final-product analysis with three blinded expert raters.
RESULTS: The group receiving VR simulation training before cadaveric dissection had a mean final-product score of 14.9 (95 % confidence interval [CI] [12.9-16.9]) compared with 9.8 (95% CI [8.4-11.1]) in the group not receiving VR simulation training before cadaveric dissection. This 52% increase in performance was statistically significantly (P < 0.0001). A single dissection mastoidectomy did not increase VR simulation performance (P = 0.22).
CONCLUSIONS: Two hours of self-directed VR simulation training was effective in increasing cadaveric dissection mastoidectomy performance and suggests that mastoidectomy skills are transferable from VR simulation to the traditional dissection setting. Virtual reality simulation training can therefore be employed to optimize training, and can spare the use of donated material and instructional resources for more advanced training after basic competencies have been acquired in the VR simulation environment.
LEVEL OF EVIDENCE: NA.
The effect of implementing cognitive load theory-based design principles in virtual reality simulation training of surgical skills: a randomized controlled trial
BACKGROUND: Cognitive overload can inhibit learning, and cognitive load theory-based instructional design principles can be used to optimize learning situations. This study aims to investigate the effect of implementing cognitive load theory-based design principles in virtual reality simulation training of mastoidectomy.
METHODS: Eighteen novice medical students received 1 h of self-directed virtual reality simulation training of the mastoidectomy procedure randomized for standard instructions (control) or cognitive load theory-based instructions with a worked example followed by a problem completion exercise (intervention). Participants then completed two post-training virtual procedures for assessment and comparison. Cognitive load during the post-training procedures was estimated by reaction time testing on an integrated secondary task. Final-product analysis by two blinded expert raters was used to assess the virtual mastoidectomy performances.
RESULTS: Participants in the intervention group had a significantly increased cognitive load during the post-training procedures compared with the control group (52 vs. 41 %, p = 0.02). This was also reflected in the final-product performance: the intervention group had a significantly lower final-product score than the control group (13.0 vs. 15.4, p < 0.005).
CONCLUSIONS: Initial instruction using worked examples followed by a problem completion exercise did not reduce the cognitive load or improve the performance of the following procedures in novices. Increased cognitive load when part tasks needed to be integrated in the post-training procedures could be a possible explanation for this. Other instructional designs and methods are needed to lower the cognitive load and improve the performance in virtual reality surgical simulation training of novices.
Retention of Mastoidectomy Skills After Virtual Reality Simulation Training
IMPORTANCE: The ultimate goal of surgical training is consolidated skills with a consistently high performance. However, surgical skills are heterogeneously retained and depend on a variety of factors, including the task, cognitive demands, and organization of practice. Virtual reality (VR) simulation is increasingly being used in surgical skills training, including temporal bone surgery, but there is a gap in knowledge on the retention of mastoidectomy skills after VR simulation training.
OBJECTIVES: To determine the retention of mastoidectomy skills after VR simulation training with distributed and massed practice and to investigate participants’ cognitive load during retention procedures.
DESIGN, SETTING, AND PARTICIPANTS: A prospective 3-month follow-up study of a VR simulation trial was conducted from February 6 to September 19, 2014, at an academic teaching hospital among 36 medical students: 19 from a cohort trained with distributed practice and 17 from a cohort trained with massed practice.
INTERVENTIONS: Participants performed 2 virtual mastoidectomies in a VR simulator a mean of 3.2 months (range, 2.4-5.0 months) after completing initial training with 12 repeated procedures. Practice blocks were spaced apart in time (distributed), or all procedures were performed in 1 day (massed).
MAIN OUTCOMES AND MEASURES: Performance of the virtual mastoidectomy as assessed by 2 masked senior otologists using a modified Welling scale, as well as cognitive load as estimated by reaction time to perform a secondary task.
RESULTS: Among 36 participants, mastoidectomy final-product skills were largely retained at 3 months (mean change in score, 0.1 points; P = .89) regardless of practice schedule, but the group trained with massed practice took more time to complete the task. The performance of the massed practice group increased significantly from the first to the second retention procedure (mean change, 1.8 points; P = .001), reflecting that skills were less consolidated. For both groups, increases in reaction times in the secondary task (distributed practice group: mean pretraining relative reaction time, 1.42 [95% CI, 1.37-1.47]; mean end of training relative reaction time, 1.24 [95% CI, 1.16-1.32]; and mean retention relative reaction time, 1.36 [95% CI, 1.30-1.42]; massed practice group: mean pretraining relative reaction time, 1.34 [95% CI, 1.28-1.40]; mean end of training relative reaction time, 1.31 [95% CI, 1.21-1.42]; and mean retention relative reaction time, 1.39 [95% CI, 1.31-1.46]) indicated that cognitive load during the virtual procedures had returned to the pretraining level.
CONCLUSIONS AND RELEVANCE: Mastoidectomy skills acquired under time-distributed practice conditions were retained better than skills acquired under massed practice conditions. Complex psychomotor skills should be regularly reinforced to consolidate both motor and cognitive aspects. Virtual reality simulation training provides the opportunity for such repeated training and should be integrated into training curricula.
Notes From the Field: Secondary Task Precision for Cognitive Load Estimation During Virtual Reality Surgical Simulation Training
Cognitive load (CL) theory suggests that working memory can be overloaded in complex learning tasks such as surgical technical skills training, which can impair learning. Valid and feasible methods for estimating the CL in specific learning contexts are necessary before the efficacy of CL-lowering instructional interventions can be established. This study aims to explore secondary task precision for the estimation of CL in virtual reality (VR) surgical simulation and also investigate the effects of CL-modifying factors such as simulator-integrated tutoring and repeated practice. Twenty-four participants were randomized for visual assistance by a simulator-integrated tutor function during the first 5 of 12 repeated mastoidectomy procedures on a VR temporal bone simulator. Secondary task precision was found to be significantly lower during simulation compared with nonsimulation baseline, p < .001. Contrary to expectations, simulator-integrated tutoring and repeated practice did not have an impact on secondary task precision. This finding suggests that even though considerable changes in CL are reflected in secondary task precision, it lacks sensitivity. In contrast, secondary task reaction time could be more sensitive, but requires substantial postprocessing of data. Therefore, future studies on the effect of CL modifying interventions should weigh the pros and cons of the various secondary task measurements.
OBJECTIVE: To evaluate the short-term stability of postoperative hearing results after tympanoplasty.
STUDY DESIGN: Prospective database study.
SETTING: Tertiary referral center.
PATIENTS: 1,367 cases of tympanoplasty I-IV were registered in the OTOKIR database between February 2004 and November 2013.
INTERVENTION: The authors included the 553 cases attending postoperative follow-ups at both 3 and 12 months.
MAIN OUTCOME MEASURE: Analysis of the changes in pure-tone average of air conduction (AC), air-bone gap, and speech reception threshold and Word Recognition Score between follow-ups were performed.
RESULTS: The overall mean change between follow-ups was 0.7, 0.5, and 0.3 dB for the AC, air-bone gap, and speech reception threshold, respectively. A majority of cases (87.7%) had a change in AC of 10 dB or less, and only 7.6% of the tympanoplasty type I cases had a decrease in AC of more than 10 dB. Of the 1,367 cases registered, 47.5% of cases were lost to follow-up at 12 months.
CONCLUSION: The changes in hearing results after tympanoplasty are minimal during 3 to 12 months after surgery. This suggests that 3-month results are as valid for reporting as 12-month results. In addition, a possible bias that compromises the validity of reported results is introduced at 12 months because half of the cases are lost to follow-up. Including results from 3-month postoperative follow-up when reporting on tympanoplasty could reduce bias in reporting and enable more centers to contribute valid results.
Cognitive Load in Mastoidectomy Skills Training: Virtual Reality Simulation and Traditional Dissection Compared
OBJECTIVE: The cognitive load (CL) theoretical framework suggests that working memory is limited, which has implications for learning and skills acquisition. Complex learning situations such as surgical skills training can potentially induce a cognitive overload, inhibiting learning. This study aims to compare CL in traditional cadaveric dissection training and virtual reality (VR) simulation training of mastoidectomy.
DESIGN: A prospective, crossover study. Participants performed cadaveric dissection before VR simulation of the procedure or vice versa. CL was estimated by secondary-task reaction time testing at baseline and during the procedure in both training modalities.
SETTING: The national Danish temporal bone course.
PARTICIPANTS: A total of 40 novice otorhinolaryngology residents.
RESULTS: Reaction time was increased by 20% in VR simulation training and 55% in cadaveric dissection training of mastoidectomy compared with baseline measurements. Traditional dissection training increased CL significantly more than VR simulation training (p < 0.001).
CONCLUSIONS: VR simulation training imposed a lower CL than traditional cadaveric dissection training of mastoidectomy. Learning complex surgical skills can be a challenge for the novice and mastoidectomy skills training could potentially be optimized by employing VR simulation training first because of the lower CL. Traditional dissection training could then be used to supplement skills training after basic competencies have been acquired in the VR simulation.
Cognitive load in distributed and massed practice in virtual reality mastoidectomy simulation
OBJECTIVES/HYPOTHESIS: Cognitive load theory states that working memory is limited. This has implications for learning and suggests that reducing cognitive load (CL) could promote learning and skills acquisition. This study aims to explore the effect of repeated practice and simulator-integrated tutoring on CL in virtual reality (VR) mastoidectomy simulation.
STUDY DESIGN: Prospective trial.
METHODS: Forty novice medical students performed 12 repeated virtual mastoidectomy procedures in the Visible Ear Simulator: 21 completed distributed practice with practice blocks spaced in time and 19 participants completed massed practice (all practices performed in 1 day). Participants were randomized for tutoring with the simulator-integrated tutor function. Cognitive load was estimated by measuring reaction time in a secondary task. Data were analyzed using linear mixed models for repeated measurements.
RESULTS: The mean reaction time increased by 37% during the procedure compared with baseline, demonstrating that the procedure placed substantial cognitive demands. Repeated practice significantly lowered CL in the distributed practice group but not in massed practice group. In addition, CL was found to be further increased by 10.3% in the later and more complex stages of the procedure. The simulator-integrated tutor function did not have an impact on CL.
CONCLUSION: Distributed practice decreased CL in repeated VR mastoidectomy training more consistently than was seen in massed practice. This suggests a possible effect of skills and memory consolidation occurring over time. To optimize technical skills learning, training should be organized as time-distributed practice rather than as a massed block of practice, which is common in skills-training courses.
CONCLUSION: Current guidelines recommend reporting short-term results of > 12 months after treatment of conductive hearing loss. This study suggests that short-term hearing results after stapedotomy recorded at the 3-month follow-up are without loss of vital information compared with data from the currently recommended > 12-month follow-up. The use of 3-month data in reporting outcome could reduce the bias inherent to the loss to follow-up at 12 months.
OBJECTIVE: To investigate the stability of short-term postoperative hearing after stapedotomy for otosclerosis.
METHODS: This was a prospective database study; 371 cases with otosclerosis were registered in the database between August 2004 and June 2013. We included the 166 primary cases and 37 revision cases that had attended both follow-ups.
RESULTS: The mean changes in postoperative hearing thresholds between the 3-month and 12-month follow-up in both primary and revision cases were minimal and clinically insignificant. In all, 3-5% of primary cases and 14-16% of revision cases experienced a change of ≥ 10 dB for the worse of one or more parameters between follow-ups. Results were also stable when considering a range of traditional success criteria. Other complications following surgery were infrequent and typically resolved long term.
Primary temporal region squamous cell carcinoma diagnosed by a superficial temporal artery biopsy
BACKGROUND: Superficial temporal artery biopsy is a frequent procedure performed in the diagnosis of giant cell arteritis.
METHODS AND RESULTS: An otherwise healthy 69-year-old man presented with 2months complaint of right temporal region pain. Giant cell arteritis was suspected and a temporal artery biopsy was performed. The histopathology revealed perineural invasion of squamous cell carcinoma (SCC). A thorough investigation revealed no other primary site for the SCC and the patient was treated with surgical excision.
CONCLUSION: Malignancy is rarely found in superficial temporal artery biopsies and lymphoma is the most common malignancy reported. In this rare case, the patient had right temporal pain explained by perineural invasion of a primary SCC in the right temporal region, which was treated with surgical excision guided by perioperative fresh frozen histology.
Peak and ceiling effects in final-product analysis of mastoidectomy performance
BACKGROUND: Virtual reality surgical simulation of mastoidectomy is a promising training tool for novices. Final-product analysis for assessing novice mastoidectomy performance could be limited by a peak or ceiling effect. These may be countered by simulator-integrated tutoring.
METHODS: Twenty-two participants completed a single session of self-directed practice of the mastoidectomy procedure in a virtual reality simulator. Participants were randomised for additional simulator-integrated tutoring. Performances were assessed at 10-minute intervals using final-product analysis.
RESULTS: In all, 45.5 per cent of participants peaked before the 60-minute time limit. None of the participants achieved the maximum score, suggesting a ceiling effect. The tutored group performed better than the non-tutored group but tutoring did not eliminate the peak or ceiling effects.
CONCLUSION: Timing and adequate instruction is important when using final-product analysis to assess novice mastoidectomy performance. Improved real-time feedback and tutoring could address the limitations of final product based assessment.
Novices perform better in virtual reality simulation than in traditional cadaveric dissection training of mastoidectomy
BACKGROUND: Temporal bone surgery requires integration of complex knowledge and technical skills. This can be difficult to accomplish with traditional cadaveric dissection training, which is often organized as single-instance participation in a temporal bone course. Simulator-integrated tutoring in virtual reality (VR) surgical simulators can visually guide the procedure and facilitate self-directed surgical skills acquisition. This study aims to explore the performances of novice otorhinolaryngology residents in a freeware VR simulator and in cadaveric dissection training of mastoidectomy.
METHODS: Thirty-four novice otorhinolaryngology residents performed a single and self-directed mastoidectomy procedure in a freeware VR temporal bone simulator before performing a similar procedure on a cadaveric temporal bone. VR simulation and cadaveric dissection performances were assessed by two blinded expert raters using final product analysis.
RESULTS: Participants achieved a higher mean final product score in VR simulation compared with cadaveric dissection (14.9 and 13.2, respectively; P = 0.02). Significantly more of the participants had their best performance in VR simulation (P = 0.04). No differences in computer experience and interest were found between the group that performed better in VR simulation and the group that performed better in cadaveric dissection.
CONCLUSIONS: Novice performance in a freeware VR temporal bone simulator was significantly better than in cadaveric dissection. The simulator-integrated tutor function and reduced complexity of the procedure in VR simulation could be possible explanations for this finding. VR simulation training could be used in the initial training of novices, reserving dissection training for more advanced training after basic competencies have been acquired with VR simulation.
Mastoidectomy performance assessment of virtual simulation training using final-product analysis
OBJECTIVES/HYPOTHESIS: The future development of integrated automatic assessment in temporal bone virtual surgical simulators calls for validation against currently established assessment tools. This study aimed to explore the relationship between mastoidectomy final-product performance assessment in virtual simulation and traditional dissection training.
STUDY DESIGN: Prospective trial with blinding.
METHODS: A total of 34 novice residents performed a mastoidectomy on the Visible Ear Simulator and on a cadaveric temporal bone. Two blinded senior otologists assessed the final-product performance using a modified Welling scale. The simulator gathered basic metrics on time, steps, and volumes in relation to the on-screen tutorial and collisions with vital structures.
RESULTS: Substantial inter-rater reliability (kappa = 0.77) for virtual simulation and moderate inter-rater reliability (kappa = 0.59) for dissection final-product assessment was found. The simulation and dissection performance scores had significant correlation (P = .014). None of the basic simulator metrics correlated significantly with the final-product score except for number of steps completed in the simulator.
CONCLUSIONS: A modified version of a validated final-product performance assessment tool can be used to assess mastoidectomy on virtual temporal bones. Performance assessment of virtual mastoidectomy could potentially save the use of cadaveric temporal bones for more advanced training when a basic level of competency in simulation has been achieved.
Learning Curves of Virtual Mastoidectomy in Distributed and Massed Practice
IMPORTANCE: Repeated and deliberate practice is crucial in surgical skills training, and virtual reality (VR) simulation can provide self-directed training of basic surgical skills to meet the individual needs of the trainee. Assessment of the learning curves of surgical procedures is pivotal in understanding skills acquisition and best-practice implementation and organization of training.
OBJECTIVE: To explore the learning curves of VR simulation training of mastoidectomy and the effects of different practice sequences with the aim of proposing the optimal organization of training.
DESIGN, SETTING, AND PARTICIPANTS: A prospective trial with a 2 × 2 design was conducted at an academic teaching hospital. Participants included 43 novice medical students. Of these, 21 students completed time-distributed practice from October 14 to November 29, 2013, and a separate group of 19 students completed massed practice on May 16, 17, or 18, 2014. Data analysis was performed from June 6, 2014, to March 3, 2015.
INTERVENTIONS: Participants performed 12 repeated virtual mastoidectomies using a temporal bone surgical simulator in either a distributed (practice blocks spaced in time) or massed (all practice in 1 day) training program with randomization for simulator-integrated tutoring during the first 5 sessions.
MAIN OUTCOMES AND MEASURES: Performance was assessed using a modified Welling Scale for final product analysis by 2 blinded senior otologists.
RESULTS: Compared with the 19 students in the massed practice group, the 21 students in the distributed practice group were older (mean age, 25.1 years), more often male (15 [62%]), and had slightly higher mean gaming frequency (2.3 on a 1-5 Likert scale). Learning curves were established and distributed practice was found to be superior to massed practice, reported as mean end score (95% CI) of 15.7 (14.4-17.0) in distributed practice vs. 13.0 (11.9-14.1) with massed practice (P = .002). Simulator-integrated tutoring accelerated the initial performance, with mean score for tutored sessions of 14.6 (13.9-15.2) vs. 13.4 (12.8-14.0) for corresponding nontutored sessions (P < .01) but at the cost of a drop in performance once tutoring ceased. The performance drop was less with distributed practice, suggesting a protective effect when acquired skills were consolidated over time. The mean performance of the nontutored participants in the distributed practice group plateaued on a score of 16.0 (15.3-16.7) at approximately the ninth repetition, but the individual learning curves were highly variable.
CONCLUSIONS AND RELEVANCE: Novices can acquire basic mastoidectomy competencies with self-directed VR simulation training. Training should be organized with distributed practice, and simulator-integrated tutoring can be useful to accelerate the initial learning curve. Practice should be deliberate and toward a standard set level of proficiency that remains to be defined rather than toward the mean learning curve plateau.
Cross-platform digital assessment forms for evaluating surgical skills
A variety of structured assessment tools for use in surgical training have been reported, but extant assessment tools often employ paper-based rating forms. Digital assessment forms for evaluating surgical skills could potentially offer advantages over paper-based forms, especially in complex assessment situations. In this paper, we report on the development of cross-platform digital assessment forms for use with multiple raters in order to facilitate the automatic processing of surgical skills assessments that include structured ratings. The FileMaker 13 platform was used to create a database containing the digital assessment forms, because this software has cross-platform functionality on both desktop computers and handheld devices. The database is hosted online, and the rating forms can therefore also be accessed through most modern web browsers. Cross-platform digital assessment forms were developed for the rating of surgical skills. The database platform used in this study was reasonably priced, intuitive for the user, and flexible. The forms have been provided online as free downloads that may serve as the basis for further development or as inspiration for future efforts. In conclusion, digital assessment forms can be used for the structured rating of surgical skills and have the potential to be especially useful in complex assessment situations with multiple raters, repeated assessments in various times and locations, and situations requiring substantial subsequent data processing or complex score calculations.
Cas clinique: Carcinome épidermoïde temporal primitif diagnostiqué par biopsie de l’artère temporale superficielle
INTRODUCTION: La biopsie de l’artère temporale superficielle est couramment pratiquée pour diagnostiquer l’artérite à cellules géantes.
MÉTHODES ET RÉSULTATS: Un homme de 69 ans, en bon état général, s’est présenté avec des douleurs de la région temporale droite évoluant depuis 2 mois. Une biopsie de l’artère temporale a été pratiquée devant une suspicion d’artérite à cellules géantes. L’histopathologie a trouvé un envahissement périneural par un carcinome épidermoïde (CE). Les explorations exhaustives n’ont pas retrouvé d’autre site de CE primitif. Une excision chirurgicale a été pratiquée.
CONCLUSION: Il est rare qu’une biopsie de l’artère temporale retrouve des signes de malignité. Les tumeurs malignes les plus fréquemment retrouvées sont des lymphomes. Dans ce cas exceptionnel, les douleurs de la région temporale droite s’expliquaient par un envahissement périneural par un CE primitif de la région temporale droite, traité par excision chirurgicale guidée par une analyse histologique de la pièce de biopsie fraîche et congelée.
[Good experiences with interactive temporal bone surgical simulator]
The Visible Ear Simulator (VES) is a freeware temporal bone surgical simulator utilizing a high-fidelity haptic and graphical voxel model compiled from segmented digital images of fresh frozen sections. A haptic device provides the 3-dimensional handling and drilling with force-feedback in real time. In a multilingual user interface the integrated tutor function provides stepwise instructions during drilling through an intuitive, volumetric approach. A censor function draws on metrics derived from the simulator to provide instant and summary feedback for the user. The VES can be downloaded from http://ves.cg.alexandra.dk.
[Gelatinous transformation of the bone marrow caused by excessive physical activity and insufficient calorie intake]
Gelatinous transformation (GT) of the bone marrow is rare and associated with a variety of clinical diseases from malignancy to chronic infection in combination with malnutrition or wasting. A new clinical picture of GT has emerged with excessive physical activity in combination with a voluntary insufficient calorie intake in the young adult male. Overtraining syndrome and systemic inflammation may play a role in the pathogenesis of GT in these patients, but further studies on GT and the effect of training and overtraining on the bone marrow are needed.
OBJECTIVE: To present a prospective ear surgery database and investigate the graft take-rate and prognostic factors for graft take-rate in tympanoplasty using the database.
STUDY DESIGN: Prospective database study.
SETTING: Tertiary referral center.
PATIENTS: A total of 1606 cases undergoing tympanoplasty types I to IV were registered in the database in the period from February 2004 to November 2013.
INTERVENTION: A total of 837 cases underwent myringoplasty/tympanoplasty type I.
MAIN OUTCOME MEASURE: Graft take-rate and prognostic factors (age, discharge at time of surgery, tuba function, technique, graft material, and revision surgery) for tympanoplasty type I were studied. A comparison with the graft take-rates for tympanoplasty types II to IV and/or cholesteatoma was made.
RESULTS: A user-friendly ear surgery database with fast data entry and direct import of audiometric data was developed. The graft take-rate was found to be 93.0% at 2 to 6 months and 86.6% at more than 12 months. Except for a discharging ear at the time of surgery, no significant differences using χ² test of association were found when comparing graft take-rates for different prognostic factors or more advanced tympanoplasty with or without cholesteatoma. A long-term graft take-rate overestimation of 6% was found if cases with defaulted follow-up because of early reperforation were not included.
CONCLUSION: A prospective database can be used to study prognostic factors and reduce bias in reporting the graft take-rate. Prospective databases are needed for high-quality longitudinal studies but require a continuous and daily effort of involved surgeons and therefore need to be convenient and fast to use.
How to exhaust your bone marrow
A 32-year-old man was admitted to the hospital because of oedema and 8 kg of gained weight. The oedema decreased spontaneously over weeks and there was no evidence for a nephrotic syndrome; however, the blood tests revealed a moderate pancytopenia. The patient practiced excessive physical activity at work and in his spare time, and kept a very thorough training and weight diary. Owing to a high intake of energy and protein drinks he tried to optimise his physical performance and kept a normal body mass index at 23.7. A bone marrow biopsy showed gelatinous bone marrow transformation, normally seen in critically ill patients or those with severe malnutrition. In this case, the cause is presumed to be excessive physical activity/overtraining in combination with relatively insufficient nutrition.
Letter on “A Structured Facial Feminization Fresh Tissue Surgical Simulation Laboratory Improves Trainee Confidence and Knowledge”