Preparing for the future of cardiothoracic surgery with virtual reality simulation and surgical planning: a narrative review
Review Article

Preparing for the future of cardiothoracic surgery with virtual reality simulation and surgical planning: a narrative review

Wouter Bakhuis^, Samuel A. Max^, Alexander P. W. M. Maat^, Ad J. J. C. Bogers^, Edris A. F. Mahtab^, Amir H. Sadeghi^

Department of Cardiothoracic Surgery, Thoraxcenter, Erasmus MC, Rotterdam, The Netherlands

Contributions: (I) Conception and design: All authors; (II) Administrative support: AJJC Bogers, EAF Mahtab, AH Sadeghi; (III) Provision of study materials or patients: None; (IV) Collection and assembly of data: W Bakhuis, SA Max, AH Sadeghi; (V) Data analysis and interpretation: W Bakhuis, SA Max, AH Sadeghi; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

^ORCID: Wouter Bakhuis, 0000-0002-3233-4182; Samuel A. Max, 0000-0002-8179-3924; Alexander P. W. M. Maat, 0000-0002-8553-2772; Ad J. J. C. Bogers, 0000-0001-6200-1998; Edris A. F. Mahtab, 0000-0003-2647-5509; Amir H. Sadeghi, 0000-0002-6118-2341.

Correspondence to: Amir H. Sadeghi, MD, PhD. Department of Cardiothoracic Surgery, Thoraxcenter, Erasmus MC, Doctor Molewaterplein 40, 3015 GD, Rotterdam, The Netherlands. Email: h.sadeghi@erasmusmc.nl.

Background and Objective: Virtual reality (VR) technology in cardiothoracic surgery has been an area of interest for almost three decades, but computational limitations had restricted its implementation. Recent advances in computing power have facilitated the creation of high-fidelity VR simulations and anatomy visualisation tools. We undertook a non-systematic narrative review of literature on VR simulations and preoperative planning tools in cardiothoracic surgery and present the state-of-the-art, and a future outlook.

Methods: A comprehensive search through MEDLINE database was performed in November 2022 for all publications that describe the use of VR in cardiothoracic surgery regarding training purposes, education, simulation, and procedural planning. We excluded papers that were not in English or Dutch, and that used two-dimensional (2D) screens, augmented, and simulated reality.

Key Content and Findings: Results were categorised as simulators and preoperative planning tools. Current surgical simulators include the lobectomy module in the LapSim for video assisted thorascopic surgery which has been extensively validated, and the more recent robotic assisted lobectomy simulators from Robotix Mentor and Da Vinci SimNow, which are increasingly becoming integrated into the robotic surgery curriculum. Other perioperative simulators include the CardioPulmonary VR Resuscitation simulator for advanced life support after cardiac surgery, and the VR Extracorporeal Circulation (ECC) simulator for perfusionists to simulate the use of a heart-lung machine (HLM). For surgical planning, there are many small-scale tools available, and many case/pilot studies have been published utilising the visualisation possibilities provided by VR, including congenital cardiac, congenital thoracic, adult cardiac, and adult thoracic diseases.

Conclusions: There are many promising tools becoming available to leverage the immersive power of VR in cardiothoracic surgery. The path to validate these simulators is well described, but large-scale trials producing high-level evidence for their efficacy are absent as of yet. Our view is that these tools will become increasingly integral parts of daily practice in this field in the coming decade.

Keywords: Cardiothoracic surgery; virtual reality (VR); simulation; preoperative planning


Received: 30 December 2022; Accepted: 29 June 2023; Published online: 28 July 2023.

doi: 10.21037/shc-22-63


Introduction

Background

Cardiothoracic surgery began as a specialism that performed procedures with no direct visualisation, such as a valve commissurotomy without extracorporeal circulation (ECC) during the 1950s. It has since evolved to include elaborate preoperative planning via echocardiography, coronary angiography, and computed tomography/magnetic resonance imagining (CT/MRI) imaging modalities (1,2). Due to improved preoperative visualisation and intraoperative support modalities including extra-corporeal circulation, the standard approach for procedures has changed from sternotomy and thoracotomy to minimally invasive techniques and video-assisted thoracoscopic surgery (VATS).

The field of cardiothoracic surgery has traditionally driven technological development on many fronts, including development of mechanical heart valve protheses, ECC, and robotic surgery which was initially designed for use in cardiac surgery (3,4). Moreover, cardiothoracic surgeons increasingly rely on perioperative visualisation techniques, currently mostly consisting of three-dimensional (3D) echocardiography and conventional MRI- and CT-scans. The range of interventions that cardiothoracic surgeons routinely perform is well defined, thereby further facilitating the creation of procedural simulations.

For less experienced surgical team members including surgical trainees, scrub nurses in training, and medical students, minimally invasive procedures can feel overwhelming due to the lack of direct anatomical visualisation and tactile feedback from thoracoscopic instruments. Familiarising staff and students with minimally invasive techniques and anatomy through simulation is therefore a useful strategy to ensure competency before they step into the Operation Room (OR). This (simulation-based) competency assessment needs to be validated using a framework, for which Messick’s validity framework (based on five pillars of evidence: content, internal structure, relationship with other variables, response process and consequence) is the current standard (5). Simulations can further help to ameliorate technique limitations for some minimally invasive procedures such as the fulcrum effect for example. Even for experienced surgeons, extra simulation tools or additional visualisation techniques can be helpful. This is because the spectrum of cardiothoracic procedures is becoming increasingly complex due to neoadjuvant treatments, reoperations, increasingly comorbid patients, and increasing proportions of minimally invasive (robotic) procedures (6). Some uncommon procedures are performed very few times per year per centre, which makes it difficult for to overcome the learning curve of such a procedure. For example, a lung-sparing congenital lung abnormality (CLA) resection was performed 4 times per surgeon per year in a study by Fascetti-Leon and colleagues (7). The relative lack of patient cases, such as was seen at the height of the coronavirus disease 2019 (COVID-19) pandemic, also makes a strong case for competency-based simulation to be able to become proficient despite relatively low exposure (8).

Increasingly complex patients and surgical techniques make pre-surgical preparations, including in-depth knowledge of both the procedure and patient-specific anatomy, essential. Improved preoperative preparations using novel digital imaging techniques have the potential to accelerate the flow of the surgery by shortening the procedure. It may also result in less perioperative surprises followed by necessary improvisations, thereby reducing intraoperative patient risk (9).

Virtual reality (VR) can help to fulfil these needs with VR-based simulation scenarios train for cardiothoracic procedures or with full-immersion VR-based procedural planning to plan complex cardiothoracic surgical procedures. VR uses a head mounted display (HMD) or other console mounted displays to show an immersive, stereoscopic, 3D view of the environment. VR has seen a relative boom in recent years, with its efficacy having been demonstrated as far as three decades ago in fields such as construction (10), aviation, and military applications (11). Within healthcare, VR is a burgeoning field, with fields including psychiatry (12), emergency medical services (13), analgaesia (14), neurorehabilitation (15), and nursing (16) amongst many others.

Rationale and knowledge gap

Immersive VR surgical simulations

Surgery is a logical application for VR simulations, as this field has for a long time struggled with the issue of how to train its junior members without exposing patients to undue risk. This is especially true in the context of an ever busier and more complex operating schedule. It was suggested in 1993 that VR was a good candidate to address these issues, although the authors admitted that there were some hurdles to overcoming the “Pacman” like nature of the current state of technology (17). Despite almost 30 years passing since this prediction, we are still grappling with the issue that the computational power and miniaturisation required to create flexible, responsive, and realistic scenarios in stand-alone VR may only just be arriving.

As with many safety-related paradigms that have been inherited by the surgical field, aviation continues to drive innovation on this front. Junior pilots who face similar training challenges to their surgical colleagues have enjoyed the benefits of VR for decades. Simulator based training, with or without VR, forms a central part of their education, and has helped to build experience whilst saving on both training costs and reducing the risk of fatal accidents (18).

Colleagues from other surgical disciplines including general surgery demonstrated in 2004 that even the relatively early simulators could improve real world performance, despite the relatively simple nature of the simulation (19). A similar small scale study by Seymour et al. showed that surgeons progressed more quickly through real-life surgery, making fewer mistakes, resulting in fewer complications in the process, after VR simulation (20). In 2007, Ahlberg et al. demonstrated that their simulator reduced the error rate and reduced operative time for trainees performing laparoscopic cholecystectomies in humans (21). Despite these promising results, cardiothoracic surgery is only beginning to implement high fidelity simulators.

To conduct a comprehensive evaluation of the surgical simulators under consideration, we will utilize de Visser’s criteria as the primary parameters for assessment (22). As outlined by de Visser et al., these criteria encompass 3 areas: physical realism, case complexity, and performance assessment. These criteria can be further defined as follows. The physical realism construct includes visual quality, instrument realism, and haptic feedback. The case complexity construct includes case variability or the ability to approach the same case in multiple ways, and complication simulation or the ability to simulate mistakes and/or unexpected events during the procedure. Finally, performance assessment includes objective measures of performance, including instrument path length or blood loss for example. By applying these criteria, we can thoroughly examine the merits of the simulators discussed below, specifically in terms of their adherence to these fundamental aspects. To ensure a well-rounded evaluation, we will consider relevant literature pertaining to each simulator’s compliance with the specified criteria.

Immersive VR perioperative visualisation and planning

The added value of spatial representation by 3D models is already proven in other industries such as engineering. Spatial 3D representation improves perception of shape and depth, decreases the mental workload, and leads to better quality and faster results of different tasks (23). In 1988, Laschinger et al. (24) for the first time manually derived 3D reconstructions from MRI images by manually segmenting the cross-sectional images and using surface recognition software. The main advantage of VR over other forms of 3D visualisation (on a 2D or 3D screen) is immersiveness and interactivity, i.e., being able to control the model with controllers. Initial pilots studies of VR in medicine were performed at the end of the twentieth century (25). With the increasing processing power of computers over the last years and the development of specific VR hardware, the quality of VR visualisation greatly increased over the last years. However, preoperative VR-based planning in medicine has been revolutionised for decades, but implementation in the clinic is often still awaited. First steps of implementation in other surgical specialties, such as urology, showed a positive effect on operative time, blood loss and length of hospital stay (26). Effective tools to preoperatively plan a procedure and even predict outcomes through modelling are therefore increasingly in demand, and are starting to be validated in literature.

Objectives

Our objective was to gather relevant literature on VR simulators and preoperative planning tools alike, in order to create an overview of the field and investigate to what extent these tools are integrated into current cardiothoracic surgery practice, and where they are likely to contribute to in the coming years as VR technology matures. In this paper, given the new possibilities to create procedural simulations and visualisation, we aim to give an overview of current innovations in VR simulation and VR-based surgical preoperative planning in cardiothoracic surgery. When evaluating each VR simulator, we assess its adherence to de Visser’s criteria concerning the levels of physical realism, case complexity, and its capability to effectively evaluate user performance. This assessment relies on a literature review dedicated to each simulator, enabling a determination of its adherence to the aforementioned criteria. We present this article in accordance with the Narrative Review reporting checklist (available at https://shc.amegroups.com/article/view/10.21037/shc-22-63/rc).


Methods

For this narrative literature review, we performed a comprehensive search through MEDLINE database for all publications that describe the use of VR in cardiothoracic surgery regarding training purposes, education, simulation, and procedural planning. The search was completed in November 2022. We excluded all papers that described the use of VR on a computer screen, augmented reality, simulated reality, and 3D visualisation/3D printing. All papers not in English or Dutch were excluded. A summary of the search strategy can be found in Table 1. In addition, studies describing VR-based bronchoscopy and oesophageal visualisations were excluded, since bronchoscopy and oesophageal procedures are not in the remit of cardiothoracic surgeons in the Netherlands. Opinions in the review reflect the opinions of the authors of various publications and the opinion of the authors of this review. Unpublished data currently in peer-review produced by authors of this review were included for context and for insights into simulators that will be released in the near-future.

Table 1

The search strategy summary

Items Specification
Date of search 2nd November 2022
Databases and other sources searched MEDLINE database
Search terms used Cardiac Surgery; Thoracic Surgery; Virtual Reality; Simulation; (Pre)Operative Planning
Timeframe No specified date range
Inclusion criteria Included all primary research studies published in English or Dutch including the above key words. Systematic reviews and meta-analyses were used to locate additional primary studies for inclusion. Unpublished materials currently undergoing peer review were also included to ensure the results of this review remain valid for an extended period of time in this fast-paced field
Selection process The selection process was performed by the first and second authors independently with suggestions from the rest of the authors, and a consensus reached through input from the corresponding author

Results

We divided all publications identified in the scoping process into two categories: (I) VR-based simulation; and (II) VR-based procedural planning.

VR-based simulation

Simulation-based VR is defined as a serious gaming environment in which two types of procedures can be performed. Procedures can be simulated to either mimic real-life surgical procedures, or other procedural interventions that are directly involved with surgical patients such as training for advanced life support, or for scrub staff training to support surgical procedures. In this narrative review we will consider both kinds of simulator and provide an overview of the current state-of-the-art, before we outline how we expect this area of technology will continue to impact surgical preparation in the coming decades.

Our literature search highlighted five VR simulators in cardiothoracic surgery. Two unpublished manuscripts simulators from our group are also briefly considered below, and summarised in Table 2.

Table 2

Overview of VR surgical simulators reported in literature, and other unpublished sources

Simulator Authors Head mounted device Advantages (+) Disadvantages (−) de Visser’s criteria
LapSim VATS Lobectomy Simulator Jensen et al. [2017], (27) Integrated into Surgical Science console Fully validated surgical simulator; realistic and useful in training junior surgeons Requires proprietary console Physical realism (+)
Case complexity (+)
Performance assessment (+)
Robotix Mentor RATS
Lobectomy Simulator
Whittaker et al. [2019], (28) Integrated into Surgical Science console Guided lobectomy procedures for novices and surgeons new to robotic surgery. Able to discriminate between skill levels Rated as “somewhat realistic” with an average realism score of 3/5 and requires proprietary console Physical realism (−)
Case complexity (+)
Performance assessment (+)
Da Vinci SimNow Turbati et al. [2023], (29) Integrated into DaVinci SimNow console Comprehensive library of robotic skills exercises to familiarise users with robotic surgery. Able to discriminate between skill levels Recently released lobectomy simulator not yet validated Physical realism (not yet investigated)
Case complexity (not yet investigated)
Performance assessment (+)
Non-surgical simulators
   VR CPR Simulator Peek et al. [2023], (30) Meta Quest 2 Flexible and immersive way to improve and democratise training for cardiac arrest after cardiac surgery, that is valid for face and construct validities VR users not yet faster than conventionally trained counterparts. Lack of haptic feedback for surgery Not applicable
   VR ECC Simulator Babar & Max et al. (31) Meta Quest 2 Novices can be trained to build and operate a heart-lung machine. Experts will be able to train for rare emergencies Heart-lung machine inputs do not update patient monitoring currently Not applicable
   VR Scrub Nurse simulator Currently unpublished Meta Quest 2 Has potential to improve and shorten the training required for new scrub staff or provide training for complex or rare procedures Currently in pre-alpha stage and therefore no validation data available Not applicable

HMD Manufacturers: Meta (Menlo Park, CA, USA); Surgical Science (Goteborg, Sweden); DaVinci SimNow (Intuitive Surgical, Sunnyvale, CA, USA). VR, virtual reality; VATS, video-assisted thoracoscopic surgery; RATS, robot-assisted thoracic surgery; CPR, cardiopulmonary resuscitation; ECC, extracorporeal circulation.

VATS simulator

The first simulator considered in this review was created by Jensen et al., and LapSim (Surgical Science, Goteborg, Sweden), and simulates a VATS lobectomy (27). This simulator comprises an HMD for participants to wear, and an endoscopic control surface by which the onscreen controls are operated, and was initially developed to simulate a laparoscopic salpingectomy and removal of an ectopic pregnancy (32). Progress has been swift, and in 2015 the lobectomy simulator followed. Images from the LapSim lobectomy simulator can be found in Figure 1A. This was initially tested by over 100 surgeons, and found to be realistic in terms of the simulator’s visual appearance, response to instrument inputs, and tissue physical properties (34). Initially, the LapSim recreated a right upper lobectomy, but have since added the possibility to remove the remaining 4 lobes, creating a complete set of lobectomy simulations (35).

Figure 1 Stills from the LapSim VATS simulator (27) (A), the VR CPR simulator (33) (B), and VR ECC simulator (31) (C). VR, virtual reality; CPR, cardiopulmonary resuscitation; ECC, extracorporeal circulation; VATS, video-assisted thoracoscopic surgery.

The validity of the LapSim VATS simulator was demonstrated by comparing the performance of experienced surgeons who had performed more than 50 VATS lobectomy procedures and beginners who had not performed any lobectomy procedures (27). Predefined end points were set including timings, damage to tissue, and instrument path length amongst others. They found that experienced surgeons did significantly better than their beginner counterparts, and “passed” the simulation by performing within one standard deviation of the expected benchmark in all but four cases. None of the beginners passed the simulation, demonstrating that the simulator was consistently able to discriminate between skill levels. Further validation studies include that of Haidari et al. where an updated simulator including haptic feedback was tested on surgeons of varying skill, this time including an intermediate category (35). Again, the simulation could discriminate between all three groups on the basis of blood loss and time taken. The most recent published assessment of competency study using the LapSim simulator found that the VR simulation consistently rated participants with high interrater reliability scores, and that it could provide a validated pass/fail threshold for users (36). These thresholds, termed the Copenhagen test, were based on time, blood loss, and instrument path length for each intervention. This simulator therefore fulfils all the criteria of de Visser as outlined above, and represents the first validated VR surgical simulator in the cardiothoracic field to do so.

Robotic surgery simulators

The number of surgical robots that are being introduced into surgical practice is rapidly increasing. To make optimal use of these new tools, training programmes are required to help surgeons become accustomed to their use.

The first validation study of the Robotix Mentor (3D Systems, Simbionix Products, Cleveland, OH, USA) lobectomy simulator in collaboration with Surgical Science was performed in 2019 (28). Novices, intermediates, and experts were familiarised with the robotic environment and controls, before performing a guided lobectomy, where they were scored on seven surgical parameters. The simulator was able to discriminate between the performance of experts, intermediates and novices, with experts handling tissue with greater care, and performing fewer movements than both groups. Interestingly, experts and novices did not differ significantly in time taken to complete the procedure, but intermediates were significantly slower than the experts, suggesting a learning curve in the shape of a parabola. Participants found the simulator to be somewhat realistic on the whole, with an average score on a Likert scale of 3 out of a possible 5. This therefore suggests that this simulator fulfils two out of three of the criteria of de Visser, with improvement in physical realism required for the third criterion.

The Da Vinci Skills SimNow (Intuitive Surgical, Sunnyvale, CA, USA) series including the basic surgical skills programme have received extensive testing from other surgical specialties. The SimNow curriculum was found to be valid in terms of construct validity, being able to discriminate between the performance of experts and novices respectively (29). This curriculum was also previously applied in 2016 to cardiac surgery, whereby 40 residents initially completed a robotic procedure as a baseline, where they completed a dissection of a model internal mammary artery, and placed 3 stitches as part of a mitral valve annuloplasty simulation (37). They then were randomised between groups receiving robotic surgery training in either a wet lab, a dry lab, or complete 9 VR robotic skills exercises in the Da Vinci simulation catalogue, with a control group receiving no training. They were then again asked to perform the robotic internal mammary artery dissection and partial mitral valve annuloplasty, and rated by experts on their performance. All participants who received training achieved expert proficiency as assessed by the Global Evaluative Assessment of Robotic Surgery scoring tool and other composite outcomes, with wet lab participants receiving the highest scores. The control group did not achieve expert proficiency for any of these metrics. This study demonstrates that high quality robotic surgery simulators, and potentially other generic VR simulators do not necessarily have to simulate an entire procedure in order for them to be of use for trainee cardiothoracic surgeons. This is also true for more senior surgeons who are familiarising themselves with the robotic surgery environment. Cowan and colleagues (38) demonstrated that although the SimNow simulator can differentiate between competency levels, a dry lab can do this more efficiently, and that the SimNow procedure was more difficult because of the less realistic tissue handling model. The Da Vinci SimNow simulator therefore fulfils two out of three of de Visser’s criteria, with improvement in physical realism specifically in terms of tissue deformation required again to fully validate the platform. A lobectomy simulator was released in an update in April 2022, but as of yet has not been validated.

Cardiopulmonary VR resuscitation simulator

The next simulator is a Cardiopulmonary VR Resuscitation simulator, or VR CPR-sim. This simulator was created by our group together with Distant Point (Skopje, North-Macedonia), and aims to train junior surgeons and allied health staff in the bespoke Cardiac Surgery Unit-Advanced Life Support algorithm (CSU-ALS) outlined by Dunning et al. for patients up to 10 days after cardiac surgery through median sternotomy (39). The benefits of applying VR in this way include that it facilitates repetitive practice in a non-stressful environment; healthcare professionals who attempt to learn the protocol ‘on the job’ are unlikely to succeed due to the high cognitive load that staff experience during cardiac arrest cases (40). This inhibits learning and slows the process of achieving competency, thereby limiting the quality of care provided (41). Combined with a low incidence of these types of arrest (around 3% of all cardiac surgery patients), this makes learning the CSU-ALS algorithm challenging in practice, hence the need for a high-fidelity simulator.

The user puts on a standalone Meta Quest 2 HMD (Meta, Menlo Park, CA, USA), and orients themselves and interacts with the virtual environment using the supplied Meta controllers. They then progress through multiple different patient cardiac arrest cases, including all three arms of the CSU-ALS algorithm. In this way, the user is trained to assist and/or perform these procedures, improving their real-life performance. Images from the simulator can be found in Figure 1B. To assess the validity of the simulator, the group used the consensus guidelines on validating VR simulators published by Carter et al. (42), and aimed to demonstrate that the simulator was valid in terms of face and construct validity. The initial publication on this simulator demonstrated that both experts and novices found the simulation to be valid for the aforementioned metrics, rated as realistic, and a useful way of training for these relatively unusual events (33).

A randomised controlled trial was performed whereby cardiothoracic surgery residents were randomised to receive conventional classroom/manikin CSU-ALS training, or VR CPR-sim training, and their abilities subsequently tested using a moulage setup. Results indicate that both groups performed resternotomy significantly faster than the target time, and the VR group, the VR group made fewer errors when applying the CSU-ALS algorithm (30). Future features of this simulator will include the possibility to collaborate with multiple users in one scenario, creating a team exercise where participants do not need to be in the same locale, or can invite (foreign) experts to teach, without the inconvenience and expense of travelling.

ECC simulator

The next simulator from this group concerns the preparation and operation of a heart-lung machine (HLM) in the context of cardiopulmonary bypass. The Virtual Reality Extracorporeal Circulation simulator or VR-ECC sim is designed for perfusionists both during their education and afterwards to maintain competency and train for rare and emergent events. Already applied to teaching in practice, this simulator helps to bring perfusion training into a sandbox environment, whereby competency and experience with rare events/machinery malfunctions can be built up without requiring the use of a physical simulator or putting patients at risk. The feasibility study (31), much like that performed for the VR CPR-sim, showed that the simulator was valid for face and construct validity as rated by experts and novices, and that both groups felt that the simulator was realistic, easy to use, and useful. Additional advantages of the VR modality include cost, as physical simulators including the ECCSIM costs upwards of $40,000 USD to purchase (43), and the Califa Patient Simulator (Biomed Simulation Inc., San Diego, CA, USA) also being similarly priced. VR by contrast requires no dedicated physical space, and costs around $500 USD for the HMD and $50USD a month for a simulator licence, representing substantial savings. Images from the simulator can be found in Figure 1C.

Extracorporal membrane oxygenator simulator

A spin-off from the above by the same group is a VR Extracorporeal Membrane Oxygenator (ECMO) simulator, which uses the same principles as above, driven by real ECMO data in order that the machine responds in a way that would be familiar to those who are experienced in ECMO operation. This simulator will feature similar troubleshooting features as the VR ECC sim, and will be aimed at intensive care unit (ICU) healthcare staff as well as junior cardiothoracic surgery residents in order to familiarise themselves with this technology. Additional applications include for special paramedic personnel who are increasingly performing extracorporeal cardiopulmonary resuscitation (eCPR) using an ECMO unit (44). Training these teams in VR holds several advantages, including being able to accurately simulate the physiologically extreme states that paramedics are likely to encounter whilst performing eCPR in the community (45).

Scrub nurse training simulator

The final simulator which is in pre-alpha stage is designed for scrub nurses, who as a profession are in increasingly scarce supply in many centres internationally (46). Creating a standardised VR curriculum to introduce staff new to cardiothoracic surgery procedures is therefore a useful application of VR. This concept has already been applied in the field of orthopaedics, where the Attune® Revision TKA Simulator v1.1 (Pixelmolkerei, Chur, Switzerland) was shown to both reduce operative times significantly by 47% over 3 training sessions, the number of verbal prompts by the surgeon reduced by 75%, the total distance of dominant hand movement was reduced by 28%, and the amount of errors made by 47% (47). Leveraging these benefits will be the aim of this simulator for cardiothoracic surgery, and is aimed to launch in 2023.

VR-based procedural planning

Preoperative planning of both cardiac and thoracic procedures can be optimised using VR (Table 3). VR affords a multitude of possibilities in visualising complex anatomy, including full 3-dimensional reconstructions, the option to visually remove structures that otherwise obscure the view of the target anatomy, being able to orient and zoom in an intuitive manner, and highlight relevant features.

Table 3

Overview of congenital and adult cardiac and thoracic surgery procedures planned by VR, that have been reported in literature

Field Authors Head mounted device Advantages (+) Disadvantages (−)
Congenital cardiac surgery
   Ventricular septal defect (VSD) Ong et al. [2018], (48) HTC Vive Flexible view of anatomy and shared imaging to create better surgical plan Currently unable to incorporate echocardiography images
Mendez et al. [2019], (49) Not Specified Good VSD size perception and relation to surrounding structures As above, and currently limited tissue resolution
Ghosh et al. [2021], (50) HTC Vive VR ensured optimal incision site and removed obscuring anatomy Small defects were not seen on cardiac MRI and not modelled
   Major aortopulmonary collateral arteries (MAPCA) Chan et al. [2013], (51) EchoPixel True3D* Quicker than a conventional 2D CT scan interpretation No improvements in accuracy or changes to operative plans
van de Woestijne et al. [2021], (52) Oculus Rift S VR 3D anatomical visualisation has added value for all cases described, and confirmed findings not found by angiography Reduced reliability with thicker CT scan slices or suboptimal contrast introduction
   Double outlet right ventricle Ayerbe et al. [2020], (53) HTC Vive Improved visualisation of the location, relation to other structures, and severity of an obstruction Single case-study level evidence with no comparator
   Valvular repair Pushparajah et al. [2021], (54) HTC Vive Better visualisation of different types of valvular lesions generally; increased confidence in surgical approach after viewing anatomy in VR 4 anatomical reconstructions out of 15 did non-match the intraoperative findings
   Left ventricular assist device (LVAD) Ramaswamy et al. [2021], (55) Oculus Rift S VR aids in positioning the inflow cannula of the LVAD, and helps to reduce imaging artefacts associated with implanted devices Single case-study level evidence with no comparator
Congenital thoracic surgery
   Congenital lung abnormalities Pelizzo et al. [2022], (56) Oculus Quest Surgical plans were adapted according to anomalies as shown in VR Suboptimal bronchial visualisation, possibly due to constraints of CT imaging used in this study
Adult cardiac surgery
   Coronary disease Sadeghi et al. [2020], (57) Oculus Rift S Port positioning and graft location determined by VR 3D reconstruction Single case-study level evidence with no comparator
   Aortic surgery Abjigitova et al. [2022], (58) Oculus Rift S Rated as useful by 100% of surgeons in this study; aided in planning open or clamped procedure No current support for MRI or echocardiographic images; no statistical difference between 2D and 3D visualisation
   Mitral valve surgery Nanchahal et al. [2022], (59) Mixed VR shows valvular & annular pathology more clearly than echocardiography Small observational studies included
Adult thoracic surgery
   Lobectomy Frajhof et al. [2018], (60) Not specified Surgical decisions made intraoperatively aided by VR visualisation Single case-study level evidence with no comparator
   Segmentectomy Sadeghi et al. [2021], (61) Oculus Rift S Surgeons reported being able to better plan segmentectomies using PulmoVR than using 2D CT Manual segmentation requires time and is prone to human error
Bakhuis et al. [2023], (62) Oculus Rift S Surgical plan was altered in 52% of cases as a result of VR visualisation; tumour localisation was altered in 14% of cases 18% of patients received a more extensive resection than was planned in VR
   Forequarter amputation and chest wall resection Peek et al. [2022], (63) Oculus Rift S VR enabled the planning of a complex operation, sparing structures that might have been resected without VR visualisation VR (and CT) imaging suggested a more radical costal resection was required that intraoperatively found to be required

HMD Manufacturers: HTC (Taoyuan City, Taiwan); Oculus/Meta (Menlo Park, CA, USA); EchoPixel (Los Altos Hills, CA, USA). *, whilst not a head-mounted device, the head tracking & stereoscopic nature of this simulator was similar enough to be included in this review. VR, virtual reality; VSD, ventricular septal defect; MAPCA, major aortopulmonary collateral arteries; LVAD, left ventricular assist device; 2D, two-dimensional; CT, computed tomography; 3D, three-dimensional; MRI, magnetic resonance imagining.

Congenital cardiac surgery

In cardiac surgery, using VR to visualise congenital abnormalities has resulted in a multitude of studies. In most cases, VR visualisation of CT imaging is used as a basis to visualise intracardiac malformations. These will be considered in turn below by the anatomy that is investigated.

Ventricular septal defect (VSD)

A case report of an 18-month-old baby with a large VSD showed that VR provided excellent visualisation regarding VSD size and relation to surrounding structures for the surgeon performing the repair (49). Ghosh et al. used cardiac magnetic resonance (CMR) images to show a 3D-VR image of a child with multiple VSDs to both a cardiac surgery team and a cardiac intervention team, and both succeeded in closing these defects via a hybrid approach, since VR showed that approaching the largest VSD via the tricuspid valve was impossible (50). Two patients in a study of Ong et al. underwent surgery, one with truncus arteriosus and the other with a large VSD and dextrocardia (48). Both cases benefited from using preoperative VR planning since both intra and extracardiac abnormalities could be visualised in VR. VR additionally offered better 3D visualization of the aortic arch for arch repair and by determining the VSD location in relation to other cardiac structures in dextrocardia.

Major aortopulmonary collateral arteries (MAPCAs)

Two pilot studies of patients with MAPCAs were conducted to assess added value of VR over conventional CT scan visualization. MAPCAs are very variable: the number of MAPCAs, the offspring and the anatomical course varies significantly between patients. Fifteen newborns with MAPCAs were visualised in VR by Chan et al. (51) and by our group using the CardioVR application from MedicalVR (Amsterdam, the Netherlands) (52). Although catheter angiography remains necessary, VR visualisation of CT angiogram (CTA) significantly enhanced the visualisation of the offspring and course of MAPCAs in relationship to surrounding structures, which was slightly faster to comprehend than using normal CT visualisation. The additional flexibility afforded by view adjustment was felt to be an advantage for planning in both studies, albeit with the precondition of suitable CT base materials on which to build a 3D model.

Double outlet right ventricle (DORV)

3D-VR visualisation of the CTA of a child with DORV helped in understanding the aetiology of left ventricle outflow tract obstruction, that was caused by subaortic conus, instead of ventricular tissue outgrowth or a subaortic membrane, as was suggested based on conventional imaging techniques (53). In addition, our group employed the CardioVR simulator and performed a trial whereby experts assessed patients with DORV in the conventional manner using 2D modalities and recorded their proposed surgical plans (64). They then re-assessed those cases two times using 3D prints and using the CardioVR simulator, and were given the opportunity to reconsider the proposed approach. The results showed that 87% of the paediatric cardiologists and congenital cardiothoracic surgeons retrospectively suggested the correct surgical plan based on 3D-VR, and 85% based on 3D printing, compared to 72% based on ultrasound and conventional CT visualisation.

Valvular repair

Pushparajah et al. similarly performed a study where patients for atrioventricular valve repair were assessed through conventional and VR visualisation means based on echocardiographic imaging data to plan atrioventricular valve repair in fifteen patients (54). With data collected through a questionnaire, surgeons reported in ten out of fifteen cases to be more confident in the surgical approach than only after reviewing conventional 2D and 3D echocardiography, and made at least one modification to the surgical approach in half of all patients after VR visualisation.

Left ventricular assist device (LVAD) implantation

A study by Ramaswamy and colleagues (55) described a case in which VR planning was helpful in the implantation of a LVAD. VR specifically contributed in determining the optimal position of the device on the patient’s heart, and ensured the inflow cannula did not impinge on the mitral valve.

Congenital thoracic surgery

Only one published study that explored VR possibilities in congenital thoracic surgery was identified (56). CLAs were resected via 3D VATS lobectomy in three infants, and preoperative virtual navigation helped the surgical team in assessing patient-specific anatomy and malformation size and location.

Our group recently performed a study to assess the added value of VR visualization in assessing the feasibility of a sublobar lung resection in CLA patients. In this study, a paediatric surgeon and thoracic radiologist located the CLA in 2D-CT versus 3D-VR. The results demonstrated that after 3D-VR visualisation, both surgeon and radiologist reached higher agreement on lung segments localization of CLA.

Adult cardiac surgery

The advantages conferred by VR visualisation in the paediatric population outlined above are becoming increasingly evident, but the application of this technology in adult cardiothoracic has also shown to be fertile ground for the application of VR.

Coronary disease

Our group produced a 3D-VR recreation of the coronary and thoracic anatomy of an 18-year-old patient who was undergoing minimally invasive coronary artery bypass grafting after suffering from Kawasaki disease (57). The visualisation aided in producing a surgical plan for graft locations, as well as optimising port placement for internal mammary artery harvesting, and mini-thoracotomy positioning.

Aortic surgery

Applying the aforementioned CardioVR to aortic surgical intervention planning was shown in a pilot study to increase perceived preparedness for surgeons in 80% of cases, and in one third of cases (2/6) altered the intervention performed (58). Further refinement of this simulator and a study with a larger sample size is currently in progress.

Mitral valve surgery

In a review of VR visualization of the mitral valve, Nanchahal et al. reported that VR not only helps visualize the mitral valve more accurately but also demonstrates associated annular pathology compared to conventional echocardiography. This facilitates more personalised interventions and likely improved outcomes, especially when predictive dynamic modelling is employed to predict flow through the valve after intervention (59).

Adult thoracic surgery

In thoracic surgery, the added value of VR in preoperative planning is shown in various publications as well. Frajhof et al. described a case report in which a left upper lobectomy via VATS was planned and performed with the aid of VR (60). Due to ingrowth in the pulmonary artery of the upper lobe, 3D-VR visualisation was initialized, and helped the surgeons in successful resection of the left upper lobe.

Our group uses PulmoVR, an application that our group co-developed together with MedicalVR (Amsterdam, the Netherlands), for planning pulmonary segmentectomies in patient with early-stage primary lung cancer, benign lesions or metastatic lesions not suitable for wedge resections (61). Recently, the results of the first fifty VR-guided segmentectomies were published (62). Additional VR visualisation of the CT-scan resulted in adjustment of the surgical plan in 52% of the cases. Resections became smaller where possible, extended if oncologically indicated, and a different segment or even conversion to lobectomy if segmentectomy was not possible. Furthermore, various anatomical variations were observed, and confirmed that the pulmonary anatomy is highly variable and patient specific.

Furthermore, our group published a case report of 3D-VR planning of a forequarter amputation with chest wall resection of a stage 3, 90 mm in diameter, squamous cell carcinoma in the left axilla (63). 3D-VR showed that vertebral artery and jugular veins were not affected, and that only the first two ribs revealed tumour invasion, which helped the surgeon to create a better preoperative plan, and help the oncologist to better inform the patient about the details of the procedure its possible outcomes (Figure 2).

Figure 2 Progress made in Virtual Reality surgical simulators and procedural planning over the last two decades, featuring (A) the MIST VR surgical simulator (20) and (B) the LapSim Lobectomy Simulator from Jensen et al. (27), and Haidari et al. (36). For the VR procedural planning simulators, (C) the dynamic VR 3D Echocardiography simulator as produced by Bol Raap et al. (65) (figure reproduced under CC BY 2.0 license) and (D) the PulmoVR simulator from Sadeghi et al. (61) are shown (figure reproduced under CC BY license). MIST, minimally invasive surgical trainer; VR, virtual reality.

Discussion

Key findings

We are now witnessing the arrival of VR in cardiothoracic surgery, and are learning how to use the technology to our advantage and finding where it is not as applicable as previously thought. For the area of simulation, despite early progress and the validated lobectomy simulators produced by LapSim and Robotix Mentor respectively, it is clear that we still await a comprehensive set of simulators that prepare surgeons for the bulk of day-to-day cardiothoracic procedures. The process of validating these simulators once they arrive however is well described, and the large scale randomised controlled trials that are currently absent according to Moglia and colleagues (66) will help to cement their role in the future of the field.

VR is starting to make a significant contribution to pre-surgical planning, where flexible visualisation of complex and/or aberrant anatomy is relatively more straightforward using VR. The surgeon can immediately review the anatomy of the patient behind their computer in their office with VR hardware and software. These VR images can be automatically produced from CT scans locally, without the need for transfer of data to external (cloud) sources. There are many areas where VR planning is having an impact on daily clinical practice, especially in the congenital cardiac surgery and sub-lobar lung resections, where surgeons have touted VR planning as the biggest advancement since the advent of CT scans in the 1970s (67). Future refinements to the process of creating these 3D visualisations will likely expand the remit of VR visualisation further. As with the surgical simulators, further large-scale multicentre trials are required to quantify whether the perceived benefits translate into clinical outcome improvements. Currently, our group is performing a multicentre trial in eight hospitals in the Netherlands to validate VR-based preoperative planning for pulmonary segmentectomies. Even if this paradigm becomes a standard of care, questions remain to be answered regarding how VR visualisation is paid for, and how it is integrated into existing picture archiving and communicating systems (PACS).

Strengths and limitations

Several challenges exist that prevent global implementation of VR for simulation and for preoperative planning. There is still a lack of evidence that VR provides better patient care. Most studies that were considered in this review based their findings on qualitative validation through questionnaires (e.g., better preparation for a surgical procedure, better preoperative visualisation), but didn’t directly quantify or define improved patient outcomes, therefore allowing the authors to conclude conclusively that better visualisation improves outcomes. A limitation of this narrative review is that some studies are included that have not been published yet. However, to provide an overview as complete as possible, we included the preliminary results of those studies. Implementation of the Medical Device Regulation in 2021 (68) and the proposed Artificial Intelligence (AI) Act (69) in Europe makes setting up research with VR more difficult from a legal perspective within the European Union. Secondly, VR visualisation efforts are almost exclusively made possible through sponsorship by the manufacturer of the VR hardware or software, or through research grants. In the coming years, discussions must be held with health insurance companies and governmental health departments about reimbursement for patient-specific VR simulations and VR visualisation to prepare for surgery. This is unlikely to proceed before concrete evidence of improved outcomes (safety, efficacy, and cost-effectiveness) is published.

Thirdly, various centres around the world are currently experimenting with different types of simulation and visualisation, based on different types of VR hardware and software (61,70-72). This is inevitable in the experimental phase of VR in medicine, but in the near future, a more nation-wide or international collaboration and/or adoption of VR techniques should be pursued. There should be more focus on multi-user environments instead of one person who is cut off from the real world by VR, or training possibilities should be provided via cloud software and be accessible on all VR hardware. This can have a positive influence on the two aforementioned issues: patient outcomes and reimbursement by health insurance firms.

On the other hand, we must accept that VR, especially for training purposes, still lacks various important features that prevent global implementation. In the last years a lot of improvements have been made of the HMDs, however, they can still be uncomfortable, causing dizziness, headaches, or forms of motion sickness (73-75). In VR users may get disoriented from being in a fully virtual world manipulating, scaling, and rotating the VR patient models extensively (74,76). In simulation, VR still lacks tactile feedback, where the participant has to hold two controllers, instead of using a real surgical instrument, which may impede learning. Recent improvements in hand tracking technology for VR HMDs as well as neural network based live object recognition and tracking make it possible that future VR simulators will feature more realistic interactions with instruments and tissues alike. Haptic feedback such as that implemented by Nakao et al. will help to ameliorate this issue (77). Additional features to improve realism such as the feeling of stress should be implemented for simulators, through a combination of visual, audio, and haptic cues.

Comparison with similar research

Reviews such as those performed by Sadeghi et al. (78), and Villanueva et al. (79) pose similar research questions, although we specifically focused on the application of VR in the field of simulation and planning. Mahtab and Egorova recently published a clinical outlook piece earlier this year where they briefly consider how VR currently impacts the field of cardiac interventions, and how this is likely to change in the future (80). We have provided an update from currently and soon-to-be available literature in this particularly fast-moving field of technology.

Future outlook

Below, we present our vision of the future of cardiothoracic surgery, portrayed through a virtual scenario set in the year 2040 and illustrated in Figure 3.

Figure 3 The journey of a cardiothoracic surgery patient in the year 2040, and the contribution of virtual reality pre-, intra-, and post-operatively. The virtual multidisciplinary team figure is adapted from Sadeghi et al. [2021] (81), and is licensed under CC BY 4.0. VR, virtual reality; ECC, extracorporeal circulation.

A patient who is due to undergo a cardiothoracic surgery is discussed in a virtual multidisciplinary team discussion (81), whereby experts attend from different geographical locations using VR headsets and gather in the communal metaverse space. From here, they will be able to independently view the patient’s 3D anatomical visualisation, with the ability to show or hide, or highlight structures, and show these alterations to their colleagues in the room. Two surgical plans will be proposed, and in order to decide on which to proceed with, the patient’s 3D anatomy is transferred into a surgical simulator, and the procedure trialled using both approaches, before settling on the superior approach based on predicted flow and haemodynamic parameters with either solution. In the outpatient setting, the patient and their family can be counselled, and the surgical options demonstrated through the VR simulation of the patient’s anatomy before and after the proposed procedure, and a shared decision reached. The patient and their family will be prepared for the experience of undergoing surgery through VR, where they are shown what the rooms will look like, who the staff members present will be and what they do, and what to expect when it is time to go to sleep, thereby reducing the stress associated with this portion of their journey (82). The surgeon who performs the operation will have honed their craft initially in the virtual surgical world, and demonstrated their ability using a proficiency-based progression strategy that they are able to safely perform each step of the operation (83). This VR simulator is an online environment in which various surgeons from different part of the world can participate, to help and train the performing surgeon with the procedure. Before performing the surgery, the patient’s visualisation will be transferred into the surgical robot, and using a combination of AI image recognition and instrument positional tracking, the anatomical model can be overlaid onto the camera’s display creating an augmented reality display that dynamically updates with tissue handling and camera positioning changes. The nursing team including the scrub nurse will have trained in VR to prepare the theatre for this specific procedure, and ensure all required tools are on hand. The perfusionist who manages circulatory support during the procedure will have experienced a rigorous VR training program that prepared them for both the type of procedure and the HLM equipment used. During the procedure, they note the reservoir level dropping and the presence of air in the venous line. The perfusionist, who has practised for the dislocation of a venous cannula in VR, and is able to calmly troubleshoot it, addressing the issue by communicating to the surgeon that the venous cannula no longer sited correctly, and in the interim uses the suctions to maintain bypass. The surgeon recannulates the venous line and completes the operation as planned, before closing and sending the patient to the ICU. There the patient experiences a sudden witnessed cardiac arrest with ventricular fibrillation seen on monitoring. The staff, who have trained together in VR for performing ALS after cardiac surgery will be able to administer stacked shocks, whereby return of spontaneous circulation is quickly restored. For the remainder of the patient’s ICU stay, VR helps to improve their mental health (84), and reduce their perception of pain (85,86).


Conclusions

In this review, we have shown that VR technology in cardiothoracic surgery is starting to bear fruit, and that for both preoperative planning and (peri)procedural simulation, the current and next generation of simulators are likely to change the way the field prepares and treat its patients. The path to validate these simulators is well described, but large-scale trials producing high-level evidence for their efficacy are absent as of yet. Despite the many challenges facing it, we anticipate that this technology will increasingly become part of daily life for many (surgical) specialities, and that cooperation between different specialities, research hospitals, tech start-ups, and more established industrial partners will result in major improvements in the usability, flexibility, and overall quality of VR applications. For a field that is growing ever more complex and minimally invasive, we propose that VR will contribute significantly into propelling cardiothoracic surgery into the next decade and beyond.


Acknowledgments

Funding: None.


Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editors (Leizl Joy Nayahangan, Lars Konge and René Horsleben Petersen) for the series “Simulation-Based Education in Cardiac and Thoracic Surgery” published in Shanghai Chest. The article has undergone external peer review.

Reporting Checklist: The authors have completed the Narrative Review reporting checklist. Available at https://shc.amegroups.com/article/view/10.21037/shc-22-63/rc

Peer Review File: Available at https://shc.amegroups.com/article/view/10.21037/shc-22-63/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://shc.amegroups.com/article/view/10.21037/shc-22-63/coif). The series “Simulation-Based Education in Cardiac and Thoracic Surgery” was commissioned by the editorial office without any funding or sponsorship. SAM and EAFM are supported by Erasmus + Grant (No. 2022-1-NL01-KA220-HED-000087770). WB, SAM, EAFM, and AHS are co-developers of the CPVR-sim and VR ECC sim, and EAFM and AHS are co-inventors of PulmoVR and CardioVR. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Gagnon ED. Commissurotomy in mitral stenosis. Can Med Assoc J 1950;63:537-40.
  2. Holmes DR Jr, Rich JB, Zoghbi WA, et al. The heart team of cardiovascular care. J Am Coll Cardiol 2013;61:903-7. [Crossref] [PubMed]
  3. Callaghan JC. Replacement of the aortic and mitral valves using the Starr-Edwards ball-valve prosthesis: a report of 50 cases. Can Med Assoc J 1964;91:411-21.
  4. Hemli JM, Patel NC. Robotic Cardiac Surgery. Surg Clin North Am 2020;100:219-36. [Crossref] [PubMed]
  5. Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Adv Simul (Lond) 2016;1:31. [Crossref] [PubMed]
  6. Bianco V, Kilic A, Gleason TG, et al. Reoperative Cardiac Surgery Is a Risk Factor for Long-Term Mortality. Ann Thorac Surg 2020;110:1235-42. [Crossref] [PubMed]
  7. Fascetti-Leon F, Gobbi D, Pavia SV, et al. Sparing-lung surgery for the treatment of congenital lung malformations. J Pediatr Surg 2013;48:1476-80. [Crossref] [PubMed]
  8. Arrighi JA, Mendes LA, McConnaughey S, et al. Competency-Based Medical Education for Fellowship Training During the COVID-19 Pandemic. J Am Coll Cardiol 2021;77:1681-3. [Crossref] [PubMed]
  9. Shirk JD, Kwan L, Saigal C. The Use of 3-Dimensional, Virtual Reality Models for Surgical Planning of Robotic Partial Nephrectomy. Urology 2019;125:92-7. [Crossref] [PubMed]
  10. Filigenzi MT, Orr TJ, Ruff TM. Virtual reality for mine safety training. Appl Occup Environ Hyg 2000;15:465-9. [Crossref] [PubMed]
  11. Lele A. Virtual reality and its military utility. J Ambient Intell Humaniz Comput 2013;4:17-26.
  12. Cieślik B, Mazurek J, Rutkowski S, et al. Virtual reality in psychiatric disorders: A systematic review of reviews. Complement Ther Med 2020;52:102480. [Crossref] [PubMed]
  13. Halabi O, Salahuddin T, Karkar AG, et al. Virtual reality for ambulance simulation environment. Multimed Tools Appl 2022;81:32119-37.
  14. Matamala-Gomez M, Donegan T, Bottiroli S, et al. Immersive Virtual Reality and Virtual Embodiment for Pain Relief. Front Hum Neurosci 2019;13:279. [Crossref] [PubMed]
  15. Massetti T, da Silva TD, Crocetta TB, et al. The Clinical Utility of Virtual Reality in Neurorehabilitation: A Systematic Review. J Cent Nerv Syst Dis 2018;10:1179573518813541. [Crossref] [PubMed]
  16. Chen FQ, Leng YF, Ge JF, et al. Effectiveness of Virtual Reality in Nursing Education: Meta-Analysis. J Med Internet Res 2020;22:e18290. [Crossref] [PubMed]
  17. Satava RM. Virtual reality surgical simulator - The first steps. Surg Endosc 1993;7:203-5. [Crossref] [PubMed]
  18. Mitha AP, Almekhlafi MA, Janjua MJ, et al. Simulation and augmented reality in endovascular neurosurgery: lessons from aviation. Neurosurgery 2013;72:107-14. [Crossref] [PubMed]
  19. Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004;91:146-50. [Crossref] [PubMed]
  20. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458-63; discussion 463-4. [Crossref] [PubMed]
  21. Ahlberg G, Enochsson L, Gallagher AG, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 2007;193:797-804. [Crossref] [PubMed]
  22. de Visser H, Watson MO, Salvado O, et al. Progress in virtual reality simulators for surgical training and certification. Med J Aust 2011;194:S38-40. [Crossref] [PubMed]
  23. Gehrsitz P, Rompel O, Schöber M, et al. Cinematic Rendering in Mixed-Reality Holograms: A New 3D Preoperative Planning Tool in Pediatric Heart Surgery. Front Cardiovasc Med 2021;8:633611. [Crossref] [PubMed]
  24. Laschinger JC, Vannier MW, Gronemeyer S, et al. Noninvasive three-dimensional reconstruction of the heart and great vessels by ECG-gated magnetic resonance imaging: a new diagnostic modality. Ann Thorac Surg 1988;45:505-14. [Crossref] [PubMed]
  25. Székely G, Satava RM. Virtual reality in medicine. BMJ Br Med J 1999;319:1305. [Crossref] [PubMed]
  26. Shirk JD, Thiel DD, Wallen EM, et al. Effect of 3-Dimensional Virtual Reality Models for Surgical Planning of Robotic-Assisted Partial Nephrectomy on Surgical Outcomes: A Randomized Clinical Trial. JAMA Netw Open 2019;2:e1911598. [Crossref] [PubMed]
  27. Jensen K, Bjerrum F, Hansen HJ, et al. Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 2017;31:2520-8. [Crossref] [PubMed]
  28. Whittaker G, Aydin A, Raveendran S, et al. Validity assessment of a simulation module for robot-assisted thoracic lobectomy. Asian Cardiovasc Thorac Ann 2019;27:23-9. [Crossref] [PubMed]
  29. Turbati MS, Goldblatt MI, Gould JC, et al. Robotic simulation: validation and qualitative assessment of a general surgery resident training curriculum. Surg Endosc 2023;37:2304-15. [Crossref] [PubMed]
  30. Peek JJ, Max SA, Bakhuis W, et al. Virtual Reality Simulator versus Conventional Advanced Life Support Training for Cardiopulmonary Resuscitation Post-Cardiac Surgery: A Randomized Controlled Trial. J Cardiovasc Dev Dis 2023;10:67. [Crossref] [PubMed]
  31. Babar ZUD, Max SA, Martina BG, et al. Virtual Reality Simulation as a Training Tool for Perfusionists in Extracorporeal Circulation: Establishing Face and Content Validity. JTCVS Techniques 2023. doi: 10.1016/j.xjtc.2023.06.004.
  32. Aggarwal R, Tully A, Grantcharov T, et al. Virtual reality simulation training can improve technical skills during laparoscopic salpingectomy for ectopic pregnancy. BJOG 2006;113:1382-7. [Crossref] [PubMed]
  33. Sadeghi AH, Peek JJ, Max SA, et al. Virtual Reality Simulation Training for Cardiopulmonary Resuscitation After Cardiac Surgery: Face and Content Validity Study. JMIR Serious Games 2022;10:e30456. [Crossref] [PubMed]
  34. Jensen K, Bjerrum F, Hansen HJ, et al. A new possibility in thoracoscopic virtual reality simulation training: development and testing of a novel virtual reality simulator for video-assisted thoracoscopic surgery lobectomy. Interact Cardiovasc Thorac Surg 2015;21:420-6. [Crossref] [PubMed]
  35. Haidari TA, Bjerrum F, Hansen HJ, et al. Simulation-based VATS resection of the five lung lobes: a technical skills test. Surg Endosc 2022;36:1234-42. [Crossref] [PubMed]
  36. Haidari TA, Bjerrum F, Christensen TD, et al. Assessing VATS competence based on simulated lobectomies of all five lung lobes. Surg Endosc 2022;36:8067-75. [Crossref] [PubMed]
  37. Valdis M, Chu MW, Schlachta C, et al. Evaluation of robotic cardiac surgery simulation training: A randomized controlled trial. J Thorac Cardiovasc Surg 2016;151:1498-1505.e2. [Crossref] [PubMed]
  38. Cowan A, Chen J, Mingo S, et al. Virtual Reality vs Dry Laboratory Models: Comparing Automated Performance Metrics and Cognitive Workload During Robotic Simulation Training. J Endourol 2021;35:1571-6. [Crossref] [PubMed]
  39. The Society of Thoracic Surgeons Expert Consensus for the Resuscitation of Patients Who Arrest After Cardiac Surgery. Ann Thorac Surg 2017;103:1005-20. [Crossref] [PubMed]
  40. Hall C, Robertson D, Rolfe M, et al. Do cognitive aids reduce error rates in resuscitation team performance? Trial of emergency medicine protocols in simulation training (TEMPIST) in Australia. Hum Resour Health 2020;18:1. [Crossref] [PubMed]
  41. Szulewski A, Howes D, van Merriënboer JJG, et al. From Theory to Practice: The Application of Cognitive Load Theory to the Practice of Medicine. Acad Med 2021;96:24-30. [Crossref] [PubMed]
  42. Carter FJ, Schijven MP, Aggarwal R, et al. Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 2005;19:1523-32. [Crossref] [PubMed]
  43. Yamada Y, Nakamura T, Yamada M, et al. Use of Augmented Reality to Assist Teaching for Future Perfusionists in Extracorporeal Technology. J Extra Corpor Technol 2019;51:244-7. [Crossref] [PubMed]
  44. Hutin A, Abu-Habsa M, Burns B, et al. Early ECPR for out-of-hospital cardiac arrest: Best practice in 2018. Resuscitation 2018;130:44-8. [Crossref] [PubMed]
  45. Wolff G, Bruno RR, Reiter M, et al. Virtual reality device training for extracorporeal membrane oxygenation. Crit Care 2020;24:390. [Crossref] [PubMed]
  46. Ball K, Doyle D, Oocumma NI. Nursing shortages in the OR: solutions for new models of education. AORN J 2015;101:115-36. [Crossref] [PubMed]
  47. Edwards TC, Patel A, Szyszka B, et al. Immersive virtual reality enables technical skill acquisition for scrub nurses in complex revision total knee arthroplasty. Arch Orthop Trauma Surg 2021;141:2313-21. [Crossref] [PubMed]
  48. Ong CS, Krishnan A, Huang CY, et al. Role of virtual reality in congenital heart disease. Congenit Heart Dis 2018;13:357-61. [Crossref] [PubMed]
  49. Mendez A, Hussain T, Hosseinpour AR, et al. Virtual reality for preoperative planning in large ventricular septal defects. Eur Heart J 2019;40:1092. [Crossref] [PubMed]
  50. Ghosh RM, Mascio CE, Rome JJ, et al. Use of Virtual Reality for Hybrid Closure of Multiple Ventricular Septal Defects. JACC Case Rep 2021;3:1579-83. [Crossref] [PubMed]
  51. Chan F, Aguirre S, Bauser-Heaton H, et al. Head Tracked Stereoscopic Pre-surgical Evaluation of Major Aortopulmonary Collateral Arteries in the Newborns. Radiol Soc North Am 2013. 2013 Scientific Assembly and Annual Meeting. Available online: http://archive.rsna.org/2013/13024673.html
  52. van de Woestijne PC, Bakhuis W, Sadeghi AH, et al. 3D Virtual Reality Imaging of Major Aortopulmonary Collateral Arteries: A Novel Diagnostic Modality. World J Pediatr Congenit Heart Surg 2021;12:765-72. [Crossref] [PubMed]
  53. Ayerbe VMC, Morales MLV, Rojas CJL, et al. Visualization of 3D Models Through Virtual Reality in the Planning of Congenital Cardiothoracic Anomalies Correction: An Initial Experience. World J Pediatr Congenit Heart Surg 2020;11:627-9. [Crossref] [PubMed]
  54. Pushparajah K, Chu KYK, Deng S, et al. Virtual reality three-dimensional echocardiographic imaging for planning surgical atrioventricular valve repair. JTCVS Tech 2021;7:269-77. [Crossref] [PubMed]
  55. Ramaswamy RK, Marimuthu SK, Ramarathnam KK, et al. Virtual reality-guided left ventricular assist device implantation in pediatric patient: Valuable presurgical tool. Ann Pediatr Cardiol 2021;14:388-92. [Crossref] [PubMed]
  56. Pelizzo G, Costanzo S, Roveri M, et al. Developing Virtual Reality Head Mounted Display (HMD) Set-Up for Thoracoscopic Surgery of Complex Congenital Lung MalFormations in Children. Children (Basel) 2022;9:50. [Crossref] [PubMed]
  57. Sadeghi AH, Taverne YJHJ, Bogers AJJC, et al. Immersive virtual reality surgical planning of minimally invasive coronary artery bypass for Kawasaki disease. Eur Heart J 2020;41:3279. [Crossref] [PubMed]
  58. Abjigitova D, Sadeghi AH, Peek JJ, et al. Virtual Reality in the Preoperative Planning of Adult Aortic Surgery: A Feasibility Study. J Cardiovasc Dev Dis 2022;9:31. [Crossref] [PubMed]
  59. Nanchahal S, Arjomandi Rad A, Naruka V, et al. Mitral valve surgery assisted by virtual and augmented reality: Cardiac surgery at the front of innovation. Perfusion 2022; Epub ahead of print. [Crossref]
  60. Frajhof L, Borges J, Hoffmann E, et al. Virtual reality, mixed reality and augmented reality in surgical planning for video or robotically assisted thoracoscopic anatomic resections for treatment of lung cancer. J Vis Surg 2018;4:143.
  61. Sadeghi AH, Maat APWM, Taverne YJHJ, et al. Virtual reality and artificial intelligence for 3-dimensional planning of lung segmentectomies. JTCVS Tech 2021;7:309-21. [Crossref] [PubMed]
  62. Bakhuis W, Sadeghi AH, Moes I, et al. Essential Surgical Plan Modifications After Virtual Reality Planning in 50 Consecutive Segmentectomies. Ann Thorac Surg 2023;115:1247-55. [Crossref] [PubMed]
  63. Peek JJ, Sadeghi AH, Maat APWM, et al. Multidisciplinary Virtual Three-Dimensional Planning of a Forequarter Amputation With Chest Wall Resection. Ann Thorac Surg 2022;113:e13-6. [Crossref] [PubMed]
  64. Bakhuis W, Kersten CM, Sadeghi AH, et al. Preoperative visualization of congenital lung abnormalities: hybridizing artificial intelligence and virtual reality. Eur J Cardiothorac Surg 2022;63:ezad014. [Crossref] [PubMed]
  65. Bol Raap G, Koning AH, Scohy TV, et al. Virtual reality 3D echocardiography in the assessment of tricuspid valve function after surgical closure of ventricular septal defect. Cardiovasc Ultrasound 2007;5:8. [Crossref] [PubMed]
  66. Moglia A, Ferrari V, Morelli L, et al. A Systematic Review of Virtual Reality Simulators for Robot-assisted Surgery. Eur Urol 2016;69:1065-80. [Crossref] [PubMed]
  67. MedicalVR. PulmoVR as a preoperative planning tool for thoracic surgery. 2022. Available online: https://www.medicalvr.eu/pulmovr/
  68. Regulation (EU) 2017/745 Of the European parliament and of the council. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32017R0745
  69. Legal analysis - European legislative proposal draft AI act and MDR/IVDR | Publication | Government.nl. Available online: https://www.government.nl/documents/publications/2022/05/25/legal-analysis-european-legislative-proposal-draft-ai-act-and-mdr-ivdr
  70. Yao F, Wang J, Yao J, et al. Three-dimensional image reconstruction with free open-source OsiriX software in video-assisted thoracoscopic lobectomy and segmentectomy. Int J Surg 2017;39:16-22. [Crossref] [PubMed]
  71. Cui Z, Ding C, Li C, et al. Preoperative evaluation of the segmental artery by three-dimensional image reconstruction vs. thin-section multi-detector computed tomography. J Thorac Dis 2020;12:4196-204. [Crossref] [PubMed]
  72. Gossot D, Lutz J, Grigoroiu M, et al. Thoracoscopic anatomic segmentectomies for lung cancer: technical aspects. J Vis Surg 2016;2:171. [Crossref] [PubMed]
  73. Byl JL, Sholler R, Gosnell JM, et al. Moving beyond two-dimensional screens to interactive three-dimensional visualization in congenital heart disease. Int J Cardiovasc Imaging 2020;36:1567-73. [Crossref] [PubMed]
  74. Monsky WL, James R, Seslar SS. Virtual and augmented reality applications in medicine and surgery–the fantastic voyage is here. Anat Physiol 2019;9:1-6.
  75. Cen J, Liufu R, Wen S, et al. Three-Dimensional Printing, Virtual Reality and Mixed Reality for Pulmonary Atresia: Early Surgical Outcomes Evaluation. Heart Lung Circ 2021;30:296-302. [Crossref] [PubMed]
  76. Goo HW, Park SJ, Yoo SJ. Advanced Medical Use of Three-Dimensional Imaging in Congenital Heart Disease: Augmented Reality, Mixed Reality, Virtual Reality, and Three-Dimensional Printing. Korean J Radiol 2020;21:133-45. [Crossref] [PubMed]
  77. Nakao M, Oyama H, Komori M, et al. Haptic reproduction and interactive visualization of a beating heart for cardiovascular surgery simulation. Int J Med Inform 2002;68:155-63. [Crossref] [PubMed]
  78. Sadeghi AH, Mathari SE, Abjigitova D, et al. Current and Future Applications of Virtual, Augmented, and Mixed Reality in Cardiothoracic Surgery. Ann Thorac Surg 2022;113:681-91. [Crossref] [PubMed]
  79. Villanueva C, Xiong J, Rajput S. Simulation-based surgical education in cardiothoracic training. ANZ J Surg 2020;90:978-83. [Crossref] [PubMed]
  80. Mahtab EAF, Egorova AD. Current and future applications of virtual reality technology for cardiac interventions. Nat Rev Cardiol 2022;19:779-80. [Crossref] [PubMed]
  81. Sadeghi AH, Wahadat AR, Dereci A, et al. Remote multidisciplinary heart team meetings in immersive virtual reality: a first experience during the COVID-19 pandemic. BMJ Innov 2021;7:311-5. [Crossref] [PubMed]
  82. Esposito C, Autorino G, Iervolino A, et al. Efficacy of a Virtual Reality Program in Pediatric Surgery to Reduce Anxiety and Distress Symptoms in the Preoperative Phase: A Prospective Randomized Clinical Trial. J Laparoendosc Adv Surg Tech A 2022;32:197-203. [Crossref] [PubMed]
  83. Mazzone E, Puliatti S, Amato M, et al. A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes. Ann Surg 2021;274:281-9. [Crossref] [PubMed]
  84. Vlake JH, van Bommel J, Wils EJ, et al. Intensive Care Unit-Specific Virtual Reality for Critically Ill Patients With COVID-19: Multicenter Randomized Controlled Trial. J Med Internet Res 2022;24:e32368. [Crossref] [PubMed]
  85. Ding L, Hua H, Zhu H, et al. Effects of virtual reality on relieving postoperative pain in surgical patients: A systematic review and meta-analysis. Int J Surg 2020;82:87-94. [Crossref] [PubMed]
  86. Trost Z, France C, Anam M, et al. Virtual reality approaches to pain: toward a state of the science. Pain 2021;162:325-31. [Crossref] [PubMed]
doi: 10.21037/shc-22-63
Cite this article as: Bakhuis W, Max SA, Maat APWM, Bogers AJJC, Mahtab EAF, Sadeghi AH. Preparing for the future of cardiothoracic surgery with virtual reality simulation and surgical planning: a narrative review. Shanghai Chest 2023;7:23.

Download Citation