Like any new technology, a lot of focus has been placed on ensuring that new users of robotic surgery are adequately trained. Simulation has had a large part to play with this. As the technology has become more mainstream, training requirements have moved from not only training existing surgeons but to ensuring that residents and fellows develop the required skill levels to ensure that they can adapt to the new technologies used in their practice.
Earlier this year we discussed a paper published by the EAU on their curriculum aimed at ensuring that fellows followed a clear curriculum at the end of which they would be deemed to be safe and competent to operate on patients independently. As with many ways of teaching surgery, the procedure is broken into specific steps that the trainee must master before being allowed to carry the whole procedure.
A typical prostatectomy is divided into the 7 following steps: bladder takedown, endopelvic fascia, bladder neck, seminal vesicle/vas deferens, pedicle/nerve sparing, apex, and anastomosis. Typically a trainee will be given a maximum time, of say 30 minutes,to complete one of these tasks during a procedure. Once they have shown that they have mastered the tasks, they will be allowed to move onto another task and eventually to the whole procedure. This is obviously easier to accomplish on parts of the anatomy and procedures that can be standardized.
Until recently, there have not been many studies looking into this practice to see what the potential patient impact could be comparing when a surgery was performed by just the one attending surgeon to one where parts of the case had been handed over to the resident.
Dr. Thiel from the Mayo Clinic in Jacksonville, Florida, has published a paper on just this topic comparing 140 cases where just an attending was involved in the surgery to 232 cases when a resident took over part of the case.
There were no differences in some key clinical outcomes such as positive margins, length of stay, catheter days, readmissions or re-operations when comparing surgeon only to resident –involved cases. There was, however, a difference seen in mean operative time between procedures that were surgeon only cases vs. resident involved (190.4 Min vs. 206.4 Min, P= 0.003)
The researchers also noted that residents were more likely to be involved with at least 1 procedural step after the purchase of the dV-Trainer.
Mimic believes in this way of training residents which is why the Maestro AR set of procedural curricula we have developed are divided into the procedural steps that a resident will be required to learn. We have been able to marry narrated 3D video content with didactic exercises that allow for a student’s ability to be tested. At the appropriate point, the correct psychomotor skill is inserted to make sure that the student can match the skills required for the procedural step.
Mimic currently has the following available:
- Right Partial Nephrectomy, Dr. Indibir Gill, USC
- Hysterectomy, Dr. Arnold Advincula, Columbia University
- Inguinal Hernia Repair, Dr. Rick Low, John C. Lincoln Hospital
- Prostatectomy (Si), Dr. Henk van der Poel, Antoni van Leeuwenhoek Hospital/Netherlands Cancer Institute in Amsterdam
- Prostatectomy (Xi), Dr. Vip Patel, Florida Hospital
- In Development for Q4 ‘16 Release:
- Lower Anterior Resection, Dr. Eduardo Parra, Florida Hospital
It has now been 15 years since Mimic Technologies was launched as a company and 12 years of providing robotic surgery simulation training. Over the years, Mimic has been involved with one of the most successful surgical simulation launches of all time with over 2,000 simulators using MSim software worldwide. We estimate that over 30,000 surgeons / residents use this simulation install base each year and that since it was launched over 6.5 million exercise sessions have been completed.
Many leading academic centers around the world have incorporated Mimic hardware and software into their training programs and there have been numerous publications that were researched and written on Mimic’s hardware and software proving validity. (A sample of these published studies can be found here.)
All of this experience has allowed Mimic to collect simulation data as well as hands-on experience in successfully implementing best practices to help fully develop new and existing robotic surgery simulation training programs. Over the years, we have found that the most common traits of a successful simluation training program include:
- Individuals (trainees) are uniquely identified and results are recorded
Data is king! It is important for simulation users to create an account and always remember to sign in so that the record of performance over time can show a progression of skill development and maintenance.
- Proficiency levels have been discussed and agreed upon
The study, Best Practices for Robotic Surgery Training and Credentialing, published in 2011 in the Journal of Urology by Jason Lee, et al., concluded that “rather than being based on a set number of completed cases, robotic surgery credentialing should involve the demonstration of proficiency and safety in executing basic robotic skills and procedural tasks. In addition, the accreditation process should be iterative to ensure accountability to the patient.” Setting institutional standards that have been both discussed and agreed upon will ensure that all clinicians who will be training using simulation are meeting the same requirements. Objective scoring is also helpful to implement a fair and accurate training environment.
- Curriculum are developed, allocated appropriately, and continuously measured
According to a 2005 study, Virtual Reality Simulation for the Operating Room: Proficiency-Based Training as a Paradigm Shift in Surgical Skills Training, by Gallagher, et al., “virtual reality training is more likely to be successful if it is systematically integrated into a well-thought-out education and training program.” Defining specific exercises, mapping out a training path, and continuously checking progress is essential for making sure that trainees get the most out of simulation to build their skills and move up the learning curve towards proficiency.
- Simulation training platforms are easily accessible to trainees
Also essential for simulation training is ensuring that trainees are able to access the simulator at times that are most convenient and conducive to their learning preferences.
- Simulation time is transferable to the real tool
Face validation shows that a training tool has a realistic look and feel. The 2015 study published in Surgical Endoscopy, Robotic Surgery Simulation Validity and Usability Comparative Analysis, concluded that, “Usability can affect the consistency and commitment of users of robotic surgical simulators.” Before simulation training is implemented, the training tool should be carefully assessed to ensure the skills trainees acquire transfer to the tool they will be utilizing.
- Cognitive and psychomotor skills can be validated
In addition to validating the training tool for Face and Content, the acquisition of both cognitive and psychomotor skills should be validated and proven to make outcomes better. Construct validity distinguishes experienced medical professionals from the inexperienced, Concurrent validity measures the extent to which the simulator correlates with the “Gold standard”, and Predictive validity goes so far as to predict future performance. These types of validation are important to consider when choosing a simulation training tool.
- Teams can train together
The ability to incorporate team training within a simulation training curricula ensure that the trainees will have well-rounded skills such as communication and movement coordination in addition to being proficient in operating the tool they are training for. The study, Teaching Surgical Skills – Changes in the Wind, published in the New England Journal of Medicine by Dr. Richard Reznick, et al., stated, “Virtual reality has the potential to enhance surgical-team training as well as technical skills training. In aviation, teamwork training with simulation has been instrumental in reducing errors. The importance of teamwork in preventing medical error is well recognized, and simulator-based team training has been advocated as a possible preventive approach. Early research results have been promising.”
Advances in technology and virtual reality simulation training can make medical training safer, more cost-effective, and efficient and building a successful program doesn’t have to be difficult if similar principles and benchmarks are applied.
Surgery has often been referred to as a team sport. The role of the surgeon is undoubtedly critical but so is the role of the entire team in ensuring patient safety and positive outcomes.
This is especially true in robotic surgery where the main surgeon is no longer at the patient’s side but is seated at a robotic console outside the operative field. The console-side surgeon now relies heavily on a first assistant or patient-side surgeon. The first assistant is a skilled laparoscopic surgeon whose prime objective is to work as a team with the robotic surgeon to maximize efficiency during the procedure.
A recent study titled, “Impact of Assistant Surgeon on Outcomes in Robotic Surgery,” by Dr. Rishi Nayyar, et al., published in the Indian Journal of Urology found that, “with increasing experience of patient-side surgeon and associated console surgeon, who form a consistent surgical team, the mean operative time for all robotic procedures shows a consistent trend of reduction across all surgical types.” (Click Here to Access the Full Study)
Typically, better outcomes in robotic surgery are associated with only the console-surgeon. According to Dr. Nayyar’s study, there is no existing objective evidence regarding the impact on outcomes with the experience of the assistant surgeon in robot-assisted surgery. Therefore, the goal of this recent study was to objectively verify the hypothesis that the experience of patient-side assistant in robotic surgery affects intraoperative outcomes.
During the study, a total of 100 cases of robot-assisted laparoscopic pyeloplasty were analyzed and on comparing outcomes between the 1st and 2nd halves of the assistant experience, the mean operative time reduced from 102.50 min to 82.80 min (P = 0.001) and mean blood loss reduced from 72.00 ml to 63.90 ml (P = 0.91). (See table below)
As the study suggests, a console-surgeon and first assistant to who work consistently together make a better team and positively influence the outcome of the procedure. Mimic’s Xperience Team Trainer (XTT) was developed specifically for this reason.
The XTT simulates the patient-side and connects with the dV-Trainer that simulates the console side, thereby allowing both the console and the patient-side surgeons to train in tandem with virtual reality simulators. Routine tasks are executed crisply and efficiently while also working on communication between both surgeons.
A 2015 study done by Dr. Jacques Hubert, et al., published in Surgical Endoscopy confirmed face, content, construct, and concurrent validity of the Xperience Team Trainer as an assessment tool of robotic surgery bed-assistance skills for the patient-side surgeon.
This study also emphasized the importance of teamwork between the patient-side and console-side surgeon in robotic surgery, which may change the paradigm of robotic surgery training in the near future. To read a past post about this study, click here.
Nayyar R, Yadav S, Singh P, Dogra PN. Impact of assistant surgeon on outcomes in robotic surgery. Indian J Urol 2016;32:204-9
There are many aspects to a training simulator that can be considered when making the initial investment in simulation training. For robotic surgery, we believe the top factors to consider are:
- Validation studies conducted on and using the simulator
- Fidelity of the controllers
- Accessibility of the simulator
- Data, data, data!
Since Mimic launched its first version of the dV-Trainer in 2007, there has been a growing number of new robotic surgery simulators entering the market. The real impetus for simulation training was made clear in 2010 when Intuitive Surgical decided to launch their own Skills Simulator, a backpack-like addition for the da Vinci® Si platform.
Intuitive Surgical chose to license 27 exercises that Mimic had already developed or were in the process of developing especially for ISI. This was made possible by the new design of the system, which allowed for the console to operate independently of the patient side cart and core. Since 2010, both the ROSS Simulator from Simulated Surgical Systems and the Robotix Mentor from Simbionix (now 3D Systems) have entered the playing field.
The installed base of da Vinci® surgical systems is now over 3,500 systems around the world and close to 2,000 simulators have been installed and used to support this installed base. The majority of training simulators are da Vinci® Skill simulators (with Mimic’s licensed software) and close to 12% of robotic surgery simulators are Mimic’s dV-Trainers.
Our estimate is that over 70% of institutions performing robotic surgery have access to a simulator of some form or another and that close to 90% of robotic surgeons will at some point have tried a simulator. In fact, since 2007 we believe that between the dV-Trainer and the da Vinci® Skills Simulator over 6.25 million exercise sessions have been completed.
So has all of this simulation training activity been valuable you may ask? One way to look assess simulation training is through validation studies. There are currently five different ways of determining validity. Starting with the basics Face, Content, and Construct and moving to more valuable validation such as Concurrent and Predictive, the definitions are:
Face validity: Does the simulator have a realistic look and feel, compared to the actual surgical system?
Content validity: Is the simulator useful as a training tool for the surgical system?
Construct validity: Does the simulator have the ability to distinguish between Novice and Expert users?
Concurrent validity: How does the simulator compare to a similar or related construct (Dry Labs, Tissue Lab, etc.) carried out on the real robotic surgical system?
Predictive: validity: Can the simulator be used to predict actual performance in the O.R.?
Face and Content are of relatively low value as they are subjective and the most highly valued validation studies are Construct and Predictive validity. The table below shows the number of papers that have been published on various types of validation. As you can see there have been over 30 papers published on Mimic software either on the dV-Trainer or the da Vinci® Skills Simulator platform.
Recently, simulation was a large part of the discussion at the FDA town hall meeting in Washington. Roger Smith from Florida Hospital presented a comparison of the different simulators led by himself (the table above is adapted from his presentation). The data presented was clear that the most focus in researching the simulators was on the controllers and how close they emulated the real robotic surgeon’s console. Obviously, the da Vinci® Skills Simulator, which uses the real console is the real thing. However for the other simulators, this is where concurrent validity because extremely important, as essentially you are replicating (using the simulator) the same activity a surgeon would be doing on the real robotic surgical system.
A direct head to head study was done by Prof. Jacques Hubert and his team at the STAN Institute in Nancy, France between Mimic’s dV-Trainer and the da Vinci® Skills Simulator. During the study, participants completed the same exercises on the both systems and researchers found that on average there was only a 3% difference in overall score between the two systems. (89.9% vs 86.8%). This varied by the type of exercise but remained consistent with some internal bench-marking carried out by Mimic. No studies have been done to the same extent on the Ross and Robotix Mentor systems.
Another component to take in consideration when choosing a robotic surgery simulator is the accessibility to the system. While the great thing about the da Vinci® Skills Simulator is that it uses the real console, this can also be very detrimental and a negative for the da Vinci® Skills Simulator that it uses the real console. Very few hospitals can afford to have a dedicated console outside the OR that is used purely for training and simulation. If an institution is lucky enough to have a dual console system they will have the simulator on the second console but that is still kept in the OR. The value of the second console is in allowing programs with residents to keep training new surgeons without interrupting the flow and efficiency of the OR. Data shows that simulation systems in the OR are used less than systems outside the OR. This is due to the simple fact that as robotic programs become more successful and utilization increases there is just not enough time for training.
All things considered, any learning experience is only as good as the objectives and goals that are being set for the student and how well they are being tracked. The MScore system allows tailored pass marks, proficiency levels and curricula to be set for the students based on their learning objectives. A multitude of metrics and data can be reviewed to allow a student to learn from their mistakes and improve their psychomotor skills.
So when looking for a simulator, make sure to find one that is validated, has high fidelity controllers, can be accessed 24/7 outside the OR, and has a flexible management and scoring system that can be tailored to meet your learning objectives. In the Tanaka study that was referred to in Roger Smith’s presentation to the FDA meeting, an observation was made that while the majority of study participants preferred the usability of the da Vinci® Skills Simulator, 70% felt the dV-Trainer was the best value for money spent when taking all things into consideration.
Growing in popularity, robotic surgery is still not without challenges. Before the benefits of robotic surgery can be fully realized, the highest level of patient safety must be ensured, while remaining cost effective and at the same time allowing new surgeons the ability to be trained and access the technology without impacting safety and cost-effectiveness.
The Halstedian Method of “see one, do one, teach one” is clearly no longer sufficient for surgical training. Many comparisons between the training of pilots and the training of surgeons have been made over the years. In 2013, the FAA updated their rules to state that to be qualified as a First Officer, a pilot needed 1,500 hours total time. This includes both real and approved simulation time. Looking at a typical Resident training program, the calculations for a general residency that will last 4 years is approximately 16,600 hours. If within this a surgical trainee chose to focus on Gynecology for 20 months they would receive 6,400 specialty hours. If a surgeon focused on minimally invasive surgery, such as robotics, the Accreditation Council for Graduate Medical Education (ACGME) guidelines recommend 105 hours exposure to a variety of cases. Even tripling this minimum, a surgeon would be only at 300 hours of surgery, which is only a fraction of the 1,500 hours the FAA recommended training time for pilots.
Just as in aviation, simulation has been seen to be a solution allowing surgeons to develop their skills without impacting patient safety. Mimic’s MSim software, found on both the dV-Trainer and the da Vinci Skills Simulator, has been one of the most researched and validated simulation software in the surgical field. Table 1 below shows the range of validation studies that have been carried out on either platform as well as other simulators.
The studies, and in a particular the predictive validity study looking at simulation and operative outcomes, carried out by Dr. Culligan, have helped shape recommendations for surgical training being developed by medical societies such as those developed by the American Association of Gynecologic Laparoscopists (AAGL) in 2014.
All of these studies and simulation programs focused on the psychomotor component of learning how to “drive the robot” and not necessarily the cognitive training requirements that would help train the next generation of surgeons. Augmented reality was developed under the Maestro AR name to help solve this issue. It is best to think of Maestro AR as a curriculum incorporating both psychomotor tasks and cognitive questions supported by a moderated guide on a procedural approach and technique.
The basic premise is that a student will learn more if the psychomotor skills that they require are placed within their procedural context as opposed to in a vacuum. As students are learning how to use the robotic device they are also being tested on tissue recognition, procedural choreography, as well as learning from the narration about the decision making process behind this specific approach.
The Benign Hysterectomy Maestro AR module, for example, is divided into 9 modules starting with a Pelvic Anatomy survey and working through clear steps on how to deal with the ligaments and uterine vasculature before finishing with the Colpotomy and the Vaginal Cuff Closure.
“Maestro AR addresses the next frontier of training by developing a pathway that incorporates Didactics with Augmented Reality through virtual reality simulation,” says Mireille Truong, Virginia Commonwealth University Medical Center, “I am confident that research will show that adding didactic elements to simulation training will continue to improve surgeon performance when they enter the OR.”
As with the airline industry, simulation is becoming a vital part of the armarmentarium required for surgeons to ensure that through all stages of their career, they have the correct level of skills for the task ahead of them. Just as pilot is able to land a plane in virtually any airport around the world on their simulator, it is hoped that surgeons will also be able to develop their skills within a procedural scenario in an augmented reality environment.