There are many aspects to a training simulator that can be considered when making the initial investment in simulation training. For robotic surgery, we believe the top factors to consider are:
- Validation studies conducted on and using the simulator
- Fidelity of the controllers
- Accessibility of the simulator
- Data, data, data!
Since Mimic launched its first version of the dV-Trainer in 2007, there has been a growing number of new robotic surgery simulators entering the market. The real impetus for simulation training was made clear in 2010 when Intuitive Surgical decided to launch their own Skills Simulator, a backpack-like addition for the da Vinci® Si platform.
Intuitive Surgical chose to license 27 exercises that Mimic had already developed or were in the process of developing especially for ISI. This was made possible by the new design of the system, which allowed for the console to operate independently of the patient side cart and core. Since 2010, both the ROSS Simulator from Simulated Surgical Systems and the Robotix Mentor from Simbionix (now 3D Systems) have entered the playing field.
The installed base of da Vinci® surgical systems is now over 3,500 systems around the world and close to 2,000 simulators have been installed and used to support this installed base. The majority of training simulators are da Vinci® Skill simulators (with Mimic’s licensed software) and close to 12% of robotic surgery simulators are Mimic’s dV-Trainers.
Our estimate is that over 70% of institutions performing robotic surgery have access to a simulator of some form or another and that close to 90% of robotic surgeons will at some point have tried a simulator. In fact, since 2007 we believe that between the dV-Trainer and the da Vinci® Skills Simulator over 6.25 million exercise sessions have been completed.
So has all of this simulation training activity been valuable you may ask? One way to look assess simulation training is through validation studies. There are currently five different ways of determining validity. Starting with the basics Face, Content, and Construct and moving to more valuable validation such as Concurrent and Predictive, the definitions are:
Face validity: Does the simulator have a realistic look and feel, compared to the actual surgical system?
Content validity: Is the simulator useful as a training tool for the surgical system?
Construct validity: Does the simulator have the ability to distinguish between Novice and Expert users?
Concurrent validity: How does the simulator compare to a similar or related construct (Dry Labs, Tissue Lab, etc.) carried out on the real robotic surgical system?
Predictive: validity: Can the simulator be used to predict actual performance in the O.R.?
Face and Content are of relatively low value as they are subjective and the most highly valued validation studies are Construct and Predictive validity. The table below shows the number of papers that have been published on various types of validation. As you can see there have been over 30 papers published on Mimic software either on the dV-Trainer or the da Vinci® Skills Simulator platform.
Recently, simulation was a large part of the discussion at the FDA town hall meeting in Washington. Roger Smith from Florida Hospital presented a comparison of the different simulators led by himself (the table above is adapted from his presentation). The data presented was clear that the most focus in researching the simulators was on the controllers and how close they emulated the real robotic surgeon’s console. Obviously, the da Vinci® Skills Simulator, which uses the real console is the real thing. However for the other simulators, this is where concurrent validity because extremely important, as essentially you are replicating (using the simulator) the same activity a surgeon would be doing on the real robotic surgical system.
A direct head to head study was done by Prof. Jacques Hubert and his team at the STAN Institute in Nancy, France between Mimic’s dV-Trainer and the da Vinci® Skills Simulator. During the study, participants completed the same exercises on the both systems and researchers found that on average there was only a 3% difference in overall score between the two systems. (89.9% vs 86.8%). This varied by the type of exercise but remained consistent with some internal bench-marking carried out by Mimic. No studies have been done to the same extent on the Ross and Robotix Mentor systems.
Another component to take in consideration when choosing a robotic surgery simulator is the accessibility to the system. While the great thing about the da Vinci® Skills Simulator is that it uses the real console, this can also be very detrimental and a negative for the da Vinci® Skills Simulator that it uses the real console. Very few hospitals can afford to have a dedicated console outside the OR that is used purely for training and simulation. If an institution is lucky enough to have a dual console system they will have the simulator on the second console but that is still kept in the OR. The value of the second console is in allowing programs with residents to keep training new surgeons without interrupting the flow and efficiency of the OR. Data shows that simulation systems in the OR are used less than systems outside the OR. This is due to the simple fact that as robotic programs become more successful and utilization increases there is just not enough time for training.
All things considered, any learning experience is only as good as the objectives and goals that are being set for the student and how well they are being tracked. The MScore system allows tailored pass marks, proficiency levels and curricula to be set for the students based on their learning objectives. A multitude of metrics and data can be reviewed to allow a student to learn from their mistakes and improve their psychomotor skills.
So when looking for a simulator, make sure to find one that is validated, has high fidelity controllers, can be accessed 24/7 outside the OR, and has a flexible management and scoring system that can be tailored to meet your learning objectives. In the Tanaka study that was referred to in Roger Smith’s presentation to the FDA meeting, an observation was made that while the majority of study participants preferred the usability of the da Vinci® Skills Simulator, 70% felt the dV-Trainer was the best value for money spent when taking all things into consideration.
Growing in popularity, robotic surgery is still not without challenges. Before the benefits of robotic surgery can be fully realized, the highest level of patient safety must be ensured, while remaining cost effective and at the same time allowing new surgeons the ability to be trained and access the technology without impacting safety and cost-effectiveness.
The Halstedian Method of “see one, do one, teach one” is clearly no longer sufficient for surgical training. Many comparisons between the training of pilots and the training of surgeons have been made over the years. In 2013, the FAA updated their rules to state that to be qualified as a First Officer, a pilot needed 1,500 hours total time. This includes both real and approved simulation time. Looking at a typical Resident training program, the calculations for a general residency that will last 4 years is approximately 16,600 hours. If within this a surgical trainee chose to focus on Gynecology for 20 months they would receive 6,400 specialty hours. If a surgeon focused on minimally invasive surgery, such as robotics, the Accreditation Council for Graduate Medical Education (ACGME) guidelines recommend 105 hours exposure to a variety of cases. Even tripling this minimum, a surgeon would be only at 300 hours of surgery, which is only a fraction of the 1,500 hours the FAA recommended training time for pilots.
Just as in aviation, simulation has been seen to be a solution allowing surgeons to develop their skills without impacting patient safety. Mimic’s MSim software, found on both the dV-Trainer and the da Vinci Skills Simulator, has been one of the most researched and validated simulation software in the surgical field. Table 1 below shows the range of validation studies that have been carried out on either platform as well as other simulators.
The studies, and in a particular the predictive validity study looking at simulation and operative outcomes, carried out by Dr. Culligan, have helped shape recommendations for surgical training being developed by medical societies such as those developed by the American Association of Gynecologic Laparoscopists (AAGL) in 2014.
All of these studies and simulation programs focused on the psychomotor component of learning how to “drive the robot” and not necessarily the cognitive training requirements that would help train the next generation of surgeons. Augmented reality was developed under the Maestro AR name to help solve this issue. It is best to think of Maestro AR as a curriculum incorporating both psychomotor tasks and cognitive questions supported by a moderated guide on a procedural approach and technique.
The basic premise is that a student will learn more if the psychomotor skills that they require are placed within their procedural context as opposed to in a vacuum. As students are learning how to use the robotic device they are also being tested on tissue recognition, procedural choreography, as well as learning from the narration about the decision making process behind this specific approach.
The Benign Hysterectomy Maestro AR module, for example, is divided into 9 modules starting with a Pelvic Anatomy survey and working through clear steps on how to deal with the ligaments and uterine vasculature before finishing with the Colpotomy and the Vaginal Cuff Closure.
“Maestro AR addresses the next frontier of training by developing a pathway that incorporates Didactics with Augmented Reality through virtual reality simulation,” says Mireille Truong, Virginia Commonwealth University Medical Center, “I am confident that research will show that adding didactic elements to simulation training will continue to improve surgeon performance when they enter the OR.”
As with the airline industry, simulation is becoming a vital part of the armarmentarium required for surgeons to ensure that through all stages of their career, they have the correct level of skills for the task ahead of them. Just as pilot is able to land a plane in virtually any airport around the world on their simulator, it is hoped that surgeons will also be able to develop their skills within a procedural scenario in an augmented reality environment.
by: Christopher Simmonds
Data, data, data. That is all we seem to hear about today in healthcare. One of the consequences of the Affordable Care Act has been to ensure that hospitals, physicians, surgeons and nurses are becoming obsessed with data and information to an extent like never before. Looking at information across large data pools, trends can be identified and behaviors that drive the trends can be discovered and, if needed, modified, including robotic surgery, which is one of the areas where there is a lot of analysis occurring.
Robotic surgery is truly a misnomer, as in reality it is a computer-assisted surgery where the computer has been placed between the surgeon and the patient, enhancing the surgeon’s capabilities as compared to other surgical techniques. If the robot was compared to a super hero, its role would be to turn the surgeon into Iron Man whose every day actions are enhanced by the power of computing.
The fact that there is a computer between the surgeon and the patient means that a lot of data can be captured. At their town hall meeting in July 2015, this was specifically noted by the FDA. In addition, a main focus of that meeting was training and simulation which also is computer-based and captures a lot of information, including a surgeon’s actions which can then be translated into a scoring system. So what can these scoring systems for robotic surgery training tell us?
If you study surgeons long enough you can identify that some surgeons will be very precise in their motions and other less so. When training new surgeons there are also certain good habits you would like them to develop such as keeping their instruments in view at all times and making sure they do not use too much force or drop things. For these reasons the MScore system, which underpins all the scoring on the dv-Trainer, looks at efficiency and good habit metrics when calculating overall scores.
When Mimic initially developed the MScore system it was calculated as a percentage-based scoring system. The scores were based on the weighted average of all individual metrics as compared to an expert base line. While this provided a simple and easy way to display the score it may not have been the best in helping an individual focus on specific areas of improvement. A high percentage in one area could compensate for a low percentage in another area while still producing an acceptable overall percentage. Mimic refers to this as the classic scoring system.
After being challenged by educators, Mimic decided to take inspiration from FLS and develop what it now refers to as its proficiency-based scoring system.
Like the classic scoring system the revised MScore system is based on expert user benchmarks, however, proficiency is measured as being within one standard deviation of the mean score of those experts. As an example, if five surgeons’ results have been pooled to produce the benchmark you have to perform better than at least one of these surgeons in order for you to pass. Instead of the overall result being a combination of the scores you have to become proficient at each individual metric before you can pass. The example below shows an individual who has passed on all other areas but failed in the area of blood loss. The number shown is a weighted addition of all the metrics together. The user would have likely passed in a percentage-based system as their superior scores in all the other metrics would have been compensated for their lower score in blood loss.
The other difference between the classic scoring system and the proficiency-based scoring system is that you can set proficiency thresholds. In FLS for example, for students to pass they need to complete the same exercise twice consecutively and ten times non-consecutively. The same principal has been introduced into MScore and defaults to two consecutive and five non-consecutive passes, though this can be modified by the end user.
Mimic realized early on that they did not have all the answers and therefore ensured that the scoring system was developed with an open architecture approach. Expert level benchmarks can be input from peer reviewed literature as well as from scores posted by surgeons within specific institutions. Weighting and proficiency levels can modified to meet specific needs. However curriculum and benchmarks such as the Morristown protocol are often used and have been implemented across many systems.
Overall, both the classic scoring system and the proficiency scoring system are helping surgeons improve their performance which is a good thing, noting that it will probably take someone longer to pass a proficiency-based curriculum than a percentage based one. In some instances this data is being used a part of annual certification programs but that will be the subject of another blog post, another day.
New Maestro AR™ 3D augmented reality brings procedure-specific content to robotic surgery simulation training for General Surgery
Mimic Technologies, Inc, announces the launch of the Maestro AR Inguinal Hernia Repair, a new augmented reality software module exclusively available on their dV-Trainer® robotic surgery simulator. Maestro AR is the first robotic surgery simulation technology that allows trainees to manipulate 3D virtual robotic instruments as a way to interact with 3D endoscopic video footage of an actual surgical case.
This full procedure simulation was developed in collaboration with Dr. Rick Low, M.D., Chairman of Surgery at John C. Lincoln Hospital, Phoenix, AZ, Chairman of the Robotic Program at John C. Lincoln Healthcare Network, and Medical Director for CAVA Robotics. Using 3D augmented video, Dr. Low guides trainees through each step of a robot assisted laparoscopic Inguinal Hernia Repair, from port placement and robot setup considerations to the final steps of the surgery. “With this Maestro AR procedure-specific training module, we present a reproducible, stepwise approach to robotic Inguinal Hernia Repair that we believe provides an excellent mechanism for the developing robotic surgeon to overcome the learning curve and mature into an expert robotic surgeon”, said Dr. Low.
At each critical juncture of the procedure, trainees experience both cognitive learning and robotic surgery skills development by identifying critical anatomical structures and surgical landmarks, simulating tissue retractions, predicting dissection planes, answering multiple choice questions, and completing virtual reality skills exercises. Embedded virtual reality tasks emphasize hand-eye motor skills critical to proficient surgical technique, including needle handling and driving, knot-tying, and closure of a peritoneal defect. For each step in the procedure, comprehensive metrics are gathered and reported, allowing trainees to objectively track their progress at learning the procedure and becoming proficient with required robotic surgery skills.
Maestro AR for Inguinal Hernia Repair divides the complete procedure into the following steps:
1. Patient Positioning and Setup
2. Exposure of Pre-peritoneal Space
3. Reduction of Hernia Sac
4. Positioning and Suturing of the Mesh
5. Closing the Peritoneum
“By augmenting real surgical video with interactive virtual content, we are able to deliver realism on a whole new level,” said Jeff Berkley, PhD, CEO of Mimic Technologies, Inc. “Our process for generating augmented reality is also extremely efficient and we expect to generate a large volume of content over the next few years that will allow trainees to walk through a wide variety of surgical scenarios as presented by world leading educators. We feel this will expose surgeons to a tremendous variety of surgical scenarios that would not normally be encountered under a normal case load.”
Maestro AR is available exclusively on the Mimic dV-Trainer. In addition to Inguinal Hernia Repair, modules for Hysterectomy (lead by Dr. Arnold Advincula of Columbia University) and Partial Nephrectomy, (lead by Dr. Inderbir Gill of USC) are also available. Prostatectomy and Lower Colon Resection will be added to the package within the next half year.
Maestro AR for Inguinal Hernia Repair will be demonstrated at the Mimic Technologies booth at the Clinical Robotic Surgery Association (CRSA) in Chicago, IL, on October 2-3, 2015 and during the American College of Surgeons Clinical Congress (ACS), Mimic Booth #756, Chicago, IL, on October 5-7, 2015. In addition, Dr. Low will be speaking about Maestro AR at CRSA on Saturday, October 3, 2:05pm in a talk titled, “How to optimize costs and time in ventral hernia repair”.
For more information: www.MimicSimulation.com/IHR
Don’t wait to start thinking about Fall training courses, register today!
Mimic will be supporting the following Fall hands-on robotic surgery simulation training courses:
- SLS Master Class #1 – Wednesday, September 2nd, New York, NY
- ERUS HOT Courses – Tuesday, September 15th, Bilbao, Spain
- AAGL Postgraduate Course Robo-604 – Sunday, November 15th, Las Vegas, NV
Can’t make it to a conference? You can earn CME credits through MimicMED at the Florida Hospital Nicholson Center. Click here to learn more about full day training courses and personalized one-on-one training.
To learn more about simulation training, please contact: training@MimicSimulation.com