Over the past few months we have talked frequently about the importance of acquiring proficiency in robotic surgery. Although this may have been evident to many, the term proficient has been based on the Dreyfus Model of Skill Acquisition. This model, developed in the 1980’s, focuses on how students acquire skills through formal instruction and practice. It was developed by the brothers, Stuart and Hubert Dreyfus, out of the University of California, Berkeley.
The diagram below is a simple representation of the Dreyfus model:
The critical factor is being able to determine where individuals sit on this progression and what are the real objective boundaries between the different skill levels.
The concept of proficiency was taken up by surgeons when beginning to learn laparoscopic surgery and incorporated into the Fundamentals of Laparoscopic Surgery (FLS) where a proficiency level of 2 consecutive and 10 non-consecutive passes have to be achieved before certification is reached.
The da Vinci® surgical system is currently the main player in the computer-assisted surgical market. Over 3M procedures have been done with the system, of which some 650,000 were in the last 12 months. What does it take to become proficient at using a da Vinci system?
First and foremost, it is important to realize that surgery is a team activity and that while the role of the surgeon is vital in the ensuring the best outcomes for the patient, the surgeon can only do so if supported by a highly skilled OR team.
The da Vinci System was designed around three key principles, 3D visualization, dexterity, and control. This essentially means that the system allows the surgeon to see and move within the operative field in a way that they were not able to beforehand while building in safe guards to ensure patients could not be inadvertently harmed.
To achieve this, the surgeon is placed on a console outside the operative field where they can control a combination of instruments using their hands and their feet. Instruments are changed by an assistant who will replace the required instrument at the correct time.
When training to use the da Vinci system, it is not only important to ensure that the surgeon is proficient at controlling the system itself but that the whole team is also comfortable with working in this new environment. What does take some time getting used to is the fact that the team leader is no longer at the bed side but is now sitting apart outside of the sterile field immersed in the console. Given the high quality of the image, they can easily become immersed in the operative field.
How do hospitals therefore ensure that their surgeons and teams are proficient and ready to perform robotic surgery? Also how do they make sure that they after their initial training they are staying up to speed with evolutions in the technology and gaining enough exposure to the system?
The table below shows a typical case distribution by surgeon over a three year period. The X axis shows the average number of cases and the Y axis represents each surgeon ranked from lowest to highest volume.
Many hospitals will typically use a minimum case volume per year to decide whether or not a surgeon should maintain privileges on a robotic system. However, looking at the distribution of cases, can the number of cases alone really be used to determine whether or not a surgeon and team are performing as advanced beginners, competent users, or are really proficient?
Thanks to its 10 years of experience in the training of surgeons and teams to run effective robotic surgical programs, Mimic has the experience to help hospitals with this issue. MimicMED’s Consultancy Services can help organizations set their thresholds to distinguish between the different skill levels, evaluate their surgical staff, and see where they currently sit on the continuum as well as develop and implement training programs to ensure overall objectives are met.
Like any new technology, a lot of focus has been placed on ensuring that new users of robotic surgery are adequately trained. Simulation has had a large part to play with this. As the technology has become more mainstream, training requirements have moved from not only training existing surgeons but to ensuring that residents and fellows develop the required skill levels to ensure that they can adapt to the new technologies used in their practice.
Earlier this year we discussed a paper published by the EAU on their curriculum aimed at ensuring that fellows followed a clear curriculum at the end of which they would be deemed to be safe and competent to operate on patients independently. As with many ways of teaching surgery, the procedure is broken into specific steps that the trainee must master before being allowed to carry the whole procedure.
A typical prostatectomy is divided into the 7 following steps: bladder takedown, endopelvic fascia, bladder neck, seminal vesicle/vas deferens, pedicle/nerve sparing, apex, and anastomosis. Typically a trainee will be given a maximum time, of say 30 minutes,to complete one of these tasks during a procedure. Once they have shown that they have mastered the tasks, they will be allowed to move onto another task and eventually to the whole procedure. This is obviously easier to accomplish on parts of the anatomy and procedures that can be standardized.
Until recently, there have not been many studies looking into this practice to see what the potential patient impact could be comparing when a surgery was performed by just the one attending surgeon to one where parts of the case had been handed over to the resident.
Dr. Thiel from the Mayo Clinic in Jacksonville, Florida, has published a paper on just this topic comparing 140 cases where just an attending was involved in the surgery to 232 cases when a resident took over part of the case.
There were no differences in some key clinical outcomes such as positive margins, length of stay, catheter days, readmissions or re-operations when comparing surgeon only to resident –involved cases. There was, however, a difference seen in mean operative time between procedures that were surgeon only cases vs. resident involved (190.4 Min vs. 206.4 Min, P= 0.003)
The researchers also noted that residents were more likely to be involved with at least 1 procedural step after the purchase of the dV-Trainer.
Mimic believes in this way of training residents which is why the Maestro AR set of procedural curricula we have developed are divided into the procedural steps that a resident will be required to learn. We have been able to marry narrated 3D video content with didactic exercises that allow for a student’s ability to be tested. At the appropriate point, the correct psychomotor skill is inserted to make sure that the student can match the skills required for the procedural step.
Mimic currently has the following available:
- Right Partial Nephrectomy, Dr. Indibir Gill, USC
- Hysterectomy, Dr. Arnold Advincula, Columbia University
- Inguinal Hernia Repair, Dr. Rick Low, John C. Lincoln Hospital
- Prostatectomy (Si), Dr. Henk van der Poel, Antoni van Leeuwenhoek Hospital/Netherlands Cancer Institute in Amsterdam
- Prostatectomy (Xi), Dr. Vip Patel, Florida Hospital
- In Development for Q4 ‘16 Release:
- Lower Anterior Resection, Dr. Eduardo Parra, Florida Hospital
It has now been 15 years since Mimic Technologies was launched as a company and 12 years of providing robotic surgery simulation training. Over the years, Mimic has been involved with one of the most successful surgical simulation launches of all time with over 2,000 simulators using MSim software worldwide. We estimate that over 30,000 surgeons / residents use this simulation install base each year and that since it was launched over 6.5 million exercise sessions have been completed.
Many leading academic centers around the world have incorporated Mimic hardware and software into their training programs and there have been numerous publications that were researched and written on Mimic’s hardware and software proving validity. (A sample of these published studies can be found here.)
All of this experience has allowed Mimic to collect simulation data as well as hands-on experience in successfully implementing best practices to help fully develop new and existing robotic surgery simulation training programs. Over the years, we have found that the most common traits of a successful simluation training program include:
- Individuals (trainees) are uniquely identified and results are recorded
Data is king! It is important for simulation users to create an account and always remember to sign in so that the record of performance over time can show a progression of skill development and maintenance.
- Proficiency levels have been discussed and agreed upon
The study, Best Practices for Robotic Surgery Training and Credentialing, published in 2011 in the Journal of Urology by Jason Lee, et al., concluded that “rather than being based on a set number of completed cases, robotic surgery credentialing should involve the demonstration of proficiency and safety in executing basic robotic skills and procedural tasks. In addition, the accreditation process should be iterative to ensure accountability to the patient.” Setting institutional standards that have been both discussed and agreed upon will ensure that all clinicians who will be training using simulation are meeting the same requirements. Objective scoring is also helpful to implement a fair and accurate training environment.
- Curriculum are developed, allocated appropriately, and continuously measured
According to a 2005 study, Virtual Reality Simulation for the Operating Room: Proficiency-Based Training as a Paradigm Shift in Surgical Skills Training, by Gallagher, et al., “virtual reality training is more likely to be successful if it is systematically integrated into a well-thought-out education and training program.” Defining specific exercises, mapping out a training path, and continuously checking progress is essential for making sure that trainees get the most out of simulation to build their skills and move up the learning curve towards proficiency.
- Simulation training platforms are easily accessible to trainees
Also essential for simulation training is ensuring that trainees are able to access the simulator at times that are most convenient and conducive to their learning preferences.
- Simulation time is transferable to the real tool
Face validation shows that a training tool has a realistic look and feel. The 2015 study published in Surgical Endoscopy, Robotic Surgery Simulation Validity and Usability Comparative Analysis, concluded that, “Usability can affect the consistency and commitment of users of robotic surgical simulators.” Before simulation training is implemented, the training tool should be carefully assessed to ensure the skills trainees acquire transfer to the tool they will be utilizing.
- Cognitive and psychomotor skills can be validated
In addition to validating the training tool for Face and Content, the acquisition of both cognitive and psychomotor skills should be validated and proven to make outcomes better. Construct validity distinguishes experienced medical professionals from the inexperienced, Concurrent validity measures the extent to which the simulator correlates with the “Gold standard”, and Predictive validity goes so far as to predict future performance. These types of validation are important to consider when choosing a simulation training tool.
- Teams can train together
The ability to incorporate team training within a simulation training curricula ensure that the trainees will have well-rounded skills such as communication and movement coordination in addition to being proficient in operating the tool they are training for. The study, Teaching Surgical Skills – Changes in the Wind, published in the New England Journal of Medicine by Dr. Richard Reznick, et al., stated, “Virtual reality has the potential to enhance surgical-team training as well as technical skills training. In aviation, teamwork training with simulation has been instrumental in reducing errors. The importance of teamwork in preventing medical error is well recognized, and simulator-based team training has been advocated as a possible preventive approach. Early research results have been promising.”
Advances in technology and virtual reality simulation training can make medical training safer, more cost-effective, and efficient and building a successful program doesn’t have to be difficult if similar principles and benchmarks are applied.
Surgery has often been referred to as a team sport. The role of the surgeon is undoubtedly critical but so is the role of the entire team in ensuring patient safety and positive outcomes.
This is especially true in robotic surgery where the main surgeon is no longer at the patient’s side but is seated at a robotic console outside the operative field. The console-side surgeon now relies heavily on a first assistant or patient-side surgeon. The first assistant is a skilled laparoscopic surgeon whose prime objective is to work as a team with the robotic surgeon to maximize efficiency during the procedure.
A recent study titled, “Impact of Assistant Surgeon on Outcomes in Robotic Surgery,” by Dr. Rishi Nayyar, et al., published in the Indian Journal of Urology found that, “with increasing experience of patient-side surgeon and associated console surgeon, who form a consistent surgical team, the mean operative time for all robotic procedures shows a consistent trend of reduction across all surgical types.” (Click Here to Access the Full Study)
Typically, better outcomes in robotic surgery are associated with only the console-surgeon. According to Dr. Nayyar’s study, there is no existing objective evidence regarding the impact on outcomes with the experience of the assistant surgeon in robot-assisted surgery. Therefore, the goal of this recent study was to objectively verify the hypothesis that the experience of patient-side assistant in robotic surgery affects intraoperative outcomes.
During the study, a total of 100 cases of robot-assisted laparoscopic pyeloplasty were analyzed and on comparing outcomes between the 1st and 2nd halves of the assistant experience, the mean operative time reduced from 102.50 min to 82.80 min (P = 0.001) and mean blood loss reduced from 72.00 ml to 63.90 ml (P = 0.91). (See table below)
As the study suggests, a console-surgeon and first assistant to who work consistently together make a better team and positively influence the outcome of the procedure. Mimic’s Xperience Team Trainer (XTT) was developed specifically for this reason.
The XTT simulates the patient-side and connects with the dV-Trainer that simulates the console side, thereby allowing both the console and the patient-side surgeons to train in tandem with virtual reality simulators. Routine tasks are executed crisply and efficiently while also working on communication between both surgeons.
A 2015 study done by Dr. Jacques Hubert, et al., published in Surgical Endoscopy confirmed face, content, construct, and concurrent validity of the Xperience Team Trainer as an assessment tool of robotic surgery bed-assistance skills for the patient-side surgeon.
This study also emphasized the importance of teamwork between the patient-side and console-side surgeon in robotic surgery, which may change the paradigm of robotic surgery training in the near future. To read a past post about this study, click here.
Nayyar R, Yadav S, Singh P, Dogra PN. Impact of assistant surgeon on outcomes in robotic surgery. Indian J Urol 2016;32:204-9
There are many aspects to a training simulator that can be considered when making the initial investment in simulation training. For robotic surgery, we believe the top factors to consider are:
- Validation studies conducted on and using the simulator
- Fidelity of the controllers
- Accessibility of the simulator
- Data, data, data!
Since Mimic launched its first version of the dV-Trainer in 2007, there has been a growing number of new robotic surgery simulators entering the market. The real impetus for simulation training was made clear in 2010 when Intuitive Surgical decided to launch their own Skills Simulator, a backpack-like addition for the da Vinci® Si platform.
Intuitive Surgical chose to license 27 exercises that Mimic had already developed or were in the process of developing especially for ISI. This was made possible by the new design of the system, which allowed for the console to operate independently of the patient side cart and core. Since 2010, both the ROSS Simulator from Simulated Surgical Systems and the Robotix Mentor from Simbionix (now 3D Systems) have entered the playing field.
The installed base of da Vinci® surgical systems is now over 3,500 systems around the world and close to 2,000 simulators have been installed and used to support this installed base. The majority of training simulators are da Vinci® Skill simulators (with Mimic’s licensed software) and close to 12% of robotic surgery simulators are Mimic’s dV-Trainers.
Our estimate is that over 70% of institutions performing robotic surgery have access to a simulator of some form or another and that close to 90% of robotic surgeons will at some point have tried a simulator. In fact, since 2007 we believe that between the dV-Trainer and the da Vinci® Skills Simulator over 6.25 million exercise sessions have been completed.
So has all of this simulation training activity been valuable you may ask? One way to look assess simulation training is through validation studies. There are currently five different ways of determining validity. Starting with the basics Face, Content, and Construct and moving to more valuable validation such as Concurrent and Predictive, the definitions are:
Face validity: Does the simulator have a realistic look and feel, compared to the actual surgical system?
Content validity: Is the simulator useful as a training tool for the surgical system?
Construct validity: Does the simulator have the ability to distinguish between Novice and Expert users?
Concurrent validity: How does the simulator compare to a similar or related construct (Dry Labs, Tissue Lab, etc.) carried out on the real robotic surgical system?
Predictive: validity: Can the simulator be used to predict actual performance in the O.R.?
Face and Content are of relatively low value as they are subjective and the most highly valued validation studies are Construct and Predictive validity. The table below shows the number of papers that have been published on various types of validation. As you can see there have been over 30 papers published on Mimic software either on the dV-Trainer or the da Vinci® Skills Simulator platform.
Recently, simulation was a large part of the discussion at the FDA town hall meeting in Washington. Roger Smith from Florida Hospital presented a comparison of the different simulators led by himself (the table above is adapted from his presentation). The data presented was clear that the most focus in researching the simulators was on the controllers and how close they emulated the real robotic surgeon’s console. Obviously, the da Vinci® Skills Simulator, which uses the real console is the real thing. However for the other simulators, this is where concurrent validity because extremely important, as essentially you are replicating (using the simulator) the same activity a surgeon would be doing on the real robotic surgical system.
A direct head to head study was done by Prof. Jacques Hubert and his team at the STAN Institute in Nancy, France between Mimic’s dV-Trainer and the da Vinci® Skills Simulator. During the study, participants completed the same exercises on the both systems and researchers found that on average there was only a 3% difference in overall score between the two systems. (89.9% vs 86.8%). This varied by the type of exercise but remained consistent with some internal bench-marking carried out by Mimic. No studies have been done to the same extent on the Ross and Robotix Mentor systems.
Another component to take in consideration when choosing a robotic surgery simulator is the accessibility to the system. While the great thing about the da Vinci® Skills Simulator is that it uses the real console, this can also be very detrimental and a negative for the da Vinci® Skills Simulator that it uses the real console. Very few hospitals can afford to have a dedicated console outside the OR that is used purely for training and simulation. If an institution is lucky enough to have a dual console system they will have the simulator on the second console but that is still kept in the OR. The value of the second console is in allowing programs with residents to keep training new surgeons without interrupting the flow and efficiency of the OR. Data shows that simulation systems in the OR are used less than systems outside the OR. This is due to the simple fact that as robotic programs become more successful and utilization increases there is just not enough time for training.
All things considered, any learning experience is only as good as the objectives and goals that are being set for the student and how well they are being tracked. The MScore system allows tailored pass marks, proficiency levels and curricula to be set for the students based on their learning objectives. A multitude of metrics and data can be reviewed to allow a student to learn from their mistakes and improve their psychomotor skills.
So when looking for a simulator, make sure to find one that is validated, has high fidelity controllers, can be accessed 24/7 outside the OR, and has a flexible management and scoring system that can be tailored to meet your learning objectives. In the Tanaka study that was referred to in Roger Smith’s presentation to the FDA meeting, an observation was made that while the majority of study participants preferred the usability of the da Vinci® Skills Simulator, 70% felt the dV-Trainer was the best value for money spent when taking all things into consideration.