by: Christopher Simmonds
Data, data, data. That is all we seem to hear about today in healthcare. One of the consequences of the Affordable Care Act has been to ensure that hospitals, physicians, surgeons and nurses are becoming obsessed with data and information to an extent like never before. Looking at information across large data pools, trends can be identified and behaviors that drive the trends can be discovered and, if needed, modified, including robotic surgery, which is one of the areas where there is a lot of analysis occurring.
Robotic surgery is truly a misnomer, as in reality it is a computer-assisted surgery where the computer has been placed between the surgeon and the patient, enhancing the surgeon’s capabilities as compared to other surgical techniques. If the robot was compared to a super hero, its role would be to turn the surgeon into Iron Man whose every day actions are enhanced by the power of computing.
The fact that there is a computer between the surgeon and the patient means that a lot of data can be captured. At their town hall meeting in July 2015, this was specifically noted by the FDA. In addition, a main focus of that meeting was training and simulation which also is computer-based and captures a lot of information, including a surgeon’s actions which can then be translated into a scoring system. So what can these scoring systems for robotic surgery training tell us?
If you study surgeons long enough you can identify that some surgeons will be very precise in their motions and other less so. When training new surgeons there are also certain good habits you would like them to develop such as keeping their instruments in view at all times and making sure they do not use too much force or drop things. For these reasons the MScore system, which underpins all the scoring on the dv-Trainer, looks at efficiency and good habit metrics when calculating overall scores.
When Mimic initially developed the MScore system it was calculated as a percentage-based scoring system. The scores were based on the weighted average of all individual metrics as compared to an expert base line. While this provided a simple and easy way to display the score it may not have been the best in helping an individual focus on specific areas of improvement. A high percentage in one area could compensate for a low percentage in another area while still producing an acceptable overall percentage. Mimic refers to this as the classic scoring system.
After being challenged by educators, Mimic decided to take inspiration from FLS and develop what it now refers to as its proficiency-based scoring system.
Like the classic scoring system the revised MScore system is based on expert user benchmarks, however, proficiency is measured as being within one standard deviation of the mean score of those experts. As an example, if five surgeons’ results have been pooled to produce the benchmark you have to perform better than at least one of these surgeons in order for you to pass. Instead of the overall result being a combination of the scores you have to become proficient at each individual metric before you can pass. The example below shows an individual who has passed on all other areas but failed in the area of blood loss. The number shown is a weighted addition of all the metrics together. The user would have likely passed in a percentage-based system as their superior scores in all the other metrics would have been compensated for their lower score in blood loss.
The other difference between the classic scoring system and the proficiency-based scoring system is that you can set proficiency thresholds. In FLS for example, for students to pass they need to complete the same exercise twice consecutively and ten times non-consecutively. The same principal has been introduced into MScore and defaults to two consecutive and five non-consecutive passes, though this can be modified by the end user.
Mimic realized early on that they did not have all the answers and therefore ensured that the scoring system was developed with an open architecture approach. Expert level benchmarks can be input from peer reviewed literature as well as from scores posted by surgeons within specific institutions. Weighting and proficiency levels can modified to meet specific needs. However curriculum and benchmarks such as the Morristown protocol are often used and have been implemented across many systems.
Overall, both the classic scoring system and the proficiency scoring system are helping surgeons improve their performance which is a good thing, noting that it will probably take someone longer to pass a proficiency-based curriculum than a percentage based one. In some instances this data is being used a part of annual certification programs but that will be the subject of another blog post, another day.
Recently, training surgeons and surgical teams has been one of the focus areas in litigation for robotic surgery. Test cases have stated that the responsibility for training, credentialing, and privileging surgeons to use new technologies belongs to physicians and the hospitals where they work. (Taylor v. Intuitive Surgical, WA State 2015).
Dr. John Lenihan, Medical Director of Robotics & MIS, MultiCare Health Systems, Tacoma, WA, is also a trained Air Force pilot and flight surgeon.
For a long time he has seen the similarities between flying and using a robotic surgical system. Both involve using systems that can have impact on human life, both have a learning curve that varies with the user, both require that the user makes sure their skills are being maintained before they use the system, and both can be validated on an annual basis.
As a Gynecologist, Dr. Lenihan also worries that given the national trend moving away from Hysterectomies toward other treatment modalities, that surgeons may not have as many opportunities as they had previously to keep their skill levels up. (Wright JD et al. Nationwide Trends in Inpat Hysterectomy in the US. Obstet Gynecol. Aug 2013 122(2), Part 1. 233-41).
A paper by Wallenstein MR, et al., titled “On Effects of Surgical Volume on Outcomes for Laparoscopic Hysterectomy for Benign Conditions,” highlighted this fact. It shows that the overall volume of laparoscopic hysterectomies has decreased by some 30% over the period 2000 to 2010, however, the percentage of cases being carried out by low volume surgeons (less than 6 laparoscopic hysterectomies a year) has increased.
Effects of Surgical Volume on Outcomes for Laparoscopic Hysterectomy for Benign Conditions
As a member of the AAGL (formerly the American Association of Gynecologic Laparoscopists) Special Interest Group on robotics, Dr. Lenihan produced with his colleagues guidelines for privileging for robotic-assisted Gynecological Laparoscopy. These were approved by the AAGL Board as an official recommendation by the Society in February of 2014. (J Minim Invasiv Gynecol, 21(2), Mar-Apr 2014).
It follows the same model and parameters you would expect of a pilot and is divided into three groups:
- Initial Credentialing:
- Establish standards to comply with learning curves
- Basic cases first with qualified assistants
- Then progress to more complicated cases
- Maintenance “Currency”:
- Establish minimum guidelines for numbers of cases to maintain surgical skills
- Encourage the use of simulators to maintain skills
- If minimums not met, require re-training and re-certification (proctoring)
- Competency Certification:
- Establish standards, utilize simulation, consider “check rides” and/or do case review
Dr. Lenihan is a firm believer in establishing proficiency through simulation. “The key to developing proficiency is practice, practice, practice,” Lenihan says, “you have to do each exercise until you have passed it at least five times with at least two consecutive passes.” All this within a structured approach with defined curriculum for basic and advanced users as well as specific curricula developed for annual credentialing.
Lenihan has taken this philosophy and applied to his own institution at MultiCare Tacoma. Between 2008 & 2009, competency was initially evaluated on outcomes. Surgeons were not expected to deviate more than 2 standard deviations away from hospital or national norms in key metrics such as operative times, blood loss, complications, etc. From 2009 to the present day, surgeons are expected to demonstrate proficiency annually using simulators. If there are poor outcomes, these are reviewed by the robotic committee and corrective actions, including simulation training, will be put in place.
More recently, Dr. Lenihan has been investigating the possibility of initiating a continuous improvement program through the use of the C-SATS platform to offer performance evaluation through crowd sourcing. Videos can be reviewed and technical weakness using a GEARS score can be calculated looking for such things as depth perception, bi-manual dexterity, efficiency, force sensitivity, and overall robotic control. When weaknesses are assessed then surgeon can be given specific curricula on their Mimic dV-Trainer that will allow them to improve and practice in those key areas. The improvement can be validate again via crowd sourcing before a surgeon operates again on a patient.
“The fundamentals are simple,” says Dr Lenihan, “you establish a credentialing and privileging system that relies in simulation. You use simulation to improve and maintain skills during period of inactivity. You can then use video review to help document proficiency and help low volume surgeons with tailored skill improvement curricula.”
A Success Story from Nancy, France
The University of Nancy and the STAN Institute in Nancy, France, has been offering a variety of courses on robotic surgery since 2008. Under the watchful eye of Prof Jacques Hubert, the center has developed into one of the leading centers of robotic training in Europe if not the world.
Their focus has always been on ensuring enough time is spent on developing adequate psychomotor skills so that once a surgeon is at the console all they need to do is to focus on the procedure. The course is now a five day course and progresses through time spent on a micro surgery workstation, to a specified curriculum using a dV-Trainer by Mimic and eventually on to dry and wet labs with da Vinci system. The recent courses in December have attracted participants from as far away as China.
In robotic procedures, the console surgeon is un-scrubbed without direct access to the patient. They can only interact with the group through audio communication, whereas in conventional surgery, more than 80% of the information exchange is realized via visual contact. The success of robotic surgery thus relies on high-quality teamwork, wherein the bedside assistant plays an important role. Some surgeons are able to develop a close working relationship with the same first assistant but in many institutions the first assistant will be constantly changing with the rotation of the OR staff. This is an area that Dr Randy Fagin from the Texas Robotic institute has often talked about and how different surgical teams can have an impact on the efficiency of the OR by extending procedural times.
Towards the middle of 2014, Mimic launched its Xperience Team Trainer (XTT), a Laparoscopic trainer that can be attached to the da Vinci console emulator (dV-Trainer) and allows a console surgeon and first assistant to work together on the same simulated exercises. Not only can they work together but their performance can also be scored on an individual basis as well as a team basis.
Professor Hubert, having a strong research focus, first wanted to ensure that the device had training validity. He therefore carried out face, content, and concurrent validity testing to ensure that activities performed on the team trainer were equivalent to the similar tasks encountered in the real world. His research published online in Surgical Endoscopy shows this to be the case1. Hubert believed more research was needed to prove concurrent validity and felt that some improvement could be made in exercises and through the addition of haptics to the device. Sufficient changes were made to the system through the addition of extra exercises and the activation of the built in haptics for the STAN team to feel confident in including the team trainer as part of their December series of courses in 2015.
They decided to introduce a team training component to the course where the surgeons attending the course would each spend time as a console surgeon and also as a first assist. This is a realistic scenario as surgeons can frequently find themselves in a first assist role particularly when going through their residency training.
Overall Prof. Hubert and the STAN Institute felt that the Xperience Team Trainer was a valuable addition to their December course line up and felt that it highlighted the importance of vocal communication and team work between the participants. They also noticed that in some groups the console operator would give advice on focal depth while the student driving the laparoscopic device would provide force feedback on the patient side-assist activities.
“We have been very pleased with the inclusion of the Xperience Team Trainer into our course,” says Alexandre Thouroude, General Manager of the STAN Institute, “it has been very good at allowing users to develop an awareness of the importance of developing non–technical verbal skills and highlighting the importance of developing team and communication strategies.”
Mimic Technologies greatly values our partnerships with the STAN Institute and Prof. Hubert, as their input into Xperience Team Trainer has been of great benefit in improving the Xperience Team Trainer product further. We thank them for the continued support and partnership.
1 Face, content, construct, and concurrent validity of a novel robotic surgery patient-side simulator: the XperienceTM Team Trainer
Song Xu, Manuela Perez, Cyril Perrenot, Nicolas Hubert, Jacques Hubert
Surg Endosc. 2015 Dec 10. [Epub ahead of print]
The Experience at Boston Children’s Hospital
As a teaching hospital, Boston Children’s Department of Pediatric Urology has a strong commitment to training both residents and fellows with a heavy emphasis on minimally invasive surgery (MIS). About 60% of its MIS cases are done robotically, mainly Pyeloplastys and Partial Nephrectomies, and usually around 16 residents and 2 fellows are trained each year.
When exploring robotic surgery training options, the staff at Boston Children’s decided to invest in the dV-Trainer because it would be available 24 hours a day and could be located close to the operating room, but not necessarily in it. They also felt there was a strong need to provide a quantitative record of each trainee’s performance, which the dV-Trainer’s MScore scoring system provides.
Next, when building out the program, their training objectives were simple. First, they wanted to teach residents the foundational skills for robotic surgery, including instrument & tissue handling. Second, they believed it would be valuable to provide them with a protected, structured time for simulation education and to also provide supervised guidance during introductory training course. Finally, they wanted to offer comprehensive, individualized feedback using the standardized scoring system.
To achieve this, a structured curriculum was developed that included the following elements:
1) Didactic components that include, Intuitive online training, AUA online – Urologic Robotic Surgery Course, FLS Training, and Video Library
2) Trainees would also be offered hands-on experience – bedside & console on the da Vinci, and simulation training using the dV-Trainer as well as components of team training
Ashley Wietsma (BCH Research Fellow 2014) was able to work with Todd Larson, Vice President & Chief Clinical Officer for MimicMED on developing a specific simulation curriculum that would meet Boston Children’s needs.
The final curriculum was divided into 6 sections including an introduction, warm up, pre-test, core, post-test and games that included approximately 20 exercises.
Since the system was acquired, 50% of the usage has been around the Boston Children’s curriculum. Twenty-five users have averaged over 80 sessions each with the power users doing significantly more than that.
“Providing training in a safe environment that helps improve handling and familiarity of the robotic console is critical,” says Richard Yu, Department of Urology, “skills such as camera control and clutching as well as developing muscle memory and improving hand-eye coordination while improving the non-dominant hand are all reasons why simulation is so important.”
For more information: www.MimicSimulation.com
New Maestro AR™ 3D augmented reality brings procedure-specific content to robotic surgery simulation training for General Surgery
Mimic Technologies, Inc, announces the launch of the Maestro AR Inguinal Hernia Repair, a new augmented reality software module exclusively available on their dV-Trainer® robotic surgery simulator. Maestro AR is the first robotic surgery simulation technology that allows trainees to manipulate 3D virtual robotic instruments as a way to interact with 3D endoscopic video footage of an actual surgical case.
This full procedure simulation was developed in collaboration with Dr. Rick Low, M.D., Chairman of Surgery at John C. Lincoln Hospital, Phoenix, AZ, Chairman of the Robotic Program at John C. Lincoln Healthcare Network, and Medical Director for CAVA Robotics. Using 3D augmented video, Dr. Low guides trainees through each step of a robot assisted laparoscopic Inguinal Hernia Repair, from port placement and robot setup considerations to the final steps of the surgery. “With this Maestro AR procedure-specific training module, we present a reproducible, stepwise approach to robotic Inguinal Hernia Repair that we believe provides an excellent mechanism for the developing robotic surgeon to overcome the learning curve and mature into an expert robotic surgeon”, said Dr. Low.
At each critical juncture of the procedure, trainees experience both cognitive learning and robotic surgery skills development by identifying critical anatomical structures and surgical landmarks, simulating tissue retractions, predicting dissection planes, answering multiple choice questions, and completing virtual reality skills exercises. Embedded virtual reality tasks emphasize hand-eye motor skills critical to proficient surgical technique, including needle handling and driving, knot-tying, and closure of a peritoneal defect. For each step in the procedure, comprehensive metrics are gathered and reported, allowing trainees to objectively track their progress at learning the procedure and becoming proficient with required robotic surgery skills.
Maestro AR for Inguinal Hernia Repair divides the complete procedure into the following steps:
1. Patient Positioning and Setup
2. Exposure of Pre-peritoneal Space
3. Reduction of Hernia Sac
4. Positioning and Suturing of the Mesh
5. Closing the Peritoneum
“By augmenting real surgical video with interactive virtual content, we are able to deliver realism on a whole new level,” said Jeff Berkley, PhD, CEO of Mimic Technologies, Inc. “Our process for generating augmented reality is also extremely efficient and we expect to generate a large volume of content over the next few years that will allow trainees to walk through a wide variety of surgical scenarios as presented by world leading educators. We feel this will expose surgeons to a tremendous variety of surgical scenarios that would not normally be encountered under a normal case load.”
Maestro AR is available exclusively on the Mimic dV-Trainer. In addition to Inguinal Hernia Repair, modules for Hysterectomy (lead by Dr. Arnold Advincula of Columbia University) and Partial Nephrectomy, (lead by Dr. Inderbir Gill of USC) are also available. Prostatectomy and Lower Colon Resection will be added to the package within the next half year.
Maestro AR for Inguinal Hernia Repair will be demonstrated at the Mimic Technologies booth at the Clinical Robotic Surgery Association (CRSA) in Chicago, IL, on October 2-3, 2015 and during the American College of Surgeons Clinical Congress (ACS), Mimic Booth #756, Chicago, IL, on October 5-7, 2015. In addition, Dr. Low will be speaking about Maestro AR at CRSA on Saturday, October 3, 2:05pm in a talk titled, “How to optimize costs and time in ventral hernia repair”.
For more information: www.MimicSimulation.com/IHR