By: Christopher Simmonds
One of the questions I have often pondered is why do patients choose certain procedures and certain surgeons. So I conducted some research which showed that there were three clear factors that influenced patient choice.
The first was the surgeon they were initially referred to. In the majority of cases, patients will do what their surgeons tells them is the best treatment modality for them. The second most important was family and friends. As soon as you have a diagnosis that will involve a surgical intervention people will talk to their extended network and listen to feedback on similar procedures or interventions. In today’s connected world it is very easy to connect with a friend of a friend who has faced the same medical challenge. The last major factor was the internet. While the majority of people will go on the web to browse health care information and to help reinforce decisions made with their healthcare providers, only a small percentage, around 10%, will change their physician and treatment choice based on the internet alone.
This is completely logical as any surgical intervention is really about trust that the surgeon and institution will carry out the procedure and that you will come out the other side as planned. Most people will choose to trust someone once they have met them or rely on the advice of family and friends and are less likely to trust what they have read on the web.
A few years ago I was able to publish a paper that looked at the aspect of patient satisfaction on patients’ willingness to recommend their treatment modality to others.
Hystersisters.com is an online community that was created in 1998 to act as a support group for women to women, dedicated to medical and emotional issues surrounding the hysterectomy experience and gynecologic-related conditions and illnesses, supporting women from diagnosis, to treatment, to recovery.
The research group sent out a questionnaire to the 300,000 members and received over 10,000 responses (however, only 6,263 met the inclusion criteria). We essentially were asking about their perception of their surgical experience and if they would recommend the procedure to someone else in the same position as themselves and, if faced with the same situation would go through the same procedure again.
The table below shows the answers across a variety of modalities and the % who would definitely recommend or choose the same procedure again.
As you can see, there was a significant difference between the types of surgical treatment offered and the willingness to recommend or have the surgery again.
What does this have to with Surgeon skill? I would hope this is obvious. Surgeons who have higher skills and have higher volumes will tend to have better outcomes. We have discussed this fact in earlier blogs.
Better outcomes lead to more satisfied patients who are more likely to recommend the surgeon and institution to family and friends. This will lead to more referrals which will continue as a virtuous circle. The reverse is also true when outcomes are not so positive they can have a negative impact on referrals.
At Mimic, we have always believed that helping surgeons master their tools through simulation is ultimately about helping patients have a better surgical experience and allowing them to get on with their lives as quickly as possible.
“The Impact of Different Surgical Modalities for Hysterectomy on Satisfaction and Patient Reported Outcomes” Published on 17/17/04 in Interactive Journal of Medical Research
Michael C Pitter1*, MD; Christopher Simmonds2*; Usha Seshadri-Kreaden3*, MS ; Helen Hubert4*, MPH, PhD
There are many aspects to a training simulator that can be considered when making the initial investment in simulation training. For robotic surgery, we believe the top factors to consider are:
- Validation studies conducted on and using the simulator
- Fidelity of the controllers
- Accessibility of the simulator
- Data, data, data!
Since Mimic launched its first version of the dV-Trainer in 2007, there has been a growing number of new robotic surgery simulators entering the market. The real impetus for simulation training was made clear in 2010 when Intuitive Surgical decided to launch their own Skills Simulator, a backpack-like addition for the da Vinci® Si platform.
Intuitive Surgical chose to license 27 exercises that Mimic had already developed or were in the process of developing especially for ISI. This was made possible by the new design of the system, which allowed for the console to operate independently of the patient side cart and core. Since 2010, both the ROSS Simulator from Simulated Surgical Systems and the Robotix Mentor from Simbionix (now 3D Systems) have entered the playing field.
The installed base of da Vinci® surgical systems is now over 3,500 systems around the world and close to 2,000 simulators have been installed and used to support this installed base. The majority of training simulators are da Vinci® Skill simulators (with Mimic’s licensed software) and close to 12% of robotic surgery simulators are Mimic’s dV-Trainers.
Our estimate is that over 70% of institutions performing robotic surgery have access to a simulator of some form or another and that close to 90% of robotic surgeons will at some point have tried a simulator. In fact, since 2007 we believe that between the dV-Trainer and the da Vinci® Skills Simulator over 6.25 million exercise sessions have been completed.
So has all of this simulation training activity been valuable you may ask? One way to look assess simulation training is through validation studies. There are currently five different ways of determining validity. Starting with the basics Face, Content, and Construct and moving to more valuable validation such as Concurrent and Predictive, the definitions are:
Face validity: Does the simulator have a realistic look and feel, compared to the actual surgical system?
Content validity: Is the simulator useful as a training tool for the surgical system?
Construct validity: Does the simulator have the ability to distinguish between Novice and Expert users?
Concurrent validity: How does the simulator compare to a similar or related construct (Dry Labs, Tissue Lab, etc.) carried out on the real robotic surgical system?
Predictive: validity: Can the simulator be used to predict actual performance in the O.R.?
Face and Content are of relatively low value as they are subjective and the most highly valued validation studies are Construct and Predictive validity. The table below shows the number of papers that have been published on various types of validation. As you can see there have been over 30 papers published on Mimic software either on the dV-Trainer or the da Vinci® Skills Simulator platform.
Recently, simulation was a large part of the discussion at the FDA town hall meeting in Washington. Roger Smith from Florida Hospital presented a comparison of the different simulators led by himself (the table above is adapted from his presentation). The data presented was clear that the most focus in researching the simulators was on the controllers and how close they emulated the real robotic surgeon’s console. Obviously, the da Vinci® Skills Simulator, which uses the real console is the real thing. However for the other simulators, this is where concurrent validity because extremely important, as essentially you are replicating (using the simulator) the same activity a surgeon would be doing on the real robotic surgical system.
A direct head to head study was done by Prof. Jacques Hubert and his team at the STAN Institute in Nancy, France between Mimic’s dV-Trainer and the da Vinci® Skills Simulator. During the study, participants completed the same exercises on the both systems and researchers found that on average there was only a 3% difference in overall score between the two systems. (89.9% vs 86.8%). This varied by the type of exercise but remained consistent with some internal bench-marking carried out by Mimic. No studies have been done to the same extent on the Ross and Robotix Mentor systems.
Another component to take in consideration when choosing a robotic surgery simulator is the accessibility to the system. While the great thing about the da Vinci® Skills Simulator is that it uses the real console, this can also be very detrimental and a negative for the da Vinci® Skills Simulator that it uses the real console. Very few hospitals can afford to have a dedicated console outside the OR that is used purely for training and simulation. If an institution is lucky enough to have a dual console system they will have the simulator on the second console but that is still kept in the OR. The value of the second console is in allowing programs with residents to keep training new surgeons without interrupting the flow and efficiency of the OR. Data shows that simulation systems in the OR are used less than systems outside the OR. This is due to the simple fact that as robotic programs become more successful and utilization increases there is just not enough time for training.
All things considered, any learning experience is only as good as the objectives and goals that are being set for the student and how well they are being tracked. The MScore system allows tailored pass marks, proficiency levels and curricula to be set for the students based on their learning objectives. A multitude of metrics and data can be reviewed to allow a student to learn from their mistakes and improve their psychomotor skills.
So when looking for a simulator, make sure to find one that is validated, has high fidelity controllers, can be accessed 24/7 outside the OR, and has a flexible management and scoring system that can be tailored to meet your learning objectives. In the Tanaka study that was referred to in Roger Smith’s presentation to the FDA meeting, an observation was made that while the majority of study participants preferred the usability of the da Vinci® Skills Simulator, 70% felt the dV-Trainer was the best value for money spent when taking all things into consideration.
by: Christopher Simmonds, VP Business Development & Marketing, Mimic Technologies
While Mimic has been actively focused on simulation for robotic surgery over the past 15 years, I thought it would be interesting to see how simulation was valued for medical training, in general. While trawling through the internet I came across a study published by the Association of American Medical Colleges (AAMC) in 2011. The survey was sponsored by a number of other societies including: IMSH, ASPE and AACN. While it is five years old, I do believe it probably still holds true.
The questionnaire was sent to 133 AAMC member medical schools and 263 teaching hospitals in January through March of 2010. It is interesting to note that the use of simulation increased over time with medical students in both medical school or a teaching hospital environment. While with residents the reverse pattern was seen to occur with more simulation taking place in the first years of residency than in the later years.
These observations reflect what we have seen in many of the teaching hospitals using Mimic’s dV-Trainer. Residents are asked to develop psychomotor skills on the simulator before being allowed to migrate to the OR. Many institutions set a specific curriculum with proficiency levels that must be attained before the resident can sit down on the real robotic surgery console and start performing only very specific steps of a procedure.
An interesting part of this AAMC survey looked at how simulation is being used for education and assessment as well as part of a quality improvement program. What sparked my interest was the fact that the researchers differentiated between a number of skills that are very important to Mimic, such as psychomotor skills in addition to clinical thinking/decision making, team training and interpersonal communication skills.
Teaching hospitals were asked to indicate how simulation is used across the three domains of education, assessment, and quality improvement or research. All 64 respondents answered this question. Similar to medical schools, overall responses demonstrate simulation is largely used for educational purposes at 87 percent average usage across all competencies, less so for assessment at 61 percent, and much less frequently for quality improvement and research at only 34 percent.
Teaching Hospital Use of Simulation by selected areas:
In online questionnaires carried out by Mimic Technologies, we were able to see that over 90% of robotic surgeons had used Mimic simulation products either on Mimic’s dV-Trainer or on the da Vinici® Skills Simulator. This simulation training was primarily for the development of psychomotor skills as part of the surgeons’ initial training on robotics. In our experience fewer hospitals are using simulation for assessment, though we do know of some residency programs who include simulation in their recruitment process. We are also aware of institutions that have implemented a short curriculum that all surgeons need to pass annually to prove that they have the maintained their skill level for the surgical robot.
When it comes to quality improvement the picture is less clear. Given surgeons’ the time constraints, very few hospitals have initiated QI programs that leverage simulation to help improve the skill sets of lower performing surgeons.
As mentioned previously, this paper is five years old and I am sure the situation has continued to evolve. The implementation of the affordable care act is shining a spot light on patient outcomes and thus indirectly on variations in surgical performance. We can see that many institutions are trying to see how they can help improve the outcomes of their lower performers and we believe simulation will have a key role to play.
The gold standard treatment option for men under 70 with early-stage, organ-confined cancer is surgical removal of the prostate using nerve-sparing radical prostatectomy. Since its introduction, surgical robotics has achieved widespread acceptance for performing radical prostatectomies in both the United States and Europe, and is increasing in adoption worldwide. In the U.S., robotic-assisted radical prostatectomy (RARP) is now the most common treatment for localized prostate cancer. Faster operating times, less blood loss, fewer complications, and shorter hospital stays are just a few of the reasons for its popularity among both patients and surgeons. In the coming years, it is anticipated that relative case volumes for the surgical robots will grow significantly as newer robotic systems are added or replaced in hospitals worldwide.
There is a debate going on currently about the importance of simulated procedural training and the best way to approach it. One approach is to have a complete virtual reality environment where students can “play “at learning the procedure and carry out any task they wish. A second way of learning, which Mimic believes works best is to learn from industry leading surgeons/proctors who can guide students through their own approaches and challenge the learner’s knowledge acquisition. This learning process is then augmented by specific virtual reality tasks that are key parts of the procedure to develop critical psychomotor skills.
There are currently no studies that have been carried out that have been able to validate either approach to procedural training. This is partly due to the recognition that the technology is still not fully advanced enough to simulate the complexity of human tissue and surgical interaction and include all the myriad of potential outcomes as surgeons learn through trial and error.
Clinical societies such as EAU have published guidelines for a structured training program and curriculum for teaching surgeons how to perform a RARP (see European Urology, August 2015, Volume 68, Issue 2, Pages 292–299). Although there is wide acceptance for key steps and strategies for the procedure, differences between cases (e.g. patient considerations, cancer location, desire for neurovascular bundle preservation, need for sentinel lymph node dissection, etc.), differences between robotic platforms (e.g. Xi vs Si), and surgeon preference or experience may warrant some variations in the surgical strategy. For this reason, Mimic has decided to simultaneously release two versions of their Maestro ARTM RARP training curricula. One has been developed based on the da Vinci® Si platform with Dr. Henk Van der Poel (filmed at the Netherlands Cancer Institute, Amsterdam, The Netherlands); the other is based on the da Vinci® Xi platform with Dr. Vip Patel (filmed at Florida Hospital, Orlando, FL, USA).
Both procedures offer a complete training solution for RARP, from initial patient and robot setup, to the final urethrovesical anastomosis. Each Maestro ARTM training curriculum integrates high definition 3D video footage from an actual RARP (narrated by the surgeon who performed the case) with augmented reality interactive tasks such as recognition of key anatomical structures, identification of surgical planes and landmarks, energy application, tissue retraction, and multiple choice questions. Additionally, each Maestro curriculum includes a set of virtual reality exercises selected by the surgeon and designed to teach specific robotic skills that are important at the various stages of the procedure.
The table below highlights the variation between the two curricula:
Although the major steps of the procedures are the same, the Si and Xi versions of the Mimic Maestro ARTM RARP curricula are differentiated at several key moments. These include: the location and technique for the initial peritoneal dissection and entry into the Space of Retzius, the timing and suturing technique used to ligate the Dorsal Venous Complex, the approach for the dissection of the posterior bladder neck and seminal vesicles, the strategy used during the preservation of the neurovascular bundle, and the type of suture and technique for the posterior reconstruction (Rocco stitch) and urethrovesical anastomosis.
Additionally, the Xi module highlights a new feature giving the surgeon the ability to rotate a 30 degree endoscope from a 30-down view to a 30-up view with the push of a button in order to gain greater visualization beneath the prostate during posterior dissection. Ultimately, the different styles, teaching preferences, and words of wisdom from our two surgeon collaborators offer a complete and well-rounded training pathway for any surgeon wanting to learn best practices for RARP on either robotic platform.
A Case Study of the BAUS Audit on Prostatectomy
Recently, there has been ongoing debate around the impact of case volumes on surgical outcomes. A previous blog post (The Cost Debate in Robotic Surgery and the Impact of Skills) discussed a 2013 study published in the New England Journal of Medicine by Dr. John Birkmeyer, et al, which looked at skill levels between surgeons and identified that surgeons in the lower quartile completed three times fewer operations compared to surgeons in the top quartile.
In December 2012, the UK Government outlined plans to publish surgeon-level outcomes data, taken from national clinical audits, in ten specialty areas, which included Urology. This is known as the Consultant Outcomes Publication (COP) programme.
The British Association of Urological Surgeons (BAUS) has since published a number of audits on surgical outcomes in areas such as Prostatectomy, Nephrectomy, Stress Urinary Incontinence, and Urethroplasty. These audits are available for the general public to review the volumes and outcomes of a wide variety of surgeons.
The 2015 Prostatectomy audit, which looked at 2014 cases was published in September 2015 and the results are summarized below (as published on the BAUS website).
- The data collection period was from January 1, 2014 to December 31, 2014
- 6,161 cases were submitted in total, of which 5,814 were from England; these 5,814 cases came from 147 consultants at 62 sites, and include 230 private patients from 37 consultants
- Hospital Episode Statistics (HES) for 2014 indicate that there were 6,651 radical prostatectomies undertaken in England, so data was collected from 87% of the radical prostatectomies undertaken in England in 2014
- 5% were robotic assisted, 26.7% laparoscopic, 13.4% open and in 1.4% of cases the technique was not recorded
- Median number of cases per consultant: 32 (range 1 – 157)
- Median number of cases per center: 85 (range 1 – 250)
- The overall transfusion rate was 7% – for England only, 2.6%. In England only, the transfusion rates by technique were: open 5.4%, laparoscopic 0.8% and robotic 2.9%.
- 5,174 of the entries recorded whether there had been adverse events. The total post-operative complication rate was 5% (491 / 5174). Of these 491 cases, 364 recorded the Clavien Dindo grade (i.e. 127 or 26% did not). Complications classified as Clavien Dindo Grade III or above were seen in 1.6% of cases.
“Another interesting point to note was that there were differences in surgical positive margin rate between the three approaches,“ says Mr. Ben Challacombe MS FRCS (Urol) Consultant Urological Surgeon & Honorary Senior Lecturer, Guy’s Hospital & King’s College London. “Robotic surgery had an average of 13% while both the open and Laparoscopic approaches were at 19%. The length of stay was also seen to be lower for the robotic approach at a median of one day post operative”
We decided to go into each of the individual surgeons recorded on the web site and try and see if we could give any further insight into volumes carried out by the differing surgeons. Given that 12% of the cases do not have complications rate reported I did not try and see of there was a linkage between volume and complications due to the incompleteness of the data set available.
There were a number of interesting patterns.
Volume by procedural type:
As the table shows, more surgeons did Robotic cases and on average did more cases per year than the other approaches. We therefore decided to an analysis of the difference in procedures between the top and bottom quartile. A quartile was based on the number of surgeons doing the procedures so for example in the robotic cohort we compared the volume of the top 21 surgeons against the bottom 21 Surgeons.
The table below highlights the differences:
25% of the surgeons (37) with the highest volumes carried out over 50% of the cases. Interestingly the concentration was greatest in Open surgery where they carried out 69% of the cases. At the other end of the spectrum 37 surgeons with the lowest volumes only did just below 9 cases each or 5% of the total volume.
This picture is made slightly more complicated as clearly some surgeons will do more than one technique. In this sample 106 of the surgeons (73%) used only on technique while 36 used two techniques and three surgeons used all three techniques.
The overall surgical volumes increased as the number of techniques used increased. Those using one technique averaged 39 cases in the time period, those using 2 averaged 45 and those using 3 averaged 49 procedures. It is only natural that this occurs as surgeons move from one technique to another or believe that different patients are better suited to different techniques.
One of the big advantages of the 21st century is that data from surgical performance is becoming more transparent. This transparency is going to allow medical professionals to have visibility on a number of factors that affect patient outcomes, which will allow them to put in the correct protocols to ensure that the highest quality of care is always delivered. We believe that the increasing amount of data is showing that the use of validated simulation protocols and curriculum can ensure best results for patients.