Health Technology Assessments (HTAs) of Robotic-Assisted Surgery (RAS) systems have proven difficult due to challenges with ethical data collection, the complexity of these products, and a lack of standardized methodology. As a result, these HTAs often come to very different conclusions or are unable to reach any firm conclusion. As the development of new RAS systems continues to grow, this situation is increasingly untenable.

To learn more about the challenges in HTAs for RAS systems, as well as best practice recommendations that can help to overcome the problems, we spoke to Jamie Erskine, Senior Consultant in medtech at Alira Health and one of the authors of our recent global review, Robotic-Assisted Surgery (RAS) in Orthopedic Procedures.

What are the key challenges HTAs face when evaluating the utility of RAS systems?

The first challenge is evidence generation through randomized control trials (RCTs). It is difficult to conduct an RCT in the same manner as you would for a new drug compared with a placebo. Surgeons cannot be blinded as they will know if they are using a robot to operate, and it is unethical not to inform patients that they will be operated on with a robot. RCTs are also impacted by the surgeons’ learning curve with using RAS platforms. Particularly when platforms are new to the market, most surgeons are still learning how to operate with that robot. When designing a trial, it’s hard to account for the various levels of expertise. One study might use expert surgeons in both arms of a trial, another might use a mixture of novice, intermediate, and expert users of RAS systems. And HTAs historically have not prioritized other types of study designs such as observational or real-world evidence (RWE) because they’re considered to be of lower quality. These complications make it difficult for HTAs to reach a conclusion.

The second challenge is that HTAs traditionally assess drugs, and the methods they’ve used since the 1970s and ‘80s are based on evaluating one drug for a single indication. With RAS systems, the robot is a single platform, but surgeons can use it for many different procedures on a variety of patient types to treat a range of clinical indications. The standard HTA methods were not developed for the challenge of evaluating a product that is applicable to different patient populations with many indications. Device lifespan also complicates the situation. Health systems and hospitals will buy a robot once and it will last for many years. But other factors go into the cost, including necessary consumables and the footprint the robot requires in an operating room.

The third issue is that different HTAs have developed their own approaches, leading to very different results, because there’s no standard framework for assessing RAS systems. I was the lead author on a recent paper, Best practice considerations on the assessment of robotic assisted surgical systems: results from an international consensus expert panel. During our research, we examined every HTA on RAS systems that we identified through the end of 2021, a total of 98 from 25 countries around the world. When we studied the economic analysis performed by these HTA bodies, we saw that the results were wildly different, even when multiple HTAs were conducted within the same country. The HTAs used different economic analysis methods and made different assumptions, e.g., different procedure volume numbers and different time horizons. This disparity in assumptions led to cost-effectiveness ratios that were different by several orders of magnitude. Some HTAs tested their assumptions with scenario analyses, but typically concluded that they didn’t have enough information to make a recommendation.

How did this research paper come about and what were the objectives?

This project came out of the Health Technology Assessment International medical device interest group; Alira Health is an active member. To research HTA methods on RAS, the group brought together 15 experts with a broad range of experience from four continents representing multi-stakeholder groups: health economists, HTA agencies, industry, payers, and surgeons. We met three times over a six-month period to discuss the issues and form a consensus for best practices when conducting HTAs of RAS systems. We published the resulting research paper in the International Journal of Technology Assessment in Healthcare.

The first objective of the paper was to inform HTA groups about the differences between standard HTA methodologies and what’s needed to assess RAS systems. The second objective was to provide them with best practice recommendations for their consideration. Because RAS systems vary, there will never be a one-size-fits-all method, but we hope that this guidance will help to address the challenges.

What were the top recommendations to improve HTAs of RAS systems?

One of the most important is that RWE needs to be one of the main sources of evidence. We recommend that manufacturers generate RWE and that HTAs critically appraise it to ensure that it’s high quality.

Another key point is to focus on the perspectives of both the surgical team and the patient. HTAs should look at patient-reported outcomes and patient satisfaction measures. Plus, they should understand how the RAS system affects the whole surgical team and their workflow. Using the robot may help surgeons and their teams to work longer and possibly do more surgeries in a day. Because there’s currently a lack of surgeons and long wait times for procedures, this could be extremely helpful and could factor into the system’s value.

HTAs should also consider the learning curve effect and correct it where necessary. Our research showed that some HTAs have done this, while others did not.

Will the HTAs of RAS systems change with the arrival of the new EU HTA Regulation (EU HTAR)?

There’s nothing specific to RAS in the EU HTAR, but one of the recommendations we made in the paper, driven by the fact that RAS systems can do multiple types of procedures, aligns well with it. We suggested that at the national level, HTAs should assess the robot for all potential procedures that have been established for that platform (i.e., prostatectomy, hysterectomy, cholecystectomy, etc. for soft-tissue robotics or knee and hip arthroplasty in orthopedics). That will result in the most comprehensive answer about the RAS system’s clinical value. Then, individual hospitals can perform a local economic analysis, based on their local population and local costs, to make a more informed decision about purchasing that system based on the procedures and volumes for which they would use it.

The EU HTAR recommends a similar approach. Clinical assessments of new technologies will happen at the EU level, and then at the national or local/hospital level, they can assess the technology’s economic value based on their local situation. Rather than every nation conducting separate clinical assessments, EU HTAs will not only save time and effort, but also, particularly in the case of RAS systems, will result in one standardized answer of total clinical value. And then each hospital can assess the likely economic impact of introducing a robot without having to repeat an onerous clinical assessment. So, our recommendations and those of the EU HTAR are quite harmonized.

Assessing the value of RAS systems is a complex decision-making problem. Our research confirmed our belief that there needs to be standardization for this process.

Download our Robotic-Assisted Surgery (RAS) in Orthopedic Procedures report for a deep dive into the current clinical landscape, including emerging players and technologies, trending market dynamics, and rates of adoption of orthopedic RAS.