TY - JOUR

T1 - Evidence following conditional NICE technology appraisal recommendations: A critical analysis of methods, quality and risk of bias

AU - Pijeira Perez, Yankier

AU - Hughes, Dyfrig

PY - 2024/7/14

Y1 - 2024/7/14

N2 - Background: The National Institute for Health and Care Excellence (NICE) may approve health technologies on condition of more evidence generated only in research (OiR) or only with research (OwR). NICE specifies the information needed to comply with its request, although it may not necessarily guarantee good quality and timely evidence for re-appraisal, before reaching a final decision.Aim: To critically appraise the methods, quality and risk of bias of evidence generated in response to NICE OiR and OwR technology appraisal (TA) and highly specialised technologies (HSTs) recommendations.Methods: NICE TAs (between March 2000 and September 2020) and HST evaluations (to October 2023) of medicines were reviewed. Conditional recommendations were analysed to identify the evidence requested by NICE for re-appraisal. The new evidence was analysed for compliance with NICE’s request and assessed using the Cochrane Collaboration’s tools for risk of bias in randomised trials and the ROBINS-I tool for non-randomised evidence.Results: NICE made 54 conditional recommendations from TAs (13 OiR and 41 OwR) and 5 conditional recommendations for HSTs (all OwR). Of these, 16 TAs presented additional evidence for re-appraisal [9 OiR (69%) and 7 OwR (17%)] and 3 HSTs [3 OwR (60%)]. Two of the 9 re-appraised TAs with OiR recommendation and 4 of the 7 OwR complied fully with NICE’s request for further evidence, while all 3 from the HSTs complied. The majority of re-appraised TAs and HSTs included evidence that was deemed to be at serious, high, moderate or unclear risk of bias. Among the 26 randomised controlled trials from TAs assessed, 8 were categorised as having low risk of bias in all domains and 10 had at least one domain as a high risk of bias. Reporting was unclear for the remainder. Twenty-two non-randomised studies, primarily single-arm studies, were susceptible to biases mostly due to the selection of participants and to confounding. Two HSTs provided evidence from randomised controlled trials which were classified as unclear or high risk of bias. All non-randomised evidence from HSTs were categorised moderate or serious risk of bias.Conclusions: There is widespread non-compliance with agreed data requests and important variation in the quality of evidence submitted in response to NICE conditionalapproval recommendations. Quality standards ought to be stipulated in respect to evidence contributing to re-appraisals following NICE conditional approval recommendations.

AB - Background: The National Institute for Health and Care Excellence (NICE) may approve health technologies on condition of more evidence generated only in research (OiR) or only with research (OwR). NICE specifies the information needed to comply with its request, although it may not necessarily guarantee good quality and timely evidence for re-appraisal, before reaching a final decision.Aim: To critically appraise the methods, quality and risk of bias of evidence generated in response to NICE OiR and OwR technology appraisal (TA) and highly specialised technologies (HSTs) recommendations.Methods: NICE TAs (between March 2000 and September 2020) and HST evaluations (to October 2023) of medicines were reviewed. Conditional recommendations were analysed to identify the evidence requested by NICE for re-appraisal. The new evidence was analysed for compliance with NICE’s request and assessed using the Cochrane Collaboration’s tools for risk of bias in randomised trials and the ROBINS-I tool for non-randomised evidence.Results: NICE made 54 conditional recommendations from TAs (13 OiR and 41 OwR) and 5 conditional recommendations for HSTs (all OwR). Of these, 16 TAs presented additional evidence for re-appraisal [9 OiR (69%) and 7 OwR (17%)] and 3 HSTs [3 OwR (60%)]. Two of the 9 re-appraised TAs with OiR recommendation and 4 of the 7 OwR complied fully with NICE’s request for further evidence, while all 3 from the HSTs complied. The majority of re-appraised TAs and HSTs included evidence that was deemed to be at serious, high, moderate or unclear risk of bias. Among the 26 randomised controlled trials from TAs assessed, 8 were categorised as having low risk of bias in all domains and 10 had at least one domain as a high risk of bias. Reporting was unclear for the remainder. Twenty-two non-randomised studies, primarily single-arm studies, were susceptible to biases mostly due to the selection of participants and to confounding. Two HSTs provided evidence from randomised controlled trials which were classified as unclear or high risk of bias. All non-randomised evidence from HSTs were categorised moderate or serious risk of bias.Conclusions: There is widespread non-compliance with agreed data requests and important variation in the quality of evidence submitted in response to NICE conditionalapproval recommendations. Quality standards ought to be stipulated in respect to evidence contributing to re-appraisals following NICE conditional approval recommendations.

M3 - Article

JO - Pharmacoeconomics

JF - Pharmacoeconomics

SN - 1170-7690

ER -