Abstract
Background
Few studies to date have explored the stated preferences of national decision makers for health technology adoption criteria, and none of these have compared stated decision-making behaviours against actual behaviours. Assessment of the external validity of stated preference studies, such as discrete-choice experiments (DCEs), remains an under-researched area.
Objectives
The primary aim was to explore the preferences of All Wales Medicines Strategy Group (AWMSG) appraisal committee and appraisal sub-committee (the New Medicines Group) members (‘appraisal committees’) for specific new medicines adoption criteria. Secondary aims were to explore the external validity of respondents’ stated preferences and the impact of question choice options upon preference structures in DCEs.
Methods
A DCE was conducted to estimate appraisal committees members’ preferences for incremental cost effectiveness, quality-adjusted life-years (QALYs) gained, annual number of patients expected to be treated, the impact of the disease on patients before treatment, and the assessment of uncertainty in the economic evidence submitted for new medicines compared with current UK NHS treatment. Respondents evaluated 28 pairs of hypothetical new medicines, making a primary forced choice between each pair and a more flexible secondary choice, which permitted either, neither or both new medicines to be chosen. The performance of the resultant models was compared against previous AWMSG decisions.
Results
Forty-one out of a total of 80 past and present members of AWMSG appraisal committees completed the DCE. The incremental cost effectiveness of new medicines, and the QALY gains they provide, significantly (p < 0.0001) influence recommendations. Committee members were willing to accept higher incremental cost-effectiveness ratios and lower QALY gains for medicines that treat disease impacting primarily upon survival rather than quality of life, and where uncertainty in the cost-effectiveness estimates has been thoroughly explored. The number of patients to be treated by the new medicine did not exert a significant influence upon recommendations. The use of a flexible-choice question format revealed a different preference structure to the forced-choice format, but the performance of the two models was similar. Aggregate decisions of the AWMSG were well predicted by both models, but their sensitivity (64 %, 68 %) and specificity (55 %, 64 %) were limited.
Conclusions
A willingness to trade the cost effectiveness and QALY gains against other factors indicates that economic efficiency and QALY maximisation are not the only considerations of committee members when making recommendations on the use of medicines in Wales. On average, appraisal committee members’ stated preferences appear consistent with their actual decision-making behaviours, providing support for the external validity of our DCEs. However, as health technology assessment involves complex decision-making processes, and each individual recommendation may be influenced to varying degrees by a multitude of different considerations, the ability of our models to predict individual medicine recommendations is more limited.
Similar content being viewed by others
References
Stafinski T, Menon D, Philippon DJ, et al. Health technology funding decision-making processes around the world: the same, yet different. Pharmacoeconomics. 2011;29(6):475–95.
George B, Harris A, Mitchell A. Cost effectiveness analysis and the consistency of decision-making: evidence from pharmaceutical reimbursement in Australia (1991 to 1996). Pharmacoeconomics. 2001;19(11):1103–9.
Grégoire J-P, McNeil P, Skilton K, et al. Inter-provincial variation in Government drug formularies. Can J Public Health. 2001;92:307–13.
PausJenssen AM, Singer PA, Detsky AS. Ontario’s formulary committee: how recommendations are made. Pharmacoeconomics. 2003;21(4):285–94.
Tilson L, O’Leary A, Usher C, et al. Pharmacoeconomic evaluation in Ireland: a review of the process. Pharmacoeconomics. 2010;28(4):307–22.
Mason A, Drummond M, Ramsel S, et al. Comparison of anticancer drug coverage decisions in the United States and United Kingdom: does the evidence support the rhetoric? J Clin Oncol. 2010;28:3234–8.
Chim L, Kelly PJ, Salkeld G, et al. Are cancer drugs less likely to be recommended for listing by the Pharmaceutical Benefits Advisory Committee in Australia. Pharmacoconomics. 2010;28(6):463–75.
Devlin N, Parkin P. Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 2004;13:437–52.
Dakin HA, Devlin NJ, Odeyemi IAO. “Yes”, “No” or “Yes, but”? Multinomial modelling of NICE decision-making. Health Policy. 2006;77:352–67.
Linley WG, Hughes DA. Reimbursement decisions of the All Wales Medicines Strategy Group: influence of policy and clinical and economic factors. Pharmacoeconomics. 2012;30(9):779–94.
Harris AH, Hill SR, Chin G, et al. The role of value for money in public insiurance coverage decisions for drugs in Australia: a retrospective analysis 1994–2004. Med Decis Making. 2008;28:713–22.
Stafinski T, Menon D, Davis C, et al. Role of centralised review processes for making reimbursement decisions on new health technologies in Europe. Clinicoecon Outcomes Res. 2011;3:117–86.
Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoeconomics. 2008;26(8):661–77.
Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy. 2003;2(1):55–64.
De Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–72.
Tappenden P, Brazier J, Ratcliffe J, et al. A stated preference binary choice experiment to explore NICE decision making. Pharmacoeconomics. 2007;25(8):685–93.
Koopmanschap MA, Stolk EA, Koolman X. Dear policy maker: have you made up your mind? A discrete choice experiment among policy makers and other health professionals. Int J Technol Assess Health Care. 2010;26:198–204.
Whitty JA, Scuffham PA, Rundle-Thiele SR. Public and decision maker stated preferences for pharmaceutical subsidy: a pilot study. Appl Health Econ Health Policy. 2011;9(2):73–9.
All Wales Medicines Strategy Group. 2012. Guidelines for appraising medicines. http://www.wales.nhs.uk/sites3/Documents/371/Guidelines%20for%20appraising%20medicines%20July%202012.pdf. Accessed 22 Feb 2013.
Johnson FR, Backhouse M. Eliciting stated preferences for health technology adoption criteria using paired comparisons and recommendation judgements. Value Health. 2006;9:303–11.
World Health Organisation. The global burden of disease: 2004 update. Geneva: WHO; 2008. http://www.who.int/healthinfo/global_burden_disease/2004_report_update/en/index.html. Accessed 30 Apr 2010.
Orme B. Getting started with conjoint analysis: strategies for product design and pricing research. 2nd ed. Madison (WI): Research Publishers LLC; 2010.
Hann GJ, Shapira SS. A catalogue and computer programme for the design and analysis of orthogonal symmetric and asymmetric fractional factorial experiments. Report 66-C-165. Schenectady (NY): General Electric Research and Development Centre; 1996.
Bateman IJ, Carson RT, Day B, et al. Choice set awareness and ordering effects in discrete choice experiments. Centre for Social and Economic Research on the Global Environment. Working Paper EDM 08-1; 2008. http://www.cserge.ac.uk/sites/default/files/edm_2008_01.pdf. Accessed 01 Mar 2012.
All Wales Medicines Strategy Group. AWMSG policy on appraising life extending, end-of-life medicines. 2011. http://www.wales.nhs.uk/sites3/Documents/371/AWMSG%20Policy%20on%20appraising%20life-extending%2C%20end%20of%20life%20medicines.pdf. Accessed 15 Feb 2012.
All Wales Medicines Strategy Group. AWMSG policy relating to ultra-orphan medicines. 2012. http://www.wales.nhs.uk/sites3/Documents/371/AWMSG%20policy%20relating%20to%20Ultra-orphan%20Medicines%20July%202012.pdf. Accessed 22 Feb 2013.
Department of Health. 2010 Oct 27. The cancer drugs fund: a consultation. http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_120931.pdf. Accessed 30 Sep 2011.
Rawlins M, Barnett D, Stevens A. Pharmacoeconomics: NICE’s approach to decision-making. Brit J Clin Pharmaco. 2010;70:346–9.
Department of Health. 2010. A new value-based approach to the pricing of branded medicines: a consultation. http://www.dh.gov.uk/en/Consultations/Closedconsultations/DH_122760. Accessed 30 Sept 2011.
Hughes DA. Value-based pricing: incentive for innovation or zero net benefit? Pharmacoeconomics. 2011;29(9):731–5.
Linley WG, Hughes DH. Societal views on NICE, Cancer Drugs Fund and value-based pricing criteria for prioritising medicines: a cross-sectional survey of 4118 adults in Great Britain. Health Econ. doi:10.1002/hec.2872
Ryan M, Gerard K, Amaya-Amaya M, editors. Using discrete choice experiments to value health and health care. Dordrecht: Springer; 2008.
Acknowledgments
We wish to thank all pilot respondents and all AWMSG and NMG appraisal committee members who kindly participated in this study. We are grateful for the co-operation of the AWMSG Steering Committee and the AWTTC who supported this project. We are grateful for advice on experimental design issues from Emily Fargher, and the constructive comments of Professor Rhiannon Tudor Edwards, Professor Philip Routledge and two anonymous referees. The authors alone are responsible for the resulting paper.
Author contributions
WGL and DAH conceived the study. WGL designed the survey, managed questionnaire administration and data collection, analysed responses, interpreted the results and drafted the manuscript. WGL and DAH critically revised the manuscript for intellectual content and approved the final version of the paper. DAH had full access to all data, had final responsibility for the decision to submit for publication and is guarantor.
Funding disclosure
Funding for the project was provided by a Bangor University PhD studentship awarded to WGL. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. WGL and DAH have support from Bangor University for the submitted work; WGL (operating via newmedinfo Ltd at the time of the study) and DAH produced the economic components of the AWMSG assessment reports for, and on behalf of, the AWTTC, which may have an interest in the submitted work. DAH is past deputy member of the AWMSG, which may have an interest in the submitted work. Their spouses, partners, or children have no financial relationships that may be relevant to the submitted work. WGL and DAH have no non-financial interests that may be relevant to the submitted work.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Linley, W.G., Hughes, D.A. Decision-Makers’ Preferences for Approving New Medicines in Wales: A Discrete-Choice Experiment with Assessment of External Validity. PharmacoEconomics 31, 345–355 (2013). https://doi.org/10.1007/s40273-013-0030-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40273-013-0030-0