Skip to main content
Log in

Decision-Makers’ Preferences for Approving New Medicines in Wales: A Discrete-Choice Experiment with Assessment of External Validity

  • Original Research Article
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Background

Few studies to date have explored the stated preferences of national decision makers for health technology adoption criteria, and none of these have compared stated decision-making behaviours against actual behaviours. Assessment of the external validity of stated preference studies, such as discrete-choice experiments (DCEs), remains an under-researched area.

Objectives

The primary aim was to explore the preferences of All Wales Medicines Strategy Group (AWMSG) appraisal committee and appraisal sub-committee (the New Medicines Group) members (‘appraisal committees’) for specific new medicines adoption criteria. Secondary aims were to explore the external validity of respondents’ stated preferences and the impact of question choice options upon preference structures in DCEs.

Methods

A DCE was conducted to estimate appraisal committees members’ preferences for incremental cost effectiveness, quality-adjusted life-years (QALYs) gained, annual number of patients expected to be treated, the impact of the disease on patients before treatment, and the assessment of uncertainty in the economic evidence submitted for new medicines compared with current UK NHS treatment. Respondents evaluated 28 pairs of hypothetical new medicines, making a primary forced choice between each pair and a more flexible secondary choice, which permitted either, neither or both new medicines to be chosen. The performance of the resultant models was compared against previous AWMSG decisions.

Results

Forty-one out of a total of 80 past and present members of AWMSG appraisal committees completed the DCE. The incremental cost effectiveness of new medicines, and the QALY gains they provide, significantly (p < 0.0001) influence recommendations. Committee members were willing to accept higher incremental cost-effectiveness ratios and lower QALY gains for medicines that treat disease impacting primarily upon survival rather than quality of life, and where uncertainty in the cost-effectiveness estimates has been thoroughly explored. The number of patients to be treated by the new medicine did not exert a significant influence upon recommendations. The use of a flexible-choice question format revealed a different preference structure to the forced-choice format, but the performance of the two models was similar. Aggregate decisions of the AWMSG were well predicted by both models, but their sensitivity (64 %, 68 %) and specificity (55 %, 64 %) were limited.

Conclusions

A willingness to trade the cost effectiveness and QALY gains against other factors indicates that economic efficiency and QALY maximisation are not the only considerations of committee members when making recommendations on the use of medicines in Wales. On average, appraisal committee members’ stated preferences appear consistent with their actual decision-making behaviours, providing support for the external validity of our DCEs. However, as health technology assessment involves complex decision-making processes, and each individual recommendation may be influenced to varying degrees by a multitude of different considerations, the ability of our models to predict individual medicine recommendations is more limited.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Stafinski T, Menon D, Philippon DJ, et al. Health technology funding decision-making processes around the world: the same, yet different. Pharmacoeconomics. 2011;29(6):475–95.

    Article  PubMed  Google Scholar 

  2. George B, Harris A, Mitchell A. Cost effectiveness analysis and the consistency of decision-making: evidence from pharmaceutical reimbursement in Australia (1991 to 1996). Pharmacoeconomics. 2001;19(11):1103–9.

    Article  PubMed  CAS  Google Scholar 

  3. Grégoire J-P, McNeil P, Skilton K, et al. Inter-provincial variation in Government drug formularies. Can J Public Health. 2001;92:307–13.

    PubMed  Google Scholar 

  4. PausJenssen AM, Singer PA, Detsky AS. Ontario’s formulary committee: how recommendations are made. Pharmacoeconomics. 2003;21(4):285–94.

    Article  PubMed  Google Scholar 

  5. Tilson L, O’Leary A, Usher C, et al. Pharmacoeconomic evaluation in Ireland: a review of the process. Pharmacoeconomics. 2010;28(4):307–22.

    Article  PubMed  Google Scholar 

  6. Mason A, Drummond M, Ramsel S, et al. Comparison of anticancer drug coverage decisions in the United States and United Kingdom: does the evidence support the rhetoric? J Clin Oncol. 2010;28:3234–8.

    Article  PubMed  Google Scholar 

  7. Chim L, Kelly PJ, Salkeld G, et al. Are cancer drugs less likely to be recommended for listing by the Pharmaceutical Benefits Advisory Committee in Australia. Pharmacoconomics. 2010;28(6):463–75.

    Article  Google Scholar 

  8. Devlin N, Parkin P. Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 2004;13:437–52.

    Article  PubMed  Google Scholar 

  9. Dakin HA, Devlin NJ, Odeyemi IAO. “Yes”, “No” or “Yes, but”? Multinomial modelling of NICE decision-making. Health Policy. 2006;77:352–67.

    Article  PubMed  Google Scholar 

  10. Linley WG, Hughes DA. Reimbursement decisions of the All Wales Medicines Strategy Group: influence of policy and clinical and economic factors. Pharmacoeconomics. 2012;30(9):779–94.

    Article  PubMed  Google Scholar 

  11. Harris AH, Hill SR, Chin G, et al. The role of value for money in public insiurance coverage decisions for drugs in Australia: a retrospective analysis 1994–2004. Med Decis Making. 2008;28:713–22.

    Article  PubMed  Google Scholar 

  12. Stafinski T, Menon D, Davis C, et al. Role of centralised review processes for making reimbursement decisions on new health technologies in Europe. Clinicoecon Outcomes Res. 2011;3:117–86.

    Article  PubMed  Google Scholar 

  13. Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoeconomics. 2008;26(8):661–77.

    Article  PubMed  Google Scholar 

  14. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy. 2003;2(1):55–64.

    PubMed  Google Scholar 

  15. De Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–72.

    Article  PubMed  Google Scholar 

  16. Tappenden P, Brazier J, Ratcliffe J, et al. A stated preference binary choice experiment to explore NICE decision making. Pharmacoeconomics. 2007;25(8):685–93.

    Article  PubMed  Google Scholar 

  17. Koopmanschap MA, Stolk EA, Koolman X. Dear policy maker: have you made up your mind? A discrete choice experiment among policy makers and other health professionals. Int J Technol Assess Health Care. 2010;26:198–204.

    Article  PubMed  Google Scholar 

  18. Whitty JA, Scuffham PA, Rundle-Thiele SR. Public and decision maker stated preferences for pharmaceutical subsidy: a pilot study. Appl Health Econ Health Policy. 2011;9(2):73–9.

    Article  PubMed  Google Scholar 

  19. All Wales Medicines Strategy Group. 2012. Guidelines for appraising medicines. http://www.wales.nhs.uk/sites3/Documents/371/Guidelines%20for%20appraising%20medicines%20July%202012.pdf. Accessed 22 Feb 2013.

  20. Johnson FR, Backhouse M. Eliciting stated preferences for health technology adoption criteria using paired comparisons and recommendation judgements. Value Health. 2006;9:303–11.

    Article  PubMed  Google Scholar 

  21. World Health Organisation. The global burden of disease: 2004 update. Geneva: WHO; 2008. http://www.who.int/healthinfo/global_burden_disease/2004_report_update/en/index.html. Accessed 30 Apr 2010.

  22. Orme B. Getting started with conjoint analysis: strategies for product design and pricing research. 2nd ed. Madison (WI): Research Publishers LLC; 2010.

    Google Scholar 

  23. Hann GJ, Shapira SS. A catalogue and computer programme for the design and analysis of orthogonal symmetric and asymmetric fractional factorial experiments. Report 66-C-165. Schenectady (NY): General Electric Research and Development Centre; 1996.

  24. Bateman IJ, Carson RT, Day B, et al. Choice set awareness and ordering effects in discrete choice experiments. Centre for Social and Economic Research on the Global Environment. Working Paper EDM 08-1; 2008. http://www.cserge.ac.uk/sites/default/files/edm_2008_01.pdf. Accessed 01 Mar 2012.

  25. All Wales Medicines Strategy Group. AWMSG policy on appraising life extending, end-of-life medicines. 2011. http://www.wales.nhs.uk/sites3/Documents/371/AWMSG%20Policy%20on%20appraising%20life-extending%2C%20end%20of%20life%20medicines.pdf. Accessed 15 Feb 2012.

  26. All Wales Medicines Strategy Group. AWMSG policy relating to ultra-orphan medicines. 2012. http://www.wales.nhs.uk/sites3/Documents/371/AWMSG%20policy%20relating%20to%20Ultra-orphan%20Medicines%20July%202012.pdf. Accessed 22 Feb 2013.

  27. Department of Health. 2010 Oct 27. The cancer drugs fund: a consultation. http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/digitalasset/dh_120931.pdf. Accessed 30 Sep 2011.

  28. Rawlins M, Barnett D, Stevens A. Pharmacoeconomics: NICE’s approach to decision-making. Brit J Clin Pharmaco. 2010;70:346–9.

    Article  Google Scholar 

  29. Department of Health. 2010. A new value-based approach to the pricing of branded medicines: a consultation. http://www.dh.gov.uk/en/Consultations/Closedconsultations/DH_122760. Accessed 30 Sept 2011.

  30. Hughes DA. Value-based pricing: incentive for innovation or zero net benefit? Pharmacoeconomics. 2011;29(9):731–5.

    Article  PubMed  Google Scholar 

  31. Linley WG, Hughes DH. Societal views on NICE, Cancer Drugs Fund and value-based pricing criteria for prioritising medicines: a cross-sectional survey of 4118 adults in Great Britain. Health Econ. doi:10.1002/hec.2872

  32. Ryan M, Gerard K, Amaya-Amaya M, editors. Using discrete choice experiments to value health and health care. Dordrecht: Springer; 2008.

    Google Scholar 

Download references

Acknowledgments

We wish to thank all pilot respondents and all AWMSG and NMG appraisal committee members who kindly participated in this study. We are grateful for the co-operation of the AWMSG Steering Committee and the AWTTC who supported this project. We are grateful for advice on experimental design issues from Emily Fargher, and the constructive comments of Professor Rhiannon Tudor Edwards, Professor Philip Routledge and two anonymous referees. The authors alone are responsible for the resulting paper.

Author contributions

WGL and DAH conceived the study. WGL designed the survey, managed questionnaire administration and data collection, analysed responses, interpreted the results and drafted the manuscript. WGL and DAH critically revised the manuscript for intellectual content and approved the final version of the paper. DAH had full access to all data, had final responsibility for the decision to submit for publication and is guarantor.

Funding disclosure

Funding for the project was provided by a Bangor University PhD studentship awarded to WGL. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. WGL and DAH have support from Bangor University for the submitted work; WGL (operating via newmedinfo Ltd at the time of the study) and DAH produced the economic components of the AWMSG assessment reports for, and on behalf of, the AWTTC, which may have an interest in the submitted work. DAH is past deputy member of the AWMSG, which may have an interest in the submitted work. Their spouses, partners, or children have no financial relationships that may be relevant to the submitted work. WGL and DAH have no non-financial interests that may be relevant to the submitted work.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dyfrig A. Hughes.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 109 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Linley, W.G., Hughes, D.A. Decision-Makers’ Preferences for Approving New Medicines in Wales: A Discrete-Choice Experiment with Assessment of External Validity. PharmacoEconomics 31, 345–355 (2013). https://doi.org/10.1007/s40273-013-0030-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-013-0030-0

Keywords

Navigation