Patient group participation in the appraisal process of the National Institute for Clinical Excellence
Pauline Quennell, Department of Applied Social Science, University of Manchester
This paper brings together two policy elements prominent in recent UK government health care policy: evidence-based medicine (EBM) and patient participation. As part of the Labour government’s clinical governance agenda, NICE was launched in April 1999, with a remit to advise the National Health Service (NHS) in England and Wales on the clinical and cost-effectiveness of health care technologies, and to produce guidelines for a range of conditions. NICE guidance is underpinned by EBM, and involves a range of health care stakeholders. NICE’s committees include representatives of patient/carer groups, and NICE involves patient organisations in its technology appraisals, offering an almost unprecedented chance to have a ‘say’ in which technologies are available in the NHS. In NICE’s appraisal process, systematic reviews of scientific research are combined with submissions of evidence from manufacturers, health professionals and patient organisations, and considered by a multi-stakeholder Appraisal Committee. However, there may be inherent tensions here: Harrison et al suggest that “… giving people what they want may not be compatible with what the evidence says they should have” (2002, p1).
As part of doctoral research into study of patient involvement in NICE, this paper uses material from interviews with representatives of patient groups involved in the first waves of NICE’s technology appraisals and patient representatives on the committees of NICE, and also NICE Board members, Appraisal Committee members, academics and Department of Health staff.
Patient groups may participate at several stages in NICE’s appraisals process, including recommending topics; commenting on the appraisal scope; submitting written evidence and giving oral evidence; commenting on consultation documents; and appealing against NICE guidance. This range of options was not available when NICE began its work, so most activity to date has related to submissions of evidence, and appeals, together with strategies such as lobbying and campaigning. Initially, formal detail about patient involvement was vague, so patient groups formed a group separate from NICE’s formal structures, known as PIN (Patient Involvement in NICE). The following concerns were raised by patient interviewees: similar issues were raised at a meeting convened by the Consumers’ Association in July 2001, and at the House of Commons (HOC) Health Select Committee hearings in January 2002.
1. The nature of evidence
A key concern was what counts as ‘evidence’: NICE’s approach is based upon EBM, but patient groups were critical of the perceived emphasis on a hierarchy of evidence and a focus on randomised controlled trials (RCTs). Some patient groups were critical of the teams producing the assessment reports (systematic reviews of published data), and were concerned about their interpretations of the evidence. It was suggested that patient evidence could fill ‘gaps’ in the scientific evidence base. It was argued that because clinical trials did not take account of patient-defined outcomes, NICE’s criteria for effectiveness should include the perspective of those affected and the outcomes that they valued. For example, RCTs of treatment for Alzheimer’s disease examined measurable cognitive changes such as improved memory, rather than improvements valued by people with Alzheimer’s and their carers relating to mood, confidence and social skills. Also, the reviewers of RCTs for Riluzole (a drug for motor neurone disease) observed that trial data showed limited evidence of “modest benefit” in tracheostomy-free survival, which “…at best only postpones death for a few months”. However, for those with MND and their families, this extension to life was highly valued, and not ‘modest’ (cited in NICE, 2002).
2. The role of cost-effectiveness
There were calls from patient groups for the relative contributions of clinical and cost-effectiveness to be explicitly stated in NICE’s guidance, since they suspected that the cost component carried undue weight. For example, some felt that NICE was “overly impressed by the magnificence of health economics” and that cost-effectiveness data had “spurious accuracy”. However, it is interesting to note that some members of PIN have received training in health economics. Some had strong views on the value of QALYs, arguing that they discriminate against those with chronic illnesses or disabilities, and that there were problems in attaching costs to improved quality of life and reduced carer burden. Cost-effectiveness became a high profile issue in NICE’s appraisal of beta interferon for treatment of multiple sclerosis (MS), and there were allegations that NICE used a ‘threshold’ of £30,000, above which an intervention was not considered cost-effective. Both the provisional and final guidance on beta interferon issued by NICE invited the Department of Health to consider a strategy to acquire any or all of the beta interferons “… in a manner which could be considered to be cost effective”, arguing that cost-effectiveness of the medicines “can only be improved if there is a significant reduction in the total cost of their acquisition by the NHS” (NICE, 2001b, p9).
3. Format of evidence
In the absence of initial guidance, patient groups did not know what constituted an effective submission, being unsure whether to use quantitative methods such as questionnaires and surveys, or qualitative methods such as focus groups, interviews, and testimonies. In the Health Service Journal NICE chief executive Andrew Dillon argued that patient views were powerful, but NICE had to be seen to be grounded in the evidence, and the article’s author contended that patient organisations’ data needed to be soundly based on research if it was to count (Eaton, 2000, p12). Some patient groups felt that their contributions may not fit neatly into NICE’s view of evidence. Initially, patient witnesses did not give evidence at NICE Appraisal Committee meetings, so there was an emphasis on written evidence. However, there was a gradual change from a representative of a patient group (such as a chief executive or staff member) to a patient with the condition, but some wondered how oral evidence from patients was perceived.
4. ‘Weight’ of evidence
There was a perceived need for greater transparency about how evidence had been used, particularly that provided by patient groups, who were concerned about how their data weighed up against other forms of evidence, and felt that lack of feedback left them unable to evaluate the effectiveness of their submissions. More recently, NICE Appraisal Consultation Documents have sections explaining how evidence of clinical and cost-effectiveness has been used, but the utility of patient evidence is less easily discerned. In response to comments made at the HOC Health Select Committee, NICE has reported that written and oral evidence from patient groups such as the Alzheimer’s Society and the MND Association had a significant effect on the Appraisal Committee’s decisions (NICE, 2002). However, some patient groups were sceptical about how credible patient evidence was to those with a ‘scientific’ background, and felt that despite increased opportunity for participation, the patient voice was thought to be inherently disadvantaged.
Conclusions
Patient groups perceive that NICE has goodwill towards the patient perspective, but they are concerned about how their evidence is handled, weighted and valued. Changes in the appraisal process may have answered some of the patient groups’ concerns about administrative procedures, but there is still the question of the potential incompatibility of patient evidence and EBM. To avoid alienation of patient groups, NICE needs to demonstrate that patients’ views have had more than a marginal impact. There have been some attempts by NICE to do this, as demonstrated in their evidence to the HOC Health Select Committee. The effect of interventions upon patient-defined outcomes such as quality of life is perceived to be missing from RCTs, and some patient groups argue that patient evidence fills such gaps in the evidence base. Patient involvement in NICE appraisals has started to question definitions of expertise and knowledge, potentially providing a counterbalance to EBM. Patient-defined outcomes may be incorporated into future clinical trials, but the problem of how NICE handles evidence from disparate sources needs more immediate resolution, and has implications for other aspects of the clinical governance agenda. Patient participation in appraisals has been a learning process for both NICE and patient groups. NICE claims to treat all stakeholders equally, but patient groups feel that others such as health professionals, health economists, and the pharmaceutical industry have more influence. It could be asked why, if patient groups feel that they are not equal partners (Hogg, 1999) and have other avenues of influence, they become involved in appraisals. Also, initiatives such as the beta interferon risk-sharing scheme, negotiated between pharmaceutical companies and the Department of Health, could be viewed as undermining the legitimacy and authority of NICE’s advice, though NICE argues that such a scheme would not have happened without the appraisal. NICE’s audiences may be persuaded by different forms of knowledge, particularly those who implement NICE guidance: some dispute the primacy of EBM, while for others the bottom line is affordability. Thus success in NICE’s appraisal process is not the end of the story for patient groups, and the struggle to be heard continues.
Useful reading
Consumers’ Association (2001) National Institute for Clinical Excellence: a Patient-centred Inquiry. London: Consumers’ Association.
Department of Health (1999) Faster Access to Modern Treatment: How NICE Appraisals Will Work. London: Department of Health.
Department of Health (2000) The NHS Plan: a Plan for Investment: a Plan for Reform. Cm4818. London: The Stationery Office.
Department of Health (2002) Cost Effective Provision of Disease Modifying Therapies for People with Multiple Sclerosis. HSC 2002/004 (www.doh.gov.uk/publications/coinh.html).
Eaton, L. (2000) ‘A dose of scepticism’, Health Service Journal, 20th July, p12.
Edelman, M. (1971) Politics as Symbolic Action. New York: Free Press.
Harrison, S. (2002) ‘New Labour, modernisation, and the medical labour process’, Journal of Social Policy, 31 (3) 465-485.
Harrison, S., Dowswell, G., Milewa, T. (2002) ‘Public and user ‘involvement’ in the UK National Health Service’, Health and Social Care in the Community, 10 (19) 1-4.
Hogg, C. (1999) Patients, Power and Politics. London: Sage.
Kendrick, S. (2001) ‘Using all the evidence: towards a truly intelligent National Health Service’, Health Bulletin, 59 (2) 71-76.
NHS Management Executive (1992) Local Voices: the Views of Local People in Purchasing for Health. London: HMSO.
NICE (2001a) Guidance for Patient/carer Groups on Making a Submission to a Technology Appraisal. London: NICE.
NICE (2001b) Appraisal of Beta Interferon and Glatiramer Acetate for the Treatment of Multiple Sclerosis. Final Appraisal Determination. London: NICE.
NICE (2002) Supplementary Submission to the House of Commons Health Select Committee. (Papers 1 and 2). London: NICE.
Raftery, J. (2001) ‘NICE: faster access to modern treatments? Analysis of guidance on health technologies’, British Medical Journal, 323 1300-1303.
Sackett, D., Straus, S., Richardson, W., Rosenberg, W., Haynes, R. (2000) Evidence-based Medicine: How to Practise and Teach EBM (2nd ed). Edinburgh: Churchill Livingstone