Getting a word in edgeways?
Patient group participation in the appraisal process of the National Institute
for Clinical Excellence
Pauline Quennell, Department of Applied Social Science, University of Manchester
This paper brings together two policy elements prominent in recent UK government
health care policy: evidence-based medicine (EBM) and patient participation.
As part of the Labour government's clinical governance agenda, NICE was launched
in April 1999, with a remit to advise the National Health Service (NHS) in England
and Wales on the clinical and cost-effectiveness of health care technologies,
and to produce guidelines for a range of conditions. NICE guidance is underpinned
by EBM, and involves a range of health care stakeholders. NICE's committees
include representatives of patient/carer groups, and NICE involves patient organisations
in its technology appraisals, offering an almost unprecedented chance to have
a 'say' in which technologies are available in the NHS. In NICE's appraisal
process, systematic reviews of scientific research are combined with submissions
of evidence from manufacturers, health professionals and patient organisations,
and considered by a multi-stakeholder Appraisal Committee. However, there may
be inherent tensions here: Harrison et al suggest that "... giving people
what they want may not be compatible with what the evidence says they should
have" (2002, p1).
As part of doctoral research into study of patient involvement in NICE, this
paper uses material from interviews with representatives of patient groups involved
in the first waves of NICE's technology appraisals and patient representatives
on the committees of NICE, and also NICE Board members, Appraisal Committee
members, academics and Department of Health staff.
Patient groups may participate at several stages in NICE's appraisals process,
including recommending topics; commenting on the appraisal scope; submitting
written evidence and giving oral evidence; commenting on consultation documents;
and appealing against NICE guidance. This range of options was not available
when NICE began its work, so most activity to date has related to submissions
of evidence, and appeals, together with strategies such as lobbying and campaigning.
Initially, formal detail about patient involvement was vague, so patient groups
formed a group separate from NICE's formal structures, known as PIN (Patient
Involvement in NICE). The following concerns were raised by patient interviewees:
similar issues were raised at a meeting convened by the Consumers' Association
in July 2001, and at the House of Commons (HOC) Health Select Committee hearings
in January 2002.
1. The nature of evidence
A key concern was what counts as 'evidence': NICE's approach is based upon EBM,
but patient groups were critical of the perceived emphasis on a hierarchy of evidence
and a focus on randomised controlled trials (RCTs). Some patient groups were critical
of the teams producing the assessment reports (systematic reviews of published
data), and were concerned about their interpretations of the evidence. It was
suggested that patient evidence could fill 'gaps' in the scientific evidence base.
It was argued that because clinical trials did not take account of patient-defined
outcomes, NICE's criteria for effectiveness should include the perspective of
those affected and the outcomes that they valued. For example, RCTs of treatment
for Alzheimer's disease examined measurable cognitive changes such as improved
memory, rather than improvements valued by people with Alzheimer's and their carers
relating to mood, confidence and social skills. Also, the reviewers of RCTs for
Riluzole (a drug for motor neurone disease) observed that trial data showed limited
evidence of "modest benefit" in tracheostomy-free survival, which "...at
best only postpones death for a few months". However, for those with MND
and their families, this extension to life was highly valued, and not 'modest'
(cited in NICE, 2002).
2. The role of cost-effectiveness
There were calls from patient groups for the relative contributions of clinical
and cost-effectiveness to be explicitly stated in NICE's guidance, since they
suspected that the cost component carried undue weight. For example, some felt
that NICE was "overly impressed by the magnificence of health economics"
and that cost-effectiveness data had "spurious accuracy". However, it
is interesting to note that some members of PIN have received training in health
economics. Some had strong views on the value of QALYs, arguing that they discriminate
against those with chronic illnesses or disabilities, and that there were problems
in attaching costs to improved quality of life and reduced carer burden. Cost-effectiveness
became a high profile issue in NICE's appraisal of beta interferon for treatment
of multiple sclerosis (MS), and there were allegations that NICE used a 'threshold'
of £30,000, above which an intervention was not considered cost-effective.
Both the provisional and final guidance on beta interferon issued by NICE invited
the Department of Health to consider a strategy to acquire any or all of the beta
interferons "... in a manner which could be considered to be cost effective",
arguing that cost-effectiveness of the medicines "can only be improved if
there is a significant reduction in the total cost of their acquisition by the
NHS" (NICE, 2001b, p9).
3. Format of evidence
In the absence of initial guidance, patient groups did not know what constituted
an effective submission, being unsure whether to use quantitative methods such
as questionnaires and surveys, or qualitative methods such as focus groups, interviews,
and testimonies. In the Health Service Journal NICE chief executive Andrew Dillon
argued that patient views were powerful, but NICE had to be seen to be grounded
in the evidence, and the article's author contended that patient organisations'
data needed to be soundly based on research if it was to count (Eaton, 2000, p12).
Some patient groups felt that their contributions may not fit neatly into NICE's
view of evidence. Initially, patient witnesses did not give evidence at NICE Appraisal
Committee meetings, so there was an emphasis on written evidence. However, there
was a gradual change from a representative of a patient group (such as a chief
executive or staff member) to a patient with the condition, but some wondered
how oral evidence from patients was perceived.
4. 'Weight' of evidence
There was a perceived need for greater transparency about how evidence had been
used, particularly that provided by patient groups, who were concerned about how
their data weighed up against other forms of evidence, and felt that lack of feedback
left them unable to evaluate the effectiveness of their submissions. More recently,
NICE Appraisal Consultation Documents have sections explaining how evidence of
clinical and cost-effectiveness has been used, but the utility of patient evidence
is less easily discerned. In response to comments made at the HOC Health Select
Committee, NICE has reported that written and oral evidence from patient groups
such as the Alzheimer's Society and the MND Association had a significant effect
on the Appraisal Committee's decisions (NICE, 2002). However, some patient groups
were sceptical about how credible patient evidence was to those with a 'scientific'
background, and felt that despite increased opportunity for participation, the
patient voice was thought to be inherently disadvantaged.
Conclusions
Patient groups perceive that NICE has goodwill towards the patient perspective,
but they are concerned about how their evidence is handled, weighted and valued.
Changes in the appraisal process may have answered some of the patient groups'
concerns about administrative procedures, but there is still the question of the
potential incompatibility of patient evidence and EBM. To avoid alienation of
patient groups, NICE needs to demonstrate that patients' views have had more than
a marginal impact. There have been some attempts by NICE to do this, as demonstrated
in their evidence to the HOC Health Select Committee. The effect of interventions
upon patient-defined outcomes such as quality of life is perceived to be missing
from RCTs, and some patient groups argue that patient evidence fills such gaps
in the evidence base. Patient involvement in NICE appraisals has started to question
definitions of expertise and knowledge, potentially providing a counterbalance
to EBM. Patient-defined outcomes may be incorporated into future clinical trials,
but the problem of how NICE handles evidence from disparate sources needs more
immediate resolution, and has implications for other aspects of the clinical governance
agenda. Patient participation in appraisals has been a learning process for both
NICE and patient groups. NICE claims to treat all stakeholders equally, but patient
groups feel that others such as health professionals, health economists, and the
pharmaceutical industry have more influence. It could be asked why, if patient
groups feel that they are not equal partners (Hogg, 1999) and have other avenues
of influence, they become involved in appraisals. Also, initiatives such as the
beta interferon risk-sharing scheme, negotiated between pharmaceutical companies
and the Department of Health, could be viewed as undermining the legitimacy and
authority of NICE's advice, though NICE argues that such a scheme would not have
happened without the appraisal. NICE's audiences may be persuaded by different
forms of knowledge, particularly those who implement NICE guidance: some dispute
the primacy of EBM, while for others the bottom line is affordability. Thus success
in NICE's appraisal process is not the end of the story for patient groups, and
the struggle to be heard continues.
Useful reading
Consumers' Association (2001) National Institute for Clinical Excellence: a Patient-centred
Inquiry. London: Consumers' Association.
Department of Health (1999) Faster Access to Modern Treatment: How NICE Appraisals
Will Work. London: Department of Health.
Department of Health (2000) The NHS Plan: a Plan for Investment: a Plan for Reform.
Cm4818. London: The Stationery Office.
Department of Health (2002) Cost Effective Provision of Disease Modifying Therapies
for People with Multiple Sclerosis. HSC 2002/004 (www.doh.gov.uk/publications/coinh.html).
Eaton, L. (2000) 'A dose of scepticism', Health Service Journal, 20th July, p12.
Edelman, M. (1971) Politics as Symbolic Action. New York: Free Press.
Harrison, S. (2002) 'New Labour, modernisation, and the medical labour process',
Journal of Social Policy, 31 (3) 465-485.
Harrison, S., Dowswell, G., Milewa, T. (2002) 'Public and user 'involvement' in
the UK National Health Service', Health and Social Care in the Community, 10 (19)
1-4.
Hogg, C. (1999) Patients, Power and Politics. London: Sage.
Kendrick, S. (2001) 'Using all the evidence: towards a truly intelligent National
Health Service', Health Bulletin, 59 (2) 71-76.
NHS Management Executive (1992) Local Voices: the Views of Local People in Purchasing
for Health. London: HMSO.
NICE (2001a) Guidance for Patient/carer Groups on Making a Submission to a Technology
Appraisal. London: NICE.
NICE (2001b) Appraisal of Beta Interferon and Glatiramer Acetate for the Treatment
of Multiple Sclerosis. Final Appraisal Determination. London: NICE.
NICE (2002) Supplementary Submission to the House of Commons Health Select Committee.
(Papers 1 and 2). London: NICE.
Raftery, J. (2001) 'NICE: faster access to modern treatments? Analysis of guidance
on health technologies', British Medical Journal, 323 1300-1303.
Sackett, D., Straus, S., Richardson, W., Rosenberg, W., Haynes, R. (2000) Evidence-based
Medicine: How to Practise and Teach EBM (2nd ed). Edinburgh: Churchill Livingstone
