Unlike drugs, are implemented despite paucity of evidence

In contrast to pharmaceuticals, new surgical and other invasive procedures often enter clinical practice without assessment of their safety and efficacy. There is usually no guidance on their use. As a result, patients are offered treatments without access to adequate information on the nature and likelihood of benefits and harms; clinicians are left uncertain as to which innovations to adopt; and hospitals and healthcare systems have difficulty managing the risks that these procedures pose.1

Why does this happen? A fundamental reason is the poor evidence base for new interventional procedures, which has a number of causes. Unlike new pharmaceuticals, procedures do not usually have a commercial sponsor, which makes research harder to fund. When procedures entail a medical device, manufacturers have some incentive to fund evaluations, but differences in the regulatory regimes for drugs and devices, and in manufacturers' priorities for investment in research, mean that randomised trials are scarce. For example, novel designs of hip prostheses came to market with limited evaluation of their clinical performance.2 When research exists, widespread reliance on study designs without randomisation or even controls makes it hard to assess the validity of a procedure's apparent benefits. Follow up is commonly too short to show how often the condition relapses or late complications arise. Patients in evaluations are sometimes highly selected and the researchers are particularly skilled pioneers, limiting the results' generalisability and making reporting bias a concern. Particularly important is the fact that evaluations often have too few patients to detect important but less frequent complications of treatment. For example, the available evidence on laparoscopic helium plasma coagulation for endometriosis consists of three case series, none of which has been published in peer reviewed journals; they report on fewer than 100 patients in total.

Uncertainties may prevail about the meaning and value of the outcomes reported in studies of new interventional procedures. Data on outcomes that matter to patients and their doctors are of prime importance but often elusive. For example, radiofrequency ablation shrinks liver tumours, but little research exists on its effects on patients' symptoms and survival.3 Another difficulty is that a procedure's clinical impact is often highly dependent on the skill of the clinician doing it. Clinicians working in settings less favourable than those from which major series have been reported will have to climb the learning curve and may not even then achieve such good results. Also, procedures may evolve and change appreciably, especially when they are new.

There may also be cultural and educational reasons why interventional clinicians do not find rigorous evaluation attractive. These may include the clinicians' decisive approach to clinical practice and their personal involvement in procedures.4 They may then be less willing to recognise the existence of uncertainty and undertake research to reduce it. The problem is not confined to new interventional procedures: more widely, treatments in general surgery are half as likely as treatments in general medicine to be based on randomised controlled trials.5 6

The benefits and harms of new interventional procedures need to be considered together, and compared with existing treatments and no treatment. For example, laser in situ keratomileusis (LASIK) is used to treat myopia, a usually minor impairment corrected with great safety and effectiveness with spectacles or contact lenses. Even a low rate of complications might lead some patients to have qualms about the procedure. By contrast, people with an otherwise untreatable and rapidly fatal cancer might accept a procedure with a high immediate mortality after which survivors are likely to be cured.

Many innovations bring positive advantages which patients, clinicians, and healthcare systems value. However, lack of information on their benefits and harms brings unnecessary risk and may militate against their widespread use. What can be done? Firstly, clinicians need to do better research, from their first use of a new procedure. They need more training in how to evaluate their innovations, more collaboration with researchers skilled in evaluating health technologies, and more funding. Secondly, manufacturers of medical devices need to do more than the minimum required by their regulators and to show by research if their products change patients' outcomes. Thirdly, healthcare systems need to support responsible clinical innovation by providing guidance, not least to ensure that patients are fully informed before deciding whether to accept an interventional procedure.

In the United Kingdom, the National Institute for Clinical Excellence (NICE) has an interventional procedures programme (www.nice.org.uk/ip), which aims to improve care for patients by specifying the conditions under which a new interventional procedure should be used.7 For procedures with uncertain safety and efficacy, these include actively managing the uncertainty in a way that minimises risk. This means enhancing consent arrangements for patients so that they understand what is known and what is not; working with hospital clinical governance staff to manage the risks that the procedures may create; and monitoring outcomes by clinical audit or research. NICE guidance describes the limitations in the existing evidence, and, for selected procedures, NICE commissions systematic reviews and primary research.

No clinical guidance can remedy the underlying reasons for the limited attention given to evaluating procedures. Their potential for helping and harming patients is similar to that of pharmaceuticals, yet investment in gauging their benefits and harms is much less. Until this is addressed patients will continue be exposed to poorly understood risks, and uncertainty will continue to hamper the safe introduction of interventional procedures into routine clinical practice.

Tom Dent, director, interventional procedures programme

(tom.dent@nice.nhs.uk )

Sally Wortley, analyst, interventional procedures programme, Bruce Campbell, chairman, interventional procedures advisory committee

National Institute for Clinical Excellence, London WC1V 6NA
---

Competing interests: TD and SW are employed by NICE. BC's employer is reimbursed for time he spends on NICE's work.

References

1. Strasberg SM, Ludbrook PA. Who oversees innovative practice? Is there a structure that meets the monitoring needs of new techniques? J Am Coll Surg 2003;196: 938-4.[CrossRef][ISI][Medline]

2. Faulkner A, Kennedy LG, Baxter K, Donovan J, Wilkinson M, Bevan G. Effectiveness of hip prostheses in primary total hip replacement: a critical review of evidence and an economic model. Health Technol Assess 1998;2: 6.

3. Medical Services Advisory Committee. Radiofrequency ablation of liver tumours: assessment report. Canberra: Department of Health and Ageing, 2003. www.health.gov.au/msac/pdfs/reports/msac1052.pdf (accessed 7 Jun 2004).

4. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ 2002;324: 1448-51.[Free Full Text]

5. Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient general medicine is evidence based. Lancet 1995;364: 407-10.[CrossRef]

6. Howes N, Chagla L, Thorpe M, McCulloch P. Surgical practice is evidence based. Br J Surg 1997;84: 1220-3.[CrossRef][ISI][Medline]

7. Campbell B, Maddern G. Safety and efficacy of interventional procedures. BMJ 2003;326: 347-8.[Free Full Text]




PDF of this article

Email this article to a friend

Respond to this article

PubMed citation

Related articles in PubMed

Download to Citation Manager

Search Medline for articles by:
Dent, T. || Campbell, B.

Alert me when:
New articles cite this article

Collections under which this article appears:
Other Surgery
Other Health Policy