Ben Goldacre posted last week about the new edition of Testing Treatments by Evans et al. for which he’s written the preface. He links in his post to a site where you can download the entire PDF of the book for free (!!!), and it’s pretty good. It has much to offer, and I’ll discuss its strongest aspects first. Then I’ll make my criticisms.
The authors openly confront medical uncertainties, calling on medical professionals to acknowledge and address them rather than trying to project a false image of confident authority when that is unwarranted. They point out that much medical practice is based on limited knowledge concerning prescribed treatments, illustrating the point with the stories of several treatments used for years and even decades without being properly evaluated, and argue for a transformation of practice that integrates research with treatment in cases of uncertainty.
One of the more insightful aspects of the book is its elucidation of the ethical and regulatory “double standard” with regard to treatment and research:
In practice, what this means is that clinicians can give unproven treatments to patients, as long as patients consent, if therapies are given within the context of ‘routine’ clinical practice. By contrast, conducting any study of the same treatments to evaluate them properly would involve going through the protracted regulatory process. So clinicians are discouraged from assessing treatments fairly, and instead can continue to prescribe treatments without committing to addressing any uncertainty about them… (pp. 108-9).Essentially, physicians are rather perversely prevented by this double standard from advancing knowledge in the course of their practice. Moreover, the authors argue, the current system of research is unfair to research subjects. The tests of treatments are often unnecessary - they discuss at length the problem that ethical reviews do not focus on the need for or contribution of the research in light of existing knowledge or the concrete usefulness of the information potentially obtained. They are also poorly designed and conducted, often twisted by those with an agenda, and the authors discuss (including linking to the eye-opening article discussed and available here) the numerous techniques used by drug and device makers to manipulate the research process to their ends.
They point to the fact that patients themselves rarely participate in the process of deciding research priorities. This quotation sums up the challenge:
‘Who has the power to see that research questions actually address the greatest needs of patients in all their misery and diversity? Why aren’t the most relevant questions being asked? Who is currently setting the questions? Who should be? Who shall direct this prioritisation? Patients are best able to identify the health topics most relevant to them and to inform their comfort, care, and quality of life, as well as its quantity. The patients are the David, who must load their slings against the Goliaths of the pharmaceutical companies who need evidence to market goods and make profits, and trialists who are driven by curiosity, the need to secure research money, professional acclaim, and career development. Profit, scientific inquiry, grant money, and research papers are acceptable only if the central motivation is the good of patients. Independent patients and organisations that advocate good quality research should ready their sling, carefully choose their stone, take aim, and conquer.’ - Refractor. Patients’ choice: David and Goliath. Lancet 2001; 358:768 (131)Evans et al. make the case for eliminating the double standard and integrating research with practice so that information gained in the care of patients contributes to the growth of medical knowledge. Of course, in order for this to be productive, research needs to be designed and performed competently and fairly. The authors call for fair tests of treatments, and define well and in detail what they mean by this term. Their vision for fair tests encompasses the entirety of the research process - from the questions asked through the research design to the dissemination of results and systematic reviews and meta-analyses. They propose a system of research designed to be fair to and benefit the important participants, particularly patients. (I’m pleased that they appreciate and discuss the role for social scientists in advancing this program.)
So the book makes a solid case about what’s wrong with the current research system and some original and sound suggestions for changing it. Of course, I appreciate their objection to paternalism and emphasis on participation, openness, and the basic value that science should serve human needs and be consistent with democracy. I also like that they substantially recognize the negative and counterproductive influence of corporations in this context, although others have perhaps explained it better and more comprehensively. An aspect of the book I enjoyed particularly was the short discussions of the history of movements within science (e.g., of doctors and patients in relation to breast cancer and HIV/AIDS), and the authors’ acknowledgement of the complexities of these movements and the hidden role of corporations in some. And it shouldn’t need mentioning that I appreciate that they’ve made it available for free download.
However, I do have a few criticisms (which does seem ungrateful given the free download and all). The first deals with form rather than content. It takes great finesse to make text boxes blend seamlessly with the body text. Even when what they present is relevant and interesting, those bordered boxes tend to interfere with reading, especially when they carry over from one page to the next or appear mid-sentence. In most cases, including this one, the information they contain would be better integrated into the regular text. (The graphics are nothing spectacular, either.) Also, the lists of suggestions and proposals at the end are a bit excessive and become confusing.
The second set of criticisms relates to the authors’ proposals for overhauling the research system. While I agree that the current double standard should go and that research should be integrated with care, I can’t accept the basis for their call to weaken ethics regulations. The authors argue that
[t]he regulatory system for research, in its preoccupation with risk and protecting potential research participants, has become over-protective and overlooks the fact that patients and the public are increasingly involved as partners in the research process... (p. 109).I certainly think that’s the ideal, and it may be more the case in the UK, but I’m not convinced at all that patients and the public are sufficiently involved as research partners in the US or many other countries that ethical standards could be relaxed while they remain protected. This is an odd claim given that, while they don’t go into the work of people like Harriet Washington and Sonia Shah, the authors clearly recognize the power that pharmaceutical companies hold over research. This isn't going to wither away if regulations are relaxed. They continue:
Requirements relating to provision of information and consent for studies are one of the ways in which the regulatory system acts to discourage rather than encourage research to address uncertainties about treatments. It is important – and ethical – to consider the interests of everyone currently receiving treatment, not just the few who participate in controlled trials. The standard for informed consent to treatment should therefore be the same whether people are being offered treatment within or outside the context of formal treatment assessments. To come to a decision that accords with their values and preferences, patients should have as much information as they want, and at a time that they want it (p. 110).I agree with the general sentiment here, which goes to their point about double standards. Essentially, patients in care settings receiving treatments about which there’s uncertainty are not being provided with enough information or a say. They’re not even unwitting research subjects, since their use of “experimental” treatments isn’t contributing to scientific knowledge. I know the authors are arguing for a change in ethical standards and regulations that informs and empowers patients, which I support; but calling for the weakening of any ethical standards is dangerous in the existing context.
Furthermore, the book doesn’t really address a major problem: that many doctors simply lack the knowledge of science and research methods necessary to integrate research and care properly. I think it’s important to get doctors engaged with this as part of an effort to increase their scientific understanding and competence, but it’s going to require a good deal of education and practice.
Finally, although Evans et al. discuss the what they consider an overemphasis on lab and animal research – suggesting that this is ethically problematic in that such research is far less likely to respond to the immediate needs of sick people – they don’t have anything to say about the ethical issues this research raises in terms of our moral relationship to nonhuman animals. The same criticisms the authors make about the ethical problems with bad research involving human subjects apply to animal research as well, perhaps more so because they can’t consent.
Overall, a good book that I’d recommend, maybe together with Goldacre’s Bad Science and a few others.