Several recent studies on acupuncture – some of which have even been published in top journals – have reached positive conclusions. One, for example, suggests that acupuncture improves language function, quality of life and neurological impairment in stroke patients. Another suggests that acupuncture will alleviate the neurological problems that often accompany diabetes. Like almost all studies reporting positive results from acupuncture or other alternative therapies, these trials share major flaws that seriously call into question their optimistic conclusions. This leads me to conclude that acupuncture is in fact little more than a theatrical placebo that does not merit further research.
Not surprisingly, acupuncture advocates find my verdict unacceptable. This raises two related questions that have plagued researchers for decades: 1) When can we assume that a therapy is not working? 2) Why do we still see so many studies on treatments that are not effective? As we will see, the answers to these questions vary from person to person.
Supporters of alternative medicine find it difficult to accept that a positive clinical trial is invalidated simply because it has methodological weaknesses. Even downright negative studies do not convince them. On the contrary, they will point out that the study was poorly done, that the treatment was poorly applied, that the therapists were poorly trained, that the patients were poorly chosen, etc. Because there are dozens of variations of a given alternative therapy and hundreds of conditions for which it purports to work, it is virtually impossible to persuade enthusiasts that their beloved treatment is ineffective. Even an overwhelming amount of evidence will therefore be brushed aside.
Waste of resources
At the other extreme are the intransigent scientists for whom alternative medicine is a laughable aberration. They tend to emphasize basic science and not clinical trials. They thus emphasize that, for many alternative therapies, there is no biologically plausible mode of action. Homeopaths, for example, defy science when they claim that their dilutions lacking a single active molecule can affect our health, and acupuncturists fall victim to wishful thinking when they believe that sticking a needle into our body rebalances the two imaginary vital forces, yin and yang, believed to determine our health.
Most scientists believe that any claim that goes against science should not be subjected to scientific testing, as this would not only be a waste of resources, but would also give undeserved credibility to quackery. In other words, they argue that testing implausible hypotheses cannot lead to effective therapies.
Proponents of evidence-based medicine often find themselves somewhere between these two attitudes. On the one hand, they point out that rejecting anything that lacks plausibility might cause us to miss the weird needle in the haystack. On the other hand, they warn that the belief of enthusiasts is not a reliable guide to progress. They therefore advocate a critical evaluation of existing evidence. For acupuncture or homeopathy, for example, we now have hundreds of clinical trials. Of course, many of them have serious flaws. Therefore we should not accept their conclusions without subjecting them to critical evaluation. Defenders of evidence-based medicine therefore believe that it is necessary, firstly, to retrieve all available clinical trials and, secondly, to examine them carefully. This methodology – which they call systematic review – should allow us to get as close as possible to the truth.
In theory, this approach seems reasonable, but there is no denying that it also has pitfalls. Chief among them is that researchers can never be completely free of bias. This is especially true for those working in the field of alternative medicine. On my blog, I created what I satirically call “The alternative medicine hall of fame” [NDLR : le temple de la renommée de la médecine alternative] for researchers who, throughout their professional life, manage to publish only positive results for their preferred therapy. It seems obvious that such success is only possible with an almost total lack of objectivity.
In China, studies still positive on acupuncture
Probably the most extreme example of this is acupuncture research in China. Several groups have independently shown that virtually all trials in this country with this therapy show positive results. In other words, either acupuncture cures all of humanity’s ills, or something is seriously wrong with this body of research. The first hypothesis seems impossible, while the second is supported by the fact that around 80% of search results from China are fabricated.
If these researchers then publish a systematic analysis of their therapy – as is often the case – one cannot expect an impartial result. In other words, even systematic reviews should be taken with a grain of salt. My advice is to examine them critically, learn more about the authors, the journal and the sponsors of the research.
As we have seen, the answers to the question of when we can assume that a therapy is ineffective and when it is absurd to continue funding new research are very different depending on who is being asked. we address. Enthusiasts will invariably want the research to continue, even if it is hopelessly useless. Scientists want to immediately ban funding for improbable research. As for supporters of evidence-based medicine, they want reliable systematic analyzes to be carried out to inform their decisions.
The crux of the matter is, of course, that science is not a good tool for proving a negative answer. In fact, there is no ideal tool for this. You can, for example, do as much research as you want, but you will never be able to prove that UFOs or ghosts don’t exist. In the case of alternative therapies such as acupuncture or homeopathy, the verdict comes down to an informed judgment by a panel of people who are both competent and independent.
In a way, this also answers the question of why we continue to fund studies of treatments that clearly aren’t effective: panels of people who are both knowledgeable and independent are rare. In my experience, they outnumber self-appointed, misinformed, and biased expert groups. If I’m right, this means that in the future we will continue to see more research into alternative therapies that are, in fact, useless. Worse still, this useless research will often be carried out by staunch supporters of these treatments, funded by incompetent funding agencies. The end result is that the public will be increasingly misled about the value of alternative medicine.
And unfortunately, there is no easy solution to this dilemma. Perhaps we should ensure that before embarking on a research project, a person is adequately trained not only in methodology, but also in research ethics. And perhaps we should prohibit those who have a habit of publishing one implausible result after another from continuing to do research.
.