This story highlights the shortcomings of the scientific publication system. At a time when studies are unveiled on social networks, collaborative reviewing must be strengthened and expanded.
Made in cooperation with our partners at esanum.fr and esanum.it. You can read the original interview (in French) here.
Dr. André Gillibert is a physician working in public health and biostatistics at the University Hospital of Rouen (a town in the north of France). He is part of a group of French physicians and researchers who, since 2018, have fought to obtain the retraction of two studies that emphasize the effectiveness of Artemisia annua and Artemisia afra against bilharzia (or schistosomiasis) and malaria. They were successful in August 2020.
esanum.fr: How did you get involved in this story?
Dr. André Gillibert: I am a public health physician at the Rouen University Hospital (Rouen’s Centre Hospitalier Universitaire or CHU), and a specialist in the statistical analysis of medical data. I help colleagues write research protocols. A CHU intern contacted me in 2018 to work on her thesis. Lucile Cornet-Vernet, the founder of the Maison de l'Artemisia, had proposed that she take part in the design of a randomized clinical trial on a treatment for malaria in infants using a trans-rectal artemisia-based treatment.
This was in October 2018, just before the publication in the journal Phytomedicine of a study on artemisia annua tea as a treatment for bilharzia. A pre-publication version of this study was sent to me by Lucile Cornet-Vernet. I noticed aberrant results. Seeing, in the electronic exchanges with Lucile Cornet-Vernet, that the infectologist Xavier Argemi was also questioning their validity, I contacted him personally.
We began to dig into the subject, with Xavier and other infectologists and methodologists, such as Jordi Landier, Nicolas Meyer, Florian Naudet, and others. We are a small informal group of physicians and methodologists, without academic links but with a common cause. Of course, I immediately stopped getting involved in the infant study project, after also dismissing my intern colleague.
esanum.fr: What was wrong with the data?
Dr. André Gillibert: There were in fact two different articles from two clinical trials, conducted in 2015 in the Democratic Republic of Congo with the help of Dr. Munyangi and the support of the Maison de l'Artemisia. One on bilharziasis1, and another on Plasmodium falciparum malaria2. However, Lucile Cornet-Vernet only mentions one large clinical study in her book, which would have been related to malaria and would have been concealed from the ethics committee under the label of a study on bilharziasis in order not to arouse “big pharma”'s suspicions. However, as patients with bilharzia and malaria are distinct, it was necessarily two independent studies that led to these two articles. Lucile-Cornet Vernet's book only presents the results concerning malaria.
For the article published on bilharziasis, the data did not show the variance of living organisms to which all biostatisticians are accustomed. It is a basic concept in biostatistics: we do not all have the same height, the same weight, etc. However, among the 400 patients suffering from bilharziasis and who were following the reference treatment, absolutely all of them recovered very late, precisely between D21 and D28. This is very astonishing. First, because this treatment is normally effective within a few days. Then because the patients were synchronous: out of 400, none healed before D21, and all of them, 400 out of 400, healed between D21 and D28. Same in the group treated with artemisia tea: all healed synchronously between D4 and D7.
Another aberration, after the exclusion of the empty categories, 17 out of 21 categories of adverse effects concerned multiples of five patients: 130 headaches, 175 abdominal pains, etc. According to the binomial law, the probability of this happening by chance is less than four chances in a billion. This phenomenon was observed in both groups, even if in the "herbal tea" group there were practically no undesirable effects [Author's note: Florian Naudet presented and commented on these data on twitter]. This too is odd. Usually, even when a placebo is given, there are many mild side effects corresponding to common symptoms such as abdominal pain, fatigue, or loss of appetite.
After a request for re-analysis, the authors provided me with the original databases of the two studies; they confirmed my suspicions. Both databases had extensive "copy and paste". For example, in the malaria study, up to 100 patients could have exactly the same parasitemia (number of parasites per microlitre of blood) at D0. Even more surprisingly, dozens of patients had exactly the same parasitemia at D0 (e.g. 50,654) as well as at D1 (e.g. 8,930) and D2 (e.g. 5,100) and so on for all measurements up to D28. The probability of this happening by chance is absolutely negligible. It is just impossible.
esanum.fr: Incompetence or fraud?
Dr. André Gillibert: There are a lot of grey areas in this story. But what I am sure of is that the pro-artemisia discourse towards us has hardened over time, even reaching attempts at intimidation. Initially, the data was kindly provided to me by the American biochemist who was participating in the study, Pamela Weathers. So she was probably bona fide. Later she explained that a physician student working in mathematics from her university, Ms. Lu Chen, had helped her with the statistical analysis.
The pro-artemisia discourse at first was: "These are roundings, imperfections, it does not question the validity of the studies." As justification, a succession of unverifiable facts was evoked by Lucile Cornet-Vernet or Pamela Weathers. For example, Pamela Weathers stated that Jérôme Munyangi had to recount the parasites on the slides because his laptop had been stolen. According to Lucile Cornet-Vernet, by 2015, a zone chief under the pay of the pharmaceutical company Shalima had already tried to obstruct the therapeutic trial and then the return of the results to the local population. This pressure on researchers would have harmed the quality of the data. According to her, Dr. Munyangui has also been the target of two assassination attempts. The press3 also reports imprisonment for three days and torture. For the pro-artemisia, these are all arguments to support the conspiracy theory.
What is clear is that the stakes are high for the Maison de l'Artemisia, a non-governmental organization that promotes the local culture of artemisia use as a weapon against malaria. If it doesn't work, it no longer has a reason to exist. Its conflict of interest is ultimately stronger than that of a lab. But the association was able to count on the support of Prof. Christian Perronne, co-signatory of the two studies. This was enough to ensure strong media coverage of this beautiful story of a "miracle" treatment, all the more so as his academic title serves as a scientific guarantee. Finally, the reaction time of the editor-in-chief of the journal Phytomedicine had extended the duration of the scientific endorsement that these two publications gave to artemisia.
esanum.fr: What happened with the journal Phytomedicine?
Dr. André Gillibert: It is a journal with a scientific editorial board, peer-reviewed articles, and a relatively high impact factor. It is classified as Category B in the SIGAPS4 system. In short, it is a fairly serious journal a priori.
In 2018, following the publication of the first study, we wrote a letter to the editor5 to list all the inconsistencies. Normally, when an editor receives this, he checks the data and retracts the study. There he published our letter and a response from the authors at the same time, completely "off the mark".
Same scenario in 2019 for the malaria study. A publication, a letter from us6, and response from the authors, but no retraction. But after receiving the original data and seeing the extent of the inconsistencies, we also found out that the ethics committee did not give its approval until 2016 when the trial dates back to 2015. So this time we explicitly asked the publisher to retract the two studies, attaching the anonymous data as evidence of the problems, on Elsevier's secure platform.
In response, the editor asked us to provide him with a complete file, providing a detailed guide allowing each of our accusations to be verified by external evaluators. At the same time, I received a formal notice from the Maison de l'Artemisia for "publishing" their data. This made it more difficult for us to communicate the complete file to the editor in October 2019. Moreover, the editor deleted our first two letters by a procedure called "temporary withdrawal" which is even worse than retraction since the content of the letter itself is removed rather than simply displayed with a "RETRACTED" watermark. After that, it was radio silence.
The studies were retracted in August 2020, probably because a lengthy article7 from Retraction Watch put pressure on Elsevier, the publisher of Phytomedicine. I didn't believe in the retraction any more. I even feared it might cause the “Galileo syndrome” [Author's note: a perception of the researchers as misunderstood and martyred genius]. But in the end, I thought it was important that a scientific consensus should clearly state: "These studies should never have been published."
esanum.fr: Why was the retraction so important?
Dr. André Gillibert: Artemisia is a well-known treatment in Africa and probably ineffective in the form of herbal tea against malaria. Only the variety of artemisia annua contains artemisinin, and according to the World Health Organisation (WHO), it should only be used as a combination therapy. It is possible that thousands, even hundreds of thousands or millions of Africans take these herbal teas to treat themselves instead of taking effective treatments. The stakes are high, even if it is difficult to quantify because the frequency of use and its consequences are not known.
Another problem, described by the WHO, is the risk of accelerating resistance to treatment. Artemisia annua contains a low concentration of artemisinin. It may not kill the parasite at this concentration but may exert selection pressure leading to the spread of artemisinin-resistant Plasmodium. Conventional treatments already generate resistance, with the appearance of strains of Plasmodium falciparum with reduced sensitivity to artemisinin: this has been proven in Asia but the phenomenon is still modest in Africa. It is likely that artemisinin will eventually cease to be effective, like chloroquine, so let's avoid speeding up the process.
esanum.fr: Are you whistleblowers?
Dr. André Gillibert: I had been reading up on whistleblower status, which could protect me legally when I received the formal notice. I think that this could qualify us, even if we limited ourselves to criticizing the scientific aspect and alerting the competent authorities. Furthermore, with Florian Naudet (a meta-researcher from Rennes, France) we continue to keep a close watch on the studies that are published, apart from artemisia.
The two of us have written another letter to the editor denouncing the methodological problems of an article on Vitamin D and COVID-19, published by Joachim Son-Forget among others. Our group also remains in contact, notably to alert the authorities to the excesses of the Maison de l'Artemisia. If we see dubious publications on artemisia and COVID-19, for example, we will mobilize.
[Author’s note: The pro-artemisia groups have rushed into the race for promoting treatments against COVID-19. In March, the Malaysian president himself launched a campaign to distribute COVID-Organics, a "cure" based on artemisia annua. Dr. Muyangui, who believes he is persecuted in his country, had applied for asylum in France in 2019. However, in May 2020 he joined the Congolese task force to fight against COVID-19, at the request of President Tshisekedi. In France, the Maison de l'Artemisia launched a crowdfunding operation to support research on an "artemisia protocol" against COVID-19].
With Florian, we think about ways to improve the quality of published studies. Even in the most prestigious journals, I believe that several studies should be retracted and they are not. What happened on The Lancet about hydroxychloroquine should make us aware of what is at stake when it comes to fraudulent studies, and the population is beginning to become aware of the stakes involved.
esanum.fr: How can the publication system be cleaned up?
Dr. André Gillibert: Beyond these media affairs, the scientific community needs a resilient social research system. Research is not just about scientific methods, it is a social organization that enables the acquisition of knowledge. The publication system is part of this, and we need to think about how to improve it. To do this, scientific journals and their editors must come out of their ivory towers.
They could call on reviewers who specialize in scientific fraud. It is not the same skill to detect methodological errors, such as those often found in studies on hydroxychloroquine, and fraud: photoshopped images, overly balanced data... On 17 September 2020, Florian Naudet and I wrote to The Lancet to suggest that they integrate an editor specialized in fraud. This would avoid them a new "LancetGate'', like the one that is emerging with the study on the Russian vaccine Sputnik V.
Peer review is still a peer review, and the editor does not necessarily take it into account. Since blind trust in peer review is impossible, one should be able to check the reliability of an article for oneself, based on critical reading. This requires a background, but above all a certain amount of self-confidence. One must dare to consider oneself more competent than the person who has peer-reviewed. A not so simple task.
I think journal editors should keep a constant watch on open reviewing sites. That would enable them to see whether their reviewers have let a blunder slip through their fingers. On PubPeer, for example, comments are often relevant. Authors are not targeted, only methodologies or results suspected of fraud. At the moment, editors of scientific journals do not seem to use them.
What we also understood is that getting a retraction is still a long and arduous fight. Especially when the editor is ambivalent. In the end, perhaps it was only thanks to the journalists at Retraction Watch that things moved. From my point of view, a quick retraction is a guarantee of the quality of the magazine. We saw it with The Lancet: the editor-in-chief could not afford to get bogged down in controversy.
Yet retraction remains a rare practice. The tragedy is that even seasoned researchers still find it difficult to say that an article is completely wrong. They will invoke methodological problems, but as the critical reading of a scientific article is subjective, as long as there is not this red "retracted" banner many readers will continue to believe it.
Resources:
1. RETRACTED: Effect of Artemisia annua and Artemisia afra tea infusions on schistosomiasis in a large clinical trial
2. RETRACTED: Artemisia annua and Artemisia afra tea infusions vs. artesunate-amodiaquine (ASAQ) in treating Plasmodium falciparum malaria in a large scale, double blind, randomized clinical trial
3. Paludisme : un chercheur congolais se réfugie en France, s’estimant inquiété pour ses travaux, Le Monde, 8 juillet 2019.
4. Système d’interrogation de gestion, d’analyse des publications scientifiques
5. Comment on “Effect of Artemisia annua and Artemisia afra tea infusions on schistosomiasis in a large clinical trial”
6. Comment on “A. annua and A. afra infusions vs. Artesunate-amodiaquine (ASAQ) in treating Plasmodium falciparum malaria in a large scale, double blind, randomized clinical trial”
7. A bitter aftertaste: Legal threats, alleged poisoning muddy the waters for a trial of a tea to treat malaria