Studies on using the previously existing medications in COVID treatment yielded inconsistent results. Researchers from the Jagiellonian University have indicated that these discrepancies could have been caused by random differences between treatment and control groups and improper statistical analysis, instead of errors in research practice.
In an article published in the journal Studies in History and Philosophy of Science scientists from the Jagiellonian University, Dr Mariusz Maziarz and Dr Adrian Stencel also indicated that the evidence to start clinical studies had been weak and suggested putting more emphasis on negative, instead of positive, mechanistic evidence.
As stated in an announcement by the JU Interdisciplinary Centre for Ethics (INCET), the SARS-CoV-2 pandemic has been one of the greatest public health challenges in recent years and its impact has been felt by millions of people all over the world. Today, more than 2.5 years after the outbreak of the disease, there are vaccines limiting the risk of severe illness after the infection, and to some extent, also the risk of transmission. Besides, there are certain medications which contain chemical molecules specifically designed to prevent the replication of the coronavirus by causing mutations in its RNA.
Yet, in 2020 no effective drug capable of limiting the severity of the disease or the transmission of the virus was available. As the development of new forms of treatment is very time consuming, testing the possible effectiveness of the already existing medications seemed the most promising course of action.
As explained by Dr Mariusz Maziarz from the JU Interdisciplinary Centre for Ethics, attempts to change the purpose of the drug usually begin from understanding how a given substance can influence the virus replication process.
Such studies usually consist in infecting cell cultures with a given virus and then treating them with specific chemical substances which can potentially reduce their replication. Only after positive results of such laboratory tests, the drug can be tested on animal models, and then – if the results are satisfactory – on patients in clinical environment.
Due to the pandemic situation and the urgent need to find a cure for COVID-19, clinical trials started right after positive results of laboratory trials, or based only on the theoretical analysis of the mechanisms of virus replication and medication effect.
The haste also hindered the coordination of clinical trials, resulting in such situations as a dozen research teams simultaneously tasting the same substance and obtaining inconsistent or even contradictory results, with some indicating that the drug reduces the mortality or infection time and others that it has no impact on COVID-19 whatsoever.
According to Dr Maziarz, the initial publication of positive results followed by research reports that contradicted them led, for instance, to the initial permission of the US Food and Drug Administration (FDA) first permitting to treat COVID patients with hydroxychloroquine and then revoking the permission.
Remdesivir followed a similar path. As stressed by the author of the article, this drug (initially developed as a medication against Ebola virus) was also administered to intensive care COVID patients in Poland, but despite the initially promising results, the treatment, costing over 2 thousand euros per person, was found to be ineffective by later research.
The discrepancies in the results have so far been explained by flawed research methods (e.g. no randomisation), calculation/statistical errors or, questionable research practices in the some studies that apparently proved the effectiveness of the tested substance. But the analysis conducted by researchers from the JU Interdisciplinary Centre of Ethics and the JU Institute of Philosophy indicated that among studies that yielded positive as well as negative results some used correct methodology, while others involved flawed or even possibly fraudulent research practices.
According to Dr Mariusz Maziarz, it should be stressed that several dozen falsely positive results (i.e. reporting a statistically significant difference between the study group and the control group, despite actual inefficiency of the drug) should have been expected because of the sheer number of studies carried out.
The researcher also adds that under normal circumstances, new medications become available after positive results from two large clinical trials. But in the case of COVID-19, the lack of other treatment options led to the drugs being accepted for use based on positive results from just one study.
‘In such a case, researchers and agencies deciding on the introduction of drugs to the market (such as the FDA or the EMA – European Medicines Agency) should statistically control the probability of falsely positive results by lowering the statistical significance threshold. Such a solution was not used, because it would increase the possibility of obtaining falsely negative results, which would mean overlooking an effective treatment against COVID’, explains the scientist.
The solution proposed by the JU researchers is based on an entirely different approach to theoretical reasoning and laboratory research, trying to understand why a given medication cannot be effective (negative mechanistic evidence).
So far, a major focus was put on positive mechanistic evidence: a drug was considered promising if it blocked the replication of SARS-COV-2 in laboratory environment. The problem with such an approach, Dr Maziarz argues, was that the drug’s mechanism of action in vivo (i.e. in the human body) may be entirely different from how it works under in vitro conditions, due to the multitude of potential interactions in the former case.
Hence, the definitely better approach proposed by the authors of the abovementioned article consists in searching for mechanisms that could indicate that a given medication would not work in vivo, instead of discovering more and more substances that would work in controlled laboratory environment. Hence, the focus is put on negative, instead of positive, mechanistic evidence.
The authors of the study claim that by adopting such an approach a lot of unnecessary clinical trials, which offered false hopes to many, could have been avoided.
For instance, the initial tests of hydroxychloroquine were conducted on the so-called Vero cells. The later tests showed that the viral infection mechanism in these cells significantly differs from the way in which the virus infects human lungs. Hence, the choice of a better suited model could have produced negative mechanistic evidence, preventing unnecessary clinical trials.
‘Therefore, scientists need to do their best to find negative mechanistic evidence, and only in the absence of these, clinical trials may begin as a result of positive evidence’, claim the authors of the paper.
‘The philosophical principle of Ockham’s razor states that “entities should not be multiplied beyond necessity”. When applied to clinical studies, this means that we should not assume that a drug is effective in more systems than it has been proved, especially in the case of much more complicated systems. Focusing on negative mechanistic evidence should help scientists follow this principle’, says the co-author of the paper, Dr Adrian Stencel from the JU Institute of Philosophy.
