Is Avoidable Experiment Expenditure Draining Billions from COVID-19 Research?

August 26 2020

The rise of COVID-19 represents an unprecedented situation for pharma. With the scramble to find a vaccine, R&D productivity is more important than ever, which makes it especially concerning that pharma still struggles with a widely recognized productivity crisis. 

Over the last decade, pharma R&D costs steadily increased, while drug approval rates stagnated. In 2018, it cost ~$2.17 billion to bring a new drug to market, almost double the ~$1.18 billion recorded in 2010. In the same period, annual forecast peak sales were halved, from ~$816 million to ~$407 million. The result was a decrease in returns from 10.1% to 1.9%.

Perhaps even more troubling, given the threat COVID-19 poses to public health, research is also taking longer. Average project lengths increased from 9.7 years in the 1990s to 10-15 years through the 2000s and 2010s. Industry experts commonly blame clinical trial inefficiencies and the industry’s shift into more high-risk, high-reward research areas.

There is, however, another key contributor that has long gone overlooked. Avoidable Experiment Expenditure (AEE) significantly contributes to increased project lengths and decreased ROI. AEE refers to all inefficiencies and productivity challenges in designing and carrying out experiments. Experiments are the lifeblood of preclinical R&D, yet irreproducibility rates exceed 50%, costing life science organizations ~$48 billion annually worldwide.

Given the tens of billions being invested in COVID-19 research, there is increased urgency to address AEE. Up to half of the global investment may be wasted, significantly delaying vaccines and treatments. 

Inappropriate reagents are responsible for 36% of all unproductive experiments

A prime example of AEE is hydroxychloroquine, which was one of several candidates rushed to clinical trials. Hydroxychloroquine was thought to hold promise as a treatment but has since been found ineffective and potentially fatal.  Initial studies were conducted in a monkey kidney cell line but not verified in representative human cells where it later failed. Worse, that study has been the basis of several other research efforts worldwide, causing further delays.

Multiple factors contribute to AEE, including inappropriate reagents, poor experimental design, unreliable or variable protocols, lack of details in reporting, and lack of transparency in internal data. However, inappropriate reagents are the most significant individual contributor, responsible for over a third of AEE, or more than $17 billion. 

Finding an appropriate reagent is challenging—experimental context must be taken into account, data for which is scattered throughout many sources. This is exacerbated by unreliability in reagent data and reagents themselves; up to 50% of reagents don’t work as intended. For context, our analyses have found over 540 reagents relevant to COVID-19. Based on the data mentioned earlier, over 270 of those reagents won’t work, but scientists may not be able to determine which ones without investing the time and resources to test and validate them. Add this to the fact reagent selection for COVID research is already especially challenging, as we still know so little about the virus.

The US alone spent nearly $5 billion on coronavirus research between February 11 and July 9, averaging approximately $1 billion per month. Over 90% of coronavirus research is preclinical, meaning the US is spending around $900 million every month on coronavirus-related preclinical R&D. Knowing over 50% of experiments are irreproducible, we can infer AEE drains American COVID research funds at a rate of $450 million per month. Further, given over 36% of AEE results from inappropriate reagents, the US may be losing over $162 million of COVID funding every month from these products alone.

Standard processes for selecting reagents are flawed and unreliable at best. However, the general perception is the exploratory nature of preclinical R&D makes it inherently inefficient. Thus, the use of inappropriate reagents is deemed an unavoidable part of the scientific process. However, reagent selection can be significantly improved with emerging technology.

Inefficiencies in preclinical R&D not only affect an organizations’ ROI. They also, especially in COVID times, impact millions of lives. Thus, we must take any steps to identify and resolve Avoidable Experiment Expenditure. We suggest the industry align on standards for quantifying the impacts of AEE, then empower scientists with information to design more successful experiments. Research is already showing better information leads to increased experimental success.

BenchSci applies machine learning to both quantify and reduce reagent-related AEE. 

Our machine learning algorithms have been trained by Ph.D. biologists to identify patterns at a scale no human could. With this help, scientists can reliably select the most appropriate reagents and increase their experimental success.

BenchSci has also leveraged its machine learning technology to analyze COVID-19 data and is freely sharing it in the hopes it will empower scientists with the best information for developing a vaccine.