When is a Negative a Positive?

If one browses through the preclinical cancer research literature, past and present, it becomes apparent that thousands of papers have been published reporting highly encouraging therapeutic outcomes of new drugs in mice. In striking contrast, the majority of randomized phase III oncology clinical trials (over 60 percent) are negative (1). Such clinical failures add to the enormous cost of the cancer drugs that do make it to market. Not to mention that they also highlight how huge numbers of patients enrolled in trials end up being treated with ineffective therapies. What is the source of this discrepancy? How can mouse cancer therapy models be improved to minimize it? Moreover, what are the implications for publication when using such improved models?

There are numerous proposed reasons for the failure of mouse studies to more faithfully predict clinical outcomes. Only three will be mentioned here, not only because they are rarely considered, but also because of the financial and publication problems they pose. First, there is the issue of sample size and statistical power. Typically, groups of 6–10 mice (or fewer) are used in most studies, resulting in a high risk of false positives. Second, there is the problem of age. The vast majority of mouse therapy studies involve treatment of mice that are around 2–4 months old – the clinical equivalent of pediatric oncology patients. The way much older adult cancer patients metabolize and handle drugs can be vastly different to their pediatric counterparts. Yet using mice that are over one year of age is rare – and prohibitively expensive. Third, most mouse therapy studies involve treatment of localized primary tumors. In contrast, the therapeutic target in phase I, II, and most phase III trials is distant, difficult-to-treat metastatic disease in sites such as the brain, liver, lungs, and bone. Replicating treatment of late-stage disease in mice is much more complex, cumbersome, and expensive. Researchers don’t like to do it.

So let’s suppose that we consider dealing with all three of these problems in an experimental design. For example, we might start with a large number of older mice (12–15 per group) and direct treatment towards established visceral metastatic disease, using clinically relevant endpoints, such as progression-free or overall survival. Three serious problems would emerge. First, the financial cost would be daunting – what funding agency would support such studies? Second, the therapy outcomes, in most cases, would likely be negative or somewhat disappointing. If so, how do we ask graduate students or postdoctoral fellows to undertake such studies? The second point also drives the third problem: how do you get “negative” results, no matter how rigorous they are, to be published in respected journals?

Considering the cost of a single failed randomized phase III oncology trial (usually over $100 million) we must not let significantly increased preclinical costs deter enthusiasm for such a rigorous approach. Regarding the second and third problems, a change in attitude by editors and reviewers about publishing “negative” results based on excellent preclinical studies, would be a good start.

We all need to accept that the high rates of failure in oncology drug development are the norm and ought to be reflected in more realistic preclinical studies, which will make the “positive” exceptions all the more promising.