DREAM Big

Ensuring patient adherence to medication is an ongoing healthcare problem – and it is exacerbated by drug-related side effects; discontinued use of docetaxel for metastatic castration-resistant prostate cancer (mCRPC) is just one example. Could the problem be improved by predicting adverse treatment events? The Dialogue for Reverse Engineering Assessment and Methodology (DREAM) Challenge was established to encourage wide-scale collaboration to tackle the problem. To learn more, we speak with James Costello, Assistant Professor in the Department of Pharmacology at the University of Colorado Anschutz Medical Campus, who has been involved with DREAM for over 7 years.

What was the motivation behind the DREAM Prostate Cancer Challenge?

The aim of this Challenge was to evaluate computational models in an unbiased and rigorous manner, and to bring together data scientists, clinicians, biomedical researchers, as well as industry, non-profit, and academic partners to focus on addressing challenges in prostate cancer.

Based on a long discussion with a scientific advisory committee and non-profits Sage Bionetworks and Project Data Sphere, we decided that open-sharing and mining of clinical trials data presented an important challenge in the field. Open-sharing of clinical trial data is a challenge in and of itself, and one that Project Data Sphere is addressing head on. Their efforts allowed us to develop research questions that could be addressed with the data they acquired, cleaned, and normalized.

The Challenge itself consisted of two questions: Can we develop better models for predicting overall survival for patients with mCRPC? And can we develop models to predict which patients with mCRPC will discontinue treatment of docetaxel due to adverse treatment events? Both questions were motivated by our goal to use clinical trial data to develop better computational models that can help clinicians to make more informed decisions resulting in better patient outcomes.

To what extent is discontinuation a problem?

The four clinical trial datasets we had access to through Project Data Sphere showed that between 10-20 percent of mCRPC patients discontinued their docetaxel treatment within three months of starting treatment, specifically because of adverse effects, which is a problem for several reasons. First, if the treating physician knew a patient would respond so negatively to Docetaxel, they could consider an alternative strategy. Second, if the treating physician had some method to determine the likelihood of adverse events, then the physician could change how closely the patient is monitored. And third, clinical trials consume many resources, so if they could be designed with consideration to likelihood of treatment discontinuation, then it should be possible to design more effective clinical trials. While we focused our efforts on one specific drug treatment in patients with mCRPC, this is a general problem that has the potential to affect any patient treatment plan.

What was the Prostate DREAM Challenge outcome?

The Challenge results and the post-challenge analysis showed that the current set of models do not provide sufficient predictive accuracy to be used in the clinic (1). There are several reasons for this, but a major one is that even though this was the biggest study to address the question of patient discontinuation due to adverse events in mCRPC, we still need more data. Working with Project Data Sphere will hopefully allow us to increase the number of clinical trials that we can leverage.

In the future, we hope to expand our analysis past mCRPC by using the data for nearly 100,000 patients that have been acquired by Project Data Sphere. Ultimately, we hope that our work makes it to the clinic, where these kinds of models could be used by clinicians to improve patient care.

How important was collaboration in this process?

One of the key outcomes of this Challenge has to be the amazing community effort that the individual teams showed. There was no previous track record between any of the groups that co-published the results of the Challenge (1), but when asked if they wanted to work together in a post-challenge effort, there was agreement. There are always going to be some pains with working with a diverse group of individuals – including logistical factors; scheduling a time when people from Washington state and Finland can be on the same call, for example. Other complications included getting everyone on the same page simply in terms of terminology – someone in machine learning uses different words than a medical oncologist, and different fields have different research practices. Despite these initial pains, I must say that the post-challenge community phase was extremely rewarding. It was very gratifying to see that no matter what training someone had or from which country you were from, we shared the common goals of doing great research and improving patient lives.

And the DREAM Challenge model is also being applied to other drugs...

Absolutely! There have been DREAM Challenges both completed and newly launched that aim to address issues of treatment response and developing drug combinations that would be most effective to patients. The DREAM Challenges present an alternative modality of research compared to more traditional, individual lab/research group models. Challenges complement the research that is currently being done and have major advantages under certain conditions, where the data is large, complex, and the question presents both an interesting biomedical and data science challenge.

All past and current DREAM Challenges can be found here.