SPH receives $2.5 million to address research reproducibility crisis

Biomedical research is facing a reproducibility crisis, with major scientific journals estimating more than 80% of published biomedical research is negatively impacted by various defects causing results that are non-reproducible and useless.

Funded by a new $2.5 million R01 grant from the National Institutes of Health, a study out of the School of Public Health at Augusta University is aimed at investigating and improving this pressing issue by identifying best research practices and pitfalls that scientists unknowingly fall into when conducting research that lead to non-reproducible, misleading or informative conclusions.

“It’s a very timely topic because two or three years ago, it was not very commonly considered or recognized that there is this huge problem with wasted research that is either flawed or defective,” said E. Andrew Balas, MD, PhD, a professor in the Department of Health Management, Economics, and Policy within SPH and principal investigator of the study.

Balas has been conducting research on research for more than a decade, and the grant from NIH’s National Institute of General Medical Sciences will allow his team to create a database consisting of prevalent research defects and develop prevention measures that researchers can use to ensure their studies are accurate and reproducible. 

“Research quality is not only a matter of detecting errors after they occur,” Balas noted. “The critical question is, how can we understand the conditions that produce research defects and develop effective ways to prevent them before they compromise scientific results?”  

Every research project is unique in some way, which makes them more vulnerable to errors. For instance, most clinical trials on Alzheimer’s disease treatments are not successful, primarily because preclinical research is not sufficiently accurate. This upcoming study should help reduce the frequency of those errors. 

Flawed research being published can result in lasting negative impacts: ineffective patient care, funding concerns and diminishing public trust. 

“Unreliable research results can also send other researchers in the wrong direction. Years of work and substantial funding may be invested in studies based on results that were never dependable,” said Wendy Burnett, PhD, an assistant professor in the School of Public Health and a co-investigator on the study.

Currently, there are no comprehensive collections of errors in research, although the Biomedical Research Innovation Laboratory led by Balas has already developed a significant collection of hundreds. He plans on analyzing which phases of research are more prone to have defects and determining where more attention should be paid and then uploading these findings into the created database. 

Burnett also wants to help create a website where researchers can input information about their methodology in a study, and the database will then identify possible errors in the study before they complete and publish it.

“The idea is similar to websites that help researchers find journals or funding opportunities. I would love to make this available to everyone, from students working on their first research projects to experienced researchers,” she said. 

Melissa Powell, PhD, a professor in the Department of Social Sciences in the Katherine Reese Pamplin College of Arts, Humanities, and Social Sciences at Augusta University, is also serving as a co-investigator on the study. Her expertise in qualitative research methodologies will contribute to understanding the perspectives and lived experiences of key contributors to research in a nationwide sample. 

She will implement focus groups and conduct surveys on basic science and clinical researchers, peer reviewers and journal editors, which will then help the team better understand the “why” behind the decisions researchers make. 

“It would be difficult to identify the sources and risks of research defects, much less understand their implications for the validity and usefulness of research findings, without incorporating the perspectives and lived experiences of those who conduct, review and publish scientific research,” Powell said. “This data will help us better understand the factors that contribute to research defects, identify barriers that may prevent researchers from recognizing or addressing them and explore potential solutions that could be implemented to reduce their occurrence.” 

Along that same vein, Powell emphasized the fact that “research is created by human actors who are shaped not only by the technologies and theoretical assumptions of their time, but also by their own biases, limitations, and weaknesses, often while relying on training and practices that may themselves be incomplete or outdated. For this reason, it is important that we critically evaluate ourselves, our peers, disciplines, and the research process itself.” 

The interdisciplinary project also involves collaborators from the Jacobs School of Medicine and Biomedical Sciences at the University at Buffalo. A group led by Peter Elkin, PhD, professor and chair of the Department of Biomedical Informatics, will develop a framework for identifying research quality issues and conduct a large-scale study of peer review across multiple journals using artificial intelligence and data analysis tools.

With scientific and technological advancements constantly being developed, methodological errors will continue to pop up far after the completion of this study. The project team has already started to develop ideas for creating a long-term, self-sustaining mechanism for the database to include and advise against research errors as they arise. 

“Dr. Balas continues to do path-breaking work related to the “science of science.” This project has the potential to strengthen the credibility and impact of biomedical research by helping ensure that scientific findings are reliable, informative and worthy of public trust,” said SPH Dean Teresa Waters, PhD.

After the project is complete, Balas and his team hope the results will ultimately impact science and society in many significant ways.

“Research has made wonders in our lives – we have life-saving drugs, we have a lifestyle that is healthier than ever before and so forth. If research could produce more valuable, useful, trustworthy, reproducible results, then we will all be better, and it should also increase public trust in research.”