Category Archives: Research

Navigating the Reproducibility Rapids to a “Learning Research System”

Reproducibility has emerged as a touchstone controversy within the scientific community, the public and the Congress over the past few years. In the wake of recent reports and headlines, science has come under intense scrutiny. But revelations about failure to reproduce experiments, alone, should not shake our trust in the legitimacy of the medical research enterprise. In fact, the attention may be a good thing. As scientists, we understand the complexities of science, but understanding among ourselves will not reassure the public and Congress. If we truly want to engender the public trust and enhance our own learning, let’s welcome this opportunity to address limitations and uncertainties and improve our processes and communication in science rather than overlooking, ignoring, suppressing or railing against the topic.

The term itself, “reproducibility,” can provoke an array of reactions – from immediate confidence (the results were reproducible and so are credible!) – to consideration of the context of the results (differences due to biological variables and applicability of the model, e.g. Were the results obtained in male humans, animals, and cells not generalizable to female species? Was the animal model simply not applicable to humans? Are there discrete differences in the population studied that masked important differences?) – to unfortunate misperceptions (science is fickle. The results must be fraudulent).

Perceptions in the public and in Congress shape policies. To that end, the NIH has just taken a big step toward ensuring reproducibility by increasing the requirements for rigor and transparency in the research that it funds.

At the 2015 GREAT Group and GRAND Professional Development Meeting, prior to the release of the new policy, NIH principal deputy director, Larry Tabak described the NIH’s proposed solutions within the policies to enhance reproducibility: 1) being accountable for adequately describing the methods or materials; 2) noting limitations in the study design; 3) considering all relevant biological variables in the scope of the research (the NIH specifically called out that sex is a biological variable and focusing studies on male animals and cells may obscure the importance of sex on biological processes and responses to interventions); 4) authenticating key biological and chemical resources; and 5) bearing in mind implicit biases in how the results are interpreted or disseminated. Thankfully, from the evidence available, fraud or misrepresentation of findings accounts for only a small percentage of the problems identified, and the scientific community and federal sponsors have effective means to address misconduct.

Navigating the reproducibility rapids goes beyond this policy. There are additional steps the entire research community can take to advance rigor and transparency; and that is by promoting and sharing negative findings as a rule. Beyond enhancing a sense of openness, there may be vital scientific knowledge residing in those negative findings – a legitimate source of “irreproducibility” that could help dissect when and for whom certain interventions may be promising and when and for whom they may not. And, isn’t that a clear signpost for building public trust?

This will take all of us. Sharing data (both negative and positive) would need to be a consideration in merits and promotions, welcomed for publication in respected journals, and established in trusted infrastructures and venues where data sharing is not a prohibitive time and resource burden for the scientists who do the work, or a concern for not protecting privacy in human studies.

In the end, navigating the reproducibility rapids may lead us not only to public confidence and supportive policies, but also to a “learning research system.”

This post originally appeared on


Dr. Bonham picture

Ann Bonham, PhD, is chief scientific officer at the Association of American Medical Colleges. She can be reached at





Investing in Medical Research: Resource-Intensive, Long-Term, and Worth It

By Ann Bonham, PhD

As Congress heads into summer recess, we will continue to see some heady days for medical research. The 21st Century Cures Act has provided a light at the end of a long tunnel of stagnant growth in NIH funding. While not a miracle “cure,” the bill is a good start to reversing the trend of stagnant federal support for medical research — setting aside $1.75 billion a year, for five years, for an NIH innovation fund. (See bill’s text here.) Continue reading Investing in Medical Research: Resource-Intensive, Long-Term, and Worth It

Making Research Careers Sound Cool, Jimmy Kimmel-Style

By Samantha Ngooi, MPP, and Vineet Arora, MD, MAPP

When we got an NIH grant last year to ascertain how teens could get their peers interested in research careers, we did not anticipate turning to Jimmy Kimmel for inspiration.

Kimmel’s “Lie Witness News” segment is notorious for asking unsuspecting pedestrians to share their opinions about ridiculous topics from a new “scented” iPhone to the “appointment” of Judge Judy to the U.S. Supreme Court. While the interviews are purely for amusement, at times they reveal an embarrassing lack of knowledge among members of the public, adding weight to the saying, “Don’t ask questions you don’t really want the answer to.” Continue reading Making Research Careers Sound Cool, Jimmy Kimmel-Style

Starting with Cells: Rethinking Science as a Place for Women

By Ann C. Bonham, PhD and Diana Lautenberger, MA

The recent Science article on stereotypes about innate genius and the impact it has for women in science fields is the latest in a series of discouraging reports. It brings to mind the 2012 Yale study published in PNAS, which concluded that both male and female scientists regard female undergraduates as less competent than male students with the same accomplishments and skills. Why are we still grappling with the differential assignment of competence by gender? What will it take to upend the stereotypes about the capacities of women that remain in our cultural subconscious? It’s not that women don’t have an innate talent for science: Women working on important discoveries challenge this notion every day. One answer may be that our cultural blinders inhibit us from seeing women as authorities, and this lack of visibility for women in science fields perpetuates a cycle of cultural stereotypes.

Continue reading Starting with Cells: Rethinking Science as a Place for Women

Responding to the Need for Innovation in GME

By Jennifer Kesselheim, MD, MEd & Ayres Heller, MEd

One of the most exciting developments in medical education happened last summer when the Institute of Medicine (IOM) released a new report, “Graduate Medical Education That Meets the Nation’s Health Needs.” The report was requested in 2012 by the Josiah Macy Jr. Foundation and supported by 11 other private foundations as well as 11 United States Senators. The Report Brief outlines the IOM’s intriguing conclusions about the governance and financing of Graduate Medical Education (GME).  The recommendations highlight the immense need for innovation in the way Graduate Medical Education is structured, distributed, designed, and implemented. The report suggests that a portion of the Medicare GME fund be designated as a “Transformation Fund,” intended to finance innovation and experimentation in GME.

Continue reading Responding to the Need for Innovation in GME

Scholarship in Medical School: Skating to Where the Puck Will Be

By Rachel K. Wolfson, MD, and Vineet M. Arora, MD, MAPP

When we were in medical school, medical students could opt to participate in traditional research at some point, generally the summer between the first two years of medical school, or during the fourth year. There were no required scholarly projects, semi-annual progress reports, or specific concentrations or tracks for students to choose. Some of our classmates took an extra year to do research, but there was no year-off forum, quarterly newsletter announcing student dissemination of scholarly work, or faculty with protected time to promote student research.

Times have changed. We co-direct the University of Chicago Pritzker School of Medicine Scholarship and Discovery program, in which students can pursue scholarly work in a broad array of areas, including non-traditional research such as global health, medical education, community health, and quality and safety. We are not alone…

Continue reading Scholarship in Medical School: Skating to Where the Puck Will Be

Interprofessional Education Should Include Researchers Too

By Michael J. Friedlander, PhD

The article in the most recent issue of the AAMC’s Analysis in Brief, “Interprofessional Educational Opportunities and Medical Students’ Understanding of the Collaborative Care of Patients,” by Drs. Grbic, Caulfield, and Matthew, provides an interesting and informative look at interprofessional education for medical students interacting with many health professions.

Continue reading Interprofessional Education Should Include Researchers Too

My view: Now is the time to invest in NIH

As the senior vice president of the University of Utah Health Sciences (UUHS), I have the privilege of working with some of the top researchers in the country who have devoted their lives to making discoveries that will change others’ lives for the better. With funding from the National Institutes of Health (NIH), the nation’s primary medical research agency, leading scientists in the United States and our own state of Utah are closing in on discoveries that would save and improve countless lives. A snapshot of discoveries made by researchers at UUHS — including genetic causes for breast, ovarian and colon cancers and preventive and novel treatments for each — speaks volumes about the importance of federal funding for NIH.

This summer, University of Utah researchers identified a previously unknown but crucial component in the process of platelet generation, a discovery that could help ease serious side effects of treatment for multiple myeloma patients. At the same time, the NIH funded University of Utah Molecular Medicine and conducted studies to learn how metabolic stress, inflammation, bleeding and clotting contribute to diseases like diabetes and obesity. Findings like these are revolutionary because they translate into lives saved, enhanced and even lengthened. We are on the right track, but without proper funding for the NIH, our progress could be derailed.

Little attention has been given to the fact that NIH receives nearly 25 percent less in funding than it did in 2003, when adjusted for inflation. The immediate impact of this funding gap is the loss of funding for hundreds of promising proposals for research on cancer, Alzheimer’s and other diseases. Stagnant funding translates into stalled research, something neither Utah nor the rest of the country can afford. And as a leading academic research institution, this can translate to fewer researchers entering the profession. The U.S. has long been the global leader in medical research, but other countries are catching up quickly. If current trends continue, China will outspend the U.S. on medical research by the year 2022.

Utah’s leading research institutions and biotech companies received more than $159 million in research funding from NIH in 2013 alone, and the return on this investment has been significant and measurable. This federal investment not only furthers our scientific endeavors, but it also bolsters our economy. Utah is home to 911 bioscience businesses, and our residents held 23,406 bioscience industry jobs in 2010. From 2010-2013, University of Utah innovation has led to the formation of 75 start-up companies. The University has also issued 265 patents and executed 307 licensing agreements during those years. Without continued investment in NIH, future statistics will not be so bright.

Our lawmakers are at a crossroads when it comes to investing in the health of our country. We are on the cusp of major breakthroughs for diseases that have reached epidemic proportions in the U.S. – cancer, diabetes, heart disease, obesity and Alzheimer’s. If we do not make the investment now in finding cures and treatments, we will pay later. Consider that Alzheimer’s currently costs the U.S. $200 billion a year. Without a breakthrough treatment or cure, that cost will reach $1 trillion by the year 2050. This is a price our country cannot afford.

Now is the time to make up for the past decade of stagnant funding for NIH. As the old adage goes, “You do not have anything if you do not have your health.” Without a healthy funding investment for NIH, nothing else matters.

-Vivian S. Lee, MD, Ph.D, MBA is the senior vice president, University of Utah Health Sciences, CEO, University of Utah Health Care, and Dean, University of Utah School of Medicine.

Continue reading My view: Now is the time to invest in NIH