John Oliver Segment Addresses the Shortcomings of Scientific Research Methods and Reporting

In a recent segment of his show “Last Week Tonight,” comedian John Oliver took on the issue of research and scientific studies. He satirized the outlandish reporting of dubious research, such as news stories alleging that a glass of wine has the same health benefits as an hour at the gym.

While these claims are blown out of proportion by sensationalist media outlets, they can also stem from poor research methods in scientific studies. Journals are more likely to publish new, interesting, and statistically significant work, incentivizing researchers to unfortunately neglect sound research practices or disregard principles of transparency and replicability. Other researchers – often without ill intentions – fall into the dangerous trap of “p-hacking,” or data fishing for patterns that are statistically significant but practically meaningless.

Here’s an example of “p-hacking” from xkcd.

These poor research methods can create evidence about nearly anything, reducing faith in scientific discoveries. A reporter on the Today show summed up the problem of scientific studies, saying “the way to live your life is to find the study that sounds best to you, and you go with that.”

J-PAL North America promotes proper transparency and reproducibility as well as sound research design. When conducted with proper care, randomized evaluations can be used to reliably quantify the impact of social programs. J-PAL North America also works to help policymakers and donors interpret social science research correctly and synthesize lessons across different studies.

Watch the full John Oliver segment below:

Author: Carolyn Ruh

Carolyn Ruh is an Undergraduate Policy Intern at J-PAL North America. As a Policy Intern, Carolyn helps write summaries of research articles and updates the J-PAL North America blog. Carolyn is currently studying economics at Boston College with a minor in Faith, Peace and Justice. She has previously worked with ACCION East, a nonprofit in Cambridge that provides small business loans throughout the United States.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s