A Lesson In Bias During Plant-Based Interventions

From protecting animal welfare to promoting health and fighting climate change, the benefits of plant-based eating are well-established. However, research on the most effective interventions to reduce meat and animal product (MAP) consumption remains incomplete. Many studies have explored plant-based messaging and behavioral nudges to encourage dietary shifts, but these studies usually measure participants’ intentions instead of their actions. They may also fail to consider things like social desirability that can cause bias in the results.

This research aimed to correct the shortcomings of previous MAP reduction studies by looking at the impact of a documentary film. The researchers chose a film produced by The Humane League, which followed best practices in designing behavioral interventions. Specifically, it provided educational information, it aligned MAP reduction with existing social norms, and it offered concrete examples to help viewers shift away from MAPs. It also incorporated meat-animal reminders and messaging to invoke disgust, two important aspects of MAP reduction psychology.

The research took place in three studies. Study 1 asked viewers to watch the documentary and measured their MAP consumption two weeks later, compared to a control group. It resembled many previous studies on MAP interventions, but with changes to prevent bias. For example, the study asked viewers to report their actual MAP consumption, not their intended consumption, and the researchers hid the study’s purpose to eliminate social desirability bias. Study 2 removed these precautions, while Study 3 enhanced the film to further encourage dietary change and focused on a receptive target audience from Study 1: people with at least two years of college education who identify as Democrats.

After the first study, the researchers found no significant difference in the self-reported MAP consumption between those who watched the documentary and the control group. Likewise, in Study 3, there was no meaningful reduction in MAP reduction among those who watched the documentary, even when the film included more resources to support dietary change.

However, Study 2 showed a different result. When the researchers removed precautions from the study meant to prevent things like social desirability bias, 68% of people who watched the documentary expressed an intention to reduce their MAP consumption, compared to 20% in the control group. This was a statistically significant difference and, as the researchers note, shows a similar trend to other plant-based intervention research. 

What can we interpret from these findings? The first and third studies, with as much bias removed as possible, found that the documentary was not an effective dietary intervention, except among the most attentive participants. The second study, with these biases reintroduced, yielded more positive findings when studying intended consumption versus actual consumption. The researchers conclude that the biases common in these types of intervention studies has likely led to an overestimation of their effectiveness.

The results may seem disheartening, but for animal advocates, the study is a helpful lesson.

When engaging in veg*n advocacy, we need to be cautious about our audience telling us what they think we want to hear. Likewise, when looking at plant-based interventions and measuring our own campaigns, it’s important to look at actual animal product reduction, not just intended reduction. In other words, just because someone tells us they plan to reduce meat from their diet, that doesn’t mean they’ll do it.

It’s also important to keep in mind that this study is focused on one specific film, and its results may not apply to other films or the many other interventions out there. In fact, previous Faunalytics research has found that 37% of animal advocates in the U.S. and Canada became involved in the movement after engaging with some form of animal welfare media. Regardless, this study reminds us that advocates must continue finding new and innovative ways to help animals while also being rigorous in how we measure our existing campaigns.