“Reduce” Or “Go Veg”? Effects On Meal Choice

Background

Probably the biggest debate in dietary advocacy is around what to ask for. In this study, we tested two key strategies against one another: asking for meat reduction versus going vegetarian. 

The advocacy method was as realistic as possible: The videos described the diverse benefits of plant-based eating, showed some mild factory farm footage, and described changing social norms around meat consumption to convey a growing trend (Caldwell, Macdonald, & Boese, 2016; Sparkman & Walton, 2017). Participants were also asked if they would be willing to take a dietary pledge (either for meat reduction or vegetarianism). Pledges and other types of explicit commitment have been used effectively in environmental advocacy and other domains to increase behavioral follow-through (Lokhorst, Werner, Staats, van Dijk, & Gale, 2013).

On the basis of the existing research, many evidence-based advocacy groups already use norm messaging and/or pledges in their outreach. Some of the most notable examples of pledge campaigns include Veganuary, MeatOut, and Challenge 22. However, to our knowledge, this is the first rigorous test of the effectiveness of these strategies in a face-to-face advocacy context that uses actual dietary choices as the outcome.

Key Findings

1. Advocating for meat reduction led to more meatless meal purchases than advocating for vegetarianism. After a reduction advocacy video, 25.8% of participants ordered a meatless meal, versus 18.9% after a vegetarian advocacy video (a marginally significant difference). This result suggests that reduction advocacy will have more of a positive impact for animals than vegetarian (or, by extension, vegan) advocacy.
2. A large part of the reason was that almost four times as many people were willing to pledge to reduce as to pledge to cut out meat entirely. While the majority of participants in this study (59.4%) were willing to take the meat-reduction pledge, only 15.4% were willing to take the vegetarian pledge. This greater willingness to take the reduction pledge explained (mediated) most of the difference in the success between the reduction and vegetarian advocacy videos.
3. Advocates need to strengthen our reduction pledges to maximize psychological commitment as much as possible. This is obvious in this study, which used very weak pledges (for methodological reasons) and found that only 31.7% of people who took the meat-reduction pledge ordered a meatless meal after the study. On the Supplementary Materials tab, we provide a downloadable Effective Pledges fact sheet, with evidence-based recommendations about how to strengthen pledges as much as possible. These recommendations are also described in detail in the Implications and Recommendations section. (That said, because reduction can take different forms, this doesn’t mean that the other 68.3% of reduction pledgers don’t plan to follow through.)
4. There is far more evidence for the impact of stronger reduction pledges than for better vegetarian (or vegan) messaging. 47.8% of people who took the vegetarian pledge ordered a meatless meal, significantly more than the percentage for the reduction pledge. In other words, though uptake was almost four times higher with reduction advocacy (point #2), follow-through was higher with vegetarian advocacy. So why do we suggest reduction advocacy? There are three main reasons:
   • First, the bottom line of the study was that reduction advocacy led to more meatless orders than vegetarian advocacy—the difference in uptake outweighed the difference in follow-through. 
   • Second, there is strong evidence from many studies (as described on the Conclusions tab) that well-done pledges increase follow-through on a desired behavior, whereas the evidence for well-done advocacy increasing uptake (willingness to go vegetarian or vegan) is, at best, weakly positive.
   • Finally, while it is important to note that the differences are small and not statistically significant, this study found a small negative effect of vegetarian advocacy on meat consumption relative to the control condition. In this study, after watching the vegetarian advocacy video (which included some mild factory farm footage) and being asked for a vegetarian pledge, 18.9% of people ordered meatless meals, as noted above. Of people who instead saw a control (non-advocacy) video before being asked for the vegetarian pledge, 23.2% ordered meatless meals. Again, this difference was too small to be statistically significant, but it is possible that showing animal suffering while making an appeal to go vegetarian may sometimes be harmful to the cause. This possibility should be researched further.
5. The results show the importance of considering both uptake and follow-through when conducting research on campaign success. When considering how successful a campaign is—particularly when comparing different types of campaign (e.g., reduction vs. vegetarian vs. vegan)—researchers must take willingness to participate into account to avoid substantial bias. Examining the success of a campaign without looking at signups is the equivalent in this study of looking only at the meal orders of people who took the reduction pledge versus people who took the vegetarian pledge. That would show us 47.8% “success” for the vegetarian pledge and 31.7% “success” for the reduction pledge, but wouldn’t capture the fact that there were almost four times as many people who participated in the reduction campaign, leading to more meatless meal orders overall.

Research Team

Dr. Jo Anderson (Faunalytics), Dr. Gregg Sparkman (Princeton University), Dr. Alison Lenton (Faunalytics), Wardah Mahboob (Carleton University), Che Green (Faunalytics), Tom Beggs (Faunalytics), Julia Dauksza (Faunalytics), & Blake Miller (Carleton University).

Acknowledgements

We are very grateful for the support and assistance of Ron Barrette, the manager of Rooster’s Coffeehouse, without whom this research would not have been possible. We also extend our sincere thanks to Jenn Murphy and Mercy for Animals for creating the video we used in this study. Finally, thank you to Karol Orzechowski, Faunalytics’ Content Director, for voicing the video narration.

Method Overview
Results
Additional Analyses

Method Overview

This study was conducted on campus at Carleton University (Ottawa, Ontario, Canada), outside Rooster’s Coffeehouse, a small, student-operated café. Participants were recruited by a student researcher dressed like an animal advocate, wearing a Mercy for Animals t-shirt with the slogan “Kindness Counts” and a picture of cows on it. The student recruited passersby by asking them if they “would like to complete a video-survey for $5 off at Rooster’s today.”

Participants completed the study on a tablet with earbuds, via the Qualtrics survey platform. They watched a video, answered a short survey, and were asked whether they would take one of two pledges: either to reduce meat in their diet or to stop eating meat.

Figure 1. Study Procedure.

Experimental Conditions

This study had four conditions. Each participant was randomly assigned to one of the following combinations of video and pledge:

1. In the Reduction Advocacy condition:
   • Participants watched a video describing how, “in recent years, many people have begun to reduce how much meat they eat, a pattern that is expected to continue” (full video), and
   • A survey question asked them whether they would pledge to reduce their own meat consumption.
2. In the Vegetarian Advocacy condition:
   • Participants watched a video describing how, “in recent years, many people have stopped eating meat, a pattern that is expected to continue” (full video), and
   • A survey question asked them whether they would pledge to stop eating meat themselves.
3. In the Control (with Reduction Pledge) condition: 
   • The video simply showed white text on a black background, describing how to complete the survey (full video), and
   • A survey question asked them whether they would pledge to reduce their meat consumption.
4. In the Control (with Vegetarian Pledge) condition:
   • The video was the same as above: white text on a black background, describing how to complete the survey, and
   • A survey question asked them whether they would pledge to stop eating meat.

Figure 2. Stills From The Videos.

Video Effectiveness

The videos were pre-tested with a small sample (n = 99) to be sure the advocacy messages were distinct from one another. As intended, the Vegetarian Advocacy video was seen significantly more as vegetarian advocacy than the Reduction Advocacy video (ps < .001). 

In addition, five survey questions in the main study assessed participants’ emotions after the videos. As one would expect if the videos worked to make participants more aware of the problems with eating meat, after watching either the Vegetarian Advocacy or Reduction Advocacy video, people’s emotions were significantly more negative than after the Control video. On average, people who had watched either advocacy video reported being less calm, less hopeful, less happy, less energized, and more ashamed (all ps < .002). There was no difference in the emotions produced by the Vegetarian Advocacy and Reduction Advocacy videos (all ps > .37).

These findings show that the two advocacy videos (Reduction Advocacy and Vegetarian Advocacy) were understood and had an emotional impact on viewers. However, their emotional impact was consistently negative, suggesting that reactions may have been driven mostly by the factory farm footage near the beginning, rather than the more hopeful normative information showing that positive social change is occurring. This occurred even though the factory farm footage in these videos was quite mild compared to what is used in many advocacy videos.

More details about the pre-test and the study method, as well as the data-cleaning procedure, are provided in the Supplementary Materials. 

Participants and Sample Size

The sample size for this study was determined by the number of participants we could recruit in the fall semester, with a goal of at least 828. This sample size would give 80% power to detect 12% difference between the Reduction and Vegetarian Advocacy conditions. More details of the power analysis and other sample size considerations can be found in our pre-registration document.

Our final sample included 833 participants, meeting our goal. However, only 617 of the 833 participants used the Rooster’s voucher they received as an incentive (74.1%). 

We had not anticipated that some participants wouldn’t use their voucher, so it wasn’t factored into our power analysis. With 617 participants, we only had 80% power to detect a 14% difference between the Reduction Advocacy and Vegetarian Advocacy conditions—a larger difference than we had expected to see. Therefore, it should be noted that the Reduction Advocacy vs. Vegetarian Advocacy comparisons in this study ended up being quite conservative.

Data from the other 216 participants could not be included in the key analyses, so we also checked for the possibility of bias due to attrition (see Supplementary Materials). Differences between participants who did versus did not use their vouchers were few and small, so bias of this sort is unlikely. 

As shown below, most of the study participants were undergraduate Carleton students. The majority were female and under the age of 25.

Figure 3. Participant Demographics.

Results

Full analysis details are provided in the Supplementary Materials, and all code and data is available on the Open Science Framework.

Advocating For Reduction Produced More Meatless Orders

This study’s central research question was whether reduction or vegetarian advocacy would impact the number of participants who ordered meatless meals with their voucher. 

As a reminder, advocacy in this context meant a video that showed some mild factory farm footage, described the benefits of plant-based eating, described changing social norms around meat consumption—more and more people reducing or cutting out meat—and concluded with an appeal to reduce meat consumption or stop eating meat.

We had hypothesized (with a pre-registered directional hypothesis) that either type of advocacy would result in more meatless meal orders than the Control video. However, as you can see in the graph below, that wasn’t the case. Of participants in the Reduction Advocacy condition, 25.8% ordered a meatless meal. Of participants in the Vegetarian Advocacy condition, 18.9% did. After the Control video, meatless meal orders were in the middle, with 22.4% and 23.2% ordering meatless meals in the two control conditions. 

The hypothesized difference between the advocacy conditions (averaged) and control conditions (averaged) did not emerge and was close to zero (0.3% ± 6.6%, p = .40, one-tailed). The reason for this is clear from Figure 4 below: The Vegetarian Advocacy condition performed worse than control while the Reduction Advocacy condition performed better than control. 

We had also hypothesized that people in the Reduction Advocacy condition would be more likely to order meatless meals than people in the Vegetarian Advocacy condition. This difference of 6.9% (± 9.3%) percentage points was marginally significant (p < .08, one-tailed), allowing us to tentatively conclude that, on average, reduction advocacy of this type decreases meat consumption more than vegetarian advocacy, at least in the short-term.

Figure 4. Meatless Meal Orders By Condition.

We tested the effectiveness of the advocacy videos themselves (separate from the pledges) by looking at the differences between each advocacy condition and its matched control condition: For instance, is the rate of meatless meal orders higher following the Reduction Advocacy video condition than following the Control video, among participants who were offered the reduction pledge?

In answer to that question, although the rate was higher, the difference was not significant. Of the people who were offered the reduction pledge, the difference in meatless meal orders between those who saw the Reduction Advocacy video and those who saw a Control video was only 2.5% (± 9.5%; p = .60), in favor of the Reduction Advocacy video.

Of the people who were offered a vegetarian pledge, the difference in meatless meal orders between those who saw the Vegetarian Advocacy video and those who saw a Control video was 3.6% (± 9.2%; p = .45), in favor of the Control video.

In short, the pattern was such that having a video advocating reduction tended to increase meatless meal orders while having a video advocating vegetarianism tended to decrease meatless meal orders, but it is important to note that this pattern was not statistically significant (p = .36 for the interaction of video and pledge).

Therefore, we cannot make strong claims about the impact of the advocacy videos. The best course of action would be to conduct additional research to replicate the findings. Since that has not been done yet, we offer the most reasonable interpretation of these results at this point in time:

When using factory farm footage in one’s advocacy, asking for reduction is the option most likely to produce dietary change. Using factory farm footage to advocate for vegetarianism or veganism should be undertaken cautiously, with a careful eye on the proportion of people who might be rejecting the message. 

Again, that is because the trend in this study was for the vegetarian message to have a negative effect on meatless meal orders. Part of the reason for concern is that the Vegetarian Advocacy video had a similarly negative but non-significant effect on speciesism, as described in the Speciesism section below. Although both effects were small and non-significant, the potential harm warrants caution and further investigation. 

The Role of Pledges

As noted above, all study participants, whether they saw an advocacy video or Control video, were asked if they would make one of two pledges: to reduce their meat consumption or to stop eating meat.

People were far more likely to take a reduction pledge than a vegetarian pledge, regardless of whether they had just seen an advocacy video or not. As shown in Figure 5, people were several times more likely to pledge to reduce their meat consumption than to stop eating meat, whether you consider the pledges following the advocacy videos or the Control video (ps < .001). 

Figure 5. Pledges By Condition.

Figure 6. Meatless Meal Orders By Pledgers And Non-Pledgers.

What’s more, pledge-takers were more than twice as likely to order a meatless meal as those who refused the pledge (ps < .001), as shown in Figure 6. Without a no-pledge control condition, we can’t say whether the pledge caused people to order meatless meals, as it is likely that people willing to eat less meat are also more willing to take a pledge. However, this finding does align with previous research from environmental psychology and other fields, where there is strong evidence that committing to a behavior increases behavior change (Lokhorst et al., 2013).

As you can see in Figure 6, pledge follow-through was better for those who took the vegetarian pledge. 31.7% (± 6.6%) of meat-reduction pledgers followed through with a meatless meal order immediately after the study. This is compared to 47.8% (± 14.4%) of vegetarian pledgers who followed through. In spite of that, we know from the overall results by condition (Figure 4) that this better follow-through was outweighed by the impact of lower willingness to pledge.

It’s worth mentioning that, because reduction can take different forms, this doesn’t necessarily mean that the majority of reduction pledgers don’t plan to follow through. We can’t tell, with this study, whether long-term meat consumption was reduced for people who took the reduction pledge. For vegetarian pledgers, it is more straightforward: Unless perhaps they decided to begin their pledge at a later time, ordering meat is a clear violation of the pledge they just signed.

It’s also important to note that this finding is correlational and pledge type probably didn’t cause the difference in follow-through. People who tend to order meatless meals are probably also more likely to sign a vegetarian pledge. In other words, vegetarian pledge follow-through may be higher because it is catching the low-hanging fruit: people who are more willing to order vegetarian food anyway.

In this study, pledge-taking explained (mediated, to use the statistical term) the difference between the two advocacy conditions’ effects on meatless meal orders. People were more likely to order meatless meals after reduction advocacy than after vegetarian advocacy because it made them more likely to take the pledge than vegetarian advocacy did (indirect effect p < .001).

Additional Analyses

Emotions

We measured participants’ emotions with five questions following the video they watched. As discussed in the “Video Effectiveness” section, all five emotions were significantly more negative after either the Vegetarian Advocacy or Reduction Advocacy video than after the Control video (all ps < .001). There was no difference in the emotions produced by the two advocacy videos (all ps > .35).

However, we also explored whether the extent of one’s emotional reaction predicted signing a dietary pledge or ordering a meatless meal. Only one result was significant: People who reported feeling more ashamed were more likely to sign a dietary pledge (p < .004). However, feeling ashamed was not associated with ordering a meatless meal (p = .45), so the pledge result is not especially meaningful: shame may have made people more likely to sign a pledge but did not translate into the important behavior.

Speciesism

We also examined whether the advocacy videos might decrease participants’ level of speciesism (Caviola et al., 2018). If anything, the results indicate that the Vegetarian Advocacy video may have slightly increased speciesism relative to the Control video, though this was a non-significant trend (p = .17) and should not be taken as solid evidence. We point it out so that it can be considered for further research, particularly because this pattern mirrors the one for meatless meal orders.

Average speciesism scores in each condition were as follows (note that this was measured on a 7-point scale where higher numbers mean more speciesism):

• Reduction Advocacy: 2.83 (± 0.17)
• Vegetarian Advocacy: 3.00 (± 0.18)
• Control (with Reduction Pledge): 2.84 (± 0.15)
• Control (with Vegetarian Pledge): 2.87 (± 0.18)

Participants’ level of speciesism did predict their likelihood of pledging and ordering a meatless meal. As one would expect, people who were lower in speciesism were more likely to take either the reduction pledge (p < .001) or the vegetarian pledge (p < .001) and to order a meatless meal (p < .004). Because it was not significantly influenced by the videos, this mostly demonstrates the role of speciesism as an individual trait. 

Conclusions

The main conclusions of the study were as follows:

• Advocating for meat reduction led to more meatless meal purchases than advocating for vegetarianism. Meatless meal orders were 6.9 percentage points higher following reduction advocacy compared to vegetarian advocacy.
• Pledges explained most of the difference in meatless meals between reduction and vegetarian advocacy: 59.4% of participants were willing to take a meat-reduction pledge, versus only 15.4% of people who were willing to take the vegetarian pledge.
• Follow-through on food choices was better with vegetarian pledges (47.8%) than meat-reduction pledges (31.7%), but this advantage was outweighed by the much larger proportion of people willing to take a reduction pledge.

Below, we consider the implications of these findings, make recommendations, and discuss the limitations of the study.

Implications & Recommendations

Reduction Advocacy Most Likely Has A Positive Effect

This study’s evidence for advocating reduction is consistent but not definitive. More people ordered meatless meals after reduction advocacy than after vegetarian advocacy. 

Almost four times as many people were willing to sign a pledge to reduce their meat consumption than to cut it out entirely. Although follow-through on meatless meal orders was better with the vegetarian advocacy and pledge, uptake was so much higher with the reduction advocacy and pledge that it outweighed the difference in follow-through. 

We could interpret the findings as evidence that we need to work on improving our persuasive materials to increase people’s willingness to try vegetarianism. However, we recommend focusing on improving follow-through with reduction pledges instead, for several reasons:

1. There is strong evidence from many studies (described under “Set People Up To Succeed” below) that well-done pledges increase follow-through on a desired behavior.
2. There is only weak evidence that well-done advocacy can increase willingness to go vegetarian or vegan. This may be because advocacy research has only begun in the last two decades, and with scientific rigor in the past few years, but it remains the case that there are no strong studies showing how to turn people vegan, or even vegetarian. That is not to say that research won’t get there eventually, but the current evidence is weak and does not point to clearly promising avenues. For instance, even showing people the realities of factory farming in an immersive, 360-degree virtual reality, with a sample size of thousands, we found only a small reduction in pork consumption (Faunalytics, 2018). This finding is certainly positive, but the evidence is nowhere near as promising as the evidence for pledges.
3. Finally, while it is important to note that the differences are not statistically significant, this study found a small negative effect of vegetarian advocacy on meat consumption and speciesism relative to the control condition. At least in some circumstances—such as showing factory farm footage and asking for behavior change without any human interaction—asking people to stop eating meat has the potential to backfire. This possibility should be researched further to identify the circumstances when and if stronger appeals (for vegetarianism or veganism) might backfire. We expand on this caution in the next section, because it is very important to the movement.

Care Is Warranted When Using Vegetarian Advocacy

First, let us be clear that the evidence against vegetarian advocacy in this study is not as strong as the evidence for reduction advocacy. But any evidence of possible backfiring should be taken seriously, as it could harm our goal of reducing overall animal suffering.

This study found that vegetarian advocacy slightly increased meat consumption and speciesism relative to the control condition, though the increases were small and non-significant. In spite of the non-significance, this is cause for some concern because it occurred on both measures, and because the study was not powered sufficiently to find small effects (so we can’t say that non-significance means they aren’t “real”).

What Does This Mean For Groups Who Advocate Vegetarianism Or Veganism?

To answer this question, bear in mind the context of this study: 

1. The advocacy videos included factory farm footage that (while mild) had a negative emotional effect, and
2. The vegetarian appeal was administered without any human interaction, via the video and a digital pledge request.

In other words, even though it was conducted in person, the experience of being a participant in this study was probably most similar to watching an online video and being asked to type your name into a pledge form afterward. The strongest implication of the findings is in that kind of context, a vegetarian or vegan appeal may do more harm than good. 

This remains to be investigated in future research, but psychological reactance provides one explanation. Reactance is what occurs when a person sees a persuasive message as trying to limit their freedom and reacts negatively to it (e.g., with anger; for a meta-analytic review, see Rains, 2013). If reactance is strong enough, it can prevent a persuasive message from working or even cause it to boomerang, making the behavior worse (Byrne & Hart, 2009). 

If vegetarianism or veganism is presented as the only option, it may be seen as an attempt to limit freedom of choice, rather than encouragement to choose the animal-friendly path. In such cases, it may be causing people to consume even more meat than usual.

Encourage People To Do As Much As They Can

Advocating for meat reduction is a repugnant thought to many advocates, knowing what we know about speciesism and the suffering of animals who are farmed for food. We are not suggesting that advocates give up promoting veganism.

Instead, we suggest that advocates promote reduction or more—whatever each individual feels capable of at this moment in their lives. Reduction does not have to be your end goal, but to avoid the possibility of harmful reactance (“don’t tell me what to do”), it is important to meet people where they are. Or as Dr. Melanie Joy puts it, we can advocate for being “as vegan as possible.” To quote Dr. Joy:

“Asking others to be as vegan as possible is also beneficial because it asks for reduction, which people are more receptive to than they are to being asked for full-on veganism. But it doesn’t frame reduction as an end in itself. It makes it clear that veganism is the goal.” (watch the video)

Not only is advocating this way less likely make your message backfire, it is also a necessary first step for many people. Some people go vegan overnight, but many others do not, and people who don’t do that seem to be more likely to succeed: Faunalytics (2014) found that people who had transitioned to a vegan or vegetarian diet slowly (over months or years) were more likely to still be vegan/vegetarian than those who transitioned more quickly (over days or weeks).

People need the flexibility to adapt their diets in a way that works for them, even if that means just reducing meat consumption for now. Otherwise, many will simply not sign on to change their behavior at all, as the low vegetarian pledge rate in this study demonstrates.

We suggest that advocates consider opening conversations by asking about meat reduction. From there, you can help them set a more ambitious goal if they feel capable of attaining it. Ask them to sign a pledge for the specific goal you choose together.

Set People Up to Succeed: Increasing Pledge Follow-Through

Choosing to ask for meat reduction or “being as vegan as possible” is the first step to helping people succeed in changing their diets, because they have to agree to change in the first place. Next we need to help them follow through on that change. 

In this study, follow-through was the weakest link in reduction advocacy, but it is important to note that, for methodological reasons (to keep the researcher blind to condition), the pledge requests we used were about as weak as a pledge request can be. They were made where only the participant could see them, as part of a survey on a tablet, rather than face-to-face by a real person. Our main goal was to be able to compare pledge follow-through in the different conditions rather than to maximize it. In that we succeeded, but the price is low follow-through rates overall.

There is a strong body of evidence to say that pledges administered face-to-face by an advocate would produce higher rates of follow-through than our minimal pledges (Cialdini, 2001; Lokhorst et al., 2013). There are several key things advocates can do to increase pledge follow-through:

1. Make it an “active” pledge rather than passive: Ask them to sign an official-looking form, write out a statement, etc. (Cialdini, 2001);
2. Make the pledge in public or encourage them to publicize it after the fact: E.g., to friends and family, or on their social media (Cialdini, 2001);
3. Be very specific about what they are pledging to do: If it is a reduction pledge, help them decide exactly how they will reduce. By choosing a particular day per week or meal per day not to eat meat, a particular type of meat to avoid, or a particular location to eliminate meat (e.g., when preparing meals at home), it is possible to be flexible but still specific. There is strong evidence that having a specific “if-then” plan (“if the situation is this then I will do that”) is very helpful in increasing follow-through (Gollwitzer, 1993; Gollwitzer & Sheeran, 2006);
4. If you have the time to talk to them about specific difficulties they foresee in keeping their pledge—a difficult family member, a potluck situation, etc.—ask them about how they could deal with those situations and help them come up with solutions in advance. This uses the same “if-then” principle as in the previous point to help avoid problems before they arise (“if my mother says this then I will do that”). Let them guide this discussion, as barriers are very individual—they know best what will work for them;
5. Finally, talk about what “failure” means (or doesn’t mean): No one is perfect, and it is important to encourage people to see occasional slips as an expected part of the process. You can convey that they are an opportunity to learn and grow rather than a sign that you should give up. This perspective on goal pursuit is known as a “growth” or “incremental” mindset and it is associated with more success and positive experiences working toward a goal, so we highly recommend trying to instill it in the people you speak to (Burnette et al., 2013).

Effective Pledges Fact Sheet

You can find a fact sheet with the above information in a handier format in the Supplementary Materials section. Or download it here.

Implications For Other Researchers

The Threat Of Selection Bias

The results show the importance of considering both uptake and follow-through when conducting research on campaign success. When considering how successful a campaign is—particularly when comparing different types of campaign (reduction vs. vegetarian vs. vegan; e.g., Grassian, 2019)—researchers must take willingness to participate into account to avoid substantial bias. 

Examining the success of a campaign without looking at signups is the equivalent in this study of looking only at the meal orders of people who took the reduction pledge versus people who took the vegetarian pledge. That would show us 47.8% “success” for the vegetarian pledge and 31.7% “success” for the reduction pledge, but wouldn’t capture the fact that there were almost four times as many people who participated in the reduction campaign, leading to more meatless meal orders overall.

Due to this strong threat of selection bias, we hope that researchers who wish to make claims about the relative impact of different types of campaign will take into account willingness to participate in future research.

Using Pledges As Proxies For Meal Choice

Because both dietary pledges significantly predicted participants’ dietary behavior, researchers have the option of using them as proxies—though fairly weak ones—in future research. 

We don’t recommend this option if measuring diet directly is possible or a validated Food Frequency Questionnaire (FFQ) is available, but if not, these minimal dietary pledges can be used to infer participants’ short-term diet using the following correlations:

• The correlation between taking a minimal meat reduction pledge and choosing a meatless meal soon after was r = 0.20, CI = (0.10, 0.31). As noted in the Results section, we can expect approximately 31.7% of people who take this meat reduction pledge to order a meatless meal afterward.
• The correlation between taking a minimal meat elimination pledge and choosing a meatless meal soon after was r = 0.28, CI = (0.18, 0.39). As noted in the Results section, we can expect approximately 47.8% of people who take this meat elimination pledge to order a meatless meal afterward.

Naturally, we would expect the correlation to be higher if a stronger pledge was used, per the recommendations above. However, additional research would be needed to provide those estimates.

Caveats & Limitations

As with all studies, this one has some important caveats and limitations.

Diet Measurement

The biggest limitation of this study was that dietary choices were measured only at a single time point, immediately after the advocacy intervention. Ideally, we would like to know how diet was affected over the long term. Observing long-term dietary behavior directly was not an option for this study, so we would have been reliant on self-report data. Given concerns about attrition and social desirability bias, we chose to use a shorter-term, non-self-report measure instead.

We hope to replicate the findings with a longer-term measure to determine whether reduction and vegetarian advocacy would fare better over several months, at least. It is possible that the observed effects would wear off over time.

Social Pressure

Additionally, because the voucher used to measure dietary choices was given to participants as part of the study, they may have felt pressure to use it in a way that we—in our role as advocates—would want. If participants’ food choices were not paid for in part by the study, the rate of meatless orders may have been lower. However, this explanation does not account for the observed difference between the Reduction and Vegetarian Advocacy conditions, for two reasons. First, only two participants asked whether purchases with the voucher were being tracked. And second, even if many of them thought it was being tracked or just felt guilt over going against what we might want, we can think of no logical reason they would feel more pressure to order a meatless pita in the Reduction Advocacy condition than the Vegetarian Advocacy condition, yet that was the pattern we observed. If anything, it should be the other way around, because reduction can take a variety of forms that might not involve ordering a meatless pita then and there. 

Campus Context

This study was conducted on a university campus in Canada, so it is important to note that the findings may not generalize to other forms of advocacy. Participants were generally young, and can be presumed to be well-educated and liberal, on average.

In particular, it is plausible that the high rates of willingness to take a meat-reduction pledge—even without advocacy beforehand—would be substantially lower in more conservative regions or samples. We would also expect rates of willingness to take a vegetarian pledge to be lower, and possibly the rate of meatless orders as well. 

That said, many animal advocacy groups have campaigns focused on university campuses (e.g., Vegan Outreach, Animal Equality), so these findings still have broad applicability.

Effective Pledges Fact Sheet

Data-Cleaning Procedure

Participants were screened out of the study at the beginning of the survey if they indicated that they were under 18 years of age (n = 10), had participated in the study before (n = 17), or were vegetarian or vegan (n = 137). In addition, there were 216 participants who did not use their vouchers. Of 833 participants who completed the survey portion of the study, 617 (74.1%) used the voucher and had useable data. 

Checking For Bias Due To Attrition

Because not everyone who participated in the study used the voucher they were given, it is important to consider the possibility of bias. Bias could occur if the people who used the voucher differed systematically from the people who didn’t. 

• People in the Vegetarian Advocacy condition were marginally less likely to use the voucher than those in the Reduction Advocacy condition or Control (with Vegetarian Pledge) condition (ps < .07). One interpretation of this finding is that the video advocating vegetarianism made people feel guilty about ordering meat, but didn’t make them want to order meatless meals, so they just didn’t use the voucher;
• Being offered the reduction pledge versus vegetarian pledge did not significantly affect likelihood of using the voucher (β = 0.19, z = 1.21, p = .23);
• People who took a pledge were no more or less likely to use the voucher than those who didn’t (χ2 = 0.31, p = .58).

Detailed Method

Video

Qualtrics randomly assigned participants to watch one of three videos so that the student researcher was kept blind to condition. All three videos were the same length, and the two advocacy videos used the same footage (provided and edited by Mercy For Animals).

The purpose of including a non-advocacy video in the control condition was to control for recruitment method and length of participation. All participants could then be recruited the same way (for a video and survey), to prevent selection bias. They also have to spend the same amount of time on the study to receive the voucher, preventing any bias that might arise from a perception of getting a better “deal” in the control condition.

Survey

Emotion: The survey included five questions assessing participants’ emotions immediately after watching the video (happy/unhappy, bored/energized, proud/ashamed, calm/angry, despairing/hopeful).

Writing Prompt: In the advocacy video conditions, it also included a writing prompt asking them to think more about the message of the video (“As it said in the video… Why do you think people are beginning to change their behaviour?”). The purpose was to make sure they reflected on the message they had seen. In the control condition, the writing prompt was about why some people like to eat many types of food, while others prefer to stick to certain staples.

Speciesism: The survey also included Caviola et al.’s (2018) six-item measure of speciesism (e.g., “Morally, animals always count for less than humans”). 

Demographics: The survey also asked about diet, age, gender, and student status. The full survey is available on the Open Science Framework.

Diet Pledge

At the end of the survey, participants were asked (in text form, via the survey) if they would pledge to “reduce the amount of meat that you eat” or “stop eating meat.” Participants in the advocacy conditions were given the pledge that matched their video, whereas participants in the control condition were randomized to one of the two pledges.

Analysis Details

The following are the statistical results for each of the findings included in the body of the report, labeled by section.

Video Effectiveness

Message Clarity

We pre-tested the treatment videos on Mechanical Turk to ensure that their messages were perceived as intended. We aimed for a sample size of 102 based on a power analysis for a one-tailed t-test with a predicted medium effect size (d = .5) and 80% power. 

Participants were randomly assigned to see the Reduction Advocacy or Vegetarian Advocacy video, then answer two questions about the video they saw. After removing participants whose timing data indicated that they hadn’t watched the whole video, n = 99. 

• The first question asked participants “To what extent are the creators of this video asking you to become a vegetarian?” with response options from 1 (completely) to 5 (not at all). As intended, the Vegetarian Advocacy video scored significantly closer to completely (M = 1.5, SD = 0.87) than the Reduction Advocacy video (M = 2.8, SD = 1.05), t(97) = -6.87, p < .001.
• The second question asked participants “What does this video ask of you? Please choose the best description.” The options were “Become a vegan,” “Become a vegetarian,” “Eat less meat,” “Buy meat that doesn’t come from factory farms,” and “Eat more eggs.” Of participants who saw the Reduction Advocacy video, 94% correctly answered “eat less meat,” with the remainder indicating “buy meat that doesn’t come from factory farms” (4%, n = 2) or “become a vegetarian” (2%, n = 1). Of participants who saw the Vegetarian Advocacy video, 79% correctly answered “become a vegetarian,” with the remainder indicating “eat less meat.” The difference by video was significant, 𝜒2 (2) = 59.80, p < .001.

Emotional Impact

The videos had an emotional impact on participants, but this had no effect on their likelihood of taking a pledge or ordering a meatless meal. The results were as follows:

• Compared to the control condition, people who had watched either advocacy video reported being, on average, less calm (β = -0.24, t = -3.32, p < .001), less hopeful (β = -0.25, t = -3.31, p < .002), less happy (β = -0.33, t = -4.84, p < .001), less energized (β = -0.21, t = -3.11, p < .002), and more ashamed (β = 0.46, t = 7.81, p < .001).
• There was no difference in any of participants’ emotions between the Reduction Advocacy condition and the Vegetarian Advocacy condition (all ps > .37), which is perhaps not surprising given that the two videos were almost identical.

Advocating For Reduction Produced More Meatless Orders

• We tested our pre-registered hypotheses using logistic regression with orthogonal contrast codes for condition and the dichotomous meal choice variable as the outcome.
• Our first pre-registered hypothesis was that the odds of ordering a meatless meal would be higher in the advocacy conditions (averaged) than the control conditions (averaged). We tested and rejected this hypothesis using logistic regression (log-odds = 0.04, z = 0.21, p = .40, one-tailed). The difference between the two averages was 0.3% ± 6.6%.
• Our second pre-registered hypothesis was that the odds of ordering a meatless meal would be higher in the Reduction Advocacy condition than the Vegetarian Advocacy condition. This difference, between the 25.8% who ordered meatless meals in the Reduction Advocacy condition and the 18.9% in the Vegetarian Advocacy condition, was marginally significant (log-odds = 0.40, z = 1.43, p < .08, one- tailed). Having considered the size of the effect (6.9% ± 9.3%) and its consistency with the effect for pledges, below, as well as the power of the study, we have chosen to discuss this effect as potentially meaningful rather than not. That said, these results would strongly benefit from an attempt at replication.
• As an exploratory analysis, we tested the unique impact of the videos (separate from pledge impact) by looking at the differences between each advocacy condition and its matched control condition: that is, the meatless meal orders of people offered the same pledge but after the Control video. Of the people who were offered a meat-reduction pledge, those who saw the Reduction Advocacy video were 2.5% (± 9.5%) percentage points more likely to order a meatless meal than those who saw the Control video (log-odds = 0.14, z = 0.53, p = .60). Of the people who were offered a vegetarian pledge, those who saw the Control video were 3.6% (± 9.2%) more likely to order a meatless meal than those who saw the Vegetarian Advocacy video (log-odds = -0.22, z = -0.76, p = .45). 

The Role of Pledges

• People were far more likely to pledge to reduce their meat consumption than to stop eating meat. This was true whether they had watched an advocacy video (after which 62.0% took reduction pledge, 14.7% took vegetarian pledge; log-odds = 2.25, z = 7.86, p < .001) or the Control video (after which 56.8% took reduction pledge, 16.0% took vegetarian pledge; log-odds = 1.93, z = 7.10, p < .001). 
• People who took either pledge were more than twice as likely to order a meatless meal as those who refused the pledge (ps < .001). Specifically, 31.7% of people who took the reduction pledge followed through, versus 14.0% of people who refused it, χ2(n = 318) = 12.17, p < .001. Similarly, 47.8% of people who took the vegetarian pledge followed through, versus 15.8% of people who refused it, χ2(n = 299) = 22.37, p < .001.
• We also examined pledge-taking as a mediator of the effect of condition on meatless meal orders, using the lavaan package for R. Standard error was estimated using 5000 bootstrap iterations. The indirect effect of condition on meatless meal orders via pledge-taking was significant (est. = 0.10, z = 4.79, p < .001).

Additional Analyses

Emotions

The impact of condition on participants’ emotions is discussed above, under Emotional Impact. However, we also explored whether the extent of one’s emotional reaction predicted signing a dietary pledge or ordering a meatless meal.

• Controlling for condition, participants’ emotions did not significantly predict their likelihood of ordering a meatless meal after the study. Only energized (vs. bored) showed even a trend (log-odds = 0.19, z = 1.58, p = .11; all other ps > .45).
• Similarly, controlling for condition, most emotions did not predict likelihood of signing a dietary pledge. Only proud (vs. ashamed) was a significant predictor (log-odds = -0.38, z = -2.88, p < .004; all other ps > .39), such that feeling more ashamed was associated with greater likelihood of taking a dietary pledge. Although this is plausible, the effect may be spurious or meaningless, given the number of tests performed and the fact that shame did not predict actual meal orders.

Speciesism

The videos did not significantly influence participants’ level of speciesism (Caviola et al., 2018). If anything, there was a non-significant trend such that the Vegetarian Advocacy video appeared to slightly increase speciesism relative to the Reduction Advocacy video, as described here:

• Average speciesism scores, on a 7-point scale where higher numbers mean more speciesism, were:
  • Vegetarian Advocacy: 3.00 (± 0.18)
  • Reduction Advocacy: 2.83 (± 0.17)
  • Control (with Vegetarian Pledge): 2.87 (± 0.18)
  • Control (with Reduction Pledge): 2.84 (± 0.15)
• The difference between the advocacy conditions (averaged) and the control conditions (averaged) was not significant (β = 0.06, t = 0.64, p = .52).
• The difference between the Reduction Advocacy condition and the Vegetarian Advocacy condition was also not significant, but the direction of the trend—higher speciesism in the Vegetarian Advocacy condition—is worth noting for future research (β = -0.17, t = -1.37, p = .17).

Participants’ level of speciesism did predict their likelihood of pledging and ordering a meatless meal. Because it was not influenced by the videos, this mostly demonstrates the role of speciesism as an individual trait.

• Participants’ speciesism negatively predicted their likelihood of ordering a meatless meal after the study, controlling for condition (log-odds = -0.28, z = -2.90, p < .004). In other words, people lower in speciesism were more likely to order a meatless meal.
• Participants’ speciesism also negatively predicted their likelihood of signing a diet pledge, controlling for condition—and therefore pledge type (log-odds = -0.63, z = -6.22, p < .001). In other words, people lower in speciesism were more likely to sign a dietary pledge. Considering the pledge types separately, it negatively predicted taking either the reduction pledge (log-odds = -0.52, z = -4.27, p < .001) or vegetarian pledge (log-odds = -0.86, z = -4.67, p < .001).