Following A Datapoint Down The Rabbit Hole
In Faunalytics’ Staff Slack, it’s not uncommon to see messages posted like “Does anyone know where this statistic comes from?” or “Is there a good source for this?” When we’re working on research projects, we want to be certain we have strong citations and a thorough understanding of the subject matter. To that end, we like to turn over every stone. We want to know that we’re doing research and offering research advice that is grounded in quality information. But sometimes claims are repeated so frequently that they become generally accepted truths without being verified — we can see examples of this in concerns over MSG or soy consumption.
Recently, while working on updating a resource, we came across a claim that has become common across the animal advocacy movement: that 8% of Black Americans are vegan. When we attempted to source the claim, what we found was a shaky statistic that has spread throughout our movement and into the broader public discourse.
This blog post isn’t meant to be a fact check of the claim that Black Americans are more likely to be vegan than the rest of the population. Instead, we want to use this commonly referenced statistic as a case study, a way of exploring the potential pitfalls of repeating unverified claims so that we can be more informed and therefore more effective as a movement.
The claim that 8% of Black Americans are vegan often appears alongside a citation pointing to the respected polling and research organization Pew Research Center. But when we looked through Pew’s report to find this number, it was nowhere to be found. There were figures related to the percentage of all Americans who follow a vegan or vegetarian diet — about 3%. There were breakdowns of certain topics by race, such as feelings about genetically modified foods. But there was no information about the percentage of Black Americans who follow a vegan diet.
Further complicating the picture, the statistic that people cite is sometimes said to reflect the percentage of Black people in the U.S. who are vegan, but other times it refers to those who are either vegan or vegetarian. For example, a Vice article notes that “3 percent of people in the U.S. follow a vegan diet, but the number jumps to 8 percent for Black people” and a Buzzfeed article says “8% of African American adults identified as vegan.” Today and the BBC, on the other hand, claim this is the percentage of Black Americans who are either vegan or vegetarian.
So what’s going on? What data are these sources actually referring to?
Where The Number Came From
To begin tracking down this statistic, we first looked in the obvious place: the cited source. In 2016, Pew published a report titled The New Food Fights: U.S. Public Divides Over Food Science. This report included the results from an analysis of data obtained through the organization’s 2016 American Trends survey panel. In a document totaling about 100 pages, the authors explore the dietary habits and food beliefs of U.S. residents, with a focus on organic foods and genetically modified organisms. After reading through this entire document, however, we found no mention of the 8% statistic.
We then went through articles that reference this number to see if we could uncover a more concrete source — with no luck. This Vice piece serves as a good example of how this statistic is cited. In the article, the author says that the Pew report shows 3% of the entire U.S. is vegan and 8% of Black people in the U.S. are vegan. The 3% statistic contains a hyperlink to the Pew report, but the 8% statistic does not. Instead, it links to a Livekindly article, which ultimately links to the Pew report. Neither of these articles provide any information about the 8% statistic in any more detail. We were also unable to find any other Pew reports that contained this statistic.
In this case, we did what any curious researchers would do: we looked at the data directly. Luckily, Pew publishes much of its raw data for the public to explore. The report that’s frequently cited for this statistic relies upon data from the 2016 American Trends Panel Wave 17. We downloaded this dataset from the Pew website, imported it into R, and began poking around. Using the information Pew provides, we were able to identify the relevant columns in the data — specifically, the data related to diet, race, and survey weighting.
In Pew’s survey, the relevant question participants answered was, “Do you consider yourself a vegan or vegetarian?,” which had the following response options: “I am a strict vegan or vegetarian,” “I am mostly vegan or vegetarian,” “I am neither vegan or vegetarian,” or “No answer.” By looking at the original question, we can see that any data obtained in response to this question refers not to the rate of veganism alone, but rather to the rate of either veganism or vegetarianism. After segmenting the raw data by race and applying Pew’s survey weight methodology to make the results more nationally representative, we arrived at a familiar number: 7.55% of Black adults in the U.S. identify as “a strict vegan or vegetarian,” rounding to the 8% figure we started with.
Is This Number Correct?
At this point, we knew where the 8% figure came from and who it refers to: the weighted proportion of Black Americans who said they are a strict vegan or vegetarian. However, reconstructing this result revealed a major issue: Pew’s survey sample included 130 Black participants. All but one Black participant answered the question about being vegan or vegetarian, but only four Black participants said “I am a strict vegan or vegetarian.” This is simply too few people from which to make broad generalizations. To illustrate this, we can follow Pew’s own calculations to see that the 8% figure has a margin of error of +/- 59.4% due to the small number of responses from Black survey participants.
This works the same way as the margin of error in a political poll. If a candidate is found to have a 53% approval rating with a margin of error of +/- 3%, we know that the actual approval rating of the entire population would probably fall between 50% (53% – 3%) and 56% (53% + 3%). This means that Pew’s data can comfortably tell us that the proportion of Black vegans or vegetarians in the U.S. is within 59.4 percentage points of the 7.55% statistic on either end — or somewhere between 0% and 67%. Because of the small sample size, we can only say with confidence that the number of Black vegans or vegetarians is somewhere between 0 people and two-thirds of the U.S. Black population, a range so wide that the result is essentially meaningless. In other words, random chance could easily lead to an estimate that is higher or lower than the true number of Black vegans or vegetarians.
To confirm our understanding, we reached out to a Pew staff member who explained that “this 8% figure is not, and never was, included in that 2016 report of ours. … We only report out demographic breaks if they meet our internal effective sample size thresholds and when we feel confident they offer reliable estimates, which was not the case with this example.” In other words, Pew collected data that would allow someone to calculate a percentage of Black vegans or vegetarians in the U.S., but that number would be unreliable due to an insufficient sample size, so Pew did not report it.
How A Number Spreads
Adding to the confusion, this unreliable figure is sometimes paired with other findings in an effort to bolster the claim. For example, after using the 8% number from Pew, the BBC cited a 2015 survey commissioned by the Vegetarian Resource Group (VRG), which found that 8% of Black Americans are vegan or vegetarian. Rather than asking participants to self-identify as a strict vegan or vegetarian like the Pew survey did, this survey asked how often participants ate “meat, fish, seafood, or poultry.” VRG did not provide data about the raw numbers of Black participants in their survey or who indicated they followed a vegetarian or vegan diet. VRG did not provide the figures upon request. A similar survey VRG commissioned in 2016 found that only 3% of Black Americans were vegan or vegetarian — a drop of more than half in only a year. The next iteration of this survey with results broken down by race appears to be from 2020 and found that 5% of Black Americans were vegan or vegetarian. This fluctuation over only a few years illustrates that we lack a strong, stable estimate of the number of vegan or vegetarian Black people in the United States.
Tracing the source of this misinterpretation of Pew’s data is tricky, but the earliest mention we could find comes from a January 24, 2020 Washington Post article. At that time, the data it pulled was already more than three years old in a rapidly changing food landscape. Impossible Burgers, for example, made their splashy debut only a month after the data was collected in 2016 and were widely available in the U.S. by the fall of 2019. The same day the Washington Post article was published, it was circulated by Physicians Committee for Responsible Medicine on social media. Referencing the Washington Post article, Mercy for Animals cited the figure in a February 2020 blog post. Shortly after this, it broke out of the animal advocacy space and appeared in Buzzfeed, Vice, and the BBC. It has since appeared in Essence, the local ABC news in Tampa, Today, academic journals, and Wikipedia, among other places.
Some of the articles that reference this statistic seem to have been written with Black History Month and the murders of George Floyd and Breonna Taylor in mind. The role of white supremacy in veganism has also rightfully been a more common topic of discussion in recent years. These factors, combined with both a general interest in understanding who is vegan and why, all likely played a role in the spread of an inaccurate statistic. Trusted news sources and animal advocacy organizations repeating the figure only lodged it further in people’s minds to the point that many advocates can now reference the number reflexively off the top of their heads.
What We Should Learn From This
It’s entirely possible that Black people in the U.S. are vegan at higher rates than other groups. In fact, Faunalytics has Original Research that suggests Black people may be more open to veganism than other groups. However, looking at the Pew data that is frequently cited to support this statement, we simply do not have sufficient information to say for sure.
This post is meant to serve as a sort of case study that illustrates some of the common mistakes to watch out for when using data points, especially those that come from media. Overrelying on a single source, referencing outdated information, repeating a claim without checking the source material, mischaracterizing a finding, using a non-transparent data source, citing numbers that are inconsistent over time, and relying on insufficient sample sizes are all potential pitfalls. Each of these mistakes can cause misinformation to spread and the effect is compounded when more than one of these mistakes is made. The Pew representative we corresponded with underscored this point: “We do our best to make sure our findings are externally cited in an accurate way, but sometimes these kinds of misrepresentations gain a life of their own and are tough to mitigate unless people do their own due diligence or reach out to us directly.” Sometimes the impact is small, but other times it can result in organizations constructing entire campaigns on shaky ground.
As the use of AI spreads, this type of issue will likely become a larger problem. LLMs have already been trained on sources that use the 8% figure. For example, when asked what percentage of Black people in the U.S. are vegan, Anthropic’s Claude gave the following response: “The most commonly cited figure is 8% — from a 2016 Pew Research Center survey finding that 8% of Black Americans identify as vegan or vegetarian, compared to 3% of Americans overall. This number circulates widely in media coverage.” As inaccurate numbers such as this are repeated by AI, it will pollute future training data as well as the information environment for animal advocates and the public more broadly. We must improve our vetting process for statistics to avoid accidentally contributing to the spread of misinformation.
The ubiquity of this single erroneous data point also speaks to the pressing need for more research into the diets and animal-related behaviors of BIPOC Americans. Among both animal advocacy organizations and the public more generally, there is clearly interest in rates of veganism and vegetarianism among different groups in the U.S. and globally. However, high-quality, transparent, and consistent data like this can be hard to come by. As a movement, we should strive to use the most accurate figures possible, whether for use in public outreach or in internal decision-making around organizational strategy. Where quality information does not exist, we should invest in adding quality information to the movement’s knowledge base rather than relying on weak data points out of convenience or ignorance. The question about the true rate of veganism among Black Americans is an important one that deserves well-powered and rigorously designed studies in its own right.
It’s not always easy to spot red flags, and even if you do, not everyone has the resources or know-how to find the original data, examine the methodology, calculate weighted averages, and so on. If you want help verifying an interesting or important data point, or conducting your own research, consider dropping into Faunalytics’ free, weekly office hours.
