Getting Started

You’ll often see a margin of error (MOE), such as +/- 3% nineteen times out of 20, reported for a survey or poll. Remember that surveys are intended to estimate the frequency of an attitude or behavior in the population. All estimates are somewhat uncertain, and that phrase gives you an idea of how much uncertainty there is. The MOE for an entire survey tells you the widest range of possible values you’ll see for any estimate in the entire survey. For statistical reasons, that widest range occurs in the middle of the probabilities, at 50%. So if a survey finds that 50% of people have cats and the survey has an overall MOE of 2.2%, the percentage of the population who live with cats is 50% +/- 2.2%. But even on the same survey, that range will be smaller for estimates that are closer to 0 or 100 — for example, like the rate of veganism in a population.

Calculators like this one are useful for understanding how margin of error relates to sample size and particular estimates. As an example, consider Faunalytics’ 2019 study of people who donate to animal causes. Using the second calculator on the linked page, you can enter our sample size (1,036) and find the same MOE we reported. Leave the confidence level at 95% (the standard), leave population blank (because for a population as large as a country, population is treated as infinite and doesn’t need to be specified), and leave the percentage at 50% (because, as noted above, the widest range falls in the middle of the probabilities). You’ll get a result of 3.04%. That means that any estimates in our survey that fall close to 50% (e.g., 54% of animal donors are married) will be accurate to about +/- 3%.

However, the specific (rather than survey-wide) MOE for something like the proportion of veg*ns is a lot smaller. This study estimates that about 5.7% of animal donors are vegan or vegetarian (veg*n), and the MOE is just +/- 1.4%, so we can be confident that the percentage in the full population is in the range of 4.3% – 7.1%.]

MOE is important to bear in mind when you compare one survey to another. it’s been widely reported that veganism “grew 600%” between 2014 and 2017 in the U.S., thanks to this report. Unfortunately, that interpretation of the data is problematic even if both surveys were identical (which is a requirement because even small changes in questions about diet can lead to very different estimates). Assuming they surveyed the standard number of people (~1000), each survey would have a MOE of +/- 3.1%, which means 2014’s estimate is 0.4% – 1.6% and 2017’s is 4.5% – 7.5%. This is still good even if it’s 1.6% up to 4.5%, but not as good or simple as saying 600% growth. The best way to report year-to-year data is with a graph that shows margins of error–while also being transparent about the exact questions used and method of data collection, rather than placing those crucial details behind a paywall.