Challenge 22+ Pilot Impact Study
The aim of this study is to explore new methods for evaluating the impact of Challenge 22+, a global program that encourages people to try a plant-based diet for 22 days. Due to low response rates in the past, a new data collection method was used for the evaluation – WhatsApp messages. The new method lead to increased response rates (50.3%), which in turn provided a more representative data.
Participants were randomly chosen from six different Challenge groups in order to check the effect of three variables: reported diet prior to taking the Challenge, time passed after completing the Challenge (one month or seven months), and number of times the participants signed up for the Challenge (one time or two times).
The study found the following:
- The only significant predictor of diet change we found was diet prior to the Challenge. The time passed after completing the Challenge (one month or seven month) and the number of times a participant signed-up for the Challenge had no influence on diet change.
- Over 77% of frequent meat-eaters, 64% of reducetarians and 34% of almost vegetarians, reported that they reduced their meat consumption after completing the Challenge.
- A very small amount of participants who were veg*n prior to taking the Challenge returned to eating meat (about 3%).
A calculation of the number of portions of meat saved resulted in the conclusion the average Challenge participant reduces 86 portions of meat per year.
Challenge 22+: Project Background
Challenge 22+ is a global program that encourages people to try a plant-based diet for 22 days, while focusing on the social aspects that accompany the transition. The program was developed by Israeli organization Animals Now, and it was launched in Israel in March 2014.
The program runs all year round, and new cycles of the Challenge constantly open. The participants of each cycle are added to a secret Facebook group. This means that each participant joins the same group together with all those who start the Challenge on the same day as them. This nurtures a sense of belonging to a community of people, going through a journey together. In addition to receiving practical advice in the form of pre-written content, such as daily challenges, bi-daily emails, etc., participants are also provided with guidance from personal mentors. The mentors are required to go through extensive training before they start volunteering. Moreover, every group is supervised by registered dietitians.
The program’s strength lies in its unique ability to address the individual needs of each person, while supporting thousands of participants at a time. The number of participants in each cycle of the Challenge ranges between 300 to 1,000 and above.
In the last two years, the project has grown rapidly and now includes specialized tracks suited for audiences from different backgrounds and cultures, and in several languages. The four main ones include the International Challenge (in English), the Israeli Challenge, the U.S. Challenge and the U.K. Challenge. In 2018, 77,349 people participated in the Challenge. A Spanish Challenge will be launched in August.
The project’s staff includes seven part-time employees, all based in Israel, as well as over 1,200 volunteers from around the world. Over the years, the staff of Challenge 22+ aided other animal protection groups and organizations in developing their own local programs. Countries in which partners developed similar programs under our guidance include Denmark, Norway, Taiwan, Romania and more.
Problems With Email Surveys
Each year, Challenge 22+ conducts an internal analysis of the project’s impact. While the results are generally satisfactory, we are faced with a low response rate. For example, in a survey held in 2018 and distributed via email, most participants reported that they remained vegan (67%), and many others reported that they plan to progress towards a plant-based diet. Although these results were very encouraging, they were based on a low response rate of 14%.
There’s a risk that the low response rate affects the results of the survey. For example, there’s a chance that participants who had a better experience in the Challenge and succeeded in making the dietary change they aimed for, will tend to answer the survey more than other. An effect in the opposite direction is also possible.
For these reasons, we decided to look for better ways collect data, with the aim of increasing the response rates, thus eliminating some potential problems like a sampling bias.