On Spay And Neuter Organizations Impact And Evaluation
A study designed to assess the historical grantmaking effectiveness of the Handsel Foundation and to gauge the degree to which companion animal groups — and hence the foundations that fund them — understand their impact on the communities served.
The overall challenges companion animal groups face include the unavailability of accurate and reliable data for the community, lack of time to collect and analyze data, and/or insufficient knowledge about measuring impact.
- Community-wide data are not available to accurately assess trends, including changes in S/N surgeries performed above baseline, shelter intakes, adoptions, or euthanasia.
- Where community data are available, they are often incomplete or inaccurate.
- Where accurate and complete community data are available, it is difficult to isolate the impact of programs from other trends and environmental variables.
- It also seems that most groups simply do not know how to accurately assess their group’s performance. Some feel that it is too arduous to track a significant amount of data unless funding is proportional to their investment in program measurement.
Spay/neuter programs are faced with unique measurement challenges; groups and foundations use a variety of statistical and anecdotal indicators of performance.
- The widely accepted key performance measure for S/N programs is impact on community shelter intakes and euthanasia of healthy and adoptable animals.
- Some S/N programs track shelter intake and euthanasia data, but the analyses are typically limited in scope and unable to prove that S/N caused a positive trend.
- Some S/N agencies we contacted track dog/cat bites (and other calls to animal control) as indicators of the number of strays and unaltered animals in the community.
- For voucher programs, events, and clinics, the redemption or no-show rates are often tracked as indicators of success and also program efficiency.
- Some of the more organized S/N programs track animal and human demographic information for customers served, which improves their ability to target.
Regardless of the inability to accurately measure impact, there is widespread agreement among experts regarding a common set of S/N program best practices.
- There is strong agreement that S/N programs are most effective when targeted to low income and indigent human populations; hence there is also strong focus on low-cost S/N programs and those that make S/N more accessible to impoverished communities.
- Foundations and other experts also clearly agree that the most successful S/N programs are geographically concentrated, with significant community investment.Within this framework of best practices, however, different foundations use significantly different processes to measure grantee accountability and impact.
- All foundations contacted require a basic application or proposal from potential grantees, but they vary significantly in the level of requisite data and accountability.
- While all foundations interviewed have a strong interest in program evaluation, most do not expect grantees to conduct sophisticated data collection or analysis to demonstrate impact as a prerequisite for funding (Maddie’s Fund is an exception).
- Foundations, like the organizations they fund, are constrained by limited data, limited time, and also a desire to keep requirements manageable for grantees.
From a grantee’s perspective, despite concerns about having sufficient time or knowledge, they are willing to make an effort to meet grant requirements concerning program and impact evaluation.
- Regardless of the many challenges to program evaluation, it seems that most grantees are willing — and some are enthusiastic — about having stricter reporting requirements.
- However, these comments were often accompanied by caveats or concerns that grant requirements should also allow some flexibility and that foundations may need to provide significantly more guidance when asking for more details.
- For most foundations, it is clearly untenable to require that grantees provide a complete and accurate assessment of program impact. However, requesting such information and also requiring that grantees at least address grant impact in their follow-up reports are both viable options.
Reactions from grantees to a few “targeted grant” ideas being considered by the Handsel Foundation were mixed, with greater interest in training and conference participation than program evaluation or optimization.
- There was some confusion about the ideas discussed, but grantees are generally interested in various types of training as well as attending conferences. However, a few people feel this type of training is a waste of time.
- There was less interest in targeted grants for data analysis or program optimization, although this may be due to a lack of understanding. Interest in these grants increases somewhat when it was suggested that an outside consultant would do the work.