Earlier in my career, I consulted for a number of nonprofit organizations, helping them think about how they could collect, analyze, and interpret data. What struck me at almost every one of these nonprofits was the lack of resources and knowledge they had to work with data. Perhaps because of this, a major frustration I experienced was that organizations often called me in too late in the process. They had already collected the data they wanted me to analyze in order to answer key questions about their work or programs. Unfortunately, I often had to deliver the unsettling news that I could not answer the questions they wanted me to answer with the data they had collected. In those days, I was repeatedly reminded of a quote from a research design textbook written by Harvard professors Light, Singer, and Willett: “You can’t fix by analysis what you bungled by design.”
Years later, at CEP, my colleagues and I have collected data from foundation leaders across the country about their struggles, their goals, and their attempts to understand the difference they are making in society. We’ve learned that one of the more common challenges foundation program officers face when trying to assess the progress their foundations are making is that nonprofits don’t have the capacity or the skills to be able to assess their own progress on the grants they receive. We’ve also learned that although foundation CEOs rely on the data they collect from their grantees in order to understand their own foundation’s impact, they believe that nonprofits need to be held to higher standards to demonstrate their effectiveness.
The nonprofit perspective is too often missing in conversations about nonprofits, though. So, in April of this year, we administered a survey about nonprofit performance measurement and management to the 300 nonprofit leaders who agreed to be a part of CEP’s The Grantee Voice: Feedback for Foundations panel.
In a study we are releasing today, titled Room for Improvement: Foundations’ Support of Nonprofit Performance Assessment, my colleagues Andrea Brock, An-Li Herring, and I share what we heard loud and clear from the 177 nonprofit leaders who responded to our survey: They do care about understanding the progress their organizations are making, and they are trying to collect data to help them understand it. What they don’t have is help from foundations: Most nonprofit leaders told us that they receive no help—no monetary and no non-monetary help—from foundation funders to do this work, but they want help. They want funding to help them assess their performance, and they want to be discussing with their funders how to do this work and what they are learning from it.
Across the data we have collected from foundations and nonprofits, we see that both sides of the funding equation are experiencing frustration. Foundations have some frustration with what they perceive nonprofits not to be doing to assess their work, and nonprofits are not getting the help they want or need from foundations to do this work well. It seems that foundations and nonprofits have a crucial shared goal, but there is much room for improvement in how they work together to achieve it.
Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy. You can find her on Twitter @EButeau_CEP and join the conversation about this report at hashtag #granteevoice.
- Data Point: How Helpful Do Grantees Find Foundation Reporting and Evaluation Processes? (Not Very.)
- The Case for Foundation Performance Assessment
- Wearing it Proudly: Clarity on Being Nonprofit
- Avoiding a False Choice (Why We Need Both ‘Morality’ and Assessment)
- Funders Should Do More to Help Nonprofits Build Evidence
Thank you for the thoughtful post and research report. There are a number of important take-aways, but I wanted to highlight two that I think are sometimes misunderstood. First, while the research shows that NPs want to build some staff capacity, it does not say that they are looking for deep evaluation capacity building. The type of evaluation capacity needed within organizations is frequently more of what TCC Group has been calling a research and develop approach to evaluation–a focus on short-term outcomes that have a direct relationship to strategies implemented. When thinking about longer-term outcomes (and even some of the skills for more nuanced short-term analysis) (both things that funders are frequently more interested in) it has been our experience that there are some things that NPs need access to specific expertise or VERY deep training of staff (which may not be cost effective).
Second, the research seems to highlight the importance of a foundation’s role in not just monetarily supporting evaluation, but in playing an engaging role. Both capacity building of funders to do evaluation, such as a current effort in Kansas for funders working together relating to advocacy evaluation, and funders doing some evaluation FOR grantees/groups of grantees are important. This latter is something that funders frequently overlook as a tool–they can do/support some higher level evaluation activities that it is unreasonable to expect from individual grantees. TCC has been building this into the evaluation system design of recent funders we have been working with, with great interest and excitement from grantees.
Thanks again for raising this important issue.
Thank you for your comments, Jared.
You’re correct, in our survey we did not ask specifically about evaluation capacity. Right now, there seems to be such a specific focus on evaluation in the sector and less of a focus on overall performance assessment and how performance data is being used by nonprofits to manage the way they work. We focused on the broader issue of performance assessment to understand how an organization is progressing against its goals, the capacity it has to do that work, and, the extent to which foundations are supporting those efforts.
I agree with your thoughts about the time it takes to develop expertise to do this type of work and that there can be times when outside expertise is needed because the skills simply don’t exist inside the organizations themselves. But, if nonprofits are truly to incorporate performance assessment practices into the way they work, they would best be served to grow the skills to do performance assessment work inside their organizations where it can be woven into the fabric of the way they work and manage their work. That said, they still may not have the skills to carry out an evaluation – which is a more specific type of assessment – for which outside organizations would remain a crucial resource.