Archive for the ‘Data Point’ Category

« Older Entries
Newer Entries »

Data Point: Finding Out Whether Your Assistance to Grantees Makes a Difference

Thursday, March 1st, 2012

Believing that their foundations’ effectiveness depends on the effectiveness of those they fund, many foundation leaders have embraced the idea of adding value “beyond the money.” In the words of one program officer interviewed for our More Than Money research, “It can often mean the difference between making a grant and making an impact.” Yet strikingly little is known about the assistance beyond the grant that foundation program staff and the consultants they retain provide – or the impact of these efforts.

In our research, we learned that more than 80 percent of CEOs and 60 percent of program staff we surveyed indicated that the provision of assistance beyond the grant is important for the achievement of their programmatic goals. Even greater numbers of respondents – more than 80 percent of CEOs and program staff – view assistance beyond the grant as important for improving grantees’ abilities to achieve their own goals.

Despite the high proportion of CEOs and program staff who believe that assistance beyond the grant is important for the achievement of their goals – and their grantees’ goals – few know whether the assistance they provide is helping to achieve the intended result. We asked CEOs and program staff if their Foundation follows up with its grantees to understand the effects of the assistance provided, and the responses were sobering.

From our survey of 103 program officers and 98 CEOs at foundations with $100 million or more in assets, only about a third of each say they “always” follow up with grantees to understand the effects of the assistance they provided. Most report following up only “sometimes.” Follow-up activities include conversations with grantees, reading grantee reports, checking whether or not specific aspects of the assistance provided were in fact implemented or useful, and investigating whether or not the project or program receiving the assistance actually succeeded.

Even among those who say they “always” follow up though, their efforts do not yield much information about the actual effects of their assistance. Only 10 percent of these CEOs and program staff told us that they actually look for evidence of changes to understand the impact of assistance they provide.

* * * * * * * *

For a fuller exploration of nonmonetary assistance at large foundations, see the report More Than Money: Making a Difference with Assistance Beyond the Grant written by Ellie Buteau, Ph.D., Phil Buchanan, Cassie Bolanos, Andrea Brock, and Kelly Chang.

 

Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy.

 

 

Tags: , , ,
Posted in Assessing Performance, Data Point, Funder/Grantee Relationships | Comments Off

Data Point: Foundations’ Use of Logic Models and Theories of Change

Friday, February 3rd, 2012

The use of logic models and theories of change has been much discussed and debated in the field of philanthropy.

Some advocates for their use suggest that foundation staff members need to articulate the logic of how the foundation’s work will lead to the achievement of its goals if they are to have a chance of achieving them – and if they are ever to understand the impact of their efforts. One sector leader who holds this position is Paul Brest, president of the William and Flora Hewlett Foundation. As Brest and Hal Harvey, a former Environment Program director at Hewlett, wrote in Money Well Spent, “An intuitively plausible theory of change is better than none at all. [But] the more tested the theory of change, the sounder its use as the basis for a strategy.”

Others argue that the complexity of social problems, and the likely result of getting a logic model or theory of change wrong, means funders should not attempt to lay out the path to their desired goals. Bill Schambra of the Hudson Institute’s Bradley Center for Philanthropy and Civic Renewal Director, argues funders should not “worry about solving a problem in the abstract [or] thinking it through in advance with a bunch of flow charts and variables [because] it isn’t going to turn out the way you thought it was going to.”

So what is the state of practice among large foundations? Last year, CEP collected data on this issue through surveys of CEOs of foundations with $5 million or more in annual grantmaking and published a report that highlights trends in the field regarding the assessment of foundation performance.

The data show that:

  • 29 percent of CEOs report using a logic model or theory of change to guide all of their work.
  • 40 percent of CEOs report using a logic model or theory of change to guide some of their work.
  • 31 percent of CEOs report not using logic models or theories of change in any of their work.

CEOs who use these tools generally place assessing their foundation’s effectiveness as a higher priority than CEOs who do not. Specifically, 80 percent of CEOs who use logic models or theories of change rated the priority of assessing the foundation’s effectiveness a 6 or 7 (on a scale of 1 to 7, with 7 representing the highest priority). About 60 percent of the CEOs who do not use logic models or theories of change gave that high of a rating to this priority.

CEOs who report using logic models or theories of change also use different types of data when assessing their foundation’s programmatic work than those CEOs who do not. CEOs who use these tools are more likely to use surveys of the foundation’s grantees, conduct focus groups and convenings of their grantees, or carry out evaluations of the foundation’s program or issue areas to understand the effectiveness of their programmatic work.

To see an example of how Wilburforce Foundation has used a logic model to drive their strategic thinking, check out this recent guest post from Paul Beaudet.

What is your take on funders’ use of logic models?

Andrea Brock is Research Manager at CEP.

 

 

 

 

Tags: , , ,
Posted in Data Point, Developing Strategy | 2 Comments »

Data Point: Is Evaluation Resulting in Meaningful Insight for Foundations?

Friday, January 20th, 2012

For our recent State of Foundation Performance Assessment report, we gathered data from 173 CEOs of U.S. foundations with annual grantmaking of at least $5 million on the extent to which they conduct formal evaluations of their programmatic work. The data indicate that evaluation remains an essential component of how foundations approach assessment. More than 90 percent of CEOs report that their foundations conduct formal evaluations of their work, and a majority turns to third parties to conduct that work.

A large majority of CEOs report that formal evaluations have helped their foundations understand the effects of their programmatic work. Yet using evaluation well is not easy: fully 65 percent report that having evaluations result in meaningful insights for the foundation is a challenge.

In terms of resources, how much are foundations putting into evaluation? Most are conducting formal evaluations for half or fewer of their grants. The median spending on formal evaluation is two percent of a grantmaking budget.

 

* * * * * * * *

To read about current foundation CEOs’ attitudes toward assessment and what foundations are doing to understand their performance, see the report, The State of Foundation Performance Assessment: A Survey of Foundation CEOs written by Ellie Buteau, Ph.D. and Phil Buchanan and published by the Center for Effective Philanthropy.

Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy.

 

 

 

Tags: ,
Posted in Assessing Performance, Data Point | Comments Off

Data Point: How Can Foundations Help Grantees Secure Funding from Other Sources?

Friday, December 16th, 2011

In a recent data point post, we shared that the most significant factor program staff consider when determining what type of assistance beyond the grant to provide is specific requests from grantees. In our research, program staff also told us that the most frequent assistance beyond the grant request they receive from grantees is help raising money from other sources.

For our survey of 103 program officers at foundations with $100 million or more in assets, we listed a range of ways foundations could assist their grantees in obtaining additional funding from other sources, and asked respondents to select all the ways in which they do so.

The typical foundation provides just 22 percent of its grantees with assistance securing funding from other sources. The most frequent way program staff assist grantees in obtaining additional funding is by suggesting other funders. But, grantees do not rate the impact of this type of assistance much differently than not receiving any suggestions at all. Perhaps this is because most foundation grantees are already well aware of other potential funders.

The typical foundation goes beyond simply suggesting other funders for only 12 percent of its grantees. However, when foundations do more, either by introducing grantees to other funders or attending meetings with other funders, it makes a difference. Grantees rate the impact of this assistance securing funding higher than grantees not receiving these activities.

 

* * * * * * * *

What is the current state of assistance beyond the grant at large foundations? For a fuller exploration of nonmonetary assistance, see the report More Than Money: Making a Difference with Assistance Beyond the Grant written by Ellie Buteau, Ph.D., Phil Buchanan, Cassie Bolanos, Andrea Brock, and Kelly Chang.

Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy.

 

 

 

Tags: , ,
Posted in Data Point, Funder/Grantee Relationships | Comments Off

Data Point: What Information are Foundations Using to Assess Their Programmatic Work?

Friday, December 9th, 2011

For our recent State of Foundation Performance Assessment report, we gathered data from 173 CEOs of U.S. foundations with annual grantmaking of at least $5 million on what types of information they use to assess the effectiveness of their foundations’ work – operations, finance, and programmatic. Compared to a similar survey we completed almost a decade ago, it appears that foundation CEOs today are drawing on an increasingly broad array of performance indicators.

In this survey, we listed a range of types of information CEOs may be using to assess the effectiveness of their foundations’ work, and CEOs could select as many as applicable.

On average, CEOs report using seven types of information to understand their foundations’ programmatic effectiveness. Almost all foundations are using anecdotal feedback, written reports from grantees, site visits, and evaluations – either of individual grants, clusters of grants, or program areas.

A minority, however, seek information from their ultimate beneficiaries – the people whose lives foundations are ultimately trying to affect – either through surveys, focus groups, or convenings. When we compare responses of CEOs whose foundations do and do not seek the voices of their ultimate beneficiaries, two important differences emerge: CEOs who report that their foundation does collect beneficiary feedback rate themselves as having 1) a better understanding of the progress their foundation is making against its strategies, and 2) a more accurate understanding of the impact the foundation is having on the communities and fields in which it works.

 * * * * * * * *

To read about current foundation CEOs’ attitudes toward assessment and what foundations are doing to understand their performance, see the report, The State of Foundation Performance Assessment: A Survey of Foundation CEOs written by Ellie Buteau, Ph.D. and Phil Buchanan and published by the Center for Effective Philanthropy.

Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy.

 

Tags: , ,
Posted in Assessing Performance, Data Point | Comments Off

« Older Entries
Newer Entries »