New Efforts to Improve the Grantee Perception Report

Amber Bradley
by Amber Bradley
March 28th, 2012
 

At CEP, we are driven by the belief that feedback can fuel positive change. Since joining CEP I’ve been proud that we walk this talk with our own work; each year — as a part of our internal efforts towards assessment and ongoing improvement — we conduct a survey of funders participating in the Grantee Perception Report (GPR). Last year, we once again commissioned LFA Group, an independent third party, to collect confidential feedback from users of CEP’s GPR.

The results, available on our website here, show that, as in past years, 2011 respondents indicated a high level of satisfaction with the GPR experience overall. I am particularly pleased to see that a full one hundred percent of respondents reported they made changes in their work based on the feedback they received from grantees.

  • 97 percent changed their communications with grantees;
  • 81 percent revised their grantmaking processes; and
  • 34 percent changed their foundation strategy.

As the CEP manager in charge of our GPR process, it is gratifying to see that the GPR continues to have a meaningful impact on foundations’ work.

However, the results are not all what I would have hoped to see. We learned that 2011 respondents’ ratings dropped from past levels on several aspects of the GPR, including the usefulness of the GPR relative to other processes, the value of the GPR relative to its cost, and the clarity of graphical presentation of GPR data. Although there have been ups and downs in some of these ratings over the years, this year felt different. We saw numerous areas drop, and learned that, among the 2011 respondents, first-time users of the GPR tended to be substantially less satisfied with the experience as compared to repeat subscribers.

Needless to say, for an organization that strives for excellence, these drops in ratings are sobering. Personally, the results provide me with the challenge and opportunity that many of our assessment tool subscribers encounter when receiving feedback through a CEP tool: How can I move past disappointing feedback and digest results in a positive, forward-looking way? How can CEP make improvements that are responsive to the feedback we’ve been offered by those who understand our work best?

The results, together with comments from GPR subscribers, suggest that the value and usefulness of the GPR could be improved by a more intuitive visual display of data, and through more responsiveness to the particular contexts of individual funders. The feedback gives us a lot to consider, and we’ll continue to use it for ongoing improvement. At present, we’ve identified two steps we intend to take to respond to recent GPR users’ suggestions:

  1. Improve visual representation of data. Constructive feedback about our charts and report formats is not new, and is something we’ve been considering for a long time (see for example Kevin Bolduc’s previous post). This area has, admittedly, been challenging and is one where we’ve been hesitant to make major changes, in part because some repeat users of the GPR very much like the format. But now, based in part on recent GPR users’ feedback, we have made a firm commitment to getting it done: we will change the presentation of our data, with the goal of creating a more intuitive and accessible display of information.
  2. Increase support to GPR users. Though all 2011 respondents indicated successfully making changes based on their GPR results, we learned that “lack of time” and “unclear next steps” were the most common challenges they faced. In order to maintain our integrity as a neutral third party that remains faithful, always, to the data, it is important that CEP not adopt a traditional consulting role – spelling out must-do steps for assessment tool subscribers. However, we believe we can do more to facilitate data-driven change based on individual GPR results, our field-wide research, and the extensive network of GPR subscribers who have learned from the GPR process. A preliminary step we plan to take is to have follow-up conversations with assessment tool users a few months after the final presentation of their results. Our hope is that this simple step will create an additional opportunity for us to learn from tool users about their ongoing work and the unique challenges they face, and will create further opportunities for CEP to share insights, advice, research and case studies, and connections to other funders. Beyond this, we will continue to look for ways to increase the support we provide to tool users, while maintaining the neutrality and integrity of our assessments.

We’re excited about the opportunity that GPR users’ feedback affords us, and are pushing forward on these changes with the goal of improving the relevance and utility of our work. We will continue to seek input and candid feedback about how we can make all of our work most successful in improving foundation performance.

 

Amber Bradley is a Manager at the Center for Effective Philanthropy.

Data Point: What are Foundation CEOs’ Attitudes Toward Assessment?

Ellie Buteau, PhD
by Ellie Buteau, PhD
March 22nd, 2012
 

One of the key findings in our recent State of Foundation Performance Assessment report was that CEOs place great importance on assessing their foundations’ effectiveness. Nearly three-quarters of foundation CEOs say assessment of foundation effectiveness is among their highest priorities.

The recent focus on foundation performance assessment has provoked some backlash, however. Some have raised questions – in op eds, blogs, and on the conference circuit – about whether there should be greater emphasis on “intuition” and less on data in philanthropic decision-making. Few CEOs who responded to our survey seem to agree with this way of thinking.

We gathered data from 173 CEOs of U.S. foundations with annual grantmaking of at least $5 million on their attitudes toward performance assessment. Only a small minority, 19 percent, believe that more emphasis should be placed on intuition.

That said, the majority of CEOs we surveyed see tension between the freedom to take risks on innovative ideas and a focus on performance assessment.

Although these CEOs believe foundations have greatly improved their practices in the past decade, they also feel that further progress is needed. More than 60 percent say that too few foundations understand their overall performance today.

* * * * * * * *

To read about current foundation CEOs’ attitudes toward assessment and what foundations are doing to understand their performance, see the report, The State of Foundation Performance Assessment: A Survey of Foundation CEOs published by the Center for Effective Philanthropy.

 

Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy.

 

 

To What End?

Phil Buchanan
by Phil Buchanan
March 20th, 2012
 

The following post originally appeared on Beth Kanter’s blog.

The just-concluded GEO conference, held in Seattle last week, was an impressive assembly of hundreds of funders who gathered to focus on pushing for greater effectiveness in pursuit of their intended impact.

On the one hand, I was impressed by the quality of conversation and sessions during the time I was there. On the other hand, I kept wondering whether, in all the excitement about seemingly “new” models and approaches (which often are not so new), we are losing sight of the basics.

My worry is that, too often, we do not root our discussions in the essentials of goals and strategy.

Discussions about tactics and approaches – from PRIs to capacity building to scaling to networks to use of social media – happen outside of a context of clarity about the most fundamental questions: What are our goals? Given those goals, what are the best strategies to achieve them?

It is only in this context that funders can make intelligent choices about which of the many tools and frameworks offered up at a conference like GEO make sense for them. But, too often, that crucial context gets lost in the chase of the shiny new trend.

One effort to address this problem, in which I wish more funders would participate, is the Charting Impact initiative. Charting Impact is a strategic alliance among Independent Sector, BBB Wise Giving Alliance, and GuideStar. As my CEP colleague Kevin Bolduc, who served as an advisor to this effort, recently noted on the CEP Blog, foundations are few and far between on the list of those who have publicly addressed the five “deceptively simple questions” that Charting Impact asks of organizations.

The questions are:

  1. What is your organization aiming to accomplish?
  2. What are your strategies for making this happen?
  3. What are your organization’s capabilities for doing this?
  4. How will your organization know if you are making progress?
  5. What have and haven’t you accomplished so far?

Wouldn’t it be great to see every funder who is a member of GEO answer these questions publicly? Perhaps GEO could even make participation a prerequisite to membership?

For many, answering the questions Charting Impact asks would be easy. For others, just seeking to answer these questions would force important discussions among foundation staff and board members and, perhaps, crucial steps to gain greater clarity on the answers. It could also help funders make better decisions about which approaches are right for them. At the very least, it would send a message to nonprofits that their funders are walking the talk when it comes to clarity on these matters.

So I hope funders will follow the lead of peers like the William and Flora Hewlett Foundation and the Tustin Community Foundation and participate in this initiative. (Full disclosure: Hewlett is a grant funder of CEP and a user of our assessment tools.)

Let’s root our conversations and thinking in the crucial context of the particular goals and strategies we are pursuing. Otherwise, all our talk of innovative, new approaches will be just that: talk.

Phil Buchanan is President of the Center for Effective Philanthropy.

Walking the Talk: Collecting Feedback from Our Donor Perception Report Subscribers

Grace Nicolette
by Grace Nicolette
March 14th, 2012
 

At CEP, we are in the business of gathering feedback. A significant part of our work involves helping philanthropic funders gather comparative feedback from key stakeholders.

But we need feedback on our performance, too, and so we regularly seek to understand how users of CEP’s assessment tools perceive us – and whether or not they are making changes in response to the findings they receive. After all, if our tools do not result in meaningful change, what’s the point?

So last year, we commissioned an independent evaluator (LFA Group: Learning for Action) to survey 2009-2010 subscribers of our Donor Perception Report (DPR), a newer tool that aims to help community foundations understand how they are doing in the eyes of their donors. The DPR seeks to put community foundation performance as judged by donors in a comparative context, just as CEP’s Grantee Perception Report (GPR) does for grantee perspectives – highlighting both relative strengths and areas for improvement. Since 2009, over 30 community foundations across the U.S., large and small, ranging from the Chicago Community Trust to the Dallas Foundation and the Napa Valley Community Foundation, have participated in a DPR.

As the manager at CEP responsible for this tool, I was eager to better understand whether the tool was making a difference. LFA focused on foundations that had used the DPR at least a year prior to the survey, in order to gather data on what changes had been made, so the number of foundations studied was small – just 11. Nonetheless, the findings give us an early indication of the tool’s utility. Here is some of what we have learned about this tool from this early assessment (you can also download the full report):

  • Satisfaction: Respondents indicated a high average level of satisfaction (at least a 6 on a scale of 1 to 7, with 1 being “not at all satisfied” and 7 being “very satisfied”) on five out of six key areas of their experience, including the DPR experience overall and the extent to which the DPR helped deepen the foundation’s understanding of its donors’ needs/interests. Specific themes that emerged from the interviews were high satisfaction with the design of the donor survey instrument and with the resulting report. (See page 6 of the report)
  • Creating Change: 100 percent of respondents reported that the DPR drove some changes in their work, with 55 percent indicating they made “significant change” in at least one area of their foundation practices or strategy. Most subscribers reported at least some change in their approach to working with current donors, engagement of new donors, foundation strategy, and collaboration among donor staff and others in the foundation. (Page 13)
  • CEP Staff Helpfulness in the Process: On a 7-point scale (where a score of 7 is associated with “very responsive” or “very helpful”), the mean score for responsiveness of CEP staff to questions was 6.7, and the mean score for helpfulness of CEP staff responses was 6.9. (Page 7)
  • Opportunities for Improvement: Subscribers shared several ideas for how to enhance the value of the DPR experience, including:
  1. increasing the size of the comparative data set;
  2. having CEP provide a “refresher” or continued access to comparative data as new community foundations are added;
  3. increasing in-person contact and dialogue around how to implement changes suggested in the findings;
  4. developing more examples of how other similar community foundations had successfully implemented changes based on DPR results. (Page 7)

At CEP, we are committed to seeking feedback on an ongoing basis to help improve our processes. We are very gratified that this initial feedback about the tool and CEP’s work with DPR users were both viewed quite positively. Still, we have begun to address several of the areas for improvement for this year’s subscribers, and we will be commissioning a similar assessment next year in our effort to continue to learn and improve.

Grace Nicolette is a Manager at CEP.

The 7 Habits of Highly Ineffective Foundation Boards

Phil Buchanan
by Phil Buchanan
March 9th, 2012
 

In my work with foundations over the past decade, it has become clear to me that, while there are lots of effective and well-intentioned board members out there, it’s easy for foundation boards to get lulled into complacency and slip into bad habits. Here are what I regard as the 7 Habits of Highly Ineffective Foundation Boards, drawn from CEP’s research on governance but also from my own observations – having met with dozens and dozens of foundation boards.

  1. Assuming success in philanthropy is as simple as success in business. This habit, often fueled by the business press, is as widespread as it is damaging. A failure to recognize that philanthropy is, almost by definition, working on the toughest challenges – the ones that business and government have not solved – leads to all kinds of problems. Here is a sampling: Unrealistic expectations, failure to see that success takes time and patience, casting about for a “silver bullet,” and a reductionist and ultimately unhelpful approach to performance assessment … which leads me to my second habit.
  2. Not doing the hard work of overall foundation performance assessment. Boards often ask unrealistically for a single ratio by which to gauge programmatic performance – failing to recognize that no analog to share appreciation or profitability exists in philanthropy (see Habit 1). When that isn’t offered up by staff, because it can’t be, board members throw up their hands – perhaps gravitating instead toward that which is more easily quantified and compared: endowment performance and administrative cost ratios. CEP’s research has documented the fact that foundation CEOs actually want their boards more involved in assessment, but to get there they and their boards need to buckle down in a way that many don’t. Good foundation performance assessment requires clear goals, coherent strategies (rooted in solid logic), and a set of indicators that tie to strategies. It requires investment: financial investment as well as the time necessary to learn about the programs on which the foundation works. It requires an “indicators” approach – a recognition that a variety of data sources need to be tapped to answer the deceptively simple, but all important, question: How are we doing?
  3. Living in the “foundation bubble” by failing to create feedback mechanisms. Any approach to assessment needs to include candid, comparative feedback from grantees and other key audiences so boards don’t live in the bubble of praise that almost inevitably forms around those affiliated with large foundations. It amazes me how often I have heard foundation board members offer up cocktail-chatter vignettes – what they heard from a grantee at a reception – as evidence of effectiveness, without pausing to reflect on whether what they heard was perhaps less than the unvarnished, candid, full truth. It also amazes me that boards often don’t hear in any real or meaningful way from those who should matter most: the intended beneficiaries of their foundations’ programmatic efforts. (CEP’s YouthTruth initiative is one attempt to remedy this.)
  4. Believing their foundation’s resources are vast relative to the problems being addressed. Foundation boards tend to overestimate the resources of their foundation, focusing on asset size rather than payout (“we’re a billion dollar foundation!”) or, even when focusing on the payout, failing to properly contextualize the number. Almost inevitably, a foundation’s resources are slim relative to the problem being addressed. This makes both focus and collaboration key if goals are to be achieved. As Joel Fleishman and Tom Tierney put it in their book, Give Smart, “One of philanthropy’s great ironies is that very little can be accomplished by individuals acting on their own, even when those individuals are extraordinarily wealthy.” The same could be said of institutions, and boards are wise to remember this.
  5. Not learning from history. Harry S. Truman once said, “There is nothing new in the world except the history you do not know.” As I recently discussed in a blog post on Olivier Zunz’s important new history of American philanthropy, this feels particularly true in philanthropy. So much of what is positioned as new – from “collective impact” to policy and advocacy work to a focus on root causes – is nearly as old as American philanthropy. Why does this matter? Because we can learn from that history. Ignoring past successes and failures benefits no one except perhaps the burgeoning field of philanthropic management consultants – too many of whose business models are built on selling “new” frameworks and approaches that are, in fact, demonstrably very old.
  6. Believing it is the board’s fiduciary responsibility to discuss and approve every grant. Many boards get so bogged down in considering individual grants that they never look up and ask what it all amounts to or how it’s all coming together. Grants get discussed in the absence of clarity on goals and strategies, and, ironically, more resources are wasted than if the board delegated more grantmaking authority to staff – but with greater clarity about the criteria and logic that should guide them. Boards need to clear the time for important discussions about goals and strategies, and that requires giving up on discussing each $25,000 grant.
  7. Over-managing and over-scripting meetings. Too many foundation boards, aided and abetted by staff, over-manage and over-script meetings such that the most spontaneous thing to happen is someone standing up to get a refill on a coffee. Collegiality gets valued over debate – indeed, the Chair and CEO often try to pre-wire everything such that no real dissent and debate occurs at the full board meeting. This leads to board meetings that add little value and, incidentally, are also utterly dull. (As Patrick Lencioni points out in his book, Death By Meeting, one of the reasons people hate meetings – and tune out during them – is that there is no drama. Not the bad kind of reality-TV drama but the good kind of “I’m not sure how this is going to turn out” drama.) Often, an inner circle makes the tough decisions in advance, leaving the rest wondering what their role is. Important debates that could elucidate issues and lead to new insights either don’t happen or don’t see the light of day and, as a result, decision-making suffers. We’ve seen in our research on foundation boards that the boards that see themselves as most effective are the ones that value respectful dissent – among board members and between board and staff.

Foundations are unique in our society, as so many philanthropy scholars and observers have noted, because of the freedom they enjoy. That freedom can be a tremendous asset: liberated from the pressure to satisfy customers or shareholders or win voters, foundations can take on the issues other actors in our society can’t or won’t. But it can also be a huge liability: foundations can coast along, making little difference, sometimes even cultivating an aura of effectiveness on the conference circuit along the way.

The board, and perhaps only the board, can tip the balance in terms of which way a foundation goes. Staying away from the bad habits I have described can help assure foundations make the most of their opportunity to make a difference.

Phil Buchanan is President of CEP. He most recently blogged on the importance of looking within the nonprofit sector for examples of data-driven organizations.