Posts Tagged ‘Tactical Philanthropy’

Listening to Beneficiaries: Why Are We So Bad At It?

Thursday, September 22nd, 2011

CEP has been actively exploring the topic of beneficiary feedback through its YouthTruth project since 2008. We were very pleased to share what we are learning – and to challenge the status quo — on the Tactical Philanthropy blog this week, where the following post was originally published.

 

The power of feedback is almost universally acknowledged. Every day, millions of people turn to reviews written by end-users in Yelp, Trip Advisor, GreatNonprofits, and Global Giving to inform their purchasing and philanthropic decisions.  The end-user’s perspective on their experience is seen as an asset just about everywhere—except in the world of foundations.

Consider a recent report published by the Center for Effective Philanthropy (CEP). The State of Foundation Performance: A Survey of Foundation CEOs, updates work CEP completed a decade ago, looking at the types of data foundations use to assess their performance.  While the report shows that foundations are increasingly looking to a range of indicators, notably surveys of grantees, only about a quarter of funders gather feedback from beneficiaries in any way. This, despite analysis showing that it may be among the most valuable types of feedback. CEOs of foundations who collect beneficiary feedback rate themselves as having a better understanding of the progress their foundation is making against its strategies and a more accurate understanding of the impact the foundation is having on the communities and fields in which it works.

As head of CEP’s YouthTruth initiative, which gathers comparative feedback from students – the ultimate beneficiaries of education reform efforts, I have seen the power beneficiary feedback can have in informing local school change efforts. Consider this recent post about The California Endowment’s YouthTruth experience.  So the question I ask myself is this: How can we better understand why foundations have not seriously embraced beneficiary feedback, and how can we encourage its use?

To elevate beneficiary perspectives, I am convinced that at least three things need to happen.

  • We need to believe as a field that beneficiaries’ perspectives matter.
  • Foundation leaders need to understand the connection between beneficiaries’ perspectives and good strategy – to acknowledge that this data has the potential to inform funders’ theory of change.
  • And we need easy-to-use tools to collect and make sense of the feedback as we acquire it.

The third area is where we have made the most progress, as we are increasingly well-positioned with new tools for gathering beneficiary feedback. Now we need to push ourselves on these first two points.

With regard to the first, we need to address the biases that convince some in the field that beneficiary feedback is less valuable or objective than the opinions of say, grantees or donors.

Daniel Sid describes the situation this way on Bridgespan’s blog:  “the working assumption of the suppliers [of nonprofit human services] and their funders alike is that they know what is best for the individuals and families they are serving… reducing [beneficiaries] to passive recipients… And why bother [gathering feedback] when your beneficiaries can’t really take their business elsewhere?”

With regard to the second point, can foundations expect to learn anything that will enable them to make different decisions from beneficiary feedback? Speaking from my own experience, we don’t know for sure but we are getting close.

YouthTruth was started by CEP, with support from the Bill & Melinda Gates Foundation to test this very notion. To date, YouthTruth has gathered perspectives from more than 71,000 students from 164 schools nationwide. With this dataset, we have been able to examine the degree to which there are systematic differences in student perspectives based on the type of school they attend.

Moreover, we have the ability to identify differences in student perceptions, say about their futures, across groups implementing a common strategy, such as a STEM strategy, and provide insights about the relative effectiveness of various efforts. This to me seems like helpful and actionable data that can inform funders’ decision-making. While this is one isolated example, we need many more experiments such as YouthTruth to push our thinking about the benefits and limits of beneficiary feedback.

I would argue that since beneficiaries are ultimately the people we are all trying to serve, we need to take dramatic steps to take their perspectives into account.

We need to ask each other the tough questions about why this feedback is not being more widely used and challenge ourselves to determine the potential of beneficiary feedback once and for all.

Valerie Threlfall is vice president – youthtruth initiative at CEP.

Tags: , , , ,
Posted in Constituent Feedback | No Comments »

Has Anything Really Changed in Philanthropy?

Monday, April 12th, 2010

Sean Stannard-Stockton, CEO of Tactical Philanthropy Advisors, and founder of the terrific blog Tactical Philanthropy, invited me to join a team of bloggers sharing thoughts on the GEO conference this week in Pittsburgh.  What follows are some musings on what’s changed since the first GEO conference I attended in 2002.  To link to this and the other guest blogs on the Tactical Philanthropy site, go to http://tacticalphilanthropy.com/2010/04/has-anything-really-changed-in-philanthropy.

The first Grantmakers for Effective Organizations (GEO) conference I attended was in Washington, D.C., in 2002.  It was tough to get in: GEO wouldn’t let me register because I was the executive director of a tiny (four staff) organization that no one had ever heard of  – the Center for Effective Philanthropy (CEP). I said, “Come on, we share a middle name.  We’re practically related!”  But they weren’t having it. 

So, with a little help from Vince Stehle, then program officer at the Surdna Foundation (one of CEP’s initial funders), I registered as if I worked for Surdna and showed up.  Vince hosted a breakfast roundtable to discuss CEP’s first report, on overall foundation performance assessment, and I was thrilled when we had to pull two tables together because 17 people showed up.

Fast-forward to the eve of the GEO conference in 2010 and allow me to switch – as my 9-year old daughter often does when she writes – to the second person and say this: you are expecting me now to discuss “how much has changed” when it comes to funder effectiveness and assessment since that conference almost a decade ago.  And, I will, but it’s not quite so simple.

Fact is, I think we tend, in philanthropy (and maybe in life), to overlook historical examples and proclaim that something is “new” when it is really only new to us. 

The media, and even those within philanthropy, often speak as if the focus on impact, outcomes, measurable results – whatever we call it – is brand new when, of course, it is not.  As Ed Skloot, now of Duke University, has noted, “Our earliest American philanthropic ancestors—John D. Rockefeller, Margaret Olivia Sage . . . , Andrew Carnegie — bequeathed a kind of courage and determination to us, a confidence we can take on large-scale problems with deeply rooted causes.”  Bill Schambra, of the Hudson Institute, also argues – although for him, unlike Skloot, it is a lament –  that the “mania to measure” goes back to the “first days” of the Rockefeller Foundation.

So, let’s not pretend that, just because our friends at McKinsey have launched a Web site focused on impact assessment, the concept was invented yesterday.  But, at the same time, it’s clear there has been dramatic change over the last decade in philanthropy.  New donors have brought new ideas, new and creative approaches, and new energy to the table. Numerous organizations that were either fledgling or non-existent a decade ago are now providing crucial data on nonprofits to funders:  Guidestar, Charity Navigator, Philanthropedia, Great Nonprofits.  And the list goes on.  And then there are the organizations focused on funder effectiveness:  GEO, GrantCraft, CEP.

But are funders operating more effectively?  Are they achieving more impact?  We at CEP see real signs of hope  – funders making clear their goals, strategies, and performance indicators; funders getting – and acting on – feedback from crucial stakeholders, from grantees to intended beneficiaries; boards really holding CEOs accountable for performance. 

I can point to some inspiring examples of present-day foundation effectiveness and impact, from the Stuart Foundation’s Child Welfare Program to the Wilburforce Foundation’s work protecting wildlife habitat to the Gill Foundation’s work on gay rights to the groundbreaking and much-discussed work of the Edna McConnell Clark Foundation.  And we at CEP have some new data we’ve been analyzing that will suggest some movement in the right direction. But, quite frankly, the movement is less dramatic than many of us would hope.

The reality is that there are still far too many staffed foundations doing significant (in dollars) grantmaking that aren’t even clear on what, exactly, they’re trying to do, much less how they’ll do it or how they’ll know if they’ve been successful.  So, my question is, how much really has changed since that conference in 2002? 

In my less optimistic moments, I wonder: is it possible that there has always been a subset of foundations that really care about effectiveness and impact and that their proportion of the overall number of foundations hasn’t changed much since the early days of Carnegie, Rockefeller, and Sage – even as the resources available to help them have grown and improved?  (A related question: how common is it that the very same foundation goes through periods of being highly effective – with clear goals, coherent strategies, and good performance indicators – and then other periods of being unfocused and adrift? I have seen many examples of this, suggesting that it’s not about “new” philanthropy or “old” philanthropy but rather “effective” philanthropy.)

My fundamental question is, how much change, really, has there been? Today, we can’t really answer that question as definitively as I would hope.  I’d be curious to know what others think.

(Disclosure: The foundations I mention above are among the more than 200 we have either provided assessment tools to or received funding from – and the former CEO of Edna McConnell Clark sits on our Board of Directors.)

Tags: , , , , , ,
Posted in Assessing Performance, Developing Strategy, Funder/Grantee Relationships, Uncategorized | No Comments »

Can Failure Be the Key to Foundation Effectiveness?

Monday, January 11th, 2010

Philanthropic failure – when there is clear evidence that a project, a program, a strategy or any combination of foundation investments designed to achieve a goal doesn’t work –  is beginning to gain a critical mass of attention.  From Annie E. Casey’s New Futures program to more recent examples from Irvine, Hewlett, and Gates, foundations are sharing what they have learned about things that didn’t work.

Scholarly books – by Joel Fleishman, Paul Brest and Hal Harvey, and Peter Frumkin comment on the value of failure.  Indeed, Fleishman devotes an entire chapter to how foundations fail and why it is so important to make sure the public knows about the failures.

Bloggers, notably Sean Stannard-Stockton on Tactical Philanthropy, have been keeping the drumbeat going about the importance of failure to philanthropy.  And my own foundation, Robert Wood Johnson, recently expanded the number of examples by publishing four chapters about numerous failures in its latest Anthology (Full disclosure – I authored one of those chapters and have drawn on that work in this series of CEP posts).

Review of the many RWJF programs that failed in one way or another produced four broad categories of failure:

  • Difficult environment
  • Strategy or design flaws
  • Faulty execution
  • Inability to adapt in a timely fashion

Why is failure valuable?  A principle reason is that it promotes organizational learning.  Foundations can get better at achieving their goals by understanding what happened when they tried and didn’t achieve their goals.  Like “after-action reviews” in the Army, carefully looking at past performance improves future performance.

Another value is that failures produce important learning for the field.  Almost no foundation is alone in its aims; others can take lessons and build on the mistakes.  To do that, reports of failures must go beyond noting that an initiative failed to explain why it failed.

Gates, for example, has laudably commented in several venues about the disappointing results from their investments in small schools.  Further information that explicated the logic for investing in small schools in the first place, and what they have learned from their investments relative to that logic, would go a long way to promoting more refined investments in school improvements – perhaps, under certain circumstances, even in certain types of small schools.

In a broader framework, sharing failures can also build trust for foundations – with grantees, stakeholders, and the public.  A foundation that is honest about things that don’t work establishes a basis for receptivity to honest communications about successes.

Given the benefits of openly sharing failures, it is reasonable to ask why reports of foundation failures have been infrequent.  One possibility is that foundations are ashamed or embarrassed by their failures.

Brest and Harvey in their book Money Well Spent note it may be not that foundations want to hide their failures so much as foundations don’t spend enough on evaluation, so they don’t know if they have failed.  Neglecting to invest in evaluation coupled with a lack of clear goals (mentioned in a previous post) to use as yardsticks for performance in the first place make failure, or success, impossible to judge.  In this view, the paucity of reports of foundation failure arises from the absence of an organized way to know much about performance at all, beyond impressions and examples.

By focusing on failure, a foundation is forced to have the goals and the measurement needed to assess performance, whether it is failure, success, or – much more often the case – something in between that can form the basis of learning.  When failures are able to be detected, there are other reasons they are difficult to publicize.  Failures involve people, institutions, and reputations that may be harmed through full disclosure.  Failures have the risk of jeopardizing future funding.

And failures can puncture deeply held beliefs about what works and why in bringing about social change.  They can generate conflict and disagreement among people with common aims and values.  So failure can be beneficial, but the learning and benefits to be garnered by focusing on failure are not without risks and potential hardships.

Failure is in one sense an indication that the strategic work that many foundations espouse to is very different in character from charity.  The term philanthropy is an umbrella that covers the full range of giving.  Perhaps one way to distinguish the two, and thus be able to judge effectiveness, is to ask if the philanthropic giving could fail.  If it is not possible to fail, it is not possible to judge effectiveness.

Bob Hughes is an independent consultant on strategy and organizational learning in health and philanthropy.

*****

Disclaimers and Disclosures: The views expressed in the CEP blog by guest bloggers are entirely their own and do not necessarily reflect the opinions of the Center for Effective Philanthropy.

Tags: , , , , , , , , ,
Posted in Assessing Performance | 8 Comments »