Archive for the ‘Assessing Performance’ Category

Data Point: Is Evaluation Resulting in Meaningful Insight for Foundations?

Friday, January 20th, 2012

For our recent State of Foundation Performance Assessment report, we gathered data from 173 CEOs of U.S. foundations with annual grantmaking of at least $5 million on the extent to which they conduct formal evaluations of their programmatic work. The data indicate that evaluation remains an essential component of how foundations approach assessment. More than 90 percent of CEOs report that their foundations conduct formal evaluations of their work, and a majority turns to third parties to conduct that work.

A large majority of CEOs report that formal evaluations have helped their foundations understand the effects of their programmatic work. Yet using evaluation well is not easy: fully 65 percent report that having evaluations result in meaningful insights for the foundation is a challenge.

In terms of resources, how much are foundations putting into evaluation? Most are conducting formal evaluations for half or fewer of their grants. The median spending on formal evaluation is two percent of a grantmaking budget.

 

* * * * * * * *

To read about current foundation CEOs’ attitudes toward assessment and what foundations are doing to understand their performance, see the report, The State of Foundation Performance Assessment: A Survey of Foundation CEOs written by Ellie Buteau, Ph.D. and Phil Buchanan and published by the Center for Effective Philanthropy.

Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy.

 

 

 

The Unique and Urgent ‘Business’ of Philanthropy

Thursday, December 22nd, 2011

The following blog post is adapted from a talk presented at a meeting of the Grantmakers of Oregon and Southwest Washington.

As Suk Rhee mentioned in her introduction, I am a native Oregonian, so I know that there is no better place than the Pacific Northwest. And I am so happy to be here with you – funders who are committed to making the region even better;

  • More economically prosperous;
  • with better opportunities for all children and stronger schools in which they can learn and grow;
  • with its natural beauty protected from pollution and destruction;
  • with habitats for wildlife preserved;
  • with fewer homeless and abused;
  • with fewer struggling with mental illness and disease;
  • with discrimination – whether based on gender, race, sexual orientation, or disability – stamped out.

These are among the very toughest challenges. Yet, if you pick up the paper or read some of the increasing number of books on philanthropy, you might think it’s not so hard.  Just apply “business thinking” or “market-based solutions,” and we’re all set.

Problems solved.

In their influential 2008 book, Philanthrocapitalism: How the Rich Can Save the World, Matthew Bishop of the Economist and his co-author Michael Green argue that a new breed of “philanthrocapitalists” are working to “apply the secrets behind their money-making success to their giving.” (As if this idea never occurred to Carnegie and Rockefeller.)

They write:

“While some are skeptical about the invasion of the M.B.A.-enabled executives in suits into the Birkenstock world of charity, many philanthrocapitalists believe that the world of giving could benefit at least as much as business from a bigger role for professional intermediaries and advisors, and from the sort of transparency and accountability that exists in financial markets.”

Did I mention that this book came out in the fall of 2008? Where is Lehman Brothers when you need them? Bishop and Green thought they were really on to something, and they’re thoughtful, smart guys who meant well.

But you know charity is working on the toughest problems. If business and government could solve the problems I listed at the start, they wouldn’t exist. You know, because you do this work every day, that no sector owns the concept of effectiveness, and that there is much work in the nonprofit sector that is strategic, data-driven, and rigorous, whether those doing it are wearing Birkenstocks or loafers.

You know, because you are living it, that though people look to you and your resources jealously, they pale in comparison to the challenges you confront.  Here is some context: Annual giving in this country is $300 billion. Total state and federal spending is an estimated $5 trillion.

You know all too well, because you are experiencing it right now that you cannot possibly fill the gaps created by slashed government spending and increased need. But you also know the nonprofit sector has a crucial role to play – one that is separate and distinct from corporations and government.

And, you know, because you live on this earth – and perhaps because, like me, you are a customer of a big national bank (and wondering why), or of the local cable company – that corporations have no greater claim on effectiveness than anyone else.

As Jim Collins, author of Built To Last and Good To Great,  has said, “We must reject the idea – well-intentioned, but dead wrong – that the primary path to greatness in the social sectors is to become ‘more like a business.’”

So let’s cut through the cloud of rhetoric that has descended on philanthropy like ash from Mount St. Helens. Let’s talk about how hard it is to be effective in philanthropy. And let’s talk about what it takes to be effective in philanthropy.

I don’t have the secret formula. No one does.

What I can offer you today may well even sound like what you already know.  (I promise to try not to pretend otherwise!) But I hope that I can at least help you understand more about why doing what we already know we need to be doing is so hard.

So let’s talk about what foundation effectiveness looks like. If you want to create the most positive impact with the resources you have – and I am going to go on the assumption that if you are here, you do – then how do you make that happen?

I think it requires four things:

  • Clear goals
  • Coherent strategies to achieve those goals
  • Disciplined implementation of those strategies
  • Relevant performance indicators to assess progress

Again, I want to repeat that these are not original thoughts nor fancy new ideas – and the challenge is not in understanding that they are necessary for effectiveness, it is that they are so hard to do right.

 

Clear goals

Defining clear, specific goals is tough.  If you’re Howard Schultz, and you’re running Starbucks, the task is clear: make a profit selling coffee.  Execution might be hard, but defining the goal is not.  Sell as much coffee as you can.  Package it up in different sizes and flavors and make people feel they have to have it. You might, I hope, increase the degree of difficulty by saying you want to do it in a responsible way, with good labor practices and sourcing, and so on. But, still, the goal is straight forward.

In foundationland, it’s much tougher. How do you choose goals?

It’s subjective, after all. Values-laden.

Plus, how do you choose just one or two goals or even several, when there are so many pressing social problems, and when those problems are so interrelated?

A wonderful former foundation staff member who I had lunch with a year or so before he died once recounted to me the story of being asked for advice by the new CEO of the foundation where he worked.

“What is the greatest mistake you fear I will make?” the new CEO asked.

The answer my lunch companion gave, at least as he recounted the story, was this:

“The greatest mistake you will make is that you will be drinking your coffee and reading the paper, and you will be deeply moved by something you read. And you’ll walk into the foundation offices and you’ll say, ‘Let’s investigate this issue. We have tremendous resources. There must be something we can do.’ And you will set in motion a process that leads to the creation of a new programmatic area at the foundation.

And, then, six months later, you will be shaving, thinking about another pressing social problem.  And you’ll do the same thing.

And then you’ll look out over the foundation five years from now, or maybe ten, and there will be so many different programs, in pursuit of so many different goals, you will realize that you have squandered the opportunity that your scale offered – to really make a difference on an issue.”

And that is precisely what occurred at this particular foundation. The CEO made exactly the mistake my former colleague predicted he would.

It’s hard not to.

When Jim Collins spoke at our conference in Los Angeles a couple of years ago, he said this:

“Disciplined action begins with piercing clarity about what you choose to not do.  In a world awash with opportunity for contribution, it’s what we choose not to do – because there is so much to do.”

So ask yourself:

  • Are your goals specific, focused, and clear?
  • How many are there?
  • Are they goals that are reasonable for you to believe you can make a difference toward, given your resources?
  • How aligned are your board and staff?
  • If each person associated with the foundation wrote down the goals, would they be the same?

It seems simple.  But all too often, the alignment just isn’t there. Our research shows that a CEO’s perception that there is goal alignment among staff and board is crucial.

So, whether your foundation has one staff member or 30, job one is to get clear on your goals.

Coherent strategies

If clear goals are the “what,” strategy is the “how.”

Everyone loves “strategy.”  Everyone has one.  But do they, really?

First, let me say that strategy plays out differently in philanthropy than in business, because there are no competitive dynamics for private foundations. In business, it’s all about “unique positioning.” But if I can make tremendous impact at my foundation by doing exactly what your foundation is doing, why wouldn’t I?  I wouldn’t demur, by saying, “No, I don’t want to fund those vaccinations because it’s not our ‘distinctive position’ to do that work.”

So how to define strategy in philanthropy?

We at CEP define strategy as:

A framework for decision-making that is

1) focused on the external context in which the foundation works, and

2) includes a hypothesized causal connection between use of foundation resources and goal achievement.

But in our research, we see a disconnect between the rhetorical embrace of strategy and the reality of its actual use. This disconnect exists at both private foundations and community foundations.

Strategy in philanthropy requires a relentless focus on the logic of how you will achieve your goals.  It is about data-driven decision making, rooted in analysis and a theory of how the foundation’s efforts can contribute to the desired change. Wherever possible, it’s informed by evidence of what works and what doesn’t, the more rigorous the better. It’s also influenced by feedback loops so you can constantly iterate and improve your strategy based on a changing context.

Strategy isn’t about deciding what works on high, about being arrogant or top-down. The best strategists are always questioning assumptions – theirs and others – and getting feedback.

Take the example of the Stuart Foundation, with roughly $300 million in assets, making grants in California and the Pacific Northwest.  In their child welfare work, the foundation focused on changing outcomes for former foster kids in California.

The conventional wisdom was that older foster kids couldn’t get adopted:  couldn’t find loving families or people who would assume some responsibility. The folks at Stuart consulted widely with those on the ground, including grantees, government officials, funders and foster kids themselves. They found reason to believe this assumption might be flawed, and tested a strategy that led to older foster kids getting adopted or connected to a caring adult. The data showed that the strategy worked, so they expanded the work.

The Foundation, joined by other foundations, also invested in the kind of supports at state universities that other kids would take for granted as coming from their families. Graduation rates in some cases for kids in these programs were better than the general student population.  The Foundation also recognized that better data systems were needed to track these kids and figure out what is working. So it did something too few foundations do: it invested in a data system.

The Stuart approach became a national model.

That’s good strategy. It’s incredibly difficult, but it can produce remarkable results.

 

Disciplined implementation

But Stuart didn’t just have a good strategy.  It had good implementation.

This is where so much falls apart.

We at CEP see this all the time. Over the past nine years we have surveyed 40,000 grantees of more than 250 foundations, developing a huge comparative dataset that we can mine for our research and for the Grantee Perception Reports we provide to individual foundations, such as Northwest Health Foundation and MJ Murdock Charitable Trust, which have both participated recently.

We see foundations that work very productively with their grantees to achieve shared goals, implementing their strategies well. But we also see foundations that undermine their effectiveness by operating in ways that are not supportive of grantees;

  • they compromise the relationship and then, inevitably, their ability to hear what is really going on;
  • they place requirements on grantees that don’t serve a purpose;
  • they fail to learn from those doing the work on the ground;
  • they fail to communicate clearly about their goals and strategies, and then somehow expect grantees to be able to implement against them.

We see foundations that say they provide capacity building assistance to nonprofits, that that is crucial to their strategy, but they don’t do it in the ways our research shows are required to make a difference.  Rather than doing the kind of comprehensive or field-focused assistance that actually helps grantees, they settle for “drive-by assistance” that may do as much harm as good.

More broadly, foundations frequently don’t commit the resources that would be necessary for implementation of their strategies.  In our research, this emerges as the biggest perceived barrier to strategy implementation.  The most effective foundations recognize that major change requires major resources. They also understand that it almost always requires resources beyond what any one foundation possesses.

Lately there has been a lot of chatter about the concept of collective impact: I got an email promoting a webinar (cost: $49) that read:

“Recently published in the Stanford Social Innovation Review …. Collective Impact is an approach to solving social problems that’s based on the idea that no organization acting alone can solve large-scale issues.”

Wait, didn’t we know this already?

Effective foundations have recognized for decades that achieving significant change requires working together to make changes with other funders and with the grantees on the ground. All the great examples of impact that foundations can lay some real claim to, from the Green Revolution to reducing tobacco use to the end of Don’t Ask, Don’t Tell, involve many institutions, funders, nonprofits, sometimes other actors including companies and government, working together, under the banner of a shared strategy, coordinating implementation.

There is nothing new here.

That’s why CEP and the Monitor Institute are working together to provide a tool that Monitor Institute created, called the Strategy Landscape Tool, to groups of funders who are working toward the same goal.  This tool allows these funders to see who is funding what, by strategy, rather than by the less helpful categories by which grantmaking is typically grouped. We’re providing these for foundations working toward shared goals in specific fields or, sometimes, as for funders in Detroit, for those pursuing improvement in a specific community.

Steve Schroeder, the former CEO of the Robert Wood Johnson Foundation, has argued that “Execution trumps strategy.” I think what he means is that strategy is meaningless if it isn’t well-implemented.

 

Relevant performance indicators to assess progress.

There has been real progress on this front in the past decade.

We see, in a survey we conducted this year of CEOs of foundations that make more than $5 million in grants annually, that assessment of foundation performance is a high priority to CEOs.  They believe much progress has been made in the last decade but that more needs to be done.  We also see that foundations are using a broader range of indicators than they were when we first studied this issue a decade ago.

This is good news.

Still, there is so much confusion about assessment. Lately, there has been a bit of a backlash against experimental design and randomized control trials. But rigorous experimental design is a powerful evaluation approach when applied well. If you are going to put major funding behind a particular strategy then you should know whether it works or not. If it’s new, then support finding out whether it works.

If something has already been shown to work, then don’t assess it all over again. Take the New York State Health Foundation, whose CEO, Jim Knickman, is on CEP’s Board of Directors. The Foundation focuses on improving clinical care and patient outcomes for diabetes – doing so by supporting the proliferation of treatment approaches that have already been proven effective. Rather than funding their own massive evaluation, they have relied on the work of others who have already shown what works – and their assessment now focuses on their ability to spread what works.

So you don’t all have to fund massive evaluative studies, but it’s irresponsible not to pay attention to what is known, and not known, about what works – and to act accordingly.

Another problem is that so much assessment today puts the foundation’s needs in front of the nonprofit grantee’s needs.  But assessment should also support the work of those on the ground, giving both you and them the information to assess and improve.

In his important new book, Leap of Reason, Mario Morino writes:

“I know many nonprofit leaders who are not managing to outcomes but are strongly predisposed to do so.  They inherently know what their outcomes are and very much want to assess and manage to them.  But they are severely hamstrung by the lack of available funding to do this hard work. … At minimum funders should be supporting efforts to help nonprofits to …. (a) track the outcomes of those served; (b) undertake at least basic analysis of this information; and (c) identify how they can use the information to learn and improve their programs over time.”

When it comes to assessment, I think the first step a foundation can take is to open itself up to feedback from the outside,  recognizing that it is surrounded by those who are predisposed to say what they think foundation staff want to hear.  To have any meaning whatsoever, this feedback must be collected by a third-party that is recognized and trusted for its independence so that the feedback is candid; and it must be put in a comparative context.  Without comparative data, it is impossible to make sense of what is a good result, what are relative strengths and weaknesses.

At the Center for Effective Philanthropy, we have worked hard to create these feedback loops.  Our Grantee Perception Report has been used by foundations as big as Gates as well as those with a couple of million dollars in annual giving. Our Applicant Perception Report captures the feedback of those who were declined funding. Our Donor Perception Report gives community foundations the opportunity to understand the views of their donors. Our Stakeholder Assessment Report taps into the perspectives of policy makers and field and community leaders. Our YouthTruth project taps into the voices of those who should matter most – the people whose lives a foundation seeks to improve. YouthTruth does this for education funders through surveys of students in high schools and, soon, middle schools – and its made a powerful difference to schools, districts, and funders.

In every case, the perceptual data is put in a comparative context to make it meaningful.

And we see that foundations are improving, to a statistically meaningful degree, when they repeat tools like the Grantee Perception Report.

As you assess, remember that what you learn often has broader relevance. If you know something about what works, or what doesn’t – share it, so others can learn from it. In my view, it’s morally indefensible not to.  Again, there is no competitive dynamic here – we’re all trying to make a positive difference.

Clear goals. Coherent strategies. Disciplined implementation. Relevant performance indicators.

Not radical concepts.

But incredibly difficult – it is much, much harder to be effective in philanthropy than it is to be successful in business.  Just ask some of the folks in Silicon Valley who have recently made the transition from business to philanthropy.

Not radical concepts – not new concepts either.  In their very good new book, GiveSmart, Tom Tierney and Joel Fleishman imagine Andrew Carnegie and Bill Gates having dinner, writing, “They would quickly discover how much they had in common.”

The authors speculate that as

“rigorous, disciplined, and deeply strategic” men, “the industrial baron and the software tycoon would be highly compatible.”

They write,

“If Gates were to mention ‘strategic philanthropy,’ ‘social entrepreneurs,’ or ‘scaling what works’ in the course of the conversation, Carnegie might not recognize the phrases but he would immediately understand the concepts.”

So the concepts are neither new nor radical, but they’re incredibly hard to act on, each day.

Each one of you here today has a role in making decisions about how resources are used to influence change for the better in this amazing region we call the Pacific Northwest.  How you do that work, each day, the discipline and clarity you bring to the task…it matters.

Foundations play a role other actors in our society cannot, or will not.

When they do it well, the results can be stunning.

So do it well.

 

Phil Buchanan is President of the Center for Effective Philanthropy.

Hard Facts: Data Tells an Unexpected Story in Boston

Monday, December 12th, 2011

The city of Boston stands as a gleaming example of the renewed and widespread appreciation for urban living. Its charms, after all, are abundant, from 18th century Beacon Hill streets to dynamic and diverse SOWA, the “new” South End. Plus it has been honored for its strong public schools (2006 winner of the Broad Prize for Urban Education, no less). All of which makes the new report, The Measure of Poverty: A Boston Indicators Project Special Report, something of a bombshell.

The Boston Indicators Project is the widely honored data-and-analysis special initiative run by the Boston Foundation with a host of partners in Greater Boston’s Civic Community. It is headed by Charlotte Kahn (honored in Istanbul in 2007 for her work) who is a leader in a global movement to create indicators initiatives, using open-source data collected from a wide array of credible sources to examine current trends and realities from the grass roots to broad regions.

Justification for the use of the b-word? Here are a few selected data points for consideration—drawn, I repeat, from the heart of one of the most economically successful places in the United States:

  • As many as 340,000 Bostonians now live in poverty (excluding college students) or stand at risk of being unable to afford such necessities as housing, food, transportation, child care and energy. That represents more than half of the total population of Boston.
  • Income inequality is bad and continues to grow worse. Despite Boston’s recent economic successes, Suffolk County remains one of the 50 most unequal in the United States. In 2009, the top 5 percent of Boston earners received 25 percent of total annual income in the city while the bottom 20 percent earned just 2.2 percent of the total.
  • Health care costs, already the highest in the nation, are expected to double by 2020. Meanwhile, social investment in programs that offer a significant return on the dollar are being slashed, promising downward economic pressure in years to come. Cuts in state funding for services from fiscal years 2009 and 2012 include the following: universal pre-kindergarten, cut 38 percent; teen pregnancy prevention, cut 41 percent; health promotion and disease prevention, cut by 77 percent; and employment services, cut 80 percent.

I find this report compelling for two reasons. First, I know Kahn to be a scrupulous and rigorous curator of data (she was a colleague when I worked for the Boston Foundation) with a prophetic voice that has—unfortunately—been validated consistently in recent years. This report deserves to be read and pondered far beyond the borders of Boston.

And the second reason is because at the Center for Effective Philanthropy (CEP), I spend much of my time looking for ways to highlight the power and utility of data as a critical tool for expanding the impact of philanthropic funders.

Real tension exists between the visceral power of anecdotal presentation and the sometimes challenging reality of actual fact. In a similar work of data-driven exposure, this tension is a major theme of the book Poor Economics, by Esther Duflo and Abhijit Banerjee, which speaks powerfully about the value of scientific investigation in the search for effective strategies that address the causes and effects of global poverty.

The point here is that our untested hunches often fail to provide a credible read of the world around us. The discipline of high quality research and data analysis brings us to a better understanding. In The Measure of Poverty, the better understanding is a sobering assessment of trends that are eroding common assumptions about the United States as a land of opportunity where the American Dream is a plausible aspiration. In Poor Economics, the better understanding is about the efficacy of common strategies to improve the lot of those with the least—and offers the better news of proven success in alternative methods that are tested and found effective.

This seems to be especially important right now, as we move toward a national election and the broad issues of where we are as a country and what we should be doing to address a long list or urgent needs. Policy decisions and the design and application of strategy—in the civic arena, for foundations, and the philanthropic sector more broadly—seem critical. Yet the air is full of unsupported assertions and sweeping summaries that don’t really bear scrutiny.

The Boston Foundation’s special report makes the point in a significant American context that Poor Economics makes in a global context: we can’t afford to make important decisions about philanthropic investment based on anecdotal evidence and our own hunches, however strongly held.

In the final analysis, data should be the decider.

 

David Trueblood is Vice President – Communications & Programming at the Center for Effective Philanthropy.

 

Data Point: What Information are Foundations Using to Assess Their Programmatic Work?

Friday, December 9th, 2011

For our recent State of Foundation Performance Assessment report, we gathered data from 173 CEOs of U.S. foundations with annual grantmaking of at least $5 million on what types of information they use to assess the effectiveness of their foundations’ work – operations, finance, and programmatic. Compared to a similar survey we completed almost a decade ago, it appears that foundation CEOs today are drawing on an increasingly broad array of performance indicators.

In this survey, we listed a range of types of information CEOs may be using to assess the effectiveness of their foundations’ work, and CEOs could select as many as applicable.

On average, CEOs report using seven types of information to understand their foundations’ programmatic effectiveness. Almost all foundations are using anecdotal feedback, written reports from grantees, site visits, and evaluations – either of individual grants, clusters of grants, or program areas.

A minority, however, seek information from their ultimate beneficiaries – the people whose lives foundations are ultimately trying to affect – either through surveys, focus groups, or convenings. When we compare responses of CEOs whose foundations do and do not seek the voices of their ultimate beneficiaries, two important differences emerge: CEOs who report that their foundation does collect beneficiary feedback rate themselves as having 1) a better understanding of the progress their foundation is making against its strategies, and 2) a more accurate understanding of the impact the foundation is having on the communities and fields in which it works.

 * * * * * * * *

To read about current foundation CEOs’ attitudes toward assessment and what foundations are doing to understand their performance, see the report, The State of Foundation Performance Assessment: A Survey of Foundation CEOs written by Ellie Buteau, Ph.D. and Phil Buchanan and published by the Center for Effective Philanthropy.

Ellie Buteau is Vice President – Research at the Center for Effective Philanthropy.

 

Making Transparency Matter

Thursday, December 8th, 2011

There has been lots of talk lately about foundations and “transparency.” Perhaps the most prominent foundation transparency initiative is a website called Glass Pockets, launched about a year ago by Foundation Center. Its tagline? “Bringing transparency to the world of philanthropy.”

CEP worked as one of a number of partners to Foundation Center as it designed this effort, and we were pleased and proud to do so. We favor the general idea of foundations being more open about what they do and how they do it, and I agree with Foundation Center President Brad Smith that the record of foundations overall is not so hot when it comes to transparency. So I applaud the effort Foundation Center is making, under Brad’s leadership, to promote transparency.

In fact, as I prepared for the event on “Transparency and Effectiveness” that Foundation Center and CEP co-hosted in San Francisco this week, I started to worry about whether we’re going nearly far enough. I started to worry about whether we are settling for superficial transparency, about whether the inevitable push to create simple check-lists is resulting in good information – or misinformation.

I worry that, in the worst case, we could witness the kind of counterproductive behavior that we see in the world of colleges and universities, where preoccupation with placement on the U.S. News and World Report rankings has led some institutions to take steps to improve their standing – to game the system, really – that are counter to what is best for their students. I worry that we’re not discussing critically the information foundations are being transparent about – and I am not sure there is value in transparency without engagement. And I worry that we’re overemphasizing transparency about trivial matters and under-emphasizing transparency that really facilitates greater effectiveness.

Which brings me back to Glass Pockets. On the Glass Pockets website, 32 foundations to date are listed under the question “who has glass pockets?” because they have agreed to be assessed on whether they disclose information in categories such as “Basic Contact Information,” “Governance Policies & Information,” “HR/Staffing Policies & Information,” “Financial Information,” “Grantmaking Information,” and “Performance Measurement.”

Some don’t do too well – turns out some of their pockets may be more of a tinted glass.

But what about those who do perform well? What about those receiving magnifying glasses or bullhorns on the site (the icons Foundation Center uses to give credit, and which link to the supporting material) in the various categories as indications of their transparency? What does it take, exactly, to get credit in the various areas?

I don’t see the categories as even close to equally important. So I was particularly interested, given CEP’s focus, in foundations that received magnifying glasses on the four dimensions of the “Performance Measurement” category. The specific dimensions in that category are: “assessment of overall foundation performance,” “knowledge center,” “grantee feedback mechanism,” and “grantee surveys.”

Much of what I found was less than inspiring. One foundation receives a magnifying glass for “overall foundation performance assessment” for a consultant’s report based on interviews with leaders in the community – with absolutely no information about methodology or number of interviews conducted. Another foundation links only to the Grantee Perception Report (GPR) provided by CEP. While I think the GPR can be an important component of an overall assessment, should it be enough, on its own, to qualify as an “overall foundation performance assessment?” I am not sure.

One foundation that gets credit on Glass Pockets for conducting a grantee survey does so because it self-administered a survey several years ago and got a 22 percent response rate – and what that corresponds to in terms of number of responses is not disclosed. The questions are poorly constructed, with response option choices that virtually guarantee positive results – and there is no comparative data. Yet among the “findings” are that there are “very favorable views of [the foundation] in all areas” and that the donor “would be proud!” I wonder. (I am not exactly objective, here, I realize. We have our own approach to grantee surveys and a particular point of view about what constitutes a rigorous survey effort.)

Another foundation gets credit for having a “grantee feedback mechanism” simply because it posts contact information for a staff member that grantees can reach out to.

Some of the foundations that do well on Glass Pockets absolutely deserve to. There are funders with impressive supporting materials in the “Performance Measurement” category. But it’s impossible to distinguish between them and others whose commitment seems more cursory based only on what’s listed on Glass Pockets. You have to go to the foundations’ web sites to figure it out, and I am not sure how many of the visitors to the web site will take the time to do that. I think the natural tendency will be to assume that those receiving credit for doing performance measurement, and being public about it, are doing something really meaningful.

So what should be done? A few ideas:

  • I would like to see Glass Pockets add a space for comments on the quality of what foundations have been transparent about. If we don’t engage with what foundations are making public, if it just sits on websites, then how valuable is it?
  • I would like to see more discussions about the substance – both online and face-to-face at meetings such as the one we held in San Francisco. Is transparency a good in its own right? Or is it a means to an end of greater effectiveness? Are transparency and effectiveness ever in tension?
  • I’d like to see Glass Pockets do more to push for openness about programmatic effectiveness – so foundations are encouraged to disseminate information about what works and what doesn’t. That seems far more important to me than criteria such as whether a foundation has a staff list on its Web site, or a newsletter, or a blog.
  • I also wonder about the potential importance of the Charting Impact initiative developed by Independent Sector, BBB Wise Giving Alliance, and Guidestar USA. This effort encourages organizations to publicly answer five “deceptively simple questions,” including “What is your organization aiming to accomplish?, What are your strategies for making this happen?, and How will your organization know if you are making progress?” Only one foundation, the William and Flora Hewlett Foundation, is listed on the IS Web site as having completed a Charting Impact Report. (Full disclosure/hypocrisy check: CEP has not yet done this, although our answers to essentially the same questions can be found in our publicly available strategic plan.) Transparency (and clarity) about the answers to these questions on the part of all major foundations would be a step forward.

I think the Glass Pockets effort is a promising one. Foundation Center deserves a lot of credit for taking a stand and pushing this forward. I also like the direction of the Charting Impact initiative.

We should all work to raise the bar on what constitutes transparency that matters. I hope we can push for much, much higher standards for the depth and quality of information necessary for a foundation to get credit for its commitment to performance measurement, for example – or, for that matter, for its transparency in general.

Foundations have made progress when it comes to transparency. But there is a long, long way to go.

 

Phil Buchanan is President of the Center for Effective Philanthropy.