Posts Tagged ‘Essentials of Foundation Strategy’

Newer Entries »

What Are the Limits of Quantitative Performance Measurement?

Wednesday, January 13th, 2010

CEP’s Essentials of Foundation Strategy concludes, “Assessment of results against strategies remains a significant challenge for foundations: staff struggle to determine the right data to collect and how to collect it.”

This is an important finding, and explanations are well worth exploring.  The report suggests several possible reasons – technical challenges, inadequate resources and support, and lack of grantee capacity and skill.  I’d like to explore another possibility: The undue attraction to quantitative analysis.

The recipe for foundations is simple and well known:  assess the environment, choose a goal, align resources, implement programs, measure performance, adjust. Repeat as needed until goal is achieved.

Indeed, every foundation’s recipe will have a different mix of ingredients, but maybe we need to think about the limits of quantitative measurement. As much as I value quantitative data to assess results, the proliferation of such work and its unintended consequences suggest we consider (to extend the recipe analogy) if the basic philanthropic ingredients contain a dash too much fascination with numbers.

It might seem impolite to raise this question on CEP’s blog, given that the heading on this new Web site begins with “better data . . .”!  But I know CEP is committed to being a learning organization, which includes fostering a supportive environment for raising different points of view.  So – what better place to ask about the seemingly unending efforts to quantify performance at every level to assess impact than on the Web site of an organization devoted to empirical analysis?

What are the possible downsides of quantitative performance measurement?  One is that doing this work has its own costs, so spending resources on this ingredient has to be done as effectively as possible, and weighed against spending for other ingredients in the philanthropic recipe.  Another is the mistaken belief that quantitative measurement is the only type of measurement.

Even when qualitative measurement is considered, it is often viewed as inferior to quantitative analysis.  Yet the heart of measurement is comparative assessment, and this doesn’t require quantitative analyses.  Think of the meaning of “taking the measure of a person” or “to speak in measured terms.” These phrases denote the sense of measurement as judgment or comparison.

A related risk is the lack of fit between the measure of performance and what is being measured.  What is the nature of philanthropic practice that we want to measure?  Are foundations measuring their own performance (i.e., strategy, selection of goals) or their grantees’ performance (project results, program impact)?  One conception of philanthropy is that it is craft.  How amenable to quantitative measurement is the craft of philanthropic practice?  At what levels?

Thinking about the limits of quantitative performance measurement suggests several things that might help foundations improve assessing their impact.  One is to take into account the costs and benefits to everyone involved when undertaking this work, and what is most important to learn to accomplish strategic goals.

For example, it is probably more valuable to put performance assessment resources to broader levels of performance (e.g., is an overall strategy working?) than to discrete grantee projects.  Focusing on individual projects inhibits an understanding of how projects fit together over time and relate to the environment in which they operate.  In turn, this reinforces a focus on a project’s internal risk – implementation – and away from strategic and design risks.

Note that the failure of implementation puts the spotlight on grantees; failure of strategy and, to some extent, design, puts the spotlight on foundations.  An excellent example of a higher-level, strategic assessment is the qualitative assessment that Patti Patrizi and colleagues did on RWJF’s end-of-life grant-making from 1996-2005.

In preparing for performance assessment, thinking carefully about design, the limitations of the data to be produced, and how the data produced will be used before committing resources may also minimize an overreliance on quantitative assessment.

In undertaking performance assessment, it is useful to think of the kinds of comparisons (quantitative or qualitative) that fit with the goals.  Case studies, like the one CEP published on the Stuart Foundation, are excellent ways of using a comparative framework – in this case to a model of bringing about social change – to assess and learn about performance.

Assessing performance rigorously is a critical ingredient in strategic philanthropy.   Questioning the limits of quantitative performance assessment should not be used as a reason not to do performance assessment – just the opposite. The field needs more resources devoted to this ingredient, but they need to be used as wisely as possible.

The challenge is getting the right mixture of ingredients, including types of measurement, for each foundation.  And as long as we keep in mind that all data are not numeric, CEP’s banner that begins with “better data . . .” is the start of a good recipe for philanthropic chefs.

Bob Hughes is an independent consultant on strategy and organizational learning in health and philanthropy.

*****

Disclaimers and Disclosures: The views expressed in the CEP blog by guest bloggers are entirely their own and do not necessarily reflect the opinions of the Center for Effective Philanthropy.

Tags: , , , , , ,
Posted in Assessing Performance, Developing Strategy | 4 Comments »

Why Haven’t Foundations Made More Progress in Becoming Strategic? (part 2)

Friday, January 8th, 2010

In the previous post, I explored the difficulty of developing clear goals and being disciplined in sticking with them as one reason why foundations are not more strategic.  This post looks at a second topic, “unique positioning.” The CEP report argues that business strategy is different than foundation strategy, in part because the concept of unique positioning is less relevant for philanthropy than for business.

Why did strategy emerge in the historical period that it did?   One hypothesis is that the rise of strategy occurred in part because of the proliferation of foundations.  The basic idea is that as more and more foundations work on a finite number of issues, the problem of how all their work fits together emerges, and “strategy” is viewed as a partial solution to this.

This implies that when there were fewer foundations, strategy was, in some sense, not as hard.  And this fits with what historian Stanley Katz has observed in his essay “What does it mean to say that philanthropy is effective?” – namely that strategic philanthropy isn’t new, it’s been what American foundations have done for over a century.

Could it be that in today’s environment it is much more difficult to be strategic than it was fifty years ago, so foundations must work harder to be strategic than in the past?  Put more concretely, foundations today are much more likely to have to account for lots of other funders doing work in their domains as they think about their strategy.  The environment is more crowded.

CEP’s case study on the Flinn Foundation illustrates that point.  One factor in Flinn becoming more strategic was the formation of new foundations in Arizona with missions that overlapped Flinn’s and that had larger endowments.  And today’s foundations need to consider a lot more than newly formed foundations.  As Lucy Bernholz has analyzed, philanthropy is increasingly comprised of other institutional vehicles in addition to foundations to get things accomplished.  So the idea of unique positioning may be more applicable in today’s environment than it was a decade ago.

All of this points to the need for and value of intermediaries and networks coordinating and fostering more explicit relationships among philanthropic organizations – an under-resourced and challenging undertaking.  One effort underway at Duke University, in collaboration with Growth Philanthropy Network, is the Social Impact Exchange, which aims to be a forum for scaling effective social programs and solutions.

These musings have led me to conclude that I may have started with the wrong question.  Rather than being disappointed in the apparent lack of progress in foundations becoming more strategic, perhaps a better set of questions revolve around how strategic foundations are, and why?  And how this has changed over time, and why?  Answering these questions may lead to a better understanding of steps the field can take to make the progress so many are eager to see.

Bob Hughes is an independent consultant on strategy and organizational learning in health and philanthropy.

*****

Disclaimers and Disclosures: The views expressed in the CEP blog by guest bloggers are entirely their own and do not necessarily reflect the opinions of the Center for Effective Philanthropy.

Tags: , , , , , ,
Posted in Assessing Performance, Developing Strategy | 4 Comments »

Why Haven’t Foundations Made More Progress in Becoming Strategic?

Wednesday, January 6th, 2010

Let me say up front that I am a big admirer of CEP.  The field needs an independent voice devoted to the comparative assessment of philanthropic practice, and CEP has helped many foundations, including RWJF, improve over the last eight years.  So I was pleased when Phil Buchanan asked if I would write several posts for CEP’s blog, especially knowing that CEP welcomes critical perspectives as well as bouquets.

Each of these posts provides a welcome opportunity to share some preliminary thoughts and provoke discussion on four questions that have been on my mind recently:

  1. Why haven’t foundations made more progress in becoming strategic? (two parts)
  2. Can failure be the key to foundation effectiveness?
  3. What are the limits of quantitative performance measurement?
  4. Can foundations be learning organizations?

Why has there been so little progress in foundation strategy? It’s been a decade since the Porter/Kramer HBR article on strategy, and it seems reasonable that with the extensive attention to strategy since then we would see more signs of improvement.

So I anticipated some good news from CEP’s latest paper, Essentials of Foundation Strategy. Unfortunately, when it comes to strategy, the results left me thinking that philanthropy’s glass is half-empty.

As the report states, “Even with what is possibly a more strategic than typical set of respondents, our findings paint a sobering picture of the challenges of foundation strategy.”  Among the findings from the responding foundations:

  • Only half of the CEO’s report a shared understanding among the Board, CEO, and the staff of the foundation’s goals
  • A majority don’t have a logic model
  • Just one-fourth use performance indicators to assess all their strategies.

These results raise questions about why foundations have such difficulty with strategy, especially in the context of significant attention to adopting strategy over the past decade.  In probing for possible explanations I’ll explore two topics that the report introduces:  the problem of goals (in this post) and the concept of “unique positioning” (in the next post).

Having clear goals is essential for strategy, and for assessing performance.  It is hard to know how you’re doing if what you’re trying to do is ambiguous.  And of course, ambiguous goals inevitably make measuring performance a frustrating activity.

It is easy to overlook how difficult it is to specify clear goals that are useful for decision making and performance assessment, especially for foundations.  Foundations often take on quite ambitious goals that don’t guide decisions.

Susan Colby and colleagues acknowledged this in a 2004 article in Stanford Social Innovation Review (SSIR). “Zeroing in on Impact,” introduced the concept of “intended impact” to fill the gap between an audacious mission and actual decision making.  “Intended impact is a statement about what the organization is trying to achieve and will hold itself accountable for within some manageable period of time.”

The article notes that developing strategic clarity is a process, not a formula.  Perhaps more importantly, it recognizes that doing this work is hard, it causes disagreements, and it always means that those who don’t agree with the decisions will be unhappy.  For foundations, this can mean trustees, staff, and grantees.

Goals are almost always contentious as they are developed, and, once established, they are contested continuously.  A range of interests among participants in goal-setting often leads to adoption of overly broad goals that accommodate as many of those interests as possible.   These goals are often stretched by constant requests for expansion through both external stakeholders and internal staff and trustees with related, but off-target agendas.

Maintaining discipline in the face of these eroding pressures is hard work.  But it is necessary because goals have two important functions.  The first is to set out a positive direction to help guide decisions.  The second function, which Porter and Kramer noted in their seminal article, is to clearly say what the foundation will not be doing.  This is often a source of conflict, as any funder who has decided to leave a field well knows.

And note that very few efforts to become strategic start from scratch – most are undertaken in the context of a web of relationships with powerful stakeholders that make change daunting.  Vague or ambiguous goals impede becoming strategic, but we can see the reasons for them and why it is hard for foundations to move away from such goals once they are in place.

Bob Hughes is an independent consultant on strategy and organizational learning in health and philanthropy.

*****

Disclaimers and Disclosures: The views expressed in the CEP blog by guest bloggers are entirely their own and do not necessarily reflect the opinions of the Center for Effective Philanthropy.

Tags: , , ,
Posted in Assessing Performance, Developing Strategy | 9 Comments »

Beyond Administrative Cost Ratios

Wednesday, December 16th, 2009

A few weeks ago, I had the pleasure of being a guest blogger on a new site created by Duke University’s Center for Strategic Philanthropy and Civil Society, called the Intrepid Philanthropist.  Inspired by the blog’s name, I decided to challenge some of those inside and outside of philanthropy, who, in my view, paint the sector with too broad a brush.  You can read all five of my posts by downloading this PDF version.

While agreeing that the nonprofit sector must dramatically step up its effectiveness, I suggested in my Duke posts that it is important not to lose our appreciation of the distinct role of the sector by simply assuming that “business thinking” has all the answers.  I also suggested that we need to recognize the successes (and failures) of the past – and present – from which we can learn.  The fact is, there are excellent examples of effective philanthropic strategy that are older than any of us.

I got an overwhelmingly positive reaction to my blogs and received many emails thanking me for what I wrote – and especially for the specific examples I cited of foundation and nonprofit impact.  On the other hand, those whose articles or books I critiqued reacted in a strongly negative way, especially Dan Pallotta, the author of Uncharitable – who took umbrage (to say the least) at my critique, leading to a lively exchange both on the Duke blog and on his blog.  Following that exchange, I am even more convinced that his worldview rests on a deification of markets that isn’t grounded in reality. 

But I do not want my debate with Pallotta to obscure the fact that, on one very important point, we agree.  That point is this: there remains far too much emphasis on administrative cost ratios as the gauge by which we judge nonprofit performance. 

Too many donors seem to focus on the percentage of a nonprofit’s budget going toward “program” without sufficient attention paid toward the impact created.  In its worst extremes, some donors act as if they expect nonprofits to operate without the infrastructure and talent that are, in fact, necessary for sustained effectiveness. 

Foundations can be a big part of this problem in the way they fund and the messages they send nonprofits.  But they can also be victims of the same kind of overly simplistic thinking in the way their boards assess their own performance.

The challenge, of course, is that performance assessment is harder in the nonprofit sector because there is no universal metric – no analog to profits – and there never will be.  (You save a rainforest in Brazil, I increase graduation rates 10 percent in Boston: we could spend the rest of our lives in dueling calculations over who achieved more impact.) 

Performance assessment is especially challenging for foundations, because they are one step removed from the impact they seek.  Without easily accessible performance data, foundation boards gravitate to what is available, quantifiable, and comparative.  The easiest to obtain data that fits this bill are investment returns and administrative cost ratios. 

It’s not that boards don’t want better data that is more connected to foundation strategies for impact: our research demonstrates that, to the contrary, they do.  It’s that this data often isn’t made available, because it’s hard to get.  Our newest report, Essentials of Foundation Strategy, details just how significant the performance assessment challenges are.  

So what do boards do?  They focus, naturally, on what they can get their hands on.

Let me offer just one simple example.  One large foundation whose grantees we had surveyed had a board whose members were fixated on its administrative spending, which was high relative to peer funders.  They were targeting areas to cut, and had identified the foundation’s research efforts as a good place to start. 

But when we presented a Grantee Perception Report (GPR) based on our survey results, they learned that the foundation was among the highest rated foundations among all those whose grantees we have surveyed on dimensions such as “advancing knowledge” in the foundation’s field of funding.  Grantees also wrote eloquently, in response to open-ended questions, about the impact of the foundation’s research on improving understanding of what was working and what wasn’t – saying that it was more valuable than the grants they received (really, they did). 

The Board, recognizing the impact of these efforts, decided to preserve the spending on research.   

People often ask me, “What kind of changes do foundations make based on CEP’s assessment tools?”  It’s a good question, and I have lots to say in response.  But sometimes just as important as what has changed in response to data is what hasn’t  – what is preserved as a result of an understanding about its effect. 

Focusing on administrative spending ratios in the absence of other data can lead to the elimination of work that is having a substantially positive impact that more than justifies its cost.   We can and must do better – by pushing for indicators of foundation effectiveness that connect to goals and strategies and allow for a more holistic assessment of performance.

Tags: , , , , , ,
Posted in Managing Operations | 1 Comment »

A Welcome from Phil Buchanan

Wednesday, December 9th, 2009

Welcome to CEP’s new blog. Here, we hope to bring data and insight to bear on the pressing issues related to philanthropic effectiveness and impact. We’ll weigh in with perspectives based on our research efforts – and we’ll also draw on what we’ve learned providing assessment tools to more than 200 foundations.

Frequent CEP bloggers will include Kevin Bolduc, our vice president – assessment tools, and Ellie Buteau, PhD, our vice president – research, as well as other key CEP staff. In addition, we’ll feature guest bloggers who we have come to know as particularly thoughtful on questions of foundation effectiveness: people like Robert Wood Johnson Foundation Vice President – Learning Robert Hughes, whose posts will appear in the coming weeks; and Georgia Levenson Keohane, a former executive at the September 11 fund, McKinsey consultant, and consultant and writer whose pieces on philanthropy and social and economic policy issues have appeared in publications such as Slate, and the American Prospect.

I’ll weigh in frequently, too, and invite you to do the same by adding your comments in response to what you read here.

This is an exciting time for CEP, with the launch of our new Web site and the release of our new research report, Essentials of Foundation Strategy, and an accompanying case on the Stuart Foundation’s Child Welfare program. Also just-released is an online strategy self-assessment that allows foundation staff to reflect on how strategic they are in their work. I hope you’ll try it out and give us your feedback.

My colleagues and I will be blogging more about our strategy research in the coming days. I hope you’ll check our blog frequently or just subscribe to our RSS feed and draw on what you find here as you work to achieve your goals.

Phil Buchanan

Tags: , ,
Posted in Administration | 2 Comments »

Newer Entries »