Posts Tagged ‘funder effectiveness’

Guest Post: How the Robert Wood Johnson Foundation Seeks To Improve

Monday, October 3rd, 2011
At the Center for Effective Philanthropy, we believe that improved performance of philanthropic funders can have a positive impact on nonprofit organizations and the people and communities they serve. As part of our work, we aim to highlight stories from funders who share that vision and who value the role of data and assessment in efforts to increase their impact.

 

PROCESS PROGRESS: HOW RWJF STRIVES TO IMPROVE
by Robin Mockenhaupt, Ph.D., M.P.H., Chief of Staff,
Dee Colello, Senior Manager, Program Operations, and
David Adler, M.P.A., Communications Officer
of the Robert Wood Johnson Foundation

 

They must often change, who would be constant in happiness or wisdom. ~Confucius

Change and continual improvement is a valued part of the Robert Wood Johnson Foundation’s culture. It’s in our guiding principles, which state, “We must commit ourselves to lifelong learning and continual improvement.” In addition to change and improvement, RWJF is also committed to transparency and peer learning, and it is in that vein that we are sharing our progress on quality improvement since 2004.

As part of our 2004 annual organizational assessment, we commissioned the Center for Effective Philanthropy’s Grantee Perception Report (GPR). We learned that while our grantees rated us comparatively well in several areas, we were using up a lot of grantee time meeting our administrative demands and we weren’t moving as fast as we needed to in processing grants. We also learned we needed better clarity in our communications of goals and strategies.

A 2011 CEP case study, Frequent Checkups Make for Healthier Funding Relationships, illustrated that we changed. We wanted to share one of the ways we took that advice to heart.

With the findings from the GPR, input from an all-staff retreat, and a focus group of grantees, we developed our first Foundation-wide Quality Improvement (QI) initiative. Our process for implementing this QI project can be broken down to five steps.

  • We set the right tone. The all-staff retreat and the announcement of the QI process by our president and CEO, Risa Lavizzo-Mourey, M.D., M.B.A., helped create widespread buy-in among staff. Any organization considering quality improvement projects should recognize that having the buy-in and public support of the CEO is a crucial first step for setting the appropriate environment for change.
  • We started with something manageable. The initial focus of our quality improvement work was on a category of grants that accounts for about 20 percent of our grantmaking. We wanted to start with something manageable to see how it worked before rolling out broader efforts.
  • We gathered the troops. After the announcement, staff interested in quality improvement convened to map out current grantmaking processes; shortly after that, a smaller core staff team was chartered, supported by a group of project sponsors from senior management.
  • We jumped in. The team designed a pilot. After testing and implementation, the project moved into control (maintenance) phase. Our first QI project resulted in sequencing and prioritizing steps in our grantmaking process, as well as launching the Foundation’s Program Information Management System (PIMS).  After our first QI project, two other projects were designed and implemented, using a similar structure and process. In addition, three smaller projects were led by staff trained in the QI process.
  • We are monitoring ongoing progress. We needed a way to monitor how we were doing and for identifying new ideas for improvement. We organized a standing staff group called the Process Improvement Group to help track metrics for our grantmaking and to initiate new quality improvement initiatives. Additionally, other units within RWJF have taken up their own quality improvement initiatives.

What did we learn?

  • Communications is a key component to implementing quality initiatives and staff behavior change.
  • Staff like being involved in cross-functional improvement projects
    when they see the need for change and can be a part of the solution.
    It’s also an opportunity to involve staff at every level of the
    organization.
  • Staff need dedicated time for QI work, as opposed to trying to “fit it in” around other responsibilities.
  • Over time, managers learned better how to scope and implement QI projects.
  • The automation of our grantmaking process (which initially was
    paper) allowed for project milestones and timelines to be standardized
    and to become transparent to all staff.

 

We’re pleased that more recent CEP reports have concluded our grantee perceptions have gotten better over time, and we believe our quality improvement efforts were a factor in this change. Our responsiveness measures are now higher as well as our quality of interactions. With that said, all our quality improvement efforts were not successful and we are receptive to revising any processes that may have missed the mark. For example, we are still working toward reducing the amount of time in both selection process and improving our ongoing monitoring and reporting.

We’re happy to share additional information about our QI process and are eager to hear how our colleagues are approaching this as well.

Avoiding a False Choice (Why We Need Both ‘Morality’ and Assessment)

Thursday, July 21st, 2011

I am speaking today on a panel at the Hudson Institute in Washington, D.C. on “Reclaiming the Moral Life of Philanthropy.”  The panel includes Gara LaMarche, president of Atlantic Philanthropies, and is centered on a talk he gave at MIT in which he suggested he has:

“A disquiet about the way we in the foundation world, along with the organisations we support and the infrastructure many of us have helped to build, have mirrored trends in the political world to talk about what we do and why we are doing it in ways that have strayed too far from first principles. We have become more about the fix, the intervention – to use a horribly dominant word in the field that calls to mind invading armies – than about the reasons for doing or caring about it. In marching under the flag of what works, and in particular what can be proven or demonstrated through the rigours of evidence, we risk straying too far from what is right. I think it is time to strike a better balance.”

I am one of three responding to Gara’s comments – along with Indiana University’s Leslie Lenkowsky, Maya Wiley of the Center for Social Inclusion.  William Schambra will moderate.

Here is what I intend to say (my actual comments may vary a bit of course): 

There is much about Gara LaMarche’s speech at MIT to be admired – and that I agree with.

For example, I agree with him that our political discourse, today, is often focused on poll-driven pragmatism at the expense of moral clarity and moral courage. Whether the issue is climate change or health care or immigration – all examples he cites – we hear too much about what is feasible, or politically possible, and not enough about what, simply, is right. (Although one could easily argue that, given what is going on in Washington right now, a little more focus on what is feasible with respect to the debt ceiling would be helpful! Still, I tend to agree with Gara’s general point here.)

Where I part company with him is when he asserts that this problem infects philanthropy. Or when he suggests that there is some disconnect in philanthropy between the moral case for what we do and the quest to understand what works. Or when he seems to imply that the fact that too many foundations are entrenched in “fixed and safe positions,” which I would agree with, is attributable to a focus on effectiveness – when, in fact, I believe the opposite is the case.

Gara says:

“In marching under the flag of what works, and in particular what can be proven or demonstrated through the rigors of evidence, we risk straying too far from what is right.”

I say: we stray from what is right when we do not assess.  This is, after all, about the people we seek to help – and if we don’t do the necessary work to confirm that we are, in fact, helping, we are falling short of our moral obligation. 

  • As Mario Morino of Venture Philanthropy Partners argues in his important new book, Leap of Reason, “Every ounce of our effort on assessing social outcomes should be with one end in mind: helping nonprofits deliver greater benefits to those they serve.” 
  • It is nothing less than a moral outrage when programs like DARE and Scared Straight receive massive funding only to realize, after years and millions of dollars, that they are having no effect, or, worse, the opposite of the intended effect. Mario’s book has a powerful example of such an instance here in Washington, D.C., at the Latin American Youth Center. That organization sought to change attitudes about domestic violence – but it found its efforts were having the opposite of the intended effect.  Fortunately, the Latin American Youth Center’s metrics allowed it to learn that early, and retool the program.
  • Or consider the durable programmatic infestation of abstinence-only education, propelled by only fervent belief in the face of clear evidence that it fails to protect young people from unintended pregnancy and exposure to disease.

We are all painfully aware that resources are limited. That means ineffective or counterproductive strategies deflect attention and waste time and money urgently needed to support and expand strategies that are effective.

Gara says,

“The effectiveness movement is now finding, I believe, that there is no real constituency for effectiveness as such. … because it is values that move people.”

I say, whatever happened to doing what is right because it is right, rather than worrying about whether an idea has a “constituency?” In fact, isn’t this the very point you led off, with, Gara, in your talk at MIT? 

I would say it is morally right to learn as much as possible about whether what you are doing is having the desired effect, whether it is politically popular or not. 

And let’s take on the task of building the constituency for effectiveness. My experience at the Center for Effective Philanthropy convinces me that there is one.  And I think it is growing.  A survey we conducted this year of foundation CEOs showed that the overwhelming majority say assessment is among their highest priorities. And the constituency for effectiveness has deep roots: it is as old as philanthropy itself. Bill Schambra so often reminds us of this in discussing the “mania to measure” – although to him it is a lament – that goes back to the days of Rockefeller and Carnegie.

The constituency for effectiveness is made up of all those who are not content to assume that they are doing as well as they could be doing – those who are not content to take for granted that they are not unintentionally doing harm.  Those who want to learn and improve. Those who reject ignorance and seek knowledge.  Those, among foundations, who want to break out of the bubble of isolation and positive feedback that affiliation with large endowments inevitably creates, so they can learn how they’re really doing – and how they are really perceived. It is a growing constituency of those who believe effectiveness is a moral imperative.

So here is my central point: Assessment and morality are not in tension, never have been.  On the contrary. We are morally obliged to seek to know how we are doing, and how we can improve.  

And I would argue that the impulse to understand the efficacy of what we are doing stems exactly from our reasons for caring about it.

  • If we really care about improving the lives of former foster kids, than we must track, as the Stuart Foundation in California and its grantees do, the numbers of those with lifelong connections to caring adults and the numbers graduating from college.
  • If we really care about civil rights for gays and lesbians, as the Gill Foundation and its grantees do, than we must track in which states gays and lesbians enjoy the same legal protections – whether in marriage or the workplace – that heterosexuals do.
  • If we really care, as the Robert Wood Johnson Foundation and its grantees do, about tackling the obesity epidemic, than we must track the relevant data – including data on the aspects of the strategies employed that we believe to be crucial, just as that Foundation did in its successful work on tobacco use.
  • If we really care, as the Wilburforce Foundation and its grantees do, about preservation of habitats for wildlife, than we must be data-driven in our tracking of endangered species and seek to strengthen the organizations on the ground doing that hard work; in seeking to strengthen those organizations, the Wilburforce Foundation also seeks to find out whether its help is really deemed to be helpful by its grantees – in part by working with CEP to get candid and comparative data on that question.

These organizations assess because they bring passion to important issues.  Caring and knowledge of efficacy are not in tension, these are not qualities needing to be held in “balance,” as Gara argues.  They form, instead, a virtuous cycle.

Just as we should reject the caricatures and labels – and polarization – that cheapen our political discourse, so should we reject them in philanthropy. 

Invocation of “morality” without regard to effectiveness is often mere ideology. 

We work in the nonprofit sector because of our desire to make communities stronger, lives better, air cleaner, whatever the goal is that you or your organization fight for.  That’s why we do the work.  For that reason, we want to know – most of us deeply, desperately want to know – whether our work is making a difference. 

Of course, it’s hard to know, and some of the attempts to answer those questions are misguided or ill-designed. I would be the first to acknowledge that some assessment is done badly.  And I’d also concede that some make the foolish choice to let ease of measurement drive their selection of goals.

But let’s not let those errors in thinking and approach taint our view of the importance of assessment. 

The fact is, the alternative to measurement and assessment is flying blind.  Doing nothing, or worse than nothing, when we mean to be doing good. 

Failing to know what really helps people. 

Failing to direct resources in ways that result in real improvement in peoples’ lives. 

Failing to be our very best.

Assessment and morality are not in tension, never have been.  On the contrary. We are morally obliged to seek to know how we are doing, and how we can improve.  

Why Foundations Need to Make a Leap of Reason

Wednesday, July 13th, 2011

When it comes to nonprofit performance assessment, Foundations need to do a better job of bringing their practices in line with their attitudes.

  • A majority of CEOs said nonprofits should be held to a higher standard of evidence than they are today, in a survey of large foundations we conducted earlier this year. 
  • And fully 70 percent said they believe foundations should be placing a greater emphasis on understanding the effectiveness of the grantee programs and organizations they are considering funding, in the same survey.

These are two in a series of attitudinal questions we asked CEOs, which we will describe in a soon-to-be released-report, that paint a clear picture.  Simply put, funders are looking for more data and evidence to assess both their performance and that of their grantees.

But here is the disconnect between attitude and practice:

  • Just 11 percent of grantees from some 200 foundations surveyed by CEP over the past eight years say they received assistance beyond the grant focused on the development of performance measures.

To be sure, more than 11 percent of grantees may be receiving support – perhaps in the form of general operating support or capacity building grants – to help them assess performance.  But there isn’t much evidence to suggest this is the case.

This disconnect is troubling, to say the least.  After all, who will support nonprofits’ progress in the area of performance measurement if not foundations? 

Funders won’t ever see the fulfillment of their wish for better data to judge effectiveness if they aren’t willing to support nonprofits’ efforts in this area.  Do funders expect that capacity and skill to magically increase without resources or support?  Because it won’t.

That’s why the argument about the role of funders made by Mario Morino in his new book, Leap of Reason: Managing to Outcomes in an Era of Scarcity, is so important.  Morino writes:

“I know many nonprofit leaders who are not managing to outcomes but are strongly predisposed to do so.  They inherently know what their outcomes are and very much want to assess and manage to them.  But they are severely hamstrung by the lack of available funding to do this hard work. … At a minimum funders should be supporting efforts to help nonprofits to …. (a) track the outcomes of those served; (b) undertake at least basic analysis of this information; and (c) identify how they can use the information to learn and improve their programs over time.” 

This should not be a radical proposition.

Yet, frequently, the pressure to fund programs and not “administrative costs” leads, paradoxically, to funding approaches that surely result in worse outcomes per dollar spent because of a lack of good data to gauge effectiveness. As Morino puts it, the “understandable desire to be careful about costs can deeply undermine the pursuit of impact.”

There are too few examples of foundations investing in building the data management capacities of individual grantees or of a field – as, for example, the Stuart Foundation did in California.  Stuart recognized that its goal of improving life outcomes for foster youth in that state could never be achieved without the data systems to allow for tracking of progress.

Currently, many foundations won’t invest in this kind of infrastructure.  Leap of Reason makes the moral case for ending this dysfunctional practice, or non-practice – starting today.  Morino writes powerfully about the imperative for “managing to outcomes” and includes a set of practitioner essays that bring his arguments right down to the ground, where the proverbial rubber meets the road.  

Perhaps most poignant of the practitioner essays is one by Isaac Castillo of the Latin American Youth Center (LAYC), who writes candidly of how his organization learned that its efforts to change attitudes about domestic violence were having the opposite of the intended effect. 

How did the organization know its program wasn’t working?  Because it had invested in performance assessment – carefully tracking clients’ attitudes before and after the program. This allowed LAYC to right the ship, making the necessary changes to ensure the program is having the desired effect.

How many instances are there like the one Castillo writes of that we don’t know about – because organizations and their funders have not invested in the data systems that would allow us to know? 

It is a frightening prospect.

Morino’s book is a powerful call to action for both nonprofits and funders.  And one of its many virtues is that it doesn’t gloss over the difficulty of doing this work.

But the book also finds much reason for hope. Another of the excellent practitioner essays comes from Tynesia Boyea Robinson, executive director of Year Up, National Capital Region.

Robinson notes that nonprofit employees have the “intrinsic motivation” needed for this challenge – why else would they be doing this work?  She beautifully articulates how passionate commitment to a nonprofit’s mission is the only force that will get you through the tough slog of “managing to outcomes.” (Rather than seeing passion and a commitment to assessment as in tension, she sees them, as I do, as complementary. See my recent blog post on Tactical Philanthropy on this issue.)  She also lays out some common mistakes of those who advocate for assessment in a way that falls flat, noting the need for those coming from the for-profit sector – as she did – to become “bilingual.”    

“It’s amazing how many outcome initiatives fail simply because of language,” she writes.

The book is a must read for anyone who cares about impact.  Funders who are not currently supporting their nonprofit grantees’ efforts to assess and improve would do well to read it.  Better yet, if funders want to help grantees with this difficult task, they could, as a very modest first step, order them the book and open up the conversation about what is standing in the way of assessment – and what they can do to help.  

Indeed, Leap of Reason should be the first assignment in the foundation-grantees book club.  Here is my suggested discussion question:

  • What do grantees need from funders in order to be able to better execute the difficult work of assessing performance?

 

  Phil Buchanan is President of CEP.

How Understanding Their Fields Helped Three Foundations Make Better Decisions and Avoid Costly Mistakes

Thursday, December 16th, 2010

At CEP, we are always looking for ways that our data can help foundations forge stronger relationships with their grantees and become more effective in their work.

Recently, our research showed that when a foundation’s staff exhibits a high level of expertise in the field in which it funds, that expertise usually results in stronger relationships with grantees.

But, how exactly, do foundations gain that strong understanding of these fields? We know that in the busy lives of foundation staff, it can be difficult to carve out enough time to understand the fields in which they are making grants. What are the best source for information? How do program officers develop the necessary knowledge to advance their work? What does understanding the field really mean?

We decided to ask staff at three foundations that had participated in CEP’s Grantee Perception Report® (GPR) and were ranked in the top five percent of more than 200 foundations according to their average grantee ratings on the survey item: “How well does the foundation understand the field in which you operate?”

Our case study, Lessons from the Field: From Understanding to Impact, provides an in-depth look at how these three different foundations cultivate an understanding of their fields and then translate that understanding into more effective grantmaking.

The three foundations – the Evelyn and Walter Haas, Jr. Fund, the Energy Foundation, and the Wilburforce Foundation – want to make an impact on some of the most complicated challenges we face: civil rights, renewable energy sources, and wildlife protection.  

When we interviewed them, staff at these three foundations told us how under­standing their fields helped them make better deci­sions and avoid costly mistakes. For example:

  • Listening to stakeholders in California led the Haas, Jr. Fund to reshape a communications campaign on immigrant rights so that it took into account important regional differences.
  • A deep understanding of Chinese culture helped the Energy Foundation pilot a groundbreaking plan to significantly reduce air pollution, which the Chinese government has adopted nationwide.
  • Some extra research helped Wilburforce discover that it needed to cultivate a key constituency group in order to protect one of the largest intact rainforests on earth.

The foundations also offered very different examples of how they develop and maintain an understanding of the field. For example:

  • At Haas, Jr., understanding the field means that the foundation seeks to develop the capacity of their staff and grantees to be leaders in their fields.
  • At Energy, understanding the field involves staff developing close working relationships with experts in the field and connecting grantees with experts to enable problem-solving.
  • At Wilburforce, understanding the field means knowing as much about the people living and working near the areas it seeks to protect as it does the flora and fauna residing on those lands.

We hope that our case study provides useful ideas to foundations seeking to better understand the fields that they fund, and we welcome your feedback. The case is available for free download here.

Introducing Linda Wood of the Evelyn and Walter Haas, Jr. Fund

Tuesday, July 20th, 2010

When it comes to soliciting feedback from grantees, the Evelyn and Walter Haas, Jr. Fund in San Francisco walks the talk. Thoughtfulness, candor, feedback from grantees, and constant improvement are an integral part of how I’ve seen the Fund do its work.  

Recently, we were interviewing staff from Haas, Jr. for a new CEP case study on understanding the field – the Fund is an exemplar in gathering and sharing expertise in its fields of work. One staff member we spoke with was Linda Wood, senior director, leadership and grantmaking.

Knowing how important good leadership is for its own success, the Fund has established a program area that provides access to leadership development for nonprofits. In the course of our conversation about her work in that area, Linda mentioned videos of grantees describing their leadership coaching experience that the Haas, Jr. Fund has posted on its website.  People reacted with surprise, Linda said, at how openly the folks in these videos share what that coaching has been like for them.

Because honesty and transparency are also CEP’s watchwords, we were delighted when Linda offered to share her thoughts about these videos in a blog post.  We hope you will join the conversation as she ponders, “What would it take for truth telling and candor to catch on in philanthropy?”

Kevin Bolduc is Vice President, Assessment Tools at CEP.