I am pleased to announce a new partnership between YouthTruth and The New Teacher Project, which will bring student perspectives to teacher performance evaluations. See the video below for details or read the press release.
Valerie Threlfall is Vice President – YouthTruth Initiative at the Center for Effective Philanthropy.
CEP has been actively exploring the topic of beneficiary feedback through its YouthTruth project since 2008. We were very pleased to share what we are learning – and to challenge the status quo — on the Tactical Philanthropy blog this week, where the following post was originally published.
The power of feedback is almost universally acknowledged. Every day, millions of people turn to reviews written by end-users in Yelp, Trip Advisor, GreatNonprofits, and Global Giving to inform their purchasing and philanthropic decisions. The end-user’s perspective on their experience is seen as an asset just about everywhere—except in the world of foundations.
Consider a recent report published by the Center for Effective Philanthropy (CEP). The State of Foundation Performance: A Survey of Foundation CEOs, updates work CEP completed a decade ago, looking at the types of data foundations use to assess their performance. While the report shows that foundations are increasingly looking to a range of indicators, notably surveys of grantees, only about a quarter of funders gather feedback from beneficiaries in any way. This, despite analysis showing that it may be among the most valuable types of feedback. CEOs of foundations who collect beneficiary feedback rate themselves as having a better understanding of the progress their foundation is making against its strategies and a more accurate understanding of the impact the foundation is having on the communities and fields in which it works.
As head of CEP’s YouthTruth initiative, which gathers comparative feedback from students – the ultimate beneficiaries of education reform efforts, I have seen the power beneficiary feedback can have in informing local school change efforts. Consider this recent post about The California Endowment’s YouthTruth experience. So the question I ask myself is this: How can we better understand why foundations have not seriously embraced beneficiary feedback, and how can we encourage its use?
To elevate beneficiary perspectives, I am convinced that at least three things need to happen.
The third area is where we have made the most progress, as we are increasingly well-positioned with new tools for gathering beneficiary feedback. Now we need to push ourselves on these first two points.
With regard to the first, we need to address the biases that convince some in the field that beneficiary feedback is less valuable or objective than the opinions of say, grantees or donors.
Daniel Sid describes the situation this way on Bridgespan’s blog: “the working assumption of the suppliers [of nonprofit human services] and their funders alike is that they know what is best for the individuals and families they are serving… reducing [beneficiaries] to passive recipients… And why bother [gathering feedback] when your beneficiaries can’t really take their business elsewhere?”
With regard to the second point, can foundations expect to learn anything that will enable them to make different decisions from beneficiary feedback? Speaking from my own experience, we don’t know for sure but we are getting close.
YouthTruth was started by CEP, with support from the Bill & Melinda Gates Foundation to test this very notion. To date, YouthTruth has gathered perspectives from more than 71,000 students from 164 schools nationwide. With this dataset, we have been able to examine the degree to which there are systematic differences in student perspectives based on the type of school they attend.
Moreover, we have the ability to identify differences in student perceptions, say about their futures, across groups implementing a common strategy, such as a STEM strategy, and provide insights about the relative effectiveness of various efforts. This to me seems like helpful and actionable data that can inform funders’ decision-making. While this is one isolated example, we need many more experiments such as YouthTruth to push our thinking about the benefits and limits of beneficiary feedback.
I would argue that since beneficiaries are ultimately the people we are all trying to serve, we need to take dramatic steps to take their perspectives into account.
We need to ask each other the tough questions about why this feedback is not being more widely used and challenge ourselves to determine the potential of beneficiary feedback once and for all.
Valerie Threlfall is vice president – youthtruth initiative at CEP.
The following post by Valerie Threlfall, Director of CEP’s West Coast office, was originally published on the Huffington Post.
With Joel Klein’s resignation as New York City schools chief, close on the heels of Michelle Rhee’s departure from Washington, DC, there is no shortage of controversy in today’s education community. Disagreements abound over everything from teacher accountability to Waiting for “Superman” to funding of hot lunch programs.
But amidst all this rancor, I find myself oddly encouraged by the passion and intensity of this public conversation. Acrimony aside, we’re at least fighting about solutions. However, one obvious and significant perspective that is missing from the conversation is the voices of students themselves.
Algebra teacher Karl Fisch (known for his thought provoking video-gone-viral “Shift Happens“), wrote recently about the importance of talking with students. I agree with many of his points but would take it a step further.
Why aren’t we asking students themselves how to make our schools work better? What about the experts or “consumers” on the other side of the textbook? Is it ridiculous to think that student feedback could actually play a significant role in shaping education reform?
Three years ago, I would have answered with uncertainty. But 86 schools and 21,000 high school student surveys later, the potential blows my mind.
In 2008, I began working on YouthTruth, a nationwide effort to gather comparative feedback from the “beneficiaries” of education funders — in this case high school students – about what’s working and not working in their schools. An initiative of The Center for Effective Philanthropy, YouthTruth shares the data it collects with teachers, school administrators, district and network leaders, and education funders so that they can make better and more informed decisions to improve their schools.
I’ll admit, at first our work is often met with skepticism, and often the biggest doubters are the students themselves. Students have become understandably wary about providing time-consuming input because they typically never see, or benefit from, the results of their feedback. All too often youth are tested, studied, and evaluated like lab rats, without ever being told what was learned or concluded.
But it doesn’t have to be that way. As one student said to me early on, “You don’t need gimmicks. All I need to know is that you are going to listen and take my feedback seriously.”
At Youth Truth, we do just that. Our survey gathers student perceptions across five major themes, including:
Schools that participate in YouthTruth are able to see how student perceptions compare to both ‘peer schools’ in their district or charter network, and to our growing national sample. YouthTruth also helps participants make sense of the data. We present the results back in digestible formats, offer comparative analysis with other schools and districts, and build in focused time for reflection and planning.
This includes bringing schools from common networks together to compare their results, identify best practices, and brainstorm strategies for sharing data back with their school community. These meetings are a rare opportunity for over-burdened school staff to get input from their peers on tangible steps they can implement once they get back to campus.
Finally, participating schools share their results with students before the end of the school year — an expectation we set for everyone early on. This demonstrates a commitment to improving the student experience, and inspires students to identify what they can do to make their school better. The average response rate for the YouthTruth survey historically has been 80 percent across participating schools.
Most exciting is that we are seeing schools use this data to drive change. In response to YouthTruth, many schools have created more rigorous and relevant lesson plans, or rethought approaches to discipline that increase student input and fairness. Other schools have created stronger advisory programs–linking students with teachers as confidants and mentors–or created new professional development programs for teachers.
A recent independent evaluation of YouthTruth found that 99 percent of school administrators agree the strategy has been helpful in planning specific changes in their schools. What’s more, 100 percent of participating principals say they would recommend YouthTruth to another school. Having seen this model work in dozens of schools now, I’ve come to realize that when given the forum and the opportunity, students provide feedback that is dead-on for how to improve their experience.
As one school principal said, “The YouthTruth data provides a very powerful resource to start conversations and motivate change.” This year we will be surveying 130 additional schools and roughly 80,000 more students in cities nationwide.
While I am in no position to argue about the efficacy of specific reform efforts underway in districts all over the country, it is alarming that more education leaders are not accessing the full breadth of feedback loops available to them. We need to listen more closely to the kids we’re all trying to serve, plain and simple.
What would your students say needs to change? We’d love to help you find out.
When I joined CEP in 2008, YouthTruth was simply an idea on a piece of paper. Our goal was to bring new voices, those of the beneficiary, to bear on the decision-making of school leaders, those who manage schools, and, perhaps most importantly, those who fund schools.
I came to this project as a generalist. I was good at writing business plans and had experience helping organizations take an idea and bring it to fruition. However, I quickly saw that we were doing something different and delivering to students and schools a differentiated, unique opportunity to drive school change.
I remember walking awkwardly into our first YouthTruth assembly in a small town in North Carolina — feeling very old as I heard girls talking about their boyfriend troubles or their latest fight with their best girl friend. “Can you believe she said…” “and then he …” All the good memories — and moreover the hard feelings — of high school came washing over me. But then, kids started talking during the YouthTruth kick-off assembly. One girl stood up to speak about one of her teachers. She said that her teacher really knew her subject but frankly wasn’t good at explaining things and would get frustrated with students when they didn’t seem to understand her. Another student talked about how he had had three math teachers that year — how was he expected to learn, he described, when he had such turnover in the teacher role?
I was honestly blown away by the thoughtfulness and candor that these students brought forward when we gave them the opportunity to participate in the improvement of their school experience and felt encouraged to redouble our efforts to make this project a success.
Since that assembly, we have seen countless students respond in equally thoughtful and powerful ways through the YouthTruth survey. We have collected quantitative ratings from more than 21,000 students and have coded and categorized qualitative comments from an equally large number of students. Here’s some comments as illustration:
“The curriculum that I am experiencing right now is one that forces me to do things for myself. The classes I take are challenging. They force me to come out of my comfort zone, so I learn more.”
Contrast that with:
“There are some teachers that don’t explain the work and don’t teach us what we need to know for tests and quizzes. All they do is assign homework and expect us to learn it ourselves. Well, some students cannot learn like that, some people actually have to be taught things and shown how to do things. Not everyone does well by having to learn things on their own.”
I was not an “education person” when I joined this project. However, I have come to realize that when given the forum and the opportunity, students provide feedback that is dead-on for how to improve their experience. As we twist our education system up into a formative pretzel, I continue to be baffled that so few are gathering feedback from kids — those who we claim we are trying to serve — about whether our reform efforts actually make sense and whether they’ll achieve what we want them to achieve. Don’t discount the voices of these kids simply because they are younger. They see and experience what we don’t see day to day and deserve to be heard.
Check out the video below to hear from students about how they feel YouthTruth is helping them provide input into their school improvement efforts.
“You don’t need lots of gimmicks. All I need to know is that you are going to listen and take my feedback seriously.”
I heard this time and time again when we were talking to students early in the development of YouthTruth. Students had become skeptical about the value of providing input because they were continually taking time-consuming surveys and never seeing the results of their feedback. What a wasted opportunity.
In my last post, I talked about the role that student feedback could and should play in helping shape education reform. However, it is not enough to simply ask students questions about what’s working and what isn’t. For this kind of input to motivate change:
Given this, with YouthTruth, we strive to actively engage students and staff both before and after the survey.
To ensure students take the survey seriously, we communicate to them upfront that they will see the results of their feedback. Participating schools are required to share their YouthTruth results with students before the end of the school year. Sharing results with students demonstrates school leaders’ commitment to improving the student experience, and it inspires students to identify what they can do to make their school better. The video below highlights two schools’ experience with sharing data back with students.
YouthTruth also actively partners with school leaders, network, and foundation leaders to help them respond to their data. One of the ways we do this is that we bring schools from common networks together in regional convenings to compare their results, identify best practices, and brainstorm strategies for sharing data back with their school community. This is a rare opportunity for over-burdened school staff to get input from their peers on strategies for improving their school and ideas about tangible action steps they can implement once they get back to campus. Check out the video below for an example of a YouthTruth regional convening.
We believe that this engagement process with both staff and students complements the comparative data we provide — and is part of what has helped YouthTruth lead to demonstrated improvements and change in participating schools.
