Listening Carefully: An Argument for Considering All the Data

by Kevin Bolduc

Field Focus: Data and the Sector is intended to offer us all the opportunity to consider the ways that numbers, statistics, and trends both impact and inform the way we do our jobs. But put another way, we’re also talking about “feedback,” the qualitative data that lets us check-in with our stakeholders about the work we’re doing and the work we ask them to do.

In this blog, Kevin Bolduc of the Center for Effective Philanthropy writes about the importance of hearing every voice.

Field Focus LogoA few weeks ago, I was chatting with a vice president of a foundation that my organization, the Center for Effective Philanthropy (CEP), has worked with over the course of many years. As we caught up, she interjected, “Remember the time that you were presenting the results of our first grantee survey and you shared a grantee comment that called our processes ‘onerous’ and ‘meddling?’”

I did remember: it was challenging feedback made by a clearly angry recipient of the foundation’s funding. This vice president’s comment, though, took me by surprise. It had been years ago, and we’d worked with her on other grantee surveys in the time since then. She continued, describing how surprised she and her colleagues had been, though they ultimately took it as a call to action and consequently made some changes in their work. Subsequent surveys showed that this funder had stronger relationships with grantees and was increasingly seen to be creating significant impact on the fields in which it funded. You’d be unlikely to find them described as onerous and meddlesome today.

Still, she got me thinking, once again, about who it is that gets heard by funders. In an increasingly crowded conversation, issues of voice and power play out in subtle ways. (ICON) Twitter Logo Is it only the squeakiest wheel — the most disgruntled grantee — that gets heard? Only the loudest thank you? Does the quieter common experience get lost?

This dynamic is a challenge not only in the use of feedback in philanthropy but also in the sharing of ideas and information. In a world of information overload, whose ideas break through? The most powerful voice? The most compelling presentation? The newest idea?

Given these dynamics — and they exist well beyond philanthropy, too — it’s incumbent on all of us to ensure we’re attentive to not just the loudest voices. (ICON) Twitter Logo Those that might otherwise go unheard often provide crucial insight about where and how we should choose to act.

If we’re not careful, the most memorable extremes can crowd out the reality of the typical experience.

That’s why CEP has long believed that “listening” has to be based on a rigorous, comprehensive, and representative approach. The particular comment that vice president shared was memorable, but it was also representative of a theme that CEP identified via other comments and data points. That upset grantee was simply the articulate avatar of the aggregate experience of many others. The comprehensive nature of the feedback is what provided weight and ultimately motivated change. Had it been just that one voice, as angry as it was, generalizing from it would have been unwise.

If we’re not careful, the most memorable extremes can crowd out the reality of the typical experience. It’s the “TripAdvisor problem”: the incentive to share is really high when you’re angry or elated; when you have nothing to lose or an opportunity to curry favor. I see this play out all the time when funders tell me, “Oh, we get feedback. Our grantees trust us. In fact, I just had this one grantee who told me that…”

This dynamic is equally at play when seeking knowledge to inform new directions. In a study commissioned last year by the William and Flora Hewlett Foundation, foundation staff and board members’ number one source of knowledge and preferred means to gather knowledge was each other — their peers. So, which peers do you listen to? On what did they base their advice? It’s hard to know.

As an example, I think back to 2015 when enthusiasm about impact investing — led by some powerful, compelling voices — reached an early crescendo. Given the intensity of the conversation, it was easy to feel that impact investing was already a major emphasis at many foundations. And there was some truth to that feeling. When CEP took a broad and representative look across all private funders granting more than $10 million annually, 41 percent of funders reported having tried impact investing. But the context provided by a comprehensive look added something important: the actual dollars foundations had allocated for impact investing were very, very small. Impact investing was a tidal wave in the conversation, but still only a ripple in how foundations used their assets.

Whether it’s in getting feedback or searching for new ideas, each of us needs to motivate ourselves to apply the effort to ask whether we’ve listened carefully and comprehensively before forming a judgment and acting. And for funders, it requires a commitment to marshalling the resources needed to get beyond the loudest voices and listen in the more expansive and time-consuming ways that yield an accurate understanding. It means striving for representativeness rather than convenience in research and feedback surveys. It means being transparent about any bias when we give advice to others. It means not overreacting to the loudest critic or being satisfied by periodic anecdotes of success.

I could tell many other stories like the one I shared at the beginning of this post — funders learning from carefully executed grantee feedback (or research), responding to a challenge or an opportunity, and making real change in their work. The work philanthropy does is consequential and powerful, and so it’s our responsibility to ensure that the way we’re collecting data has a fair chance of hearing all perspectives, not just the loudest voices. (ICON) Twitter Logo

Photo: Kevin Bolduc


Kevin Bolduc is vice president, assessment and advisory services, at the Center for Effective Philanthropy. Follow him on Twitter at @kmbolduc.

Field Focus Logo
Join the conversation!
Visit johnsoncenter.org/fieldfocus to begin exploring our resources and engaging with new ways of using data to inform your work. Have questions or thoughts to share? Let us know by using #FieldFocus on social media or by emailing jcp@gvsu.edu.

Post a comment