Why learner satisfaction matters
Even if it doesn't prove 'impact'...
Last Tuesday, Mind Tools published our new report, ‘Building Better Managers’.
Written by my brilliant colleagues Dr Anna Barnett and Dr Gent Ahmetaj, the report combines survey data with findings from academic research to provide insight into the challenges managers face, and the skills they need to succeed in their roles.
Over the coming weeks, Ross G and I will be using the Dispatch to unpack some of these insights, zooming in on particular areas that piqued our interest.
For me, one of the most intriguing findings from the report related to learner satisfaction.
Of the 2,001 managers we surveyed, only 58% said they were satisfied with the leadership and management training they had received.
While that number is embarrassingly low, it isn’t what intrigued me. Rather, it was the claim that this number actually mattered.
Before reading ‘Building Better Managers’, my instinctive response to that 58% figure would have been: ‘So what?’
I mean, sure, it would be nice if people liked training a little more. But L&D isn’t in the entertainment business.
Whether a learner enjoys an experience or hates it, their emotional reaction doesn’t tell us if they’ve learned anything, and it doesn’t prove impact.
While I still believe that’s essentially true, I’ve also come to realize it’s a bit of a straw man.
As Anna and Gent write:
‘As of 2019, a substantial 83% of organizations were using satisfaction surveys (or “happy sheets”) to evaluate their learning programs – a method more popular than anything else.
Satisfaction surveys (and happy sheets, in particular) get a bad rap. And it’s not fully justified. There’s a considerable amount of research available demonstrating relationships between learner reactions and learning outcomes
A review of that research shows that satisfaction with learning improves both skill-based and factual knowledge (albeit in small quantities and only in the short-term). But learning impact is about much more than knowledge. It’s about behaviors, performance and business outcomes.’
This brings me back to the ‘intangible benefits’ argument I made a few weeks ago.
Although satisfaction may be insufficient to demonstrate impact, this doesn’t mean we should disregard it entirely.
Intuitively, a learner who enjoys an experience might be more likely to apply what they’ve learned, or to pursue additional learning opportunities in the future. They may also feel that their organization has invested in their development, and decide not to look for other jobs elsewhere.
In many ways, satisfaction surveys are the Coldplay of learning measurement — hugely popular, just not amongst the cool kids. And membership of that club demands a militant disdain for ‘happy sheets’.
Reading Anna and Gent’s report has challenged some of my more dogmatic views, and put me ‘in my place’. Maybe it can ‘fix you’, too.
Were you satisfied with this week’s Dispatch? Interested in working with the Mind Tools Custom Team? Then get in touch by emailing custom@mindtools.com or reply to this newsletter from your inbox.
🎧 On the podcast
Two weeks ago, we kicked off our three-part #BuildingBetterManagers series with an overview of the 12 capabilities that we know make a difference to performance. Now we’re asking: How do we help managers build those capabilities?
To answer this question, Dr Anna Barnett from the Mind Tools Insights team joined Ross G and I to discuss her recommendations, based on findings from the report.
We discuss: evidence-based approaches to improving manager capability; how Mind Tools factored this evidence into our product design; an example of a custom management programme that made a measurable difference to manager capabilities.
Check out the episode below. 👇
You can subscribe to the podcast on iTunes, Spotify or the podcast page of our website. Want to share your thoughts? Get in touch @RossDickieMT, @RossGarnerMT or #MindToolsPodcast
📖 Deep dive
One of the studies Anna and Gent cite in ‘Building Better Managers’ is ‘A Review and Meta-Analysis of the Nomological Network of Trainee Reactions’, published in the Journal of Applied Psychology.
Based on their findings, the study’s authors make several recommendations for practitioners regarding the measurement of learner reactions.
Broadly speaking, they conclude that the usefulness of learner reactions depends on the goals of the evaluation effort.
For instance, if an intervention is based around ‘affective learning outcomes’ (motivational speaking, diversity training, etc.), reaction can be used to gauge learners’ receptivity to attitude change.
The study also found a strong relationship between reaction and post-training motivation, self-efficacy, and declarative knowledge when technology was used to deliver the intervention. The authors write:
‘Thus, it makes sense to be particularly sensitive to reactions in technology-delivered courses. Designing technology-based courses that trainees feel are useful and interesting should improve reactions and enhance training effectiveness.’
Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K., and Zimmerman, R. D. (2008) ‘A Review and Meta-Analysis of the Nomological Network of Trainee Reactions’, The Journal of Applied Psychology, 93(2), pp. 280–295.
👹 Missing links
When John Mendola’s dog Princess was diagnosed with cancer, he was devastated. Then, after watching a television show about pet-cloning and doing some research online, he found a company in Texas called ViaGen Pets & Equine. In this article from The New Yorker, Alexandra Horowitz explores the ethical implications of cloning, and what it means for the animals involved.
🤑 Going Infinite: The Rise and Fall of a New Tycoon
I recently enjoyed a week’s vacation in Santorini, and brought Michael Lewis’ latest book along for the journey. It’s a fascinating portrait of Sam Bankman-Fried’s rise to the top of the crypto world, and his subsequent conviction for fraud. Even if you know nothing about crypto, or find it absolutely bewildering (like I do), you’ll have trouble putting the book down.
💧 The ideal place to relax on holidays and weekends
Yet another holiday-related ‘missing link’, as I’m clearly not quite ready to let go of my vacation. In this short issue of his ‘Granted’ newsletter, Adam Grant highlights new research which suggests that after just 2 minutes of viewing water outdoors, blood pressure and heart rate drop. So, if you’re still planning a relaxing summer holiday, you might want to consider lakes, pools, and streams, instead of grass, trees, and mountains.
👋 And finally…
This banger was ringing in my ears as I wrote this week’s Dispatch:
👍 Thanks!
Thanks for reading The L&D Dispatch from Mind Tools! If you’d like to speak to us, work with us, or make a suggestion, you can email custom@mindtools.com.
Or just hit reply to this email!
Hey here’s a thing! If you’ve reached all the way to the end of this newsletter, then you must really love it!
Why not share that love by hitting the button below, or just forward it to a friend?