Advertisement
Advertisement
Ruler.fw.png
ATD Blog

Performance-Focused Smile Sheets

Wednesday, August 7, 2013
Advertisement

Smile sheets are not related to learning. Smile-sheet ratings that are good don't guarantee that the learning solution was effective. But smile-sheet ratings that are bad don't necessarily mean that the learning solution was poor. Meta-analyses from scientific refereed journals show correlations at about .09, which is basically no correlation at all.

So, what does this mean to you? Well, if you trust smile sheet ratings as a barometer of how effective your learning solutions will be, you're just being foolish. And, I offer my apologies for being so straightforward.

The bad news is that most of us are going to keep using smile sheets anyway.

The good news is that a new kind of smile sheet has been devised: the performance-focused smile sheet. It is based on four major improvements in smile-sheet design.

  1. It targets on-the-job performance, not training engagement or learner satisfaction.
  2. Its design is inspired by the learning research in that it assesses how well the course or learning solution uses critical learning factors in its design.
  3. Its questions are worded to enable more precise answers from learners, better reflecting the reality of the learning.
  4. Its questions are worded to enable clearer results and, thus, better decision making by stakeholders.

World's Best Smile Sheet Question

I have a new candidate for the World's Best Smile Sheet Question. It's great because it focuses on work performance, it enables clear decision making on the part of learners, and it helps stakeholders know whether the training was good or not.

Advertisement

In regard to the course topics taught, HOW ABLE ARE YOU to put what you’ve learned into practice on the job?

  1. I’m NOT AT ALL ABLE to put the concepts into practice.
  2. I have GENERAL AWARENESS of the concepts taught, but I will need more training/practice/guidance/experience TO DO ACTUAL JOB TASKS using the concepts taught.
  3. I am ABLE TO WORK ON ACTUAL JOB TASKS, but I’ll need more hands-on experience to be fully competent in using the concepts taught.
  4. I am ABLE TO PERFORM ACTUAL JOB TASKS at a FULLY-COMPETENT LEVEL in using the concepts taught.
  5. I am ABLE TO PERFORM ACTUAL JOB TASKS at an EXPERT LEVEL in using the concepts taught.

This question also can be used on a delayed smile sheet, making it doubly powerful.

Now, compare this question to the typical smile-sheet, overarching question: "Overall, how would you rate this course? A. Excellent, B. Very Good, C. Good, etc..." Stakeholders who see results in response to this question (such as "The average response was a 4.1") don't really learn very much about the training that is being evaluated.

Advertisement

In contrast, stakeholders who answer the World's Best Smile Sheet Question will see that 80 percent of learners are "ABLE TO PERFORM ACTUAL JOB TASKS at a fully competent level." These stakeholders will know a whole lot more about the learning taking place!

Obviously, no one question can work alone. Performance focused smile sheets also can—and should—include other overarching questions, as well as questions that focus on whether the course was designed in accordance with the learning research and if the learner is likely to put the learning into practice.

Note: This blog post is excerpted from Will Thalheimer’s August issue of “Will’s News.”  You can join Will’s mail list at www.work-learning.com, or read his blog at www.willatworklearning.com.

About the Author

Will Thalheimer is a learning expert, researcher, instructional designer, business strategist, speaker, and writer. He has worked in the learning and performance field since 1985. In 1998, Will founded Work-Learning Research to bridge the gap between research and practice, compile research on learning, and disseminate research findings to help chief learning officers, instructional designers, trainers, e-learning developers, performance consultants, and learning executives build more effective learning and performance interventions and environments. He speaks regularly at national and international conferences. Will holds a BA from the Pennsylvania State University, an MBA from Drexel University, and a PhD in educational psychology: human learning and cognition from Columbia University.

2 Comments
Sign In to Post a Comment
Great article, I was looking for something like the above!
Sorry! Something went wrong on our end. Please try again later.
Amazing, thank you! I sub. to your website newsletter and I will work through you new book. I am using a lot of your ideas in a graduate setting. As a clarification of your example that, "80 percent of learners are "ABLE TO PERFORM ACTUAL JOB TASKS at a fully competent level" it seems helpful to point out that stakeholders don't actually see/prove their competency. The question tells us what they believe about their competency. We'd need more to assess actual competency (Kirkpatrick New world?)
Sorry! Something went wrong on our end. Please try again later.
Sorry! Something went wrong on our end. Please try again later.