For 15 years, learning professionals have acknowledged the importance of designing and implementing effective Level 2 tests. Successful information gathering depends on trainers' ability to leverage the basics of why the learning is important and what the goals are.
What is it?
When people who develop learning programs talk about "Level 2 tests," they are referring to the second level of training evaluation as defined by Donald Kirkpatrick. A quick paraphrase of Kirkpatrick's evaluation levels reads
Level 1: Reaction - How did those participating in the learning find the experience?
Level 2: Learning - Did the participants learn what they set out to learn?
Level 3: Behavior - Did the participants change their behavior because of what they learned?
Level 4: Results - Did we achieve the intended outcome(s) for the organization, and was it worth it?
To do a great job in one level of measurement, you need to think about all four. Here, we are going to focus on creating tests in support of Level 2: learning.
Guidelines for creating Level 2 tests
1. Know why. Although we are focusing on what people have learned because of our work with them, to measure the right things, we need to understand clearly why we have been asked to design the learning experience in the first place. The best questions we can ask our customers are, "What do you want to be different because we worked together?" and, "Why does that matter to you?"
If these two questions seem to be about Level 3: behavior and Level 4: results, it is because they are. As we determine what we will test in Level 2, we must be informed by the big picture.
You may have a customer who wants to improve customer service in the call center. Your good questions will lead to good design, and you will also know what learning will be most critical to achieve your customers' needs. Do they need to master the phone or computer systems? Do they need product knowledge? Do they need to listen better? Do they need to understand the culture of the callers?
When you know what needs to be learned and why, you know what to test for.
2. Remember that you are evaluating the effectiveness of your work. Creating Level 2 tests feels a lot like writing tests for school students, but the purpose here is not to pass or fail participants. Remember that you are testing the efficacy of your design and delivery system.
If your test results show that the participants have not learned the key lessons, it is time to redesign the experience so that they do. If you are doing safety training, it is more important that you are delivering it in such a way that learners know when and how to use the emergency eyewash station, for example, than it is for you to determine who is or who is not a "model student."
3. Measure the relevant. If you are teaching salespeople a systematic way to answer an objection to the price of your product, asking them who founded the company and in what year is probably the wrong question. Avoid falling into the trap of writing a test for the participants by asking them "tough" questions or "extra credit" questions. You are interested in asking questions that will help you determine whether or not your design and delivery has contributed to the retention of key points.
4. Check more than once. The moment the learner finishes a section or completes the entire program is a handy time to test whether key information has been captured. It's a good time to test because we have the learners present - they are in the classroom or at the keyboard, and we won't let them leave until they complete the Level 2 evaluation.
This is certainly a valid time to check in with your learners, and it is also a wonderful opportunity to let them know that you'll be back with either a follow-up meeting, email, or online session to check in a few weeks to see what they remember. (You can continue to evaluate at the other three levels at this time as well.)
5. Use Level 2 evaluation as a learning tool. "Tell them what you're going to tell them. Tell them. Tell them what you told them."
This is an old guideline for speechwriting that we can borrow for Level 2 evaluation. Be sure that the learners know the key points that they need to learn before they begin. Highlight the key learning points while they are learning by stating, "This is a key learning point that we will be testing for later."
Introduce the Level 2 evaluation as a continuation of the learning: "Now we will check to see the effectiveness of our learning together by checking what you remember about the key learning points we introduced before we began and in the body of our session."
6. Track it and tweak it. The reason that you are measuring what people are retaining is so that you can be sure that you are doing the great job you want to do and that your customer requires. Treat the results of your Level 2 evaluation as data. What key learning points are more reliably retained than others? What key learning points are retained over time? Conversely, what points are missed or lost over time?
Take what you are learning from your Level 2 data and revisit your design and delivery. What needs to be changed? What is it about the delivery of the points that are being retained that is different than that of the points that people miss?
Retool your design and delivery, do it again, and retest.
Targeted, repeated Level 2 evaluation will contribute to the continuous improvement of the learning events that you design for your customers, and will contribute mightily to success in the other three levels of evaluation. Learners who are learning will have a better experience. Learners who have their learning reinforced are more likely to apply it. Organizations that are clear about what needs to be learned and are measuring to be sure that learning is taking place are more likely to get the results they are seeking.