Did They Get It? And Should You Care?

Performance improvement professionals have become very adept at creating exceptional ways to impart knowledge or develop skills to improve job performance. In the last 10 years or so, many have also become proficient at measuring how successful they are in meeting performance improvement goals.

Professionals that credibly measure success use the ROI methodology that isolates the business impact from the training event. (Phillips 2003). This methodology has helped a somewhat skeptical learning, development, and business audience become comfortable that those programs measured for job performance are being measured accurately. But very few programs are measured at this level. Why? Because of a lack of resources or expertise and because of the intrusiveness it can cause to the employees and the organization (Phillips, Phillips, and Hodges 2004).

In fact, most organizations do not measure job performance or ROI at all. Many measure the participants' satisfaction or reaction to the program, but unfortunately, many organizations do not know if their employees actually learn from their training programs. The truth is, however, nearly all programs can be measured for learning. These are the advantages of creating tests and other techniques that measure if participants "got it":

• Program designers receive quantitative data about which parts of the program work and which do not.
• Performance improvement professionals learn whether the training program is the cause of poor performance or if something else is to blame.
• Training leaders can determine if the participants have the prerequisite skills required to participate in the training program.
• Training designers have the data to make comparisons regarding programs, such as which media type is most effective, which instruction is working best, or which participant group is getting the most from the program.
• Program stakeholders, including those who paid for the program and those supervisors who must do without their employees who are participating in the program, receive quantifiable data about the value of the program.
• The organization receives scorecard data that demonstrates how effective the training organization is in meeting its goals.

So if learning data is so valuable, why don't more organizations take advantage of it? Many do not realize the ways learning data can be used or how to design or validate tests.

Now that we have established the importance of learning data, let's address the expertise component. There are many resources that can help the training designer develop knowledge- or performance-based tests. A list of good resources to start with is included with this article.

In addition, ASTD provides testing certificate programs. Several consultants provide workshops for organizations so that their entire design and development team can learn methods to construct tests. And, the Test Construction and Validation Guidance Documents and Standards, developed by Toni Hodges DeTuncq, ensure that all tests are constructed and validated properly. These standards not only ensure the organization develops tests that are legally defensible but are standardized so that scores from one program can be compared with those of another. Once these standards have been prepared, the organization can practically, effectively, and efficiently measure their programs, and realize the benefits of doing so.