Return on expectations (ROE) is rapidly sweeping the learning and development industry. The reason is quite simple: when used properly, it never fails to show the value of training in the terms desired by key stakeholders.
This is because ROE is designed and executed in partnership with the people who will determine learning's value. The Kirkpatrick Foundational Principles explain the ROE process and the critical importance of business partnership in achieving it:
- The end is the beginning.
- ROE is the ultimate indicator of value.
- Business partnership is necessary to bring about positive ROE.
- Value must be created before it can be demonstrated.
- A compelling chain of evidence demonstrates your bottom line value.
The end is the beginning
Showing the value of training begins before the program even starts. Donald L. Kirkpatrick says it best in Evaluating Training Programs: The Four Levels:
"Trainers must begin with desired results and then determine what behavior is needed to accomplish them. Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the desired behavior(s). The final challenge is to present the training program in a way that enables the participants not only to learn what they need to know but also to react favorably to the program."
The clearer a training professional is on the desired Level 4 results of a solution (See Table 1), the better chance that his time and effort will cost-effectively accomplish that end. A common error many training professionals make is defining desired results at the level of training instead of relating training to the accomplishment of the highest level business goals. Here are some common examples of true Level 4 results:
- employee retention
- saving or extending lives
- prevention of damage to the environment.
It is also critical for the training professional to find out how important the initiative is to the organization and obtain some idea of the resources that can be allocated to it. Not every single initiative and program will warrant a complete execution plan as outlined in this article. The level of effort should be in alignment with the importance of the initiative to the success of the organization.
ROE is the ultimate indicator of value
When executives ask for new training, many learning professionals retreat to their departments and start designing and developing suitable programs. Sometimes, a training needs assessment is conducted, but it is rarely taken beyond training and into critical performance and business needs. This is where a lot of training waste occurs: training that isn't needed is conducted, and the required training or other solution that would be truly beneficial is not even identified.
Stakeholder expectations define the value that training professionals are responsible for delivering. Learning professionals must ask the stakeholders questions to clarify and refine their expectations on all four Kirkpatrick levels, starting with Level 4. This is a negotiation process in which the training professional makes sure that the expectations are satisfying to the stakeholder and realistic to achieve with the resources available.
Learning professionals then need to convert the typically broad, unquantified expectations into observable, measurable Level 4 results by asking the question, "What will success look like to you?" These Level 4 results become the targets on which collective efforts are focused.
Next, learning professionals should work with the managers of the intended training participants. Together they should identify the critical behaviors of training graduates, which, if performed consistently on the job, will most likely produce the desired results. Only then can the training professional proceed with the traditional identification of learning objectives.
At the same time, the training professional should ask what evidence will be required at each of the four levels to show that the initiative was a success. When the measurement methods, tools, and techniques are defined at the start of the initiative, it makes evaluation much easier to perform.
Business partnership is necessary to bring about positive ROE
Once there is a clear understanding of the result to be accomplished, the next step is for training professionals to work with business managers and supervisors to create a tactical execution plan. This is necessary because training events alone typically result in only 15 percent transfer of learning to on-the-job behavior, according to
Historically, learning professionals have focused most of their efforts on training events (Levels 1 and 2); however, the largest ROE occurs at Level 3, on the job. For maximum ROE, there needs to be agreement about the intensity of effort required from all departments before, during, and particularly after training. Specifically, learning professionals need to partner with supervisors or frontline leaders
- at checkpoints during training design
- before training to prepare participants
- after the training to support and monitor trainee performance on the job.
When participants return to the job after training, required drivers - processes and systems that reinforce, monitor, encourage, or reward performance of critical behaviors on the job - must be in place to provide both support and accountability for the consistent performance of critical behaviors. The degree to which drivers occur directly relates to the extent to which critical behaviors are performed. Performance of critical behaviors is what yields business-level results. Results determine the ROE from the initiative.
Value must be created before it can be demonstrated
At this point in the process, desired results are clearly defined, and a cross-functional execution plan has been created. What remains is the implementation of the plan.
Organizations spend more than 72 percent of their training evaluation resources measuring Levels 1 and 2 (ASTD, "The Value of Evaluation" report). This shows how much money training professionals are spending on employee training. It leaves just 28 percent of the training evaluation budget for demonstrating how the training supports on-the-job behavior and accomplishment of key business results (Levels 3 and 4).
Seventy percent of the time, when participants fail to consistently perform desired behaviors on the job, it is due to factors in the work environment (2006 ASTD State of the Industry report).
It is only 10 percent of the time that poor performance is due to something that did or did not happen during formal training.
Very little time and attention is being given to the critical on-the-job application environment for new knowledge and skills. To create maximum ROE, it is essential that learning professionals do everything possible to partner with supervisors to support on-the-job behaviors and executing the required drivers. To accommodate this, fewer resources should be spent on Levels 1 and 2 to allow more training involvement in Level 3.
While monitoring and encouraging on-the-job behavior, training professionals should create an "early warning detection system" in which preliminary measurements of identified metrics are taken. Areas that are not hitting targets should be identified so that adjustments to the plan can be made. This keeps major long-term initiatives on track and creates the highest likelihood that the desired results will be achieved.
A good way to communicate the status of critical behaviors, required drivers, and leading indicators (short-term observations and measurements that suggest that critical behaviors are on track to create a positive impact on desired results) is with a dashboard. A dashboard is a visual depiction of goals set for each of these items and the current status of each. Dashboards are a highly effective driver and one that executives favor.
A compelling chain of evidence demonstrates your bottom line value
Training is on trial, accused of failing to deliver sufficient business value to the bottom line. There is an urgent need for training professionals to demonstrate the value they do bring to the business by presenting a chain of evidence at the end of important initiatives.
This process is easy and fun when the Kirkpatrick Foundational Principles are followed because the evidence to be collected is clearly defined at the beginning of the project by the stakeholders, and data is collected all along the way. It can be very rewarding to assemble the story of how working together in business partnership has created the desired ROE.
The chain of evidence includes both numeric data and the testimonials and stories of those who participated in the process. This combination of evidence shapes a story that appeals to and is understood by everyone. Training professionals should prepare a compelling presentation that will be delivered in person if possible, to share this data with stakeholders and the entire project team.
Telling this story will earn training professionals a "not guilty" verdict for the accusation of failing to deliver adequate value for their efforts. Since the plan was created and executed in partnership, the training professional is then cast in a light of delivering a positive story of success.
"But we were asked to show our ROI"
It is common that training professionals are asked to "show their ROI." Return-on-investment has become shorthand for value much the way "Kleenex" has become the accepted name for facial tissue. Be sure to ask the stakeholders exactly what sort of measurement of value they require. Generally what ensues is a conversation in which the stakeholders say they want to know what they are getting for the time, money, and resources invested in learning and development efforts.
Some well-meaning training professionals don't ask questions and just assume that an ROI study to isolate the impact of training is required, and proceed to spend even more money to hire someone to deliver it. It is important to clarify exactly what ROI is to understand how it differs from ROE. ROI is defined as the ratio of money gained or lost (whether realized or unrealized) on an investment, relative to the amount of money invested:
Net Program Benefits 100 = ROI%
How ROE differs from ROI
Table 2 indicates the key differences between ROE and ROI. Brinkerhoff comments on the use of ROI in measuring the value of training as follows:
"To isolate the impact of training is methodologically impossible and strategically counterproductive. If we're trying to build partnerships with managers of trainees, we can't use an evaluation plan that isolates the impact of training alone and ignores the critical management support after the training event. It's like saying I've got a new laptop; now let's see if we can isolate the value of the B key."
Get started on building ROE in your organization today:
- Identify the key stakeholders that judge the "success" of training.
- Get to know your stakeholders and identify their priorities.
- Create a cross-functional advisory committee with representatives from business and training.
- Select a strategic initiative, and use the ROE process.
The reality is that companies are more likely to let a "contractor" go before a member of their own team. Earn your place on the team by being a cooperative member who works collaboratively towards the overall success of important initiatives. Then watch the confidence in your work grow and requests to "prove the value" decrease. T+D