Maintaining Momentum: Keeping ROI And Results-Based Training Efforts On Track

Keeping a results-based evaluation and ROI process on track is one of biggest implementation challenges for evaluation professionals. In many ways, it's equivalent to training for a marathon. You start small with manageable goals, stretch your muscles, change your habits, celebrate milestone events, and then wait for the training and preparation to kick in so you can go the distance. Yet the similarity ends there, since for us "going the distance" represents just half of the journey. Staying on track and maintaining the health and integrity of the ROI process in the face of continuous change is equally critical and an aspect of results-based evaluation work that is often overlooked and underestimated.

Stages of implementation

To best ensure that your evaluation solutions stand the test of time, critical stages in ROI process implementation must be addressed and anticipated.

Stage one: Recognition. An accountability "wake-up call" is generated and preliminary evaluation actions are taken. This stage is usually initiated by a single person or small group, and at this point ROI is not necessarily viewed as a strategic imperative by senior management. In this stage, you're likely to find

• increased demand that the training and workplace learning and performance improvement function show bottom-line results
• internal complaints or concerns about the value of training and workplace learning services
• training and workplace learning expenditures being identified as significant
• workplace learning programs have a history of more than one disaster
• increased emphasis on the linkage of training and workplace learning and performance improvement to strategic directions.

Stage two: Reservation. In this stage, there may be renewed objections about the impact of new evaluation processes on time and resources, and business demands will escalate competition for like resources. The initial support and goodwill from preliminary successes with the process have waned, and there's a prevailing sense of impatience and concern that the investment towards a results-based WLP focus is not worth the effort. It's not uncommon for the organization to either abandon evaluation efforts or significantly decrease its original commitment of resource support. Indicators that your organization is in this stage include

• reduced funding for the ROI process
• fewer mentions of the programs and processes in various company communications
• a lack of participation and involvement by the management team
• constant shuffling of people involved in the process
• fewer requests for products and services around the accountability process
• postponed or canceled review sessions aimed at keeping results-based processes on track
• complaints about the time or cost of evaluation activities.

Stage three: Renewal. Now an organization starts to move past its inhibitors and explores how to renew its initial commitment to invest in expanded evaluation solutions. The need for integrated evaluation processes and solutions is getting increased visibility and attention from internal staff, including senior management. Indicators that your project has entered this stage typically include:

• renewed mechanisms for monitoring and addressing evaluation mindsets, behaviors, and practices
• increased linkage between needs assessment and evaluation practices
• increased energy and potential chaos around identifying where further support is needed and how to get it
• some conflict as multiple solutions are explored.

Stage four: Integration. In this stage, enabling strategies and infrastructures are clearly in place to ensure that the ROI process is firmly integrated into the "DNA" of training and development work. This may range from technology-based support processes to standardized evaluation guidelines, policies, and procedures. Here, the ROI process is universally and mutually understood by all who are involved and the following indicators are evident:

• a spirit of continuous improvement, innovation, and out-of-the-box thinking around performance improvement and evaluation
• increased respect for the training and WLP improvement functions as key players in organizational change initiatives
• a growing training and workplace learning and performance improvement budget
• strengthened collaborations between workplace learning and other key functional areas
• workplace learning staff being viewed as internal experts in organizational performance evaluation
• evaluation mentoring and support of increased internal capabilities
• identified and shared best practices.

Enabling strategies

It's important to note that these implementation stages rarely occur in a linear fashion. However, the concerns and indicators of each phase are typical, in varying degrees, as organizations begin to implement results-based evaluation processes as standard practice. By understanding these stages, an evaluation professional can identify critical leverage points and assume roles that will accelerate organizational movement from one phase to the next. The key is to keep moving toward integration and avoid prolonged inertia. Specific actions that facilitate movement are called enabling strategies.

Conduct case studies. Applied, targeted case studies foster internal capabilities in results-based evaluation and build acceptance for the process. They also show

• which programs have achieved success
• what organizational enablers or barriers occurred in the process of achieving desired results
• what new program expenditures are justified.

Initiate annual reviews. This review process is typically for the senior leadership team and is designed to track targeted improvements from the ROI process, including where significant deviations from expected results have occurred; determine what evaluation priorities and potential payoffs exist for the near future; and determine what is needed to keep evaluation targets in place for the coming year. These reviews can also be used to reinforce the role of management in building and sustaining a results-based culture.

Review staff roles. If workplace learning staff lose their enthusiasm for the process or fail to complete the tasks associated with their roles, others can perceive this as a lack of commitment. This view can be contagious and cause other internal groups to lose support and commitment, as well. WLP staff must make sure that evaluation policies are implemented, practices adhered to, and data delivered in a timely manner. Periodic reviews of staff roles and how they translate into day-to-day job descriptions are essential to ensure that all members of the group understand their responsibilities in making the ROI process work as a system-wide approach. Also be sure that staff job descriptions emphasize evaluation skill sets as a core competency.

Implement continuous improvement mechanisms. Implementing the ROI process and monitoring its progress within the realm of a single impact study may be a relatively simple task. But as the process becomes more visible and integrated in the organization, continuous improvement mechanisms become more critical in order for the process to remain credible and flexible over time. Use a scorecard or dashboard approach to tie evaluation results to organizational performance measures. Publish success stories. Establish routine meetings where lessons learned and solution implementations are reviewed and tracked. Develop capabilities by integrating results-based training into core business processes and developing communities of practice for those charged with implementing and supporting evaluation efforts.

Identify and share best practices. With increased prevalence and application of evaluation methodology, much of the focus has now turned to best practices for ROI implementation. Best practices include positioning ROI as a process improvement tool and not a performance evaluation tool for the WLP staff. In addition, it is important to pace yourself and conduct ROI impact studies on a selective basis--usually involving 5 to 10 percent of all programs and solutions

Putting it all together

Ultimately, the task of sustaining results-based evaluation processes over time is not the sole responsibility of the training function. A key challenge is to secure and foster ongoing ROI support, cooperation, interaction, and dedication from individuals and groups across all organizational levels. Focus on the following to assist with these challenges.

• Remember that organizations go through predictable growth stages in the move from episodic implementation to long-term integration of the ROI process.
• Look for specific indicators that the ROI process may be off track.
• Continually renew and refresh commitment to the process across individual, process, and organizational levels so that it remains consistent, reliable, and credible in the eyes of stakeholders.
• Develop best practices for capturing organizational responses and lessons learned during your accountability journey.
• Continually seek best practice examples from professional associations, training literature, journals, case studies, colleagues, and the ASTD ROI Network.

Finally, one of the best pieces of advice for any WLP professional seeking to standardize the ROI process as a way of doing business is this: Commit to the long haul!

There are no short cuts to building a results-based measurement culture. It takes time, effort, and a dedicated desire to make a difference. Keep in mind, however, that the payoff from that commitment will help transform the strategic role of workplace learning in your organization; improve efficiency in training design, development, and delivery; and increase respect, support, and commitment from the senior executives and major program sponsors who seek and approve resources for value-added performance solutions.

2007 ASTD, Alexandria, VA. All rights reserved.