What do Mark Twain, Ernest Hemingway, and return-on-investment (ROI) have in common? All have been reported dead, prematurely. Twain was quite resilient, and reporting was so inaccurate in the 19th century that he was reported deceased twice! Almost a century later, reporting was no better: Hemingway was widely reported dead after a single plane crash in Africa, when in fact he was involved in two separate plane accidents and survived both.
Today, a general misunderstanding about measurement, rather than poor reporting, has fueled a controversy over the vitality of ROI as a viable metric in the learning and development industry.
Why the controversy?
Recent controversy has grown about the use of ROI to measure learning's impact. Detractors claim that it is too quantitative, too difficult to implement, and not relevant to some stakeholders. Jack J. Phillips, chairman of the ROI institute, indicates that recent controversy stems from three sources: evaluation consultants, end users of ROI results, and suppliers and vendors who work with learning and development professionals.
Some evaluation consultants disparage the metric because they misunderstand it, are incapable of applying the ROI methodology, or simply choose not to pursue it. End users of ROI results are skeptical because they do not fully understand how to isolate the impact of training. Finally, suppliers and vendors feel threatened by the measurement of their performance.
Back to the basics
It is worth briefly examining how ROI is defined and calculated. Essentially, ROI is a cost-effectiveness measure and is computed by monetizing the benefits (for example, converting business results such as increased sales into
revenue) and calculating the complete set of training costs. Once these
figures are obtained, the computation
is simple math:
ROI = ((Benefits - Costs)/Costs)
ROI is often viewed as the ultimate measure of effectiveness, because it sits at the apex of Phillips's ROI methodology. But the "higher is better" notion is not as compelling as several other reasons. First, ROI is a business metric that the C-suite understands. Second, it is useful because it provides a view of both effectiveness and cost, all in one metric. Lastly, learning and development professionals can use ROI for several valuable training purposes.
Using ROI
Traditionally, ROI is used to demonstrate a program's cost effectiveness after training has been delivered. However, there are three other points during the standard ADDIE course development process when ROI is
valuable. ROI should be used to
Forecast. Forecasts are best estimates - predictions of the future - but so is the budgeting process, which is widely accepted within businesses. With time and practice, learning and development professionals get better at estimating inputs and outcomes, and forecasted ROI values become more realistic and closer to the actual ROI values computed after training.
Plan. Many learning and development managers use a cost-benefit process when designing their programs. And in most cases, the decision to implement training in one form or another is driven by costs. Program managers answer the question, "What can we achieve within our budget?" However, most do not combine costs with a fully detailed estimate of the benefits. The advantage of ROI at this point is that the benefits could heavily outweigh the costs and even justify asking for an increased budget to achieve desired outcomes.
Demonstrate. Once training has occurred, it is the learning and development team's responsibility to gather the most accurate, valid, and reliable information available. They can start by asking learners to estimate the impact of training and how much it has improved their performance, if at all. By estimating training's impact, the final business results can be adjusted downward conservatively to reflect only the influence of training. If possible, the team should gather enterprise data such as actual sales figures or actual revenue so the impact is tangible and meaningful to business leaders.
Make decisions. By comparing ROI values for each program, leaders can see the relative impact of each course. But focusing on the numbers alone is myopic. Just because a program has the highest ROI does not mean it should be funded again. If the program demonstrates a positive return, and there is an ongoing business need, then it is a good candidate for funding. Discussion among leaders about strategic goals is more important than the actual ROI figures.
ROI is still alive and relevant
ROI has many more vital years to live. It is an excellent measure of cost effectiveness, and it resonates well with business leaders. The C-suite cares about investments and outcomes, not knowledge gain or satisfaction scores. ROI provides senior leaders with information about where their investments will produce the most benefit for the business.
Regardless of the audience - whether it is a CFO, a CLO, or a training manager - ROI is a potentially valuable measure. The only way to know whether it is truly valuable is to ask stakeholders, "What is your metric of interest?" Learning and development managers may want information about course quality and suggestions for improvement. CLOs and CEOs may want a view of training outcomes to determine if business objectives are being met. A CFO may want a valid and reliable measure of ROI. Depending on the audience, the desire to measure ROI will wax and wane.
After Hemingway was declared dead, he continued to do the things that made him a renowned novelist - fish, hunt, and write. He even chronicled both "fatal" crashes in an article published by Look magazine. And Twain? Well, he continued to build his reputation as America's greatest storyteller, traveling around the globe as a headline orator.
What about ROI? Thousands of workplace learning and performance practitioners receive ROI training each year, and corporations still use it to measure the cost effectiveness of training. Will it ever fade from use or die? Probably not, but who knows? Until then, may the arguments against it rest in peace.