Isolating the effects of a coaching program on business impact data is one of the most challenging, yet necessary, steps in the ROI Methodology. When addressed credibly, this step links the coaching process directly to business impact.
Other factors are always present. In almost every situation, multiple factors affect business results. During the time that coaching programs are implemented, many other functions within the organization may be attempting to improve the same metrics addressed by coaching program. For example, marketing and sales programs are designed to affect revenue, and, while coaching may help a marketing or sales employee positively affect revenue, the marketing or sales program itself also may do that. In addition to internal factors, external factors may influence business results during the time that coaching is occurring.
Without isolation, there is only evidence, not proof. Without taking steps to show the contribution of coaching, there is no clear business linkage. Instead, there is only evidence that coaching may have made a difference. When business results improve during coaching, it is possible that other factors may have contributed to that improvement. The proof comes from isolating the effects of the coaching initiative.
Other factors have protective owners. The owners of the other functions that influence business results are convinced that their processes or programs make the difference. For example, marketing and sales managers probably firmly believe that any increase in sales is entirely due to their efforts, and they often present a compelling case to management to support their claim. Likewise, the information services department likely believes that their technology changes make the primary difference in efficiency gains within an organization. They, like marketing, often present a compelling case to management. Other processes or programs, such as performance improvement, reward systems, and job redesign, all have protective owners, and their arguments are plausible. Therefore, owners of coaching programs are under pressure to build a credible argument for their case to claim value added to the organization.
To do it right is not easy. The challenge of isolating the effects of a coaching program is critical and can be done, but it is not easy for complex programs, especially when strong-willed owners of other processes are involved. It takes determination to address this situation every time a return-on-investment study is conducted.
Common myths about isolating the effects of coaching
Several myths about isolating the effects of a coaching program often create concerns, confusion, and frustration with the process. Some professionals further complicate this matter by suggesting that isolation is not necessary. Here are the most common myths related to isolation:
Programs such as coaching are complementary to other processes. Therefore, you should not attempt to isolate the effects of coaching. Coaching programs are complementary to other factors, all of which drive results. If a project sponsor needs to understand the relative contribution of a coaching initiative, this issue must be addressed. If accomplished properly, it will show how all the complementary factors are working together to drive the improvements.
Other functions in the organization do not isolate the effects of their programs and projects. While some functions do not grapple with this issue because they try to make a convincing case that the improvement is related to their own processes, others are addressing the issue. A credible approach to address this issue is necessary. Notice the next time you complete a customer satisfaction survey after you make a purchase or open a new account. Do they ask you why you made the purchase? If yes, they are trying to isolate the results of multiple variables.
If you cannot use a research-based control group, then you should not attempt this step. Although a well-constructed control group is the most credible approach, it will not apply in the vast majority of coaching situations. Consequently, other methods must be used to isolate effects. The problem does not go away just because it is not possible to use the most desired or favorite isolation technique. The challenge is to find other isolation techniques that are effective and that will work, even if they are not as credible as the control group method.
The stakeholders will understand the linkage, so you do not need to attempt to isolate the effects of coaching on business impact measures. Unfortunately, stakeholders see and understand what is presented to them. Absence of information makes it difficult for them to understand the linkage, particularly when others are claiming full credit for their improvement.
Estimates of improvement provide no value. The worst-case scenario is to tackle this issue with the use of estimates from those individuals who understand the process the most. Although this is a last choice, it may provide value and be a credible process, particularly when the estimates are adjusted for the error of the estimate. Estimates are used routinely in a variety of functions throughout organizations.
Ignore the issue. Maybe they won’t think about it. Stakeholders are becoming more sophisticated on this issue, and they are aware of multiple influences. If no attempt is made to isolate the effects of coaching, stakeholders will assume that other factors have had a tremendous effect, and maybe the entire effect. Thus, credibility of coaching results deteriorates.
When funding is provided to different functions in the organization—with different process owners—there is always a struggle to show, and sometimes even to understand, the connection between what they do and the results. If you do not tackle this issue, others will—leaving the coaching program with less than desired budgets, resources, and respect.
Note: This article is excerpted from Measuring the Success of Coaching by Jack J. Phillips, Lisa Ann Edwards, and Patricia Pulliam Phillips.
© 2012 ASTD, Alexandria, VA. All rights reserved.