To Paul Leone, talent acquisition and development at American
Express, Level 6 specifically identifies the factors in the
participant's work environment that predict a program's impact,
while looking at what might inhibit or help the transfer of
learning. Learning Executives Briefing spoke with Leone.
LXB: Can you explain the impetus of your work? Was
it driven by a company mandate?
Leone: We have, as with every large organization,
a challenge around putting a return-on-investment on training
programs. Regarding the blended learning approach, one of the
primary questions is where do we get the most value? As a global
organization, we do a lot of training. We began to (have) questions
a few years ago about instructor-led training programs, which are
usually a bit more expensive. First, were these instructor-led
programs practical? And, second, what was our return-on-investment
for different delivery modes?
We launched our flagship program, called Situational Leadership II,
which was mandated for about 10,000 to 20,000 leaders. We looked at
our different delivery approaches - web-based, instructor-led, or
blended - and one of my tasks was not only putting an ROI on
training in general, but also looking at how the different ROIs
compared across delivery modes.
LXB: Is there anything different about your
approach other than how you measure it on the back end?
Leone: The one thing that is different about our
approach is that we included the (newly developed) Kirkpatrick
Level 6 step. Levels 1 through 5 have been written about
extensively. Those measure everything from satisfaction with the
program all the way up to ROI. The big difference in this research
is that we measured what we call the "transfer climate."
We had suspected that the delivery method that employees had
(received their learning) - blended, instructor-led, or solely
web-based - is not the most important part of learning; it's the
environment that the employees are taking that training back to
that matters. That was a huge hypothesis we wanted to test. What we
found was that this transfer climate - where the employees return
with their training was important. What kind of leader do they
have? What kind of environment do they have that provides them an
opportunity to practice this training? All of these transfer
climate elements are what predict whether a training program will
have impact or not. And that is exactly what we showed in our
LXB: Had you been bouncing around this concept in
your head before putting this study into place? Or did some unique
event or opportunity at American Express trigger it?
Leone: There is some research around transfer
climate, but it has been at the conceptual level. Many learning
professionals and theorists came together and said it would
probably make sense to have a leader who supports the training. And
it would probably make sense to have continuous follow-up with
participants. That was all conceptual. There has never been a Level
6 transfer climate measured at a quantitative level. We know it
matters, but how much does it matter? How much can it predict the
impact of the program?
Beyond some of the past learning research and drawing on a major
need within one of our lines of business, (we asked) why is this
working in some areas of our organization and not working in other
areas? In some cases we had clients - participants - going through
a (learning) program who reported no impact, which means they were
getting nothing out of training. Some were coming back reporting
incredible impact. From these initial results, we added the Level 6
to determine what is in these different groups that is predicting
the impact. It was diving deeper and asking: "What is Group A doing
differently than Group B?"
LXB: Do you think your research is more credible
because you have a bigger pool? This might not be so easy to study
in a smaller organization.
Leone: Absolutely. One of the biggest strengths of
this research is the size. I have read results of research with 30
or 50 participants. Our research was conducted with more than 3,000
learners. As you report with sample sizes of 3,000-plus, you really
get people's attention.
LXB: What hasn't e-learning completely done away
with the instructor-led or blended learning approach?
Leone: The personal connection that can be made
with other learners - the sharing of learning - is something that
can't be substituted. The strictly web-based approach will always
be challenged in terms of value compared to instructor-led and
blended learning. Frankly, (instructor-led) is classically the way
we are taught from elementary school up to the college classroom. I
don't think people are ready to accept just a virtual,
self-directed, make-your-own-deadline type of learning. I don't
think I would ever say, as a result of our research, that a
web-based approach under all the same conditions would be as
effective as an instructor-led approach. However, if you maximize
your efforts around a web-based approach, you will be able to
maximize your investments and see a greater return-on-investment
than you would in an instructor-led approach in a horrible climate.
LXB: Was it difficult to get buy-in from employees
to do the kind of follow-up work and reporting to make this
research and implementation possible?
Leone: Within any company there will be some level
of adversarial relationship with an HR group that wants to
attribute a certain amount of business success to particular
training programs. I don't think any executive wants to admit that
he was completely ineffective before going through a training
program, and now, miraculously, he is a much better leader. As HR
and learning professionals, we get that (response) from many
customers and clients as well. The challenge for us, in this case,
was to convince them that we can place an ROI figure on learning.
It's a challenge in any organization, but at American Express, we
are ahead of the curve in accepting just how much human capital
matters, and how much return-on-investment you can get by investing
in human capital. While all companies think about the bottom line,
American Express buys into the idea that it is employees, and their
level of engagement and motivation, that will impact the bottom
line a lot more than investing in products and manufacturing.
LXB: Is it more important for a service company
like American Express to have a highly engaged and motivated
workforce than it might be for a manufacturing company?
Leone: Within our leadership model, we have always
known that. The short answer is that as a service company, we know
that our employees who interface with customers can literally make
or break accounts. We know that they can interact with and satisfy
their customers better if they have engaging leaders and leaders
are able to motivate them.
If we lose that and we don't have engaged and motivated employees,
we lose customers. And that affects the bottom line. We have made
the leap. We are going to invest in our leaders. We are going to
assure that our leaders are engaging and motivating, and all-around
good leaders so that it satisfies the employees. And those
employees pass on satisfaction and engagement to our customers.