At a recent learning conference, managers of two insurance
companies and a technology company shared 10 ways that
organizations measure impact of training.
We'd like to report each of the companies has a strategic,
scientific way of doing so, but that would be a stretch. What we
did find is that in one way or another, our three companies deploy
informal and formal ways of measuring learning retention or
checking for performance mastery.
Let's face it, measurement is a puzzle that we're all looking to
solve. Here is a sampling of some of the ways the organizations are
approaching measurement of learning impact:
1. Learning progress report
The report tool provides learners and their managers with an
authentic performance assessment or summary of progress - like a
"report card" in electronic format. The tool:
- enables gathering, storing, and communicating of instructor
ratings of learner achievement, and individual recommendations for
additional learning activities
- ensures field line managers are engaged in the learning process
and helps with transfer of learning to the field
- posts completed reports in a learner's LMS history with access
only available to the learner and immediate manager, who receive an
email with a direct link to the report.
Information obtained from this tool is for employee knowledge and
skill development only and is not legally defensible to base
employee career actions on the results. A different approach might
be to use your LMS if there is a "learning plans" or similar
feature to track progress.
2. Practice and skill checks
Practice and skill checks are used at varying levels and in
different ways. Integrating practice and skills checks:
- allows learners to practice the real task they will do on the
job and checks for skill development.
- demonstrates learners can DO the task, rather than hold
knowledge.
- provides criteria for the practice partner/skill checker to
help them evaluate how well the learner performed the task and
provides a way for coaches/supervisors to give great feedback!
Ask yourself if knowing about something is enoughor are you really
after performance?
3. Final test - mastery
Two of the companies are working toward implementing
performance-based training as the standard for courseware. To make
the transition, evaluating the performer's mastery is the key.
Performance-based training:
- continues the practice and skill checks, but checks for mastery
- provides a final integrated skill check that measures final
results of the learning. For instance, a sales person models a
sales conversation...from the greeting, to the conversation, to
systems used - and finally closing the sale and the referral - ties
it all together to make sure the learner can demonstrate an
end-to-end process
- provides a skill check criteria list to help your skill
checkers gain very specific feedback. By the time the learner gets
to the final, they should be ready to roll.
4. Transfer of learning techniques
Informal techniques for transfer of learning with the core
curricula courses can be used in a variety of ways resulting in
different outcomes.
- Supervisor and employee pre- and post-training discussions -
Most courses include a discussion guide for these two parties to
discuss expectations of what the learner will learn before the
training and how they will use lessons learned after the training.
- Accountability cards - Can be pre-printed or 3 x 5 index cards
to be used for the learner to list specific behaviors they pledge
to implement after training. Cards can be used by an individual,
with a partner, direct supervisor, or team member.
- Accountability partners - Used when trying to change specific
behavior that requires coaching and observation. Partners can be
sounding boards as they work through performance improvement.
5. Balanced scorecard
Scorecards are an intentional way to measure the impact of
training. This formal methodology requires commitment to analysis
and tracking. Outcomes include
- results matched to vision and strategy of company or department
- objectives tracked to action plans and steps
- measurements as detailed or as general as desired.
6. Competency development impact assessment
The adoption of enterprise competencies and the outline of expected
behavior is one way to apply them to learning, talent, and
performance management. Using assessment tools focusing on
measurement of business impact, transfer of learning to the
workforce, and enhanced productivity will allow your organization
to:
- tailor and electronically distribute questionnaires at
pre-determined times to ask participants and their managers where
they can demonstrate key learning initiatives to make a significant
difference in improving performance and results
- gain reporting capability, which reduces the show-stopper time
and resources previously required to gather this important data. It
is integrated with LMS for ease of use, ease of distribution, data
collection and reporting.
7. Adoption reporting
This reporting shows whether the solution reached the intended
learners and measures learning activity (for example, completions,
page or site visits, and learner contributions). It includes:
- compliance and general metrics - sales and production
- percentage of employees using learning methodology, service,
product, and task
- tying activity and measurements of training to enterprise
vision and strategies.
Adoption goals and measurements are important to gauge change
management of an initiative, vision, strategy or project (for
example, use of a new process)
8. 'Listening' software to measure informal
learning
A new way to measure the impact of informal learning is by using
marketing software that"listens" to the use of social media. The
investigation looks to see if the organization should:
- troll internal social learning media - learning blogs, wikis,
and other tools to analyze for trends and comments
- attempt to measure learning by factors such as strength,
sentiment, passion, and reach.
- try to discover what is the 'emotion' and use that information
to come up with learning program solution decisions.
9. Needs analysis
All three companies conduct needs analyses with a methodology to
determine what currently exists in the workforce and what skills
and behaviors are needed to achieve business goals. This analysis:
- includes asking business stakeholders what the key business
needs are and what learners need to learn and do to have an impact
on business results
- involves assessing learners to determine their current state
performance compared to the desired state, or what the business
requires them to do to be successful
- drives the objectives, content, and design of the learning
solutions and the evaluation plan
- proves necessary to determine the learning solutions return on
expectation with stakeholders.
10. What about social media and mobile learning?
We know the world is changing. They way we teach and the way we
learn change daily.
- Should we measure the impact of informal and mobile learning?
The answer is probably "that depends."
- If informal and mobile learning are a big part of your culture,
you may want to look for ways to measure the impact.
- Try using nontraditional methodsbe creative and think "out of
the box."
Changing times call for changing techniques!
------------------------------------------------------------------------------------------------------------------------------------------------------------
Factors to Consider When Measuring the Impact of
Training
It takes time and resources to measure the impact of any training.
There are several questions all three organizations used to
determine what to measure and how to measure it.
- How critical is the initiative to the success of the
organization? The more important it is to the enterprise, the more
important it is to know if the training has made an impact.
- How frequently will the content, system, or process change? If
you're constantly changing the training, then it will be difficult
to establish a base line for measurement and the data may not be
relevant.
- Are there legal concerns about measuring the impact of the
training (compliance issues)?
- Are you measuring confidence, competence or both?
- Do you have the infrastructure to provide the necessary data
for measurement analysis?
- What do you plan to do with the information? Are you REALLY
going to use it?
- Will your organization's culture embrace the idea of any kind
of measurement?
- What accountability is in place to ensure measurement and
transfer of learning takes place?
------------------------------------------------------------------------------------------------------------------------------------------------------------
Ron Cowan is manager of Learning Strategy
Execution at XEROX CORPORATE HR - Global Learning Services.
Carolyn Hansen is manager of learning development
& services, corporate learning & development for COUNTRY
Financial.