What do we mean by evaluation success? Chances are the definitions will change over time, and from situation to situation. There are some common definitions out there, however. Success with evaluation does not mean that the training or learning activity being evaluated was itself necessarily a success. One reason you may be evaluating is precisely because the training was not a success! In general there are 5 steps to evaluation success. They are:

1. Develop a goal statement and baseline.

2. Conduct a comprehensive stakeholder analysis.

3. Develop an impact model.

4. Select the right methods.

5. Implement with grace and present the results.

5. Build for the future; follow up.

Step 1: Develop a Goal Statement and Baseline

The starting point for any evaluation is the 'why'. Why intervene in the first place? Knowing this will tell us plenty about what needs to be evaluated.

For each project you tackle, you need to draft a goal statement and a stakeholder analysis. Why have clarity about the measures? Well, if you aren't clear about the measures, specifically what to measure, how the measure is defined, and how to measure, then how sure can you be that your study is actually reflecting reality?

Where it is possible, a baseline measure of performance is taken and included in the goal statement. A baseline is a key element in any evaluation that sets out to compare performance before and after an intervention. It is not always possible to develop a baseline measure and when this happens, it should be noted in any report. This does not prevent an assessment, but it will have less certainty and credibility.

Step 2: Conduct a Comprehensive Stakeholder Analysis

One of the first steps toward training success is to identify the stakeholders and find out their wants and needs. The same is true of evaluation. Who is interested in the evaluation of the training? Why? What do they want to know? What is their agenda?

If you have the answers to the above questions, you can undertake an evaluation study to meet all these requirements, likely to be lengthy and costly, or undertake a limited and targeted evaluation study that addresses the needs of just one or two primary stakeholders. But you still need to manage the major expectations of the other stakeholders!

In finding out what the various stakeholders require, you will be better placed later to select the most appropriate evaluation method. This is critical if you are to ask and answer the right questions, and to do so effectively and efficiently.

The stakeholder analysis also will tell you how the evaluation study should be reported and how often and by which methods you should keep each stakeholder informed of progress. This is often known as a communications plan.

Step 3: Develop an Impact Model

An impact model is simply a description, a forecast, of what performance will look like if the intervention is successful. It does this in more detail and depth than the goal statement in Step 1 because it makes explicit the links between the desired outcome and what actually has to be done.

The impact model provides guidance about what data may need to be collected and also how, surveys, interviews, and so forth. This detail also can help you select the appropriate evaluation approach, which is the subject of Step 4.

Whether the intervention has already taken place, or (more ideally), has yet to take place (and so evaluation can be woven in to it), the impact model is a key part of the success of the intervention and its evaluation. For interventions yet to be developed or delivered, it helps identify the key objectives, in particular the training or learning objectives. For interventions already in place, it helps establish what should have happened against what an evaluation can assess for effect and efficiency.

For the best effects the impact model should make a reasonably deep investigation in to root causes behind the gap. If you aren't clear about this, it is quite possible to measure some very clearly defined measures as described in Step 1, but that have only a passing influence on the indications of the problem you clarified at the start.

Step 4: Select the Right Evaluation Methods

Different stakeholders may have different requirements. This will have an impact on the evaluation methods you use. And there's no reason why you can't use several methods together. For example, one stakeholder may be the CFO who requires a return on investment. Other stakeholders will certainly be the trainee and the trainee's line manager. They are likely to be more interested in whether the training is helping the trainee to perform better and be better placed for career progression.

Step 5: Implement With Grace and Present the Results

Once you select the appropriate methods, you have to implement them. On many occasions where evaluation studies have been less than 100 percent effective, the methods employed have either not been the right ones or they have been poorly implemented, especially where other people are involved with data collection.

All methods require some data collection, and require some focus on individual people. Forgetting the human factor can result in approaches that are seen as annoying if not antagonistic.

There's little point in doing an evaluation if you aren't going to present your findings, but what is the best way to do this? It depends on the audience, the stakeholders identified in Step 2. They will help you decide what to present and in what format.

You can present the evaluation findings in any of the following ways:

  • detailed reports
  • executive summary
  • general audience
  • single page reports
  • scorecards.

And of course you can always deliver a stand-up slide presentation and provide opportunities for questions to be taken and so build greater understanding. In such cases, don't go overboard on the amount of detail; keep it simple.

Step 5: Building for the Future

You may wonder why this isn't step 6. This last step is not critical to the success of one specific evaluation study, but it is useful to take if you are looking to conduct future training and evaluation with this client or part of the organization.

Over time, more data will come to light about the longer term success of any training you have evaluated. From this you can learn to improve the evaluation approaches in the future as well as help improve the impact of the training itself.

The trick is to develop and maintain a relationship with the stakeholders once the initial evaluation study is completed. So, if you can, stay in touch with the stakeholders. Use it to gather information and to share the experiences of your other evaluation studies to continue to add value to all your clients/stakeholders.

Follow the 5 steps (yes, even the last step!), to ensure training contributes to better performance, and you build your reputation.