This is a complete guide on how to evaluate your training sessions.
Chapter 1: Introduction To Evaluating Training
Chapter 2: Getting To Know People
Chapter 3: Your Evaluation Strategy
Chapter 4: Sample Training Evaluation Form
How To Evaluate Training
Introduction
Every year organizations and individuals spend literally billions of dollars on training and education programs. However, experience has consistently shown that little time or expense is paid to evaluate the effectiveness of the training that has taken place.
- Did the training program work?
- Was it worthwhile?
- Did the results justify the investment?
This guide will help you as a trainer to evaluate the effectiveness of your training programs and support you in your discussions with your contracting clients whether they are within or outside your organization.
Feedback
When we look at the evaluation of training we must first understand the role of feedback. Quite simply feedback is the process of giving information to learners, to assist them to learn more about themselves, the subject matter, and the effect that their behavior has on others. Constructive feedback will help learners to progress toward their learning goals. Feedback can be positive or negative and if introduced correctly it can be extremely constructive.
- Positive feedback is the feedback that you provide learners to recognize a task well done.
- Negative feedback on the other hand is provided to highlight some behavior that requires action on or improvement
Guidelines for constructive feedback include:
- Ensure that the feedback is clear and focuses on the behavior and not the personality of the individual.
- Don’t give too many messages as it can confuse the learner.
- Ensure that you create enough space for the feedback or evaluation to take place.
- Don’t be over-negative.
- Start and end the feedback on a positive note – room for improvement.
Discussion Questions
- Why is it important not to be over-negative?
- Why is it important to create enough space for feedback?
- What other factors do you think contribute to effective constructive feedback?
Icebreakers represent an important part of a trainer’s toolkit that allows the trainer to foster communication, enhance interaction and creativity, and energize group formation.
Getting To Know People
Most evaluations of training within organizations fall within three broad categories. These are:
- Context – Evaluations within or just after the learning event.
- Input – Evaluations focused on the resources used to meet the learning event (e.g. content, methods, etc).
- Output – Evaluations of the immediate, intermediate, and ultimate outcomes of the learning event
Training professionals usually distinguish among four levels of training evaluation, which were first published by Donald L. Kirkpatrick (1959).
Level 1 – Reaction
This level measures participants’ perspectives about the training course and is the most common way to evaluate a course providing a measure of customer satisfaction. Advantages
- Low cost
- Easy to employ
- Provides insights into participants’ opinions
- Provides quick feedback regarding successes and failures
Disadvantages
- A quick measure
- Results may not be a reliable basis for changing a training strategy
Level 2 – Learning
This level measures how participants have understood and taken on board the course’s learning objectives. It can include tests of performance before and after the course. Advantages
- This provides more detailed evidence of whether a training program is effective
Disadvantages
- Requires more time and resources
Level 3 – Transfer To The Job
This level measures how the knowledge, skills, and values from a course are used on the job. In most organizations, this is measured 3 to 6 months after training. Advantages
- Provides stronger evidence that a training investment has the desired impact on job behavior
Disadvantages
- Requires significant time, expertise, and resources
Level 4 Organizational Impact
This level measures all types within an organization (performance improvement, quality improvements, and cost savings)
Advantages
- Provides the strongest possible evidence that a training program has the desired impact on an organization
Disadvantages
- Requires substantial time, expertise, and resources
The trainer evaluates training by employing a wide range of information. As a trainer, you should try to collect this information from every aspect of the session. Tools that will help you collect this information include:
- Questionnaires
- Interviews and Group Discussion
- Learning Diaries
- Attendance and Punctuality Data
- Assessment Results
- Why Individuals Are Attending the Course
- Supervisor Reports
- Productivity
- Benchmarks
- Reactions
The trainees’ reactions to training are very important – most of all to the trainees and the trainer. During the course of the training program, these reactions tend to vary and that needs to be recognized by the trainer. The most popular approach to collecting these reactions is questionnaires with rating scales. This can also be supplemented by interviews and informal discussions during the course of the training program and through follow-up contact with the participants. Participant reaction criteria include:
- Content – quality, quantity, relevance, level
- Style – pace, format, accessibility of language, interactivity, flexibility
- Media – variety, clarity, readability, user-friendliness, stimulating
- Personnel – presentation, motivation, fit for purpose, commitment, expertise, professionalism, commitment
- Location – fit for purpose, comfort, access, refreshments, facilities
- Timing – in relation to external and internal events
At the end of this document, we have supplied you with a number of reaction questionnaires that you are free to employ and modify within your training sessions.
Learning
As the primary goal of training is to transfer knowledge, skills, and abilities it is no wonder that this is one of the most important parts which needs to be evaluated. The first place to start is with the learning objectives associated with the course. These also represent the first step in the design and content of a training course. Learning objectives should have three components:
- They must be terminal in nature. To illustrate, they must specify what kind of behavior can be used as evidence that the participant has achieved the objective.
- They must define the behavior in terms of the kind of conditions the behavior will be expected to occur.
- The accepted performance criteria that the participant must perform.
In this sense, evaluation in terms of learning objectives becomes a matter of condition. To illustrate, if the objective was to understand conditions necessary for change then the evidence would be a test to whether this understanding existed. It is important to remember that if the participant does not reach this learning it is the training system that has failed and not the trainee. As a result, it will need modification to enable the participant to achieve the learning objective.
Transfer To The Job
Within work environments, it is always essential that the skills and knowledge acquired are transferable to the job behavior. In this situation, the objective of the training is to transfer or result in a change of job behavior. The measurement of this change can be difficult especially when the training undertaken relates to the development of a number of skills for application within a number of settings. This stated, it is not an impossible task. In such situations, it is important to work closely with the contracting individual (e.g., the manager) to ensure that effective measurements/markers are in place. Supervisor reports are used predominately to assess this level of learning. It is important to highlight that participants must be provided with the opportunity to employ the skills and behaviors developed during the course. In the absence of this opportunity, evaluation results become redundant.
Organizational Impact
By far the most challenging aspect of evaluating training is within the area of organizational impact. However, this is also the most important to the organization. To illustrate, most training initiatives are designed to deliver an increase in production/profitability or a reduction in some cost area. Organizational impact areas can include:
- Quality or Quantity of Production
- Absenteeism
- Turnover
- Operation Costs
- Health & Safety
- Work Attitudes
- Work Methods
It is a challenge to identify with certainty that the impact on the organization was a direct result of the training intervention. However, by working closely with the organization it is possible to attribute areas of impact and assign the ultimate value to the training. The measure of ultimate value to the organization refers to how the organization has benefited from the training in terms of increased profitability, survival, or growth. Return on investment measures can also greatly support any findings through the collection methods highlighted earlier in this training evaluation guide.
What Issues Should Be Evaluated?
The following table will assist you in determining what issues should be identified and assessed during any training evaluation. These items will also help you with the design and implementation of your training programs.
Purpose of Training
Description: The primary reason for developing the training. What performance deficit is to be targeted by the training? What strategic objectives are going to be supported by the training?
Linkage
The link between training and the needs met by them
- Are the training objectives clearly linked to identified needs?
- Are the training objectives clearly linked to identified strategic goals?
Marketing
Description: How the training is promoted.
- Are the participants clearly identified?
- Is the training promoted in terms of their interests?
Goals/Objectives
Description: The goals or outcomes to be met by the training.
- Are the goals of the training program clearly stated?
Outcomes
Description: The desired impact of the training (Individual, On-The-Job, Organizational)
- Are the individual, on-the-job and organizational outcomes identified?
- Who is accountable for the transfer of learning?
Assessment
Description: How the training will be measured.
- How will participant reactions to training be measured?
- How will behavior be measured?
- How will the on-the-job benefits be measured?
- How will the organizational impact be measured?
Setting
Description: Where the training will take place. Where the evaluation will take place.
- What is special about the training location?
- What is special about the evaluation location?
Participants
Description: The individuals to whom the training is directed.
- What assumptions are made about the participants?
- What prerequisite skills are needed?
Training
Description: The subject matter.
- Is the subject matter appropriate to the learning objectives?
- Has it been prepared appropriately?
Provision For Differences
Description: Preparing for differences between participants.
- How will differences between participants be recognized?
- How will these differences be managed within the training location?
Logistics
Description: Planning for the delivery of the training.
- Are all resources required available?
Testing The Training
Description: How training will be measured at the completion of the course.
- Are any tests clearly linked to learning objectives?
Implementation Of Training
Description: How the training is delivered.
- Are the delivery methods appropriate to learning objectives?
Application
Description: How the training will be applied on the job.
- What on-the-job application is desired?
- How will this be measured?
Effects
Description: The results of training.
- What effects are possible?
- What effects are desired?
- What effects are achieved?
Use Of Information
Description: How information on the effects of the training will be used.
- Who will need to be informed about the effects of the training?
- What decisions will this information inform?
Timelines For Evaluation
As you design your evaluation strategy it is important to recognize the timelines for your evaluation. The following will help you deploy your evaluation and training program effectively.
Pre-Training
The reason for the training should be clearly established and communicated to all the participants. This communication should include information on how the training will benefit them personally and how it will benefit the organization.
During Training
It is always important for the trainer to be aware of the reactions of the participants during the training session. This may also require the trainer to be flexible on how to evaluate training courses and accommodating in the style they adopt during the training session. Reaction sheets should be used and unanswered questions should be followed up.
Post-Training
The organization’s management should be supported in providing feedback following the training session. This will ensure that the organization is not only supporting the training but is also conscious of measuring the effectiveness of the training. Individual follow-ups should also be performed to support the participants on the course.
Who Should Be Consulted During A Training Evaluation?
In most organizations, evaluations tend to focus on the participants and their reactions to the training event. However, to evaluate all levels of evaluation outlined earlier in this guide, the following need to be consulted.
- Sponsor (Senior Management)
- Providers (Trainers)
- Participants
- Line Managers/Supervisors
- Remember – the more that are involved, the greater accuracy of and buy-in to the results of the evaluation.
Your Evaluation Strategy
The last aspect to consider in how to evaluate training courses is whether it best suits your organization or the organization where the training evaluation will take place.
Value-For-Money
Top management believes that training should only take place when it can prove its financial return.
- Identify direct and indirect costs of training
- Identify financial benefits to the organization
- Compare the cost and benefits to create a cost-benefit ratio
- Identify whether the use of other solutions or media will be more cost beneficial
Investment-Value Strategy
When the focus is not just on the immediate cost benefit but on the end results.
- Focus on cost-benefits
- Identify end result value
Objectives-Centred Strategy
The training sponsor and trainer work together to set objectives.
- Development of a learning contract that outlines outcomes, measurement, and final evaluation Auditing Strategy Requirement for a comprehensive audit of the training program
- Link to overall training and development plan
- Systematic evaluation of all training events
Business-Led Strategy
Where the organization is going in terms of workforce development and the role of training.
- Identify the direction and organizational goals of the business
- Focus on the role of training and development in the achievement of organizational goals
We have included a short sample evaluation for your consideration and use within your training settings.
Sample Evaluation Form
Name:
Job Title:
Department:
Name of Course:
Date of Course:
Instructors Name:
Did the course meet your objectives? Yes No (please circle your answer)
Please explain your answer:
Coverage of the Topic (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Organization of the Course Material (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Relevance of the Course Material Used (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment Use of Practical Examples (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment Level of Difficulty (please circle your answer)
Too Difficult 5 4 3 2 1 Too Easy
Comment
Length of Course (please circle your answer)
Too Long 5 4 3 2 1 Too Short
Comment
Facilities (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Suitability of Trainer (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment Quality of Handouts (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Quality of Overheads (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Quality of Exercises (please circle your answer)
Excellent 5 4 3 2 1 Poor
Comment
Will You Be Recommending This Course To Your Colleagues?
Yes/No (please circle your answer)
What Areas Of The Course Could Be Improved?
Conclusion
That’s it for our guide on how to evaluate training courses. Let us know if you like this guide by contacting us. And, please don’t forget to share this guide.
