Designing An Effective Training Evaluation Process

Introduction

In today’s environment of increased accountability, the training evaluation process is a critical component of an organization’s training program. Organizations administering the program not only are accountable for what employees learn, they also are accountable for ensuring that employees transfer their knowledge to their work performance. While traditional training evaluation methods focus on using the assessment process to improve training delivery, information should also be collected to determine whether training is assisting the organization to improve its business performance.

Training Evaluation Approach

Evaluation methods should be determined based on the goals of the training process and should meet the demands of the various stakeholders involved. Every organization has multiple stakeholders and not everyone within the organization has the same information needs. Typically, organizational stakeholder groups include the training department, employees and business units. Their information requirements fall into two categories: whether the competencies have been learned and whether the learning has been applied toward improved performance.

Assessment of Competency Learning

The assessment of competency learning, whether the training material has been learned, is best conducted at the individual level. It is at the individual level that organizations are able to assess employees’ reactions and increased understanding. Reactions are important because, if students react negatively to a course, they are less likely to transfer what they learned to their work. Increased understanding in terms of new or improved knowledge, skills and abilities is the primary aim of a training event.

The training department requires information about competency learning to determine the effectiveness of training delivery and approach. Data concerning the overall effectiveness of training procedures, appropriateness of media and instruction methods and other issues relating to possible revisions in instructional design may prove to be very valuable to the training department and will assist them in better serving the needs of the other stakeholders involved. This information can be obtained through:

• Questionnaires – Questionnaires offer a structured tool that may provide both quantitative and qualitative information about employee reactions to the training event. The questionnaire should focus on both training content and delivery. The training content section should target questions to ascertain whether the training materials provide useful information that will assist in performing work tasks and whether the employee is more knowledgeable about the subject matter following the training event. The training delivery section should determine whether the information was presented in a logical order, at the appropriate level of detail and in an appropriate format.

This information will assist the training department in determining how the training material should be revised or supplemented and whether the training medium used is best suited to the training content. It will also allow the training department to maintain an internal inventory of training vendors or offerings that have been rated best or least effective.

• Knowledge Review – Knowledge reviews offer an objective means of determining whether training content has been learned. Knowledge reviews refer to a general group of assessment tools in which employees read questions and respond in writing. The knowledge reviews may be administered by delivering the tool at the start and end of the training event. The tool should be developed as a short answer or multiple-choice instrument using the same questions in each administration, presented in a different order. The questions on the knowledge review should adequately address the learning objectives of the training. The results of each administration of the knowledge review can be compared to measure knowledge transfer during the training event. This information will be useful in assisting the training department to improve the course material.

• Observation – Observation is another evaluation method that provides information regarding employee reactions to the training. Training department personnel should observe employee interaction, level of engagement with training instructors and responses to course content. This evaluation technique may be informal or highly structured. Informal observation can be used to provide general information about the training structure. Alternatively, highly structured observation focuses on monitoring particular points in the training event and commonly involves the utilization of a checklist of the points to be observed.

Employees also require information about competency learning. However, the information provided to employees should be related to their ability to do their job better. These stakeholders will benefit by understanding their own strengths and weaknesses and how they have been addressed through the training process. This information can be obtained through:

• Employee Portfolio – The employee portfolio is an effective means of providing information to employees as well as their managers about the level of mastery of particular knowledge, skills or abilities following a training event. The employee portfolio should be designed to be used for informational purposes only. The mresults of the portfolio are not intended to be linked to employee performance evaluations.

To implement this concept, employees should be required to identify the learning objectives of the training and within a 60-day time frame develop a work sample for review by the manager that demonstrates the application of their learning. The business unit manager should be required to ensure that within that time frame employees are assigned activities that will allow them to produce the portfolio. After receiving the portfolio, the manager should review it and provide feedback to the employee on strengths and areas for improvement.

• Skill Gap Analysis – Another means of providing employees information about their skill development is to require business unit managers to conduct an assessment of each employee’s level of knowledge, skills and abilities relative to the level required for successful performance in the position. The Skill Gap Analysis can be administered using a survey instrument. A skill gap occurs when an employee is rated with a lower level of skill than the position requires. This assessment should be used for informational purposes only and should not be linked to the employee evaluation process. Employees can use the results of this assessment to track their skill development over time from their manager’s perspective.

Assessment of Competency Application

The assessment of competency application, whether the competencies have been applied to improve performance and justify the investment, is best conducted at the organizational level, where performance changes can be assessed and related to training costs incurred. If, as a result of training, employees are using appropriate behaviors on the job, this should have a positive impact on their performance. However, the level of performance improvement should justify the expenditure.

For the business units, the training evaluation process can provide opportunities to determine how competencies are being applied on the job. The information will assist them in making sound business decisions and determining training priorities. This information can be obtained through:

• Analysis of Organizational Performance Measures – Organizational performance measures identify the metrics against which successful business unit operations can be evaluated. These measures will enable performance tracking to identify trends and areas requiring further attention and/or interventions.

As a training evaluation method, the changes in organizational performance measures should be compared to the employee Skill Gap Analysis. This comparison will identify the relationships between skill development and business performance. Information derived from this analysis will allow business unit managers to identify performance strengths and areas for improvement and make decisions about where additional training may be required. The use of organizational performance measures to assess training allows business units managers to be results-oriented and business-focused in their prioritization of training requirements.

• Analysis of Training Return on Investment (ROI) – ROI is a measure of the monetary benefits obtained by an organization over a specified time period in return for a given investment in a training program. ROI can be used both to justify a planned investment and to evaluate the extent to which the desired return was achieved. ROI is calculated by making estimates or obtaining measurements of the costs and benefits associated with a training program. Using this information, the business units can effectively assign and apply resources to improve performance and ensure organizational success.

Summary

The training evaluation process has the potential to provide useful information to multiple stakeholder groups. By designing an effective training evaluation process, an organization can obtain the information needed to improve both training program delivery and business performance, creating opportunities for continuous organizational improvement.