Training Evaluation

Distribution of the Evaluation Forms

Evaluation completes the total quality loop.  The following highlights areas to be evaluated and how they are measured.  Evaluation of training results can cover knowledge, reaction, behavior and functional results. These are not fixed recipes; the correct approach should suit local situations and company environment.

IS EVALUATION NECESSARY?

Without evaluation, there will be no objective way to isolate and measure the impact of training.  If training is an investment like other activities, its returns on business should be measured like any other investments.  However, evaluation is time consuming and has many subjective sides.  But the more evaluation is planned and desired, the more one can reduce guesswork, and the more this makes evaluation worthwhile.  Organizations who want to fully benefit from training should do their best to apply all evaluation forms as much as they can.      

At the level of reaction:  to evaluate employees reaction to the course to know those parts of the course that need alteration.  This type of evaluation is of little value to the trainee or the customer, but it is of value to the trainer.

To measure Reaction: Interviews, impressions, scale-rating of sessions can be made at the end of the training or even after each session.  These are used for steering purposes: for rectifying and deciding what to do next.  However, long-term evaluation of the reaction may be needed to confirm the results of short-term reaction-evaluation, provided that does not mix with behavioral or functional training.  The way to evaluate should be tailor-made to be effective and objective. 

Evaluating the learning level: to obtain information on the amount of learning as a result of training. This evaluation is immediate and is done through examinations.  Immediate evaluation does not indicate ability of trainees to apply knowledge in business.

To Measure Learning:  Learning can be divided into knowledge, skill, and  attitude.  Evaluation of acquisition of knowledge can be by tests and examinations: open questions, multiple choice, oral examinations.  In order to measure improvements, tests can be administered at the beginning  then results are compared with later test results. 

To measure Skill learning are measured through practical tests in which the trainee is given the opportunity to show learned skills.  Again, improvement is determined from the difference between the first test at the beginning and subsequent tests at the end of the training.    

To Measure Attitude Learning: is difficult to measure and appreciate, measure are sometimes complex and subjective, not easy to isolate and evaluate.

To Measure Behavior: The best factor for this purpose is to get trainers actively involves themselves as far as possible in the job situation of their trainees.  This will not be possible for trainers external to company.  Possible ways for measuring behavior are activity sampling, observation of specific incidents, appraisal by superiors, and self-appraisal.  All these methods can be used before training.  Then results compared with evaluations post-training.  Evaluation before training in this case can be used to define training needs and objectives.  In long-term training where trainees return o work at regular time interval, behavior evaluation can be used to “steer” training between intervals.

Evaluation of the change in behavior measures the change in behavior at work of the trainee as a result of training, which is a measure of the effectiveness of the training if it addresses behavioral results.  This type of training is more difficult to measure than the first two types.

Evaluation at the functional level is used to measure the effect of training and changes in behavior on the performance at work, on the productivity or efficiency.  Long-term evaluation or ultimate evaluation is sometimes the name given to this type of evaluation.  It is difficult to know the amount of improvement directly tied to training.

To Measure Impact on Functioning:  Productivity, efficiency, performance, or any index of functioning can be used to evaluate the impact of training on business.  Once the effect of training is isolated, it can be measured in hard figures.

The difficult parameter is to refer improvement in productivity or efficiency to training.  In this case if the objectives of training are very clearly defined this will help evaluation at this level.

The use of control groups can help evaluate the impact of training.  Control groups are employees which does not undergo training and used as a reference.  The performance of the group who followed training can be compared to that of the control group.  However, there are practical difficulties of obtaining and matching control groups in industrial life.

Objective Setting and Evaluation: Planners of training should set objectives in one, some or all the four levels of evaluation.  If these objectives are defined and were specific, it would be possible evaluate at every level.

Evaluation in Training

Evaluation by the Customer (Who pays to receive training)

Evaluation before training focuses on the content of the training courses, the effectiveness of the vendors delivering such training, and the expected contribution of  programs to business performance.

Evaluation during training focuses on the employee’s feedback on what he/she is acquiring at the time of training delivery. The feedback is provided in a questionnaire-form. Before training starts, a copy of an evaluation questionnaire is distributed to employees to obtain their input on pre-determined evaluation indicators rather than depending on a narrative report that may not be focused on useful feedback.  The questionnaire also serves as a basis for interviews or later development. This evaluation phase is an early start of the assessment process rather than evaluating the training after delivery when employees may lose focus or forget what needs to be emphasized.

Evaluation After training is divided into short-term and long-term evaluation.  The Short-term evaluation starts when the supervisor holds an interview  with the employee to identify and clarify personal benefits derived from training.  The interview is an intermediate evaluation step to verify that the primary impressions about the course mentioned in the questionnaire are correct and that training can be translated into ideas that are beneficial to business.  The questionnaire form serves as a source data for the interview.  The Long-term evaluation is introduced to measure concrete impact of training on business: when both employee and supervisor decide to develop a plan and follow up its action steps to maximize benefits gained at work.

 At the end of the evaluation process the initial impressions will be supported by measurable scorecards of real business gains from training. The long-term evaluation addresses the difficult issue of isolating and measuring the impact of training on business.  Without long-term evaluation, the evaluation cycle is not complete.  This type of evaluation is applied for the first time ever in Gupco and in the local industry.

EVALUATING A TRAINING COURSE BY TRAINEES (During Training)

The purpose of course evaluation is to assess course delivery and content.  There are three forms of evaluation according to the time:

• Ad-hoc evaluation during instruction;

• Review at the end of a day of work;

• Seminar evaluation following completion of the subject-related part of the course.

The evaluation has various functions. 

It:

• can help participants become conscious of latent feelings and moods;

• provides the group with the opportunity to get rid of “bottled-up” tension and “blow off steam”;

• helps in the case of conflicts and disagreements to assess the approval or rejection of theses, the strength of the opposing parties, and the interest of the remaining participants in the controversy.

As diverse as the functions of an evaluation are, it can also be applied in as many ways.  The form of evaluation the moderator chooses depends on which of the aforementioned aspects he is particularly interested in.

Evaluation During Course Work

If the moderator (instructor)  has the impression that the subject-specific work is not proceeding as it should, but he is not sure why, he can conduct an ad-hoc evaluation: 

• to test the participants’ understanding of the material;

• to determine the general mood in the group and the level of motivation;

• to allow participants to comment on the current situation (passivity, lack of enthusiasm, conflict, etc.)

Evaluation With the Help of a Checklist

Can be done during a session, at the end of the day, where the instructor evaluates the course according a pre-defined list.  Answers on the items of the check list is sought from the participants.

 

EVALUATION IN GROUPS

Evaluation can be done at the level of the person or in groups.  Group evaluation will allow discussion and also permit participants to explain their views.

 

REVIEW AT END OF THE DAY

An evaluation round at the end of the day gives the participants the opportunity to state their opinions on the contents and course of the program.  It also gives them the opportunity to contribute to the structure and organization of the course.  The moderator is given the chance to reflect on the results and draw conclusions for the following days.

 

OVERALL EVALUATION

At the end of the training, a thorough evaluation should be conducted which addresses all aspects of the program.  As a rule, an evaluation of this kind consists of a written part, in which the participants evaluate individually, followed by general discussion.