In the pursuit of greater flexibility and effectiveness in training provision many companies are considering introducing or increasing their use of computer based training (CBT). This type of training may be made available via CD/DVD based PC applications or centrally via an intranet. CBT can reach large numbers of staff in a cost effective way but it can also make it difficult to ensure both sufficient uptake and the effectiveness of the training provided unless a robust process of evaluation is also implemented and monitored.

Some general assessment of the content is necessary before any CBT purchase is made as there is generally less option to amend the training program once the materials have been purchased. Most CBT is based on a generic package with little or no room for tailoring and may not be suitable for either the staff involved or the conditions under which it will be used. Prior evaluation is important even where the training materials are inexpensive enough to be abandoned if they fail to deliver, any such failures have consequences on level of available skills as they will delay training completion and may also alienate users from the replacement or from other CBT.

To be sure that you are getting what you expect when you buy CBT applications it is important to decide the benchmarks of effectiveness you wish to use before purchase against, particularly where large numbers of users may be involved. The package under consideration should, wherever possible, be piloted against these benchmarks on a representative sample of the target staff under the access conditions in which it will be used before making a commitment.

Effectiveness assessment of approach and content is particularly important where the CBT is aimed at core skills or at issues that involve statutory compliance, such as health and safety, environmental management or financial management. In terms of health and safety and environmental management the effectiveness of training is a major consideration given that ensuring competence is a key part of providing a safe system of work. Therefore it is essential that the purchaser of CBT should have sufficient knowledge to be able to ensure that the proper expertise has been captured in the materials being provided. Where the core skills are more specialist in nature, for example for risks such as COSHH, ergonomics or environmental impact, it is a wise precaution to have the CBT providers and materials assessed by a domain specialist.

Time is also a key factor in providing compliance related training and it is important that health and safety training can be and is implemented in a timely manner if the possibility of accidents/injuries and claims of negligence are to be avoided. CBT needs to be implemented quickly and reliably on the technology that is available to the intended participants if it to be considered effective.

The Benchmarks

Purchasers may chose to benchmark CBT against a wide range of performance criteria, and these will not be the same for all materials or subjects. However evaluating the effectiveness of CBT on issues that are part of statutory compliances should cover at least the following key factors.:

Prior to purchase/implementation

Measures to be considered before commitment to a CBT programme include: -

· Scope of the training materials

For a training programme to be effective the content needs to be as relevant to the business as possible. In evaluating this it is important to consider the extent to which the material directly reflects:
· those situations in which staff will need to demonstrate competence;
· the form of risk/situation that they are likely to encounter.

In some situations content material may need to be transferable across a wide range of staff and tasks and separate benchmark may need to be set for the varying context in which staff will be using the new skills. The more universal and frequent the situation involved the tighter the compliance to benchmarks needs to be.

When reviewing CBT it is important to keep the target users firmly in mind and look for: -
· the range of compliance issues covered;
· inclusion of adequate discussion of situations that are known to have occurred or be occurring;
· demonstrations of environments, situations and tasks that will be familiar to the student;
· the nature of the training vocabulary, materials should make use of words and terms that will be familiar to the workgroup being trained;
· consideration and suggestions for solutions/actions that may be directly transferable to the users normal working situations and tasks.

· Fit to individual need

Some users may have additional learning needs either because of their level of skills, their experience of learning or the jobs they do. Any CBT program used must be able to meet these if they are to be fully effective, e.g. can a DSE course meet all needs including those of staff working with PCs on production lines?

In particular the CBT content should be assessed for its ability to meet the needs of: -
· vulnerable staff, for example young staff or disabled staff;
· those working in atypical environments e.g. those who work out of the office or who work alone;
· those jobs which may include atypical or high-risk elements, for example staff whose work includes dangerous materials or other high risk situations.

Though CBT may not be required to meet all user needs, for example it might not be intended for young persons, the scope of a CBT programs use should be a deliberate decision not a default caused by unforeseen limitations of the package.

· Time for implementation

Consideration needs to be given to any technical problems and therefore time delays that may arise when considering the overall effectiveness of CBT. If changes or upgrades in hardware/software technology are required to enable the materials to be used then other options may prove to be more effective in the short to medium term. This can be a significant consideration where the training needs to be delivered in a tight time frame. In general training in statutory fields such as health and safety carries a time imperative, particularly where it is aimed at meeting duties that have been in place for some time. Time to implementation may also be an issue where the skills are necessary to develop or support new products or services, this might include things like software use or telephone skills, particularly if the staff being trained will be customer facing.

· Accessibility

For CBT to be effective it must be easily accessible, this requires that users have a reasonable degree of access in terms of training time, not working hours, to a suitable PC. This may mean that the workstation involved is unavailable to other possible users whilst people are training and can raise accessibility issues in areas where hot desking is common or where workload makes it difficult to have PCs tied up in non task related activities. This needs to be planned for before training begins and local management will need to be consulted.

The ability to deliver the CBT to staff who often work away from the office also needs to be considered and the ability to access training from a laptop or home computer may be an important part of training effectiveness for some groups of staff.

Post implementation

Once implemented CBT effectiveness needs to be reviewed against the performance benchmarks mentioned above. The benchmarks chosen may vary but are likely to include at least the following:

· successful completion rates/time

One disadvantage of CBT is that though staff may begin a course it is harder to ensure that they complete it successfully and in the intended timeframe, however time to completion is likely to be a key effectiveness benchmark. This is a particular issue where self-paced learning is allowed.

The percentage of staff required to successfully complete the course and associated tests in a fixed time frame should be agreed in advance and progression to this target should be monitored. Some form of sign off procedure will be necessary and times between implementing training and sign off needs to be tracked and delays investigated and resolved.

· assimilation of material/competency
Successful assimilation of information is critical to effectiveness regardless of the nature of the training, and in conventional training a large part of the trainers skill will be in judging candidates competence and providing remedial help to those who struggle. With CBT there is no trainer present for most periods of learning and this raises issues of how to determine that a course has been completed successfully by a learner rather than simply viewed.

Many CBT programs do incorporate some form of achievement/comprehension test to demonstrate that the material contained has been mastered to a satisfactory degree. However these tests can be compromised and success rates need to be monitored or validated to assess training effectiveness. Measures to prevent staff from cheating these tests either by getting others to complete them for them or by cribbing the answers from those around them may also be necessary.

However these internal tests may not be a sufficient measure of competency in all circumstances. Many CBT tests are tick box multiple choice formats which makes guessing, rather than knowing, a possibility. The software may also allow staff to go on trying permutations of answers until they get the right combination without recording that this has happened or reflecting that in the scores awarded. Ways of avoiding such distortion of success rates needs to be found of CBT is to be the main vehicle of teaching, particularly where safety critical skills are involved.

To ensure that CBT has been effective in situations such as health and safety where training and competence are part of statutory duties materials may need additional evaluation against:-
1. transfer of knowledge - real understanding of the material and its consequences
2. capability to extrapolate - the degree to which users can apply the information in new situations
This may require some form of additional testing/validation, particularly for key issues or higher risk topics, and will generally involve some form of additional professional assessment. Post CBT competence may be evaluated by random one to one or written assessments by health and safety staff or by an external expert. For example with DSE risk assessment training a sample of CBT trained staff may be observed conducting their workstation assessments, when evaluating manual handling CBT effectiveness a managers manual handling assessments would be reviewed before and after training.

Monitoring effectiveness is an ongoing activity. Records need to be kept and routinely reviewed of all staff successfully completing CBT.

Conclusion

CBT is a useful addition to employer’s options when planning training, particularly for large numbers of staff. However a range of measures is needed to ensure that the CBT used is reliably and is successful at developing the required skills, and also that the staff involved get sufficient time and opportunity to use it effectively. These measures include checks both before and after the CBT is purchased and implemented and should consider both the materials being taught and the learners involved.

Procedures to ensure that individuals and groups use CBT effectively will also need to be developed and implemented. Where the training has a bearing on statutory compliances all of these measures, and the use of the CBT I practice, form part of the due diligence process and must be capable of audit.

This is an extended version of an article pubished in 2004