Given the complexity described above, one could be forgiven for saying that changing practice is to an extent a leap of faith, particularly if a technique such as e-learning is unfamiliar territory. Over the last decade, this uncertainty in the outcomes and impact of e-learning has led to development of a greater level of sophistication in methodologies for evaluation that have extended broad approaches to collection and analysis of feedback (Hounsell, 1999: 161).
As a result, there are a wide range of evaluation frameworks and models evident in the literature (see appendix 1 for some annotated examples). However, you may need help in deciding what approach will suit your own needs and in constructing a model for your own evaluation. One way of providing guidance and support, which is growing in popularity, is in the form of a “toolkit”.
Toolkits are useful in cases where a range of approaches might be used. They are a resource structured around an expert model or theory of a design process, often including examples. Each decision stage is supported by an activity that can guide you towards appropriate options. Importantly, the toolkit does not decide on the best approach on your behalf or force you to adopt pre-defined, ‘generic’ solutions. Rather, they guide you through evaluation processes prompting for information and assumptions from which the practitioner can make informed, professional decisions (Oliver and Conole, 2002).
The following toolkit offers a good starting point for both small, personal evaluation activities as well as large-scale projects and distributed forms of information gathering.
- LTDI Evaluation Cookbook
From the Learning Technology Dissemination Initiative, this publication, available as a online tool, is an extremely helpful and practical guide. It assists lecturers interested in evaluating learning technologies for their effectiveness in achieving specific learning objectives. This is a resource from which you can pick and mix from a range of ideas and suggestions, and thus design an evaluation study to suit your own specific needs.