In developing your teaching and the curriculum, you may be involved in some way in introducing e-learning, that is the use of technology to assist student learning. This may involve you in creating course websites, integrating online discussions or incorporating computer-assisted assessment. You may be bringing together computer-based resources to support your subject area or developing specific software.
One might argue that all teaching involves technology of some sort – whether it’s a computer or an OHP. However, with e-learning comes new issues and responsibilities. It is important that you are sensitive to these in relation to how technology-based methods and materials are introduced and integrated into learning and teaching. Evaluation can provide a framework for thinking about such matters and to help you judge whether any changes in practice have been beneficial.
A single model for evaluating e-learning is, however, hard to define. Learning technology is still a fairly new field. Understanding of the theory and pedagogy that underpins the use of the technology is constantly being revisited in the research community. Developing e-approaches is therefore compounded by “the absence of a widely established and practiced methodology by which rigorously to evaluate e-learning, and through which to develop the secure body of knowledge on which to build learning technology as a discipline” (ALT JIG, 2003).
Nevertheless, the overarching point made in the literature is that e-learning is first and foremost about learning. “Without asking hard questions about learning, technology remains an unguided missile” (Ehrmann, 1996a). In terms of designing an evaluation, this simplifies matters, as a fundamental principle must, therefore, be that evaluating e-learning is as much about evaluating good educational design as it is about evaluating the technological approaches. Effective evaluation should therefore include a close scrutiny of the pedagogical rationale and outcomes of the curriculum and assessment design which e-learning is intended to support.
Evaluation offers a means to investigate, provide evidence, learn, share and make judgements about what we do and how we do it. However, whether or not e-learning is being used, educational evaluation involves a complex, value-laden set of issues. It incorporates principles and methods used in educational or social science research more broadly, for which an extensive literature exists. The increase in action research as a basis for educational development and in accountability to growing numbers of stakeholders means that the boundaries between research and evaluation are blurring (Macdonald, 2002: 10).
All things said evaluation does not have to be an arduous process. A well-aimed, light touch evaluation is much more valuable than a costly study that generates masses of data with unclear purpose and focus. A good evaluation will have explicit aims, address questions that are meaningful to these aims and to the intended audience, and use an appropriate mix of methods for gathering, analysing and presenting data.
This guide offers some principles and methods for effective evaluation of e-learning. It is deliberately a discursive document intended to help you think through the kinds of evidence you might wish to find, what are useful questions to ask, how to ask them, and what to do with the answers. It introduces some simple tools and further resources to support evaluation design and analysis. While it takes unashamedly a positivist approach to educational evaluation, it offers an opportunity to reflect upon your own developments and cultures of inquiry.