Jay Dempster, Centre for Academic Practice, University of Warwick
National and institutional attention is focusing increasingly on lifelong learning and distance education, including continuing professional development in industry and business. Communications and information technology (C&IT or ICT) plays a pivotal role in developing and delivering in these areas. Maintaining teaching quality yet expanding the educational market of HE is a challenge that sees income generation and building the ICT infrastructure in direct competition. In the article by the Vice-Chancellor in this issue, The role of technology in higher learning, Professor Sir Brian Follett describes the rapid pace of the capabilities of technology. He looks at the way in which Warwick is harnessing such developments in teaching, learning and assessment given the dwindling public resources to universities.
In the light of this challenge, it is hardly surprising that evaluation of the outcomes of technology-assisted teaching and learning initiatives is a key issue in prioritising expenditure on ICT. However, in both C&IT development and its implementation, evaluation is too often seen as a minor aspect due to lack of project funding, enthusiasm or understanding of effective methodologies. In the second article, Developing and Evaluating Courses to meet Learning Outcomes, Dr Martin Oliveroutlines developments at the University of North London in three key areas: creating a context in which reflective practice is valued, supporting curriculum development and encouraging evaluation. He describes an evaluation toolkit that supports staff in the design of an evaluation plan that will enable them to judge how effectively their course supports particular learning outcomes.
For institutions, departments and individual staff, evaluation should be an essential element to finding out the impact and effectiveness of teaching, particular where a significant change is made to the method used. It is also a valuable means of justifying the cost of investing in ICT. While developers feel they can recognise the benefits of integrating learning technologies, they often skip over assessing how the students' learning experience has actually been enhanced. Lecturers will remain sceptical without supporting evidence and case studies. Most lecturers engage in some form of evaluation even if just to find out what their students think of a new course or module. The motives for seeking feedback to evaluate teaching may also be extrinsic, for example through the Quality Assurance Assessment. These factors will influence the choice of feedback source (self, students, colleagues, incidental) and the time for gathering evaluation data (before, during, after a module). This in turn should assist in deciding the appropriate method of eliciting feedback. A concise overview - useful for both new and experienced academics - is provided in Chapter 13: Evaluation of Teaching in the recent book "A Handbook for Teaching and Learning in Higher Education: Enhancing Academic Practice" (Eds. Fry, Ketteridge & Marshall, Kogan Page), available in the Staff Development Section of the Warwick Central Library Student Reserve Collection).
Evaluation demands measurable outcomes and therefore requires early clarification of learning objectives. This can only bring a healthy reflection to course design and delivery. It allows us to consider more thoroughly what the ICT intervention is aiming to achieve and to what extent the desired objectives are being met.
In research-led institutions, the majority of higher order learning is achieved through debate and discovery of knowledge, requiring dialogue with tutors and students, and between students. The final article, Developing Research Capabilities through Technology Enhanced Learning, from the TELRI Project, describes how the higher learning skills of the proficient researcher might be developed through the use of particular learning technologies. The case studies for implementation of learning technology being developed from the TELRI Project evaluations will assist in demonstrating the extent to which the research capabilities of students are enhanced through the use of web publishing and discussion methods.
A final note for evaluation strategies. Whichever technology approach is used to enhance the learning experience, it must be planned in. As Laurillard says "Student use of new technology does not occur in isolation from other aspects of their course as far as they are concerned … It must be embedded as a natural part of their learning, if they are to take it seriously, and to benefit from using it." Laurillard, D. (1993) Rethinking University Teaching: A Framework for the Effective Use of Educational Technology, Routledge.
Dr Jay Dempster
Centre for Academic Practice
University of Warwick
Tel: +44 (0) 24 7657 2737