Skip to main content

Linking Evaluation Methods with Pedagogy

In these and other papers there are examples of different evaluation methods – see also LDC’s evaluation website.

Examples include

• questionnaire survey
• interviews (including stimulated recall)
• logs of messages or web 'hits'
• pre test post test
• products
• focus groups
• online discussion
• user diaries

However evaluation does not always make a good link between methods and pedagogy. This means that we do not always get a good idea of the learning objectives or how learning took place e.g. Hiltz et al (2002). In contrast a good example of linking evaluation to an explicit model of pedagogy is Fowell and Levy (1995).

If pedagogy is important then it might be possible to highlight different dimensions to online activity. These are outlined in table 2 below.

 Dimension Characteristics Some issues for evaluation Examples


(low to high)
Timetables, ground rules, tutor assessment, tutor direction, requirement to log on, assigned roles lead to high structure

Coherence of curriculum design?

Match between design and learners preferred styles?

Gap between intention and practice

Clarke (2002) an example of very low structure, Aviv, R. Erlich, Z., Ravid, G., Geva, A. (2003) provide two examples in a problematic paper


and / or content

Content heavy - extensive predetermined course material,

Communication heavy - high exchange of ideas and information

Communication reich leads to focus on messages, message analysis and relationship of learners to messages;

Content heavy leads to study of imformation processing

McConnell, D. (2000) a good example of high communication low content
Independance / collaborative learning Collaborative learning often involves teams discussing cases or scenarios and constructing a group response, cooperative collaborative contested terms here. Collaborative leads to focus on group processes

Lockhorst, D., Admiraal, W., Pilot, A. and Veen, W. (2002) discuss case based approaches, Tsui, and Ki, (2002) discuss an open forum with independent contributions

Dempster and Blackmore (2002) around student web publishing and critical analysis (see TELRI model below)


Online / off line On line learning suggests that most learning literally takes place at the machine much neglected - how do we capture what happens away from the machine

Henri and Pudelko (2003) discuss different types of online community

Chen and Hung raise questions over personal and group knowledge

Table 2 dimensions to online activity

TELRI adoptive-adaptive learning model

The model devised by the TELRI project (Roach, Blackmore & Dempster, 2000) arose from consultation in departments across five research-led institutions. It is based around distinguishing between two types of learning: knowledge and practice-based learning (adoptive) and creative and higher order learning (adaptive).



Knowledge and Practice of.... Formation and Generation of....
Facts, Assertions, Rules and Laws Personal Interpretation and Meaning
Terminology, Language and Protocols Evaluation and Decisions
Techniques and Procedures Arguments, Reasoning and Justification
Organisation and Structure Synthesis and Conceptualisation
Established Principles and Relationships Originality, Creativity and Innovation

By nature, assignments to develop higher order thinking tend to be more open, challenging and motivating than those that require the demonstration of established knowledge and procedures, characterised by adoptive learning outcomes. For courses that are particularly keen to support the higher level, adaptive type learning, tutors should aim to provide students with opportunities to obtain feedback on their ideas and their approaches through debate and discussion and sharing of work, as well as providing an environment for reflection. Extending this to peer assessment, students can learn valuable lessons when given access to the work of their peers by engaging in the evaluation of what is and what is not good quality work in their subject area, using the same open-ended approaches the tutor might use. In this way, students themselves can contribute to the course, bringing in resources that support the formation and expression of their ideas and justifying these.

The model supports the lecturer in locating the different learning resources, learning support and assessment methods used against the adoptive or adaptive learning criteria. It has the advantage of supporting the integration, or blending, of both online and classroom activities and resources. In so doing, the TELRI model offers a broad framework for course design and profiling that can be tailored to different disciplines.

Course review and e-learning mapping tools based on TELRI are provided in Annex A and available on the web at:

Conversational Model

Laurillard (1993, 2002) offers examples of how learning and teaching methods, media/tools contribute to a didactic or dialogic relationship between the lecturer and the students. For example, she illustrates how the different e-learning tools relate to the different media forms representing these ‘conversations’

Media forms

e-learning tools



Narrative Print, TV, videocassette
Interactive CD, DVD or web-based resources
Communicative Web conferencing, asynchronous or synchronous
Adaptive Manipulable model
Productive Tools for students to create models or descriptions

Using this model produces a course design profile by mapping the tools used to resulting learning outcomes based on the different kinds and directions of dialogue supported.

An online toolkit based on this model is Media Adviser (Conole & Oliver, 2002):

Content-communication-collaboration model

Once the pedagogical aspects of the course objectives are clarified, the process of setting up the learning activity can be based on informed decision-making. The simple model represented below (Dempster, 2004) can be used to map learning activities against a hierarchy of increasing dialogic richness. Tools and methods can be selected that support increasing integration of communication and collaboration across the learning activities. The resulting course profile thus indicates the extent to which learning and teaching activities are focused on: presentation or practice of knowledge or techniques (e.g. content rich: teacher-student dialogue: adoptive) or on communication and collaboration (e.g. content light: multiple dialogues: group and task based dialogues: adaptive).

content communication collaboration model

(Dempster 2004)