Skip to main content

An Online Tool to Support Development and Assessment of Interactive Skills for Researchers

Dr Rob Johnson, Centre for Student Development and Enterprise, Univesity of Warwick

The Warwick History Interactive Skills for Researchers (WHiSkRs) is a package designed to teach and assess fundamental historical skills online. It is designed to enhance History Undergraduates’ use of information resources, conventions in referencing and presentation, analysis of documents, understanding of historiography, familiarity with certain key concepts, and knowledge of the ‘Late  Modern’ era (at an interim stage of the First Year Undergraduate course). As such, it is designed to smooth the transition for students becoming undergraduate historians through a term-long induction. It consists of useful information, presented interactively, as well as formative and summative assessment elements.

Rationale and Structure

Using the existing template designed for the Whasp project (History of Art information skills) as the basis for WHISkRs, work is underway to create a program that will teach and self-assess vital historical skills. Each of the seven components represents a different set of skills:  
  • Mastering Critical Analysis
  • Referencing and Presenting
  • Making best use of the library and e-resources
  • Using Concepts and Historiography
  • Reflective and Accelerated Learning techniques
  • Mastering Statistics in History
  • Final Assessment

Each one of these is broken down into three phases. The first phase focuses on giving the student the critical information. The second phase allows the student to practice the skill and get immediate feedback. The third phase is a self-assessment, putting all the points together. However, all three elements are interactive and overlap. For example, information skills are taught through ‘access and retrieval’ interactivity (using small sections of information, tools that require a response, tips and tests). Students are then challenged with a series of progressively more difficult tasks, activities or techniques. Finally, they are given a series of self-assessed questions and activities which are split in levels of difficulty and complexity. By the end of this component, students will be able to locate and acquire both printed and electronic resources using the Library Catalogue efficiently, use online databases of periodicals and books, find good-quality information on the internet or through portals, evaluate that information and make electronic references to it.

The programme is underpinned by blending the findings of three key pedagogical approaches to eLearning, namely Salmon’s five step approach, Warwick’s TELRI findings and the work of the JISC (Joint Information Systems Committee). Salmon advocated a stepped approach to learning via tailored ‘e-tivities’ (Salmon, 1998 and 2002) or interactive sessions (see fig. 1)

fig1
Fig 1: Salmon’s Five Step Model of eLearning

WHiSkRs combines Salmon with the findings of TELRI namely the need for adaptive rather than adoptive learning (Roach, Blackmore and Dempster, 2000; Beetham, 2004; Dempster and Blackmore, 2002). This concept advocates moving towards greater interactivity, favouring ‘knowledge construction above knowledge acquisition’ (see Fig. 2). In a skills program, this might appear contradictory, but students are encouraged to make evaluations and decisions when they are presented with, say, incorrect footnotes or bibliographies, or when they are offered the chance to see the consequences of poorly presented statistics. Adaptive techniques suggest that students should be participants in learning processes, and WHISkRs gives them a safe environment in which to experiment or get support. Students can generate a personal understanding and begin the process of using concepts. Moreover, mastery of these skills allows them to move on to more original thinking, creativity, innovation and evaluation

 Adoptive Learning Outcomes

Adaptive Learning Outcomes 

 Knowledge and Practice of...  Formation and Generation of...
 Facts, Assertions, Rules and Laws  Personal Interpretation and Meaning
 Terminology, Language and Protocols  Evaluations and Decisions
 Techniques and Procedures  Arguments, Reasoning and Justification
 Organisation and Structure  Synthesis and Conceptualisation
 Established Principles and Relationships  Orginality, Creativity and Innovation

Fig 2. TELRI Adoptive-Adaptive Learning Model

Underpinning the program and the activities themselves are the principles of Effective Practice with e-Learning which are published by the Joint Information Systems Committee(JISC, 2004). JISC believe that programs should be tested to ensure that, for the student, they:

  • Extend
  • Compel decisions/self-direction
  • Connect
  • Offer interactivity
  • Motivate (through variety and use of the multiple intelligences)
  • Offer opportunity for collaboration
  • Can be accessed anywhere and at any time

The Content

Each components of the programme deal with historical skills but many could be adapted to meet wider learning objectives and the requirements of other disciplines. For example:

 
  • The critical analysis interactivity encourages the critical deconstruction and evaluation of texts and other source material;
  • The terms and concepts interactivity informs, and subsequently tests, student knowledge, understanding and application of political, philosophical, artistic and literary ideas in history, but many of these are common to other subjects.
  • The historiography interactive component encourages consideration of the key issues and debates about the study of history, but also equips students with an appreciation of the scholarship in their field and how best to use it.
  • The final component of the entire programme is a self test which brings together all the other component materials. In assessment terms, there are two types of required responses: questions that give instant feedback, and questions that require longer answers or reflections. This aspect of the program is thus summative and the idea is that results on the short answer questions are collated automatically whilst the longer student responses will be automatically tagged and transferred to a blog.

An Example

Having academic rigour, demonstrable benefit and immediate relevance to the students’ experience are essential if the program is to be accepted by its users. We anticipate the value of the presentation and referencing skills section will be obvious to both staff and students. Since correcting references is a particularly irksome duty for tired lecturers, the program will enable students to check the correct way to footnote their essays and to construct a bibliography. The interactivities in this component are progressively more challenging, building student confidence and skill together. The model here, perhaps surprisingly, has been video game technology which encourages the user to progress through a series of levels, whilst offering assistance, instant feedback and rewards. The program also possesses a strong sense that the student is directing their own studies in the style of research-led learning (Dempster, 2003). Students will be able to apply the results directly and immediately to their first essays.

fig3

Fig 3. Screenshot of a page which tests the students ability to locate and enter footnote details.

fig 4

Fig 4. Screenshot of a students' selection of endnote details for a journal article with the incorrect entry highlighted. A tutor's advice tip could be added to this result.

Evaluation Strategy

Having made adjustments to the prototype now emerging, the full version will be deployed in October 2007. The plan is to evaluate and to disseminate our findings in 2007-08. The first stage in the evaluation will be the ‘road-testing’ of the program by students before the launch in the autumn, but user questionnaires and group meetings will seek to answer the following questions in the first few months of the program’s use:

  • Were the learning goals achieved? (learning new skills, opportunity to practice, understanding of when to use existing scholarship, understanding of the process of deconstruction, rating their level of critical awareness, ability to format and footnote correctly, etc.)
  • How efficient is the program? (Are the components navigable, shareable, manageable and re-usable?)
  • Has learners’ experience been enhanced? (Has the program assisted motivation?)

These questions are based on JISC’s ‘Effective Practice Planner’ (JISC 2004)

A written evaluation report will be made by the project manager and presented to the Skills Working Group and the History Department, and to the TQEF committee who generously funded the technical staffing of the project.

The Technology

The basic template for the project is Sitebuilder and Flash. These are relatively easy to master and enable rapid construction of interactivities where previously we would have had to spend some time and monwey on a complex design or a purchased package. Now we have the ability to construct pages that allow students to enter data, respond to questions, reflect, and answer questions (some of which are blog forms) and the new Sitebuilder2 ‘quizbuilder’. However, the technology is very much harnessed to the subject, not the other way around.

WHISkRs coincides with the first phase of First Year Making of the Modern World core course by using texts and podcasts supplied by our own staff, and has the graphical ‘feel’ of the department built in.

Summary of Benefits

The programme will assist students making the transition to Higher Education:
  • It will offer developmental, reflective and interactive support over one term (rather than the short time period of ‘induction’ at present).
  • It will compliment the existing approach to skills development in history, which is an embedded programme in the core course.
  • It will enable students to avoid some of the common errors made currently (particularly in referencing, use of on-line materials, a lack of depth in conceptual understanding, and an inability to offer a critical evaluation).
  • It will be used to teach and test information skills, historical skills, and literary skills, but will also act as a reference tool for students to revisit whenever they wish. This may be useful to students who have taken a break from their studies and who want to brush up on certain skills.
  • WHISkRs may be used to replace or enhance the First Year assessment which takes place in January, and could certainly be used to reduce the marking load by offering 50% online assessment.
  • More importantly, it will encourage students to develop a reflective approach to their studies, not only on the skills required to become a historian, but also on their own approach and performance in their studies. This will mark a step change in the way the department approaches personal and academic development.

The idea is to construct learning skills interactivities that are of immediate use to First Year history students, but WHISkRs also has an inbuilt ‘reflective practice’ interactivity to encourage a critical review and a PDP-approach to students’ studies. Students will have the ability to submit sections of the program they have completed into their blog, thus building up a reference store of work. The History department are keen to be able to spot students who are struggling so we are looking at a mechanism which will record the number of (failed) attempts at a particular activity, but also a limited number of submissions that can be quickly assessed off-line. 

The anticipated outcomes are:

  • an enhancement of history students’ confidence and skills in the first term (especially in retrieval of sources, referencing, and critical-evaluative writing skills), 
  • a reduction in marking load and skills teaching which could free up time for more advanced learning techniques, 
  • a contribution to the understanding of e-learning and the advantages of a blended approach (where offline lectures and seminars are complimented by a genuinely interactive support tool), and greater ease in the updating of the skills materials as the course and skills programme evolves.

Finally, it is hoped that the project will lead to the adoption of a similar program in other arts and humanities departments, and, after sufficient evaluation, that other departments might develop the idea further, apply the model to their own subject, and engage with the issue of student induction, transition and PDP.

References

Beetham, H., (2004) Review: Developing e-learning Models for the JISC Practitioner Communities JISC.

Dempster, J.A. (2003) Developing and Supporting Research-Based Learning and Teaching Through Technology, Chapter 19, pp.128-158, in Ghaoui, C. (Ed.) Usability Evaluation of Online Learning Programs. Idea Group Publishing, USA.

Dempster, J.A. & Blackmore, P. (2002) Developing Research-Based Learning Using ICT in Higher Education Curricula: The Role of Research and Evaluation, Chapter 11, pp. 129-139, in Macdonald, R. & Wisdom, J. (Eds.) Academic and Educational Development: Research, Evaluation and Changing Practice in Higher Education. Kogan Page: London.

JISC, (2004) Effective Practice with e-Learning. www.jisc.ac.uk/elearning_pedagogy.html  

Roach, M., Blackmore, P. & Dempster, J. (2001) Supporting High Level Learning Through Research-Based Methods: A Framework for Course Development. Innnovations in Education and Training, 38, pp. 160-169. Available on-line as pdf files at http://www.telri.ac.uk/Publications/

Roach, M., Blackmore P.,  and Dempster J., (2000) Guidelines for Course Design at www.telri.ac.uk/guidelines.pdf (Accessed December 2006).

Salmon, G., ‘Developing Learning through Effective Online Moderation’, Active Learning, 9, (December, 1998), pp. 3-8.

fig5

Fig 5. Screenshot of the demonstration of how to format pages for essays 

 

Citation for this Article
Johnson. R, (2007) An Online Tool to Support Development and Assessment of Interactive Skills for Researchers. Warwick Interactions Journal 29. Accessed online at: http://www2.warwick.ac.uk/services/cap/resources/pubs/interactions/current/abjohnson/johnson
(Accessed )