Contents   Abstract 1. Introduction 2. Evaluation Framework 3. Survey Results and Analysis 4. Recent Changes to IOLISplus 5. Conclusion   Download IOLISplus - The Second Chapter David Grantham School of International Studies and Law Coventry University d.grantham@coventry.ac.uk   Abstract IOLISplus was first introduced and described in JILT 1999 (1) . This article describes what was learned from the evaluation of the pilot study of the IOLIS/IOLISplus package. Student reaction to this electronic learning environment is explained and measured against the main curriculum intentions. The importance of the role of the tutor in the process is carefully considered and examples of 'attentive' tutor electronic interventions are given. Following the evaluation, certain changes and additions were made to IOLISplus and the underpinning pedagogy and educational strategy of these is examined. Finally the article briefly describes the impact of the introduction of WebCT on IOLISplus, particularly on the strategic use of electronic fora and the potential for future developments. Keywords: IOLISplus, Pilot Evaluation, Tutor Role, Changes to IOLISplus, WebCT, Strategic use of Fora. This is a Refereed Article published on 29 February 2000. Citation: Grantham D, 'IOLISplus - The Second Chapter', 2000 (1) The Journal of Information, Law and Technology (JILT). <http://elj.warwick.ac.uk/jilt/00-1/grantham.html> .New Citation as at 1/1/04: <http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/2000_1/grantham/> 1. Introduction IOLISplus was presented at the first LILI conference in January 1999. It has since been described in some detail and put in the learning and teaching context where it operates (Grantham,1999). The purpose of this paper is to explain the outcomes of an evaluation of the project and the pedagogy underpinning subsequent developments. For those not familiar with IOLISplus let me briefly outline its history and its purpose. IOLISplus was born in the Autumn of 1997 as an idea for an electronic learning environment to support the much more familiar IOLIS. Time to develop the proposal was made available through being an original member of the Coventry University Teaching and Learning Task Force, set up in the wake of the Dearing Report ( 1997 ). The University had already decided to invest heavily in the hardware for communications and information technology (C&IT) and IOLISplus was just one of a number of projects that were intended to harness the new technology. IOLISplus was designed to meet perceived changes in student needs at a time of a shrinking unit of resource and to provide a means by which good quality learning was not tied to a particular time and place. IOLISplus, consisting of web pages created in Microsoft Front Page, partly mirrors IOLIS by providing: a) self-paced learning not bounded by rigid timetables;   b) powerful navigation around the learning environment;   c) links to articles and other web-based materials.   In addition, the early version of IOLISplus provided: a) questions designed to promote deeper learning of some matters already introduced in IOLIS, particularly a critique of the law, where appropriate;   b) more contextual material;   c) ongoing online dialogue with and between students via an electronic legal forum;   d) graphics, including some animated graphics to add interest to the pages. IOLISplus has a Home Page from which students can navigate to all other pages, a 'How to use it' instructions page and a search facility. Students move between both resources, the IOLIS data and exercises providing a foundation of learning upon which the IOLISplus questions and additional materials will build. From the tutor's perspective, links in IOLISplus to electronically published legal articles are a valuable source for exploring the kind of issues which can deepen understanding of the subject. If appropriate, through the medium of hypertext, students can be directed to concentrate on just one issue in an article. It is the more challenging issues that will be the focus for dialogue in the legal forum. Perhaps it is my good fortune that my own field of tort law holds much of interest and is so amenable to contextual and critical study. Psychiatric injury was the first area of the field that was the subject of experimental web pages. More recently, pages introducing tort and pages on the tort of nuisance have been added. Before the end of the 1999/2000 academic year it is planned to include pages on economic loss and remoteness of damage. At this point there would be in excess of 25% of the tort module in electronic form. 2. Evaluation framework A parallel project using the same Microsoft package was being developed by another Task Force colleague (a statistician) and much was learned from our combined experiences. Our underpinning pedagogy was broadly similar and we have recently described both this and our early mistakes with our respective designs ( Grantham and Hunt,1999 ). We were able to conduct some evaluation of each other's work on a collegial basis but it was the student experience that was to be the acid test of our endeavours. Analysis of the first evaluation of the pilot studies of IOLISplus was carried out in the Spring of 1999. I was interested in the opinions of students on a number of issues, including ease of navigation, assessment of the level of difficulty of questions and comparisons with traditional ways of learning. I regarded the process as a form of 'action research' as described by Carr and Kemmis( 1986 ) and Schon ( 1987 ). This casts the tutor in the role of the teacher-researcher. After an investigation of the context in which their own pedagogy is set and of the learning and teaching strategy itself, the teacher-researcher makes changes aimed at improving the student experience. These will usually be aimed at meeting criteria that have emerged from an analysis of the data produced by the investigation. An evaluation of the impact of the changes is then carried out to make an assessment of the success of the innovations against the set criteria. If the teacher-researcher finds that none, or only some, of the criteria have been satisfied then the research-reflection-change-evaluation cycle begins again. Four curriculum aims underpinned the early version of IOLISplus: a) 'user-friendliness' in terms of ease of navigation and maintaining student interest;   b) useful links with prior learning and online resources;   c) encouragement of deeper learning by introduction of context and appropriate and searching questions;   d) enabling electronic communication with and between students. To what extent these aims were achieved was an essential question for the evaluation of the experience of the pilot group. In another sense the evaluation could be seen as an essential part of what Davis ( 1981 ) has described as 'critical trialling'. IOLISplus was an experiment in a new kind of online learning and I needed to know whether it delivered the kind of deeper learning for which it was designed. If it failed to do this, even in part, then redesign would be necessary. Pilot trials can often result in at least a partial return to the drawing board and I anticipated that the trialling of IOLISplus was to be no exception. The survey questionnaire was designed to yield both quantitative and qualitative outcomes. Bar charts represent the qualitative outcomes of the survey results and are intended to provide an easily observed visual account of the student responses. On the other hand, part of my educational research background is in the qualitative tradition of 'illuminative evaluation' ( Guba and Lincoln,1981 ). Such an evaluation attempts to capture how the educational experience feels to the student by citing and interpreting student accounts of their own experience. These are intended to shed light on the quality of that experience as measured against the aims and objectives of the project. Responses to the 'free comments' questions in the survey were designed to 'illuminate' those aspects of the student experience which were not elicited by the specific questions. See copy of the survey in the Appendix . 3. Survey results and analysis There were 22 surveys returned out of a total pilot group of 26 (an 85% return). Two groups were chosen, one group of ten full-time students, most of whom had only word processing experience, and one group of sixteen students who attended the University part-time, in the evening. Though these were mainly mature students there were four younger students, two of whom had been referred in the module from the previous academic year. Two of the full-time students and three of the evening students expressed their fear of computers and only four of the latter group regularly used PC's in their daily work. None of the students had any prior experience of electronic learning. One part-time student, of more mature years than the rest, had a dual fear - of computers and of tort law. She told the group that she had had several attempts at tort examinations and had not yet been successful in any of them. She, much to everyone's surprise, embraced the technology with enthusiasm and this must have given considerable confidence to the other novices. (For the record, this student was delighted when she passed her tort examination with a mark in excess of 60%). Because students experienced a learning 'package' there were some questions relating to both IOLIS and IOLISplus. In addition to specific questions about the students' reactions to navigating around the pages there was a more general 'free comments' section. This revealed some interesting student views on the whole experience. No marked differences between the experiences of full-time and part-time students were evident. Figure 1: Results of student survey on navigating IOLIS No fewer than five students specifically commented on the navigational frustrations created by IOLIS pages not numbering backwards. This idiosyncrasy has been drawn to the attention of the producers of IOLIS. Figure 2: Results of student survey on navigating IOLISplus Since there were technical difficulties with the web server I had expected quite a negative response to the question of ease of navigation around IOLISplus. However, this does not seem to have skewed the responses. Also, my concern that most students might find navigation between IOLIS and IOLISplus quite difficult was clearly unfounded. Mindful of recent research pinpointing that student frustration with technical problems is more common than hitherto believed ( Hara and Kling,1999 , Mendels,1999 ) I continue to explore easier ways of getting around the total package. Figure 3: Results of student survey on completing task within suggested time My total estimated time for the task of completing the psychiatric injury package was 10-12 hours. This guesswork was likely to be wide of the mark since, despite my best efforts at putting myself in the place of the student, there is always a tendency to think in terms of how long it would take me to complete the study. In the event, the survey results suggest that the actual time spent was broadly in line with the estimate. Interestingly, one of the students who reported that he or she had spent longer than estimated also said: 'But the resources and links were very useful - which is why I spent more time on it!' Figure 4: Results of student survey on the difficulty of IOLIS questions Given that one of the main pedagogical purposes of IOLISplus was to generate 'deeper' learning in the students ( Marton and Saljo,1976 ) I wanted to get some kind of a grip on how they rated the difficulty of the questions. Because there were different levels of intellectual challenge in both IOLIS and IOLISplus the results were likely to reveal only a broad perception of difficulty. However, the data shown above and below was useful and shows that there is only a perceived marginal increase in difficulty in IOLISplus questions. Clearly, I needed to make some questions even more challenging! These student responses do not, of course, reveal whether deep learning did actually occur. Previously reported contributions to the discussion forum did in fact suggest that such learning was happening but a full evaluation of this outcome has yet to be conducted. ( Grantham, 1999 ). Two students liked the questions but were not keen on reading from the screen. Their feelings are summed up in the comment of one of them: 'The test questions were a good tool for learning the work. Reading long passages of text from the screen is less preferable than a number of text books to hand'. One student made an observation concerning the quality of the questions: 'The questions were difficult to answer because they were searching and required time to think and formulate an answer that was adequate'. Figure 5: Results of student survey on the difficulty of IOLISplus questions Figure 6: Results of student survey on the web design of IOLISplus No student found the IOLISplus pages 'boring' or 'very boring'. Indeed, it appears that most students were very positive about the impact of the page design. Another of the claims made for IOLISplus over IOLIS was that it is easier to update material and make necessary links to new case reports, articles, etc., as they become available online. Student responses to questions on the usefulness of links are shown next. Figure 7: Results of student survey on the usefulness of IOLIS links

Figure 8: Results of student survey on the usefulness of IOLISplus links

Because the links in IOLISplus can connect with very current and topical material it was, perhaps, no real surprise that more students found them 'very useful' when compared to IOLIS links. One student nicely summed up what a number of them had said to me about the linked material:

Finally, I wanted to know whether the students preferred electronic learning to traditional learning through 'hard copy' materials, or, indeed, whether they would like a combination of both.

As can be seen from the chart below, preference for traditional methods just about edged out electronic methods. However, by far the largest preference was for a combination of methods, thus supporting the contention of Naisbitt ( 1984 ) that if the introduction of new technology is going to be successful then there is also a need for a 'high human touch' component.

Figure 9: Results of student survey on preferences for electronic learning

At the pilot session there was a concern over possible technical difficulties so I was present with the students at the time. I decided against asking specific questions about this pilot electronic discussion forum since responses would be distorted by the fact of my being there. There were advantages in my 'real time' attendance. I was able to motivate them and give help when needed. In particular, I had to work quite hard to get the students to make initial postings to the electronic legal forum. However, once they had made that all important first step, most of them contributed quite freely. Indeed, there were some quite unexpected postings relating to the law on psychiatric injury in New South Wales which extended everyone's knowledge , including my own. Despite this real bonus, there is clearly a need to give much more thought to how hesitant students could be motivated into making that crucial first exploratory posting. For them it is often a high-risk venture where the fear of 'getting it wrong' could be exacerbated by varying degrees of technical phobia. Students need to navigate forward confidently and in the knowledge that it's 'OK to make errors', both technical and in learning the subject matter. It is the tutor's role to lead them forward, to be alive to the need for gentle persuasion and to the timing of electronic interventions. Oakshott ( 1989 ) has highlighted the importance of what he has referred to as the 'attentiveness' of the tutor:

During the current use of IOLIS and IOLISplus I am not physically present while the student engages with the online learning environment. 'Attentiveness' on my part must therefore be demonstrated electronically. This means regular monitoring of electronic discussions in the legal forum and deploying encouraging comments. My incursions into the world of the learner are strategic, often encouraging or giving praise for perceptive postings. Examples are:

These instances may be no more than an electronic equivalent of best seminar practices. A tutor may be doing no more than would be heard in the well managed discussion. Strategic tutor interventions require both timing and a sure touch. Arguably, the best electronic discussions, like the best 'live' seminars, are those where the tutor says less and the student says more. Here, the tutor acts as mentor, guide or learned friend who enters the discussion only when he or she senses that the students need encouragement, are too far 'off task' or need a new input to take the dialogue onto a different plane. Conversely, and also like the real-time seminar, the discussion can be inhibited, or even curtailed, by insensitive or ill-considered interventions by the tutor. Wherever it is taking place, tutor comments of a sarcastic, discouraging or humiliating nature are unlikely to promote an in-depth debate.

A full evaluation of the electronic legal forum will feature prominently in the next survey which will seek the views of over 140 students. One pilot student did make a specific 'free comment' on the forum, however, and I relay it here - though mainly for the pleasure I derived from reading the final part of the comment:

4. Recent Changes to IOLISplus

With Ausubel ( 1978 ) and Entwistle and Entwistle ( 1992 ) I am convinced that the most important thing the learner brings to any new learning milieu is what he or she already knows. The argument probably holds not just for topics, modules or courses but also for what students bring with them to higher education institutions generally. Each area of study within IOLISplus now begins with a list of all the prior knowledge a student should have before beginning to navigate through the less familiar waters in the new web pages. If that prior study is available in IOLISplus it will be a simple matter to hyperlink to the relevant previously charted waters. Students having doubts about their previous studies will need to revisit these before moving forward.

One of the most valuable features of a web site is the area where 'Frequently Asked Questions' ('FAQ's') appear. These can quickly guide the navigator to tips and techniques, short cuts and common errors. The latter is a kind of map of the learning minefield that is undoubtedly present in many legal topics. Every IOLISplus topic area now includes 'FOM's' - 'Frequently Occurring Misunderstandings' which students can visit, via hypertext, before proceeding into mine infested waters! These are distilled from many years of seminar experience and from assessing thousands of attempts at pieces of coursework and examination scripts. Additions will no doubt be made over time. My choice of a Homer Simpson as an animated graphic for FOM's seemed appropriate since I could not imagine anyone who seems to be more frequently misunderstood!

Finally, I wanted to really get myself into the role of the learner and to see, as far as I was able, how the learning seemed to feel from their perspective. Like Marton and Booth ( 1997 ) I perceived learning as a joint enterprise in which learners and teachers are the prime actors. To do this it becomes necessary to really touch base with the needs of the learners. These will, of course, vary with different learners and different topics. Some questions in IOLISplus, like questions in IOLIS, will be well within the capacity of some learners. Though I may wish it otherwise, those same questions will be outside the present capacity of other students. Of course, this inability will often be temporary and students may later return to a question with renewed confidence. In the interim, they could spend wasted time on questions that they cannot yet handle, but do not yet comprehend that they do not comprehend. To assist the process of student discernment I have made a professional judgment about each IOLISplus question in terms of its level of difficulty. Each question is weighted according to a rating scale:

Those students who are 'strategic learners' ( Gibbs, 1981 ) will thus be empowered to follow whichever learning plan works for them. This could be to tackle all the easy questions first before moving on to the more difficult. As an alternative, they could follow a three-question combination through all levels of difficulty before moving on to another three. Those who want no more than a good pass or even a bare pass might ignore the three star questions altogether. It gives students the choice, which they can make according to their own needs and desires.

Another new feature of IOLISplus is that it now resides within the framework of a WebCT electronic learning environment. From September, 1999 all Coventry University programs and modules have had their own WebCT site. This has allowed the more flexible electronic forum capacities of WebCT to be linked to IOLISplus so that fora can be set up for different seminar groups. Only those students within the group and the seminar leader can make and see postings to their forum. This helps both hesitant students and attentive tutors to move the discussions along. Other standard WebCT features are also available to students, including direct online links to the library catalogue, study skills web pages, email, group presentation areas, assessment record and student home pages.

WebCT has provided yet another dimension to electronic learning since it has provided an opportunity for experimenting with yet further forms of web page design. Procedures for uploading material are rather elaborate, especially for a beginner who will need to learn 'file-zipping' to upload graphics. However, once these have been mastered there are many ways in which pages can be presented to the learner, and I am currently experimenting with different designs.

Many, but not all students, are making use of a number of module WebCT sites. Although these are early days in the 'WebCT electronic world' there are favourable indicators that both students and colleagues are beginning to appreciate the power and versatility of this particular milieu. My own module site has been visited by 80% of the students registered on the module. Student tracking, an interesting if rather 'big brotherish' tool, records that the highest number of 'hits' by any one student thus far is 267. Content pages are the most frequently visited and the students make extensive use of the email facility. It is likely that the next development will be to find a way of linking IOLIS more directly with WebCT and to design pages with a facility for instant feedback after a student has made a reasonable attempt at a question. WebCT opens up other possibilities apart from the creative use of feedback mechanisms. New ways of assessing students could emerge, especially of group and individual postings to discussion fora and of work incorporated into group presentation areas. IOLISplus would be very much an integral part of any such developments and, together with WebCT could be the 'enabling technology' for the kind of collective and collaborative peer learning as advocated by Boud, Cohen and Sampson ( 1999 ), Chalmers and Volet ( 1997 ) and Slavin ( 1995 ).

5. Conclusion

Designing, testing and redesigning web-based learning brings with it a new range of professional challenges. In essence, however, at its core are the old pedagogical skills of reflecting upon what tutors can do to bring about valuable learning and acting on the professional judgments emerging from this process. These decisions will include:

Electronic learning environments such as IOLISplus and WebCT and the power they give to professionals and students alike, will, to varying degrees, mean a change of role for both. But pedagogy must always lead the technology, not vice-versa. In this brave new world educators will need to master both pedagogical and technical skills and to combine both in meaningful ways to bring about better learning for their students. If lectures are simply about imparting information then electronic methods of doing this will become infinitely more effective. On the other hand, seminars or tutorials will continue to be the mainstay of a university education. Whether these are face-to-face or electronic it is where a tutor needs to be creative, innovative and versatile.

For their part, students will have to accept much more of an active role in the learning process. They too will also need to master the technical skills needed to navigate around the electronic learning environment and make useful postings to discussion fora. If students willingly and positively embrace active learning in the electronic milieu then they should be better prepared than ever for the dialogue of the face-to-face environment. One of the main aims of IOLISplus and WebCT is that the web-based resources, questions and forum exchanges should better prepare students for even more complex matters to be discussed in real time seminars.

IOLISplus is getting more sophisticated in its aims and objectives, more strategic in the options it affords to the learners and more complex in its pedagogy. As an electronic environment it has moved on from its original form but is still in developmental mode. This is perhaps as it should be - learning and teaching in any discipline is never the finished article.

References

Ausubel, D.P., Novak,J.S. and Hanesian, H. (1978) 'Educational Psychology: A Cognitive View'. Holt, Rinehart & Winston, New York.

Boud, D., Cohen, R. and Sampson, J. (1999) Peer Learning and Assessment. Assessment and Evaluation in Higher Education, Volume 24, No. 4. pp413-426.

Carr,W. and Kemmis, S. (1986) 'Becoming Critical' (Education Knowledge and Action Research). The Falmer Press, London.

Chalmers, D. and Volet, S. (1997) Common misconceptions about students from South-East Asia in Australia. Higher Education Research and Development, 16(1) pp87-98.

Davis, E. (1981) 'Teachers as evaluators. George Allen and Unwin, Sydney

Entwistle, A.C. and Entwistle, N.J. (1992) ' Experiences of Understanding in Revising for Degree Examinations'. Learning and Instruction, 2, 1-22.

Gibbs, G. (1981) Teaching Students to Learn; A Student Centred Approach. Open University Press, Buckingham.

Grantham, D.J. (1999) IOLISplus- Extending the Electronic Learning Environment. The Journal of Information Law and technology, 1999,1, (<http://www2.warwick.ac.uk/fac/soc/law/elj/jilt/1999_1/grantham/>).

Grantham, D.J. and Hunt. N (1999) ' Web Interfaces to enhance CAL materials; case studies from Law and Statistics. ALT-J Volume 7, Part 3.

Guba, E.G.and Lincoln, Y.S. (1981) Effective Evaluation. Jossey Bass, San Francisco.

Hara, N., and Kling, R . (1999). Students' Frustrations with a Web-based Distance Education Course: A Taboo Topic in Discourse.(< http://www.slis.indiana.edu/CSI/wp99_01.html >)

Marton, F. and Booth, S. (1997) 'Learning and Awareness'. Mahwah, N.J. Lawrence Erlbaum.

Marton, F and Saljo, R (1976) 'On qualitative differences in learning I: Outcome and process', British Journal of Educational Psychology, 46 4-11.

Mendels, P. (1999). Study Finds Problems With Web Class. New YorkTimes. (< http://www.nytimes.com/library/tech/99/09/cyber/education/22education.html >)

Naisbitt, J. (1984) 'Megatrends'. MacDonald & Co, London.

Oakshott, M. (1989). Learning and Teaching. In Oakshott, M., and Fuller, T. (Eds), TheVoice of Liberal Learning. New Haven, CT: Yale University Press.

Report of the National Committee of Inquiry into Higher Education 1997 (The Dearing Report).

Schon, D.A. (1987) 'Educating the Reflective Practitioner'. Jossey Bass, San Francisco.

Slavin, R.E. (1995) Cooperative Learning. Allyn and Bacon, Boston.

Appendix

Iolisplus Evaluation