Graham Lewis, Centre for Academic Practice, University of Warwick
The Centre for Academic Practice was approached by the Sociology department at Warwick to deliver a workshop on searching the Web to fill a gap in the provision of skills on a postgraduate course: MA in Researching Society. The students were from a range of birth disciplines, nationalities, age and experience of using the Web, although most had used a search engine before, some having had a number of years’ experience of doing so. The students had already attended sessions offered by the Library concerned with electronic library resources and bibliographical databases.
Clearly, the dividing line between local and remote resources is increasingly less distinct, as it seemed that the students had not received guidance in tackling the wider Web.
There are a number of subject gateways dedicated to the social sciences and these include excellent guidance and advice so it was decided not to replicate this in a face-to-face session, but rather to look at the wider Web as a source of information. Students were provided with information about these subject gateways to be pursued outside of the workshop.
It remained to be decided whether the presentation and exercises should focus on resources specific to this course and subject area or whether to take a broader, generic view. I t was decided to take the latter approach for a number of reasons:
- The presenters were not specialists in the subject area;
- Complex subject-specific examples and exercises might distract from the lessons to be learned, i.e. taking the issues out of context was intentional;
- Good practice in terms of planning a search apply equally to all subject areas and is a key skill component of information literacy;
- Searching the Web is now a commonplace leisure activity for many outside of the work environment.
The session begins with a fairly lengthy presentation looking at the benefits and pitfalls of the Web as a repository and source of information and the range of tools and resources that attempt to catalogue it. Some insight into how these tools function and how they are best applied to different search tasks was believed to be important if the students are to make efficient use of them. The presentation finished by looking at some guidelines on assessing the worth of information recovered from the Web and pointed to some Web resources such as The Internet Detective that deal with this issue. No examples of evaluating quality were given and none of the exercises dealt explicitly with the quality of resources although it was an issue with all of them (see below).
Students are reminded that the Web is not merely data but also through the various communication protocols, a network of people; accessing people can be a very powerful way to locate information particularly if one is new to an area of study. However, it was not possible to explore this within the time-span of the workshop.
The participants are then given an exercise sheet with a number of search tasks. They are not given additional guidance as a class, beyond that in the presentation although several experienced Web searchers circulated about the room hinting at or suggesting approaches.
The tasks are designed to be fun and relatively easy to achieve in a short time but also for each to illustrate a number of aspect of search strategy. The tips and advice given in the presentation and summary handouts should assist the students to complete the exercises efficiently. Most of the students work in pairs or small groups.
At the end of the session the students are asked to reflect on their experiences by posting a brief summary to a Web based discussion list. 30 submissions were received.
Posting of formative assessments on module website for peer/staff review – write about a page on your impressions of the exercises that you carried out in class. For each of the exercises that you attempted, detail your strategy, the search tool that you chose (and why) and the queries that you used. Where you successful?
The question is intentionally open to encourage reflective thinking.
As we shall see, the power of second generation search engines and the very interlinked nature of the Web ensures some degree of success, or at least an impression of success, even with poor search strategies. However, well-planned and formulated searches will still ensure faster and, critically, more comprehensive coverage on any topic. To achieve this, it is essential that users understand the optimal application of the different types of tools and resources as well as the most efficient way to employ each of these.
As these students clearly appreciated:‘One of the most important things I learned from the workshop is that you need to investigate how each search engine and subject directory operates and how it interprets your requests. I also learned about the difference between search engines and subject directories and how they collect data, which is crucial knowledge when using the tools.’
‘Although I was already quite familiar with using search engines, I was unaware that you could use quotation marks to search for exact results or use NEAR in a search.? I was also unaware that the search engines were so different in the way they searched.’
In this article, by referring to and quoting from the submitted assignments, I examine how the students tackled the exercises and how much they benefited from the previous presentation. Overall the aim of the workshop is to increase the amount of time that students spent in planning and structuring their searches. I will try to determine if this was achieved with the Sociology group.
All resources used or referred to in the workshop including Powerpoint presentation and exercise sheets are available at: http://cap.warwick.ac.uk/searching/.
Overall the workshop was lively and although no formal workshop evaluation forms were used, most students indicated satisfaction in their assignments. A small number who felt that they already had a high level of competence with search engines were dissatisfied, feeling that the level was far too low, deceived perhaps by the apparent simplicity of the exercises. It is not, however, entirely clear from the assignments whether this confidence reflected competence. A larger number suggested that examples and exercises specific to the course could have been better.
‘In conclusion, this web assessment provided some useful information, however it could have been improved by gearing the exercises more towards the researching society course or particular subject areas.’
‘ It might have been more advantageous if the exercise search areas had been linked to our course. However it is clear that the strategies that we have been shown will be very useful for future Internet searches.’
A typical response was:
‘ Like most of us doing this exercise, I have been using the Internet and search engines regularly for at least three years. While the class on Internet searches refreshed my memory, I’m afraid I didn’t really learn anything new from it.’
Most of the students were new to using Boolean expressions and other command line filters in searches
‘The most useful information in the session for me was about creating a search statement, as I was not familiar with the technique using signs, such as +, -, “”, AND and OR. General information of the Internet and website was also interesting.’
‘In tackling the following exercises I found the information given during the presentation regarding creating a search statement particularly useful.’
Most of the students had used a search engine before. Despite being introduced to a wide range of search tools and resources, most of the students chose the search engine they were most familiar with – almost exclusively Google - and stuck with that tool for all the exercises. Despite advice to the contrary by search experts, this is in fact what most of us do. It could be that they perceive the workshop was too short to allow them to explore new tools. Most were, however, able to achieve their goals without resorting to a new tool so it may be that some of the exercises should have been designed to fail completely in Google (see Exercise 1).
‘For most of the exercises I used google.com, which is the same search engine I used before the workshop.’
Ask Jeeves” was used by some usually with limited success, it seems largely since its mode of operation was ill-understood. Future workshops will use exercises designed to illustrate the differences from other search engines. '
Choice and construction of Search Tool
The following quote is probably typical of the way most of the students used search engines before and after the workshop.
‘The strategy that I generally used was that of typing many key words into the search engine named ‘Google,’ among others such as Yahoo and AOL.’
‘The guidance given in the presentation assisted my search strategies as, with previous untutored usage, I usually just typed one item into the search engine; I did not use AND or NEAR.’
Despite the detailed look at syntax in the presentation and handouts reinforcing this, most of the students still used incorrect search terms such as
yoga + back pain
when what was intended was probably
+yoga +”back pain”
such terms will return some useful sites but do not achieve exactly what is intended. Without understanding the correct syntax it is not very clear to the user that this is the case. Google will interpret this term as:
+yoga +back +pain
which is similar to, but not exactly, what is intended.
Search engines like Google, due to the default/implied Boolean terms it employs are fairly forgiving. Badly arranged terms will usually return a good selection of relevant sites, but the list (and order of match) will not always be the same list that would have been returned had the terms been compiled correctly.
Evaluating the quality of results
It was not possible within the time available for the students exhaustively to evaluate the usefulness and fitness for purpose of the resources they found. Most did attempt to place a value on their results but some merely reported having found a list of sites and disappointingly did not refer back to the aim stated in the exercise.
At best, perhaps the impact of the workshop was expressed by this student:
‘Having used the Internet for a few years, I approached the workshop on web resources with some experience of conducting searches. However, I was by no means an expert. Essentially, I learnt that my previous search strategies constituted a limited means of exploring the vast amount of information available. By selecting a good search engine, utilising specific terms and Boolean operators I found that one can hone in on required information in a way that is both efficient and effective. I also became aware of additional resources such as subject directories and subject gateways, which can be more relevant when searching on academic topics. The guidance given during the presentation thus proved to be invaluable since I now feel confident to refine my searches and get more out of the Internet.’
Only one student was very critical of the workshop:
‘Being honest, the guidance in the presentation did not really aid me. To begin with, I already knew how to use search engines. Secondly, a lot of the presentation was on the theory behind search engines or the different types one could use. This is of very little relevance as most people are going to use Google.‘
Construction of search terms
The students did make good use of Boolean expressions, which was new to most of them, although many were sloppy about the syntax and, due to the forgiving nature of Google, quite unaware of this. Overall the students only began to appreciate the guidance given in the presentation after making mistakes in the exercises. Some struggled.
Assessing quality of resources
‘The workshop never the less made me more aware that there is actually some information on the web that can be useful for an academic purpose, even though I am still sceptical of the quality of most web pages since most of them do not go through a review process.’
‘Another relevant point to mention is that I was not sure whether or not to trust some of the things I saw on the web, not necessarily relating to the successfully accomplished exercises I engaged in. Also, I noticed that some of the sites that appeared in searches were absolutely nothing like those I was looking for. These are all important issues I have dealt with before, but it is interesting to think about how it may relate to future sociologically-related searches I do on the web.’
‘However, I believe that more could have been said about validating web resources and on specifically how to upload data (since this was the ultimate object of the assessment).’
The following links give details of each exercise and aims and examples of how the students tackled them.
Did the students become more reflective in their searching?
It seemed that those with little experience of Web searching absorbed more from the presentation. Perhaps it is all too easy to fall back on established practice especially if those practices work even when they are not optimal. However, the following quotes were encouraging:
‘This was an interesting set of tasks, although I knew how to use search engines, being given some instruction on how to define what you are searching for was particularly useful and I have already used what I learnt!’
‘I found the Internet task very useful because the importance of thinking before embarking upon a search upon was highlighted. Since the exercise, I have saved lots of time that would have been spent sifting out useless information by simply thinking about what I want from the internet before I use a search engine.’
How should the workshop change?
- More difficult and perhaps subject related exercises for confident users.
- Exercises that demonstrate the uses of different search tools such as ‘Ask Jeeves’
- Subject specific examples to examine methods of evaluating the quality of resources.