The methods presented here are aimed at staff who need to perform basic statistical analyses on data from sample surveys. The focus is on the post-course ‘student evaluation questionnaire’, which is widely used in small scale evaluation of educational developments. However, in order to enhance traditional practices, we would argue that a range of methods be considered for collecting information, from highly structured methods, such as self-completed questionnaires typically used for post-course student feedback, to those with little or no structure, such as in-depth interviews and focus groups.
- face-to-face interviewing
- postal surveys
- telephone surveys
- direct observation.
There are three key stages involved in survey research, planning, design of instruments and analysis. Much of the work in using surveys is therefore ‘up front’. In thinking about using surveys, you might consider:
Critical appraisal of survey research
- What sort of evidence surveys provide
- The need for and importance of written protocols
Sources of questions
- note that evaluation questions and survey questions are not the same thing
- get feedback on questions from typical participants to test out phrasing and jargon
- make use of pre-existing questionnaires used for similar purposes to save time and effort
What constitutes representative sampling in surveys
- Use of standardised instruments
- Response rates and numbers
- How to design questionnaires that minimise bias
Statistical analysis of survey data
- Quantitative statistics
- Qualitative analysis
Presentation of survey results
- who are the target audiences/stakeholders for your findings.
For a rehearsal of these, a good starting point is Colin Robson’s book, Small-Scale Evaluation.