Test Item Analysis for English Teachers:
Applying Classical Test Theory using Excel

Workshop Facilitators:

Chihiro Inoue, CRELLA, University of Bedfordshire, chihiro.inoue(a)beds.ac.uk
Sathena Chan,CRELLA, University of Bedfordshire, sathena.chan(a)beds.ac.uk
Carolyn Westbrook,Southampton Solent University, carolyn.westbrook(a)solent.ac.uk

Summary of workshop:

This workshop aims to provide foreign language teachers with hands-on sessions on conducting basic item analysis using classical test theory (CTT). Using sample data sets in Excel spreadsheets, various functions will be introduced to calculate descriptive statistics (e.g., frequency tables, histograms, central tendency and dispersion) and reliability, as well as to conduct item analysis (e.g., facility values and discrimination, i.e. point biserial, distractor analysis).

Along with the hands-on session, lecture sessions are also given (using sample items from different item types) to raise awareness in test design, item writing, scoring methods and inputting data into spreadsheets. The intended learning outcomes are:
- An understanding of key concepts in language testing
- An understanding of the thinking behind item analysis in CTT
- An understanding of the functions for item analysis in Excel.

By using Excel, this workshop can provide English teachers with more accessible means for conducting their own item analysis in CTT. Teachers will also benefit from being able to take home the Excel templates for their own future use.

Provisional schedule and contents:

Day 1
14.00 – 15.30 Introducing types of tests and testing / Item types and scoring methods / Guidelines for writing good items
15.45 – 17.15 Entering data into Excel

Day 2
9.00 – 10.30 Calculating and displaying descriptive statistics using graphs
10.30 – 10.45 BREAK
10.45 – 12.15 Calculating facility values, discrimination and distractor analysis
12.15 – 13.30 LUNCH
13.30 – 15.00 Calculating reliability (1) Cronbach’s Alpha
15.00 – 15.15 BREAK
15.15 – 16.45 Calculating reliability (2): Rater reliability / Guidelines to make tests more reliable

Day 3
9.00 – 10.30 Improving test items that are identified as problematic
10.30 – 10.45 BREAK
10.45 – 12.00 Interpreting the results to feed back to teaching and future tests / Round-up and close

Prior knowledge required:

- Basic knowledge about Excel (i.e. what it looks like and how text / numerical data can be entered)
- Some experience of writing tests and reporting scores at their school
All information (handouts and datasets) will be provided during the workshop. However, if participants have data sets from tests they have run, they can use those to practise the activities we will be doing or for additional practice between sessions.

Please bring your laptop, with Microsoft excel installed (part of the Office package).

 

Bio-data of Workshop Facilitators:

	Chihiro Inoue is a post-doctoral research fellow at Centre for Research in English Language Learning and Assessment (CRELLA), University of Bedfordshire. She has a PhD in Language Testing from Lancaster University in 2011, and her main research interests lie in the task design of speaking tests and criterial features of learner language. She has been involved in various projects in the development and validation of language tests for testing organisations in Japan, USA and UK, and is experienced in carrying out workshops on statistical analysis using Excel and SPSS for English teachers and postgraduate students. Chihiro also has taught English at senior high school, college, and university levels in Japan. She holds an MA in TEFL from Tokyo University of Foreign Studies and a PG Cert in Teaching EAP from Sheffield Hallam University.
	Sathena Chan is a post-doctoral research fellow at Centre for Research in English Language Learning and Assessment (CRELLA). She holds a PhD in Language Testing from the University of Bedfordshire and a MA in Language Testing and Programme Evaluation from the University of Essex. Her major areas of research include reading-into-writing assessments, test development and validation, and cognitive processing of language use. She is experienced in statistical analysis for both language test development and academic research. She is proficient in using statistics software such as Excel, SPSS and Winsteps. She teaches postgraduate modules of Research Methods and Language Testing, and conducts SPSS workshops for postgraduate students in the UK. She is the principal investigator for a few test validation research studies and test revision projects for examination boards and educational organisations in the UK and worldwide.
	Carolyn Westbrook is a Senior Lecturer in EFL at Southampton Solent University. She is currently studying for a PhD in language testing at the University of Bedfordshire and her main areas of research are listening-into-writing in EAP tests, design and validation of in-house tests for EAP at university, and plagiarism and international students. She has been involved in a number of small in-house test validation projects for university entrance examinations.

 

Please bring your laptop, with Microsoft Excel installed.

Venue:
Ramphals Building
Room R1.15

Online Registration:
Costs: £65
Registration is on a first-come first-serve basis, so please book early to avoid disappointment.
We will accept 30 participants.
To register click here.

Accomodation:
Warwick Conferences will hold rooms for you at a special price.
A link for booking will be provided in due course.