Computer programs are - in principle - ideal subjects for automated assessment. Not only can the correctness of a program be measured, but also its quality. However, there is no single correct approach to the problem of assessing programming assignments, and different practitioners may adopt different strategies, depending on the specific aims and objectives of the course they are teaching, and on their own style and preferences.

Our interest is both technical and educational, and we have investigated how new technologies can be used in an assessment system in ways which will benefit the teaching and learning processes. Our pedagogic foundation is that automatic assessment should not constrain the teacher to present or deliver their assessment material in any given style.

Our research has resulted in the production of a software package (further information available from boss.org.uk) which deals with the whole process of managing a programming assignment. Students can submit their work securely, and teachers can automatically test students' work against defined criteria, including unit tests, using a simple interface for marking and delivering feedback. Support for software metrics and for unit testing is included, allowing for a rigorous software engineering based approach, and new technologies for support the complex administrative components of the process.

Investigators: Mike Joy and Nathan Griffiths

Publications

Where the text of publications has been made available online, please respect the following copyright statement:

"This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder."

M.S. Joy, N.E. Griffiths and R. Boyatt, "The BOSS Online Submission and Assessment System", ACM Journal on Educational Resources in Computing 5(3), pp. 1-28 (2005). (DOI 10.1145/1163405.1163407)

P-S. Heng, M.S. Joy, R. Boyatt and N.E. Griffiths, Evaluation of the BOSS Online Submission and Assessment System, Research Report RR-415, Department of Computer Science, University of Warwick, Coventry, UK (2005)

M.S. Joy, "Automating the Process of Skills-based Assessment", in Effective Learning and Teaching in Computing, ed. Alastair Irons and Sylvia Alexander, RoutledgeFarmer, pp. 76-85, 2004.

M.S. Joy and N.E. Griffiths, "Online Submission of Coursework - a Technological Perspective", Proceedings of the 4th IEEE International Conference on Advanced Learning Technologies (ICALT2004), Joensuu, Finland, pp. 430-434 2004. (DOI 10.1109/ICALT.2004.1357451)

M.S. Joy and N.E. Griffiths, "The BOSS Online Submission System", Forum 26, The University of Warwick, p. 15, 2004. (Web)

N.E. Griffiths and M.S. Joy, An Integrated Approach to Courseware", Proceedings of the World Federation of Engineering Organizations (WFEO) and American Society for Engineering Education (ASEE) Electronic Conference, 2003. (PDF)

S. Rawles, M.S. Joy and M. Evans, Computer-Assisted Assessment in Computer Science: Issues and Software, Research Report RR-387, Department of Computer Science, University of Warwick, Coventry, UK, 2002. (Web)

M.S. Joy, P.-S. Chan and M. Luck, "Networked Submission and Assessment", Proceedings of the 1st Annual Conference of the LTSN Centre for Information and Computer Sciences, LTSN-ICS, 2000. (PDF)

M. Luck and M.S. Joy, "A Secure On-line Submission System", Software - Practice and Experience 29(8), pp. 721-740, 1999. (DOI 10.1002/(SICI)1097-024X(19990710)29:8%3C721::AID-SPE257%3E3.3.CO;2-S)

M.S. Joy and M. Luck, "The BOSS System for On-line Submission and Assessment of Computing Assignments", Computer Based Assessment (Volume 2): Case studies in Science & Computing, ed. Dan Charman and Andrew Elmes, SEED Publications, University of Plymouth, pp. 39-44, 1998. (PDF)

M.S. Joy and M. Luck, "The BOSS System for On-line Submission and Assessment", Monitor: Journal of the CTI Centre for Computing 10 pp. 27-29, 1998. 

M.S. Joy and M. Luck, "Effective Electronic Marking for On-line Assessment", Proceedings of the 6th Annual Conference on the Teaching of Computing / 3rd Annual Conference on Integrating Technology into Computer Science Education (ITiCSE '98), Centre for Teaching Computing, Dublin City University; ACM SIGCSE 30(5) pp. 134-138, 1998. (DOI 10.1145/290320.283096)

M.S. Joy and M. Luck, "Software Standards in Undergraduate Computing Courses", Journal of Computer Assisted Learning 12(2), pp. 103-113, 1996. (PostScript)

M.S. Joy and M. Luck. "A User-Friendly On-line Submission System", Proceedings of the 4th Annual Conference on the Teaching of Computing, ed. Rory O'Connor and Sylvia Alexander, Centre for Teaching Computing, Dublin City University, pp. 92-95, 1996. (PostScript)

M. Luck and M.S. Joy, "Automatic Submission in An Evolutionary Approach to Computer Science Teaching", Computers and Education, 25(3), pp. 105-111, 1995. (DOI 10.1016/0360-1315(95)00056-9)

M.S. Joy and M. Luck, "On-line Submission and Testing of Programming Assignments", in Innovations in Computing Teaching, ed. J. Hart, SEDA, pp. 97-103, 1995. (PostScript)