Partial credit in mathematics exams - a comparison of traditional and CAA exams

With the growing trend in many subjects to deliver at least some part of examinations by computer, it is important to know whether there are any differences in the results obtained by candidates sitting examinations taken by computer compared to those obtained by candidates sitting conventional examinations using pen and paper. The purpose of this paper is to describe a pilot project to compare the traditional type of assessment with assessment done by computer in mathematics examinations and in particular to investigate the role of partial credit in these examinations. In paper based examinations full marks are awarded for a completely correct answer. If, however, a student obtains an incorrect answer but gets some parts of the working correct then in mathematics examinations partial credit is normally awarded. In a computer examination an incorrect answer to a question is normally awarded no marks with no consideration of any partial credit. The mechanism for giving partial credit in the computer examinations of this project was to break the question down into Steps. The project compared results of students taking computer tests in three different formats (either no Steps, compulsory Steps or optional Steps) and the partial credit they would have obtained by taking the corresponding examinations on paper. The tests were at the level of Scottish Higher school examinations and were taken by school students who were about to sit their Higher examinations. This level was chosen as it was high enough to test the students on strategy and mathematical working, while the questions were not too long so that a clearer analysis of the results was possible.