Loughborough University
Leicestershire, UK
LE11 3TU
+44 (0)1509 263171
Loughborough University

Loughborough University Institutional Repository

Please use this identifier to cite or link to this item: https://dspace.lboro.ac.uk/2134/17243

Title: The problem of assessing problem solving: can comparative judgement help?
Authors: Jones, Ian
Inglis, Matthew
Keywords: Assessment
Problem solving
Comparative judgement
Issue Date: 2015
Publisher: © Springer Verlag
Citation: JONES, I. and INGLIS, M., 2015. The problem of assessing problem solving: can comparative judgement help? Educational Studies in Mathematics 89(3), pp.337-355.
Abstract: School mathematics examination papers are typically dominated by short, structured items that fail to assess sustained reasoning or problem solving. A contributory factor to this situation is the need for student work to be marked reliably by a large number of markers of varied experience and competence. We report a study that tested an alternative approach to assessment, called comparative judgement, which may represent a superior method for assessing open-ended questions that encourage a range of unpredictable responses. An innovative problem solving examination paper was specially designed by examiners, evaluated by mathematics teachers, and administered to 750 secondary school students of varied mathematical achievement. The students’ work was then assessed by mathematics education experts using comparative judgement as well as a specially designed, resourceintensive marking procedure. We report two main findings from the research. First, the examination paper writers, when freed from the traditional constraint of producing a mark scheme, designed questions that were less structured and more problem-based than is typical in current school mathematics examination papers. Second, the comparative judgement approach to assessing the student work proved successful by our measures of inter-rater reliability and validity. These findings open new avenues for how school mathematics, and indeed other areas of the curriculum, might be assessed in the future.
Description: This definitive version of this paper is available at Springerlink: http:dx.doi.org/10.1007/s10649-015-9607-1
Sponsor: This work was supported by a Royal Society Shuttleworth Research Fellowship to IJ a Royal Society Worshipful Company of Actuaries Research Fellowship to MI, and the Nuffield Foundation.
Version: Accepted for publication
DOI: 10.1007/s10649-015-9607-1
URI: https://dspace.lboro.ac.uk/2134/17243
Publisher Link: http:dx.doi.org/10.1007/s10649-015-9607-1
ISSN: 0013-1954
Appears in Collections:Published Articles (Mathematics Education Centre)

Files associated with this item:

File Description SizeFormat
final.pdfAccepted version6.79 MBAdobe PDFView/Open


SFX Query

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.