Linking on-line assessment in mathematics to cognitive skills

The project, undertaken in 2000/2001, investigated the ability of on-line assessment to test the skills in Bloom’s cognitive domain (Bloom et al, 1956) in the Scottish Qualifications Authority (SQA, 2002) Advanced Higher in Mathematics. There were three main sources of data. 1. The course information: The SQA learning outcomes, content detail and performance criteria. 2. The assessments: The National Assessment Bank questions (NABs), SQA specimen exam papers and the SCHOLAR project (Paterson, 2001; SCHOLAR 2002) on-line assessments. 3. The computer software and data: The online questions and features of assessment engines. The intention was to match the cognitive skills required in the learning outcomes with those tested in the questions through examination of the key verbs used in each. It was envisaged that the limitations of using one assessment system could be addressed by focussing on features that were available elsewhere. Many difficulties arose and led to a complete rewrite of the on-line questions with closer reference to the performance criteria, the paper based specimen exams, the features of on-line assessment systems and the marking scheme used. This paper addresses: a) the subjective nature of the choice of which cognitive skills are expected to be tested in each learning outcome b) the use of cognitive skills to identify levels of learning in Mathematics c) the use of key verbs in Mathematics to identify the cognitive skills d) the approach taken of trying to convert paper based systems for on-line use e) the features and limitations of existing on-line assessment in producing summative assessment f) the need for further development in particular areas to increase the scope of summative on-line assessment