Measuring staff attitude to an automated feedback system based on a computer adaptive test

In Higher Education today, increasing reliance is being placed upon the use of online learning and assessment systems. Often these are used to manage learning, present information and test learners in an entirely undifferentiated way, all users having exactly the same view of the system. With the development of increasingly large and complex computer applications and greater diversity in learner groups, consideration of individual differences and greater efficiency in learning and testing have become important issues in designing usable and useful applications. Our initial findings, reported at CAA 2005, suggested that students valued this approach to providing automated feedback and considered it to be a fast, effective and reliable method. In the study presented in this paper, the attitude of staff to our automated feedback tool is presented. Three presentation sessions involving more than 80 university lecturing staff were undertaken and their views of the feedback tool were captured using video recordings. Initially a small group of computer scientists took part in a short presentation followed by a discussion where they presented their views on the CAT approach, the adaptive nature of the system and the provision of feedback. The second study involved a presentation and feedback session with more than 50 lecturers from all sectors of the university who provided their opinions of the approach in general. A short questionnaire was administered at the end of this session. The results of this, which broadly support our approach to automated feedback, are presented in this paper. A third study is reported, which involved 20 lecturers with special interests and roles in online and blended learning within the university. Subsequent analysis of the sessions using qualitative data analysis methods showed that teachers in general were receptive to the idea of automated feedback based on CAT. Several interesting ideas arose from the discussions, which are presented here. Computer based testing and automated feedback are becoming increasingly important in Higher Education. It is important that the views of teachers are considered when developing and implementing such systems if they are to be accepted and hence effective.