Pebbles in the Pond: Evaluation of the CAA Centre

This paper provides a summary of the evaluation of the Computer-assisted Assessment Centre TLTP project, which ran from September 1998 – October 2001. During it’s three-year life the Centre undertook a range of activities including co-hosted the International Computer-assisted Assessment (CAA) conferences with Loughborough University, researching good practice, and providing a range of institutional and national staff development workshops. As part of this work the Centre conducted an extensive survey of current practice and perceptions of CAA and ran a series of pilot projects at consortium institutions. A web-based resource was developed to support the uptake of CAA ,a series of newsletters ,papers and the Blueprint for CAA were published to help to create a community of practitioners. Ultimately, the Centre provided a national support service for CAA activity in the higher education sector. This paper will consider the extent to which the project aims have been achieved – both from an internal and external perspective. Throughout the project has been evaluated formatively by project members and practitioners. Formative evaluation activities have included: feedback from workshops; practitioner evaluation of the Blueprint for CAA; steering group meetings, surveys, interviews and focus groups. Between July 2001 and March 2002 the project has been evaluated summatively by an external evaluator. The focus of this evaluation has been to explore impact of the project on consortium institutions and the wider community. The evaluation has consisted of three main activities: analysis of project documentation; a focus group with project members, Management and Steering Groups; and a series of interviews with a range of staff at consortium and other institutions. The outcomes of the evaluation indicate that the project has had considerable impact, beyond the field of CAA. The success and failures of the project will be discussed. The evaluation has revealed insights about the potential of CAA, as well as identifying a number of issues concerning the future development of CAA and role of funded projects. This paper reports on selected highlights from the evaluation and synthesises some emergent themes. The conference presentation will compliment this and provide further in depth analysis of the results.