Utilizing a realist evaluative research approach to investigate complex technology implementations: an e-learning lecture capture exemplar

Purpose - This study aims to investigate the effectiveness of a theory-driven realist evaluative research approach to better understand complex technology implementations in organisations. Approach – An institution wide e-learning implementation of Lecture Capture (LC), within a UK University, was chosen and a realist evaluation framework was used, tailored for educational technology. The research was conducted over 4, increasingly focused, evaluation cycles combining engagement analytics, user interviews and theory to refine what works (or does not work), for whom, in which contexts and why. Findings - Despite explicit demand and corresponding investment, overall student engagement is lower than expected. Increased student use appears linked to particular staff attitudes and behaviours and not to specific disciplines or course content. The main benefits of LC are; providing reassurance to the majority, aiding revision and understanding for the many, and enabling catch-up for the few. Recommendations for future research are based on some unexpected outcomes uncovered, including; evolving detrimental student behaviours, policy development based on technological determinism and future learner-centred system development for next-generation LC technologies. Practical implications – The realist approach taken, and evaluation framework used, can be adopted (and adapted) for future evaluative research. Domain specific reference models, categorizing people and technology, supported analysis across multiple contexts. Originality/value – This study responds to a call for more theory-based research in the field of educational technology. We demonstrate that a theory-driven approach provides real and practical recommendations for institutions and allows for greater insight into the political, economic and social complexity of technology implementation.