Extracting more meaning from CAA results using machine learning

This work describes a novel approach to the problem of extracting knowledge from the results obtained via a CAA system by adopting a Machine Learning paradigm. The basic idea guiding our research was to investigate the existence of association rules among the topics covered in a course. The data used came from the exams administered to the freshmen in electronic engineering attending the course of Foundation of Computer Science at the University of Ancona. Ten Multiple Choice Questions with four possible answers constituted an exam. Questions have been classified according to the topic covered in a taxonomy derived from the course syllabus. Each question has an absolute weight representing its relative importance inside the curriculum. The data have been filtered by removing low-end and high-end achievers to obtain a subset containing information free from border effects. Each questionnaire has been coded into a vector of features (one for each element of the questions’ taxonomy) representing the student’s answers (right, wrong, not given). The feature vectors are further classified with respect to the final score obtained by the student (poor, average or good) and analysed using C4.5, a classification system based on top-down induction of decision trees that allows generating production rules. We classified the generated rules into three categories: “straightforward”, “reasonable” and “unexplainable”. Rules are considered “straightforward” when they put in relation topics that we believe are related. “Reasonable” rules put in relation topics that although not being predictable by our experience, may be understood after a deeper analysis of the questions. “Un-explainable” rules put in relation topics that do not appear to be related in any way. A first interesting result of the method discussed is represented by the so-called “reasonable rules” that may be used to better tune the teaching of the topics that appear to be related.