Development and Evaluation of a Learner Behavior Model for a Learning Management System

Abstract

Most Learning Management Systems (LMS) lack automated components that analyze data and generate information on learner behavior. Majorly traditional manual tools and methods such as the administration of questionnaires relating to a specific learning style and cognitive psychometric tests have been used to identify specific behavior. The problem with such methods is that a learner can give inaccurate information, is time-consuming, and is prone to errors. While there are existing automated models as reviewed in literature predicting learning style and cognitive traits, most of them are based on single behavior and tested on specific learning platforms. The primary objective of this study was to design, develop and evaluate a model complementing Learning Styles with psychology-based ones such as Cognitive Traits. An automatic model based on Felder-Silverman Learning Style Model and Cognitive Trait Model was designed. Approximately 200,000 log records of 389 students who had accessed e-Learning courses for a 15-week semester were extracted from LMS to create a dataset. From this, a prototype estimating learning behavior based on the two theories was developed and evaluated with students in a classroom environment. The model can estimate learning behavior and display relevant learning content matching the learning styles and cognitive traits of an individual student. The evaluation of the model done using Kappa statistics demonstrated that the inter-rater reliability results were moderately in agreement with the traditional psychometric methods. Further, machine learning methods involving artificial neural networks and cluster algorithms were used to analyze the dataset collected from the log records. The artificial neural network model was trained to predict learner behavior and classify each student. The prediction success rate of 0.63, 0.67, 0.64, 0.65, 0.26, and 0.64 for accuracies, precision, recall, f-score, kappa, and Area Under the Curve (AUC) respectively were recorded. On the other hand, the cluster evaluation analysis showed positive silhouette coefficients for all clusters indicating that all the students with similar browsing behavior had been assigned to the right clusters matching their learning styles and cognitive traits. Cumulatively, these findings demonstrate that students with similar content navigation behavior share common learning styles and cognitive traits. These results suggest that it is possible to automatically estimate the learning styles and cognitive traits of a learner in a learning management system with fair accuracy. This research brings forth a generic modeling architecture that developers can integrate with existing learning management system platforms to improve learner characterization. Furthermore, a course lecturer can use the information generated by the model to provide learning materials matching identified characteristics for each student and also apply appropriate teaching methods.