Leveraging Ensemble Model and Optimized Feature Selection to Boost Prediction Accuracy in Educational Data Mining

Leveraging Ensemble Model and Optimized Feature Selection to Boost Prediction Accuracy in Educational Data Mining

연구 분야: Databases

학회: SN Computer Science

In contemporary research, educational data mining (EDM) has become a captivating field for data mining and machine learning experts, focusing on identifying factors influencing students' academic performance and predicting the likelihood of students dropping out. To uncover these influential factors, feature selection methods are employed, while various machine learning models are used to predict students at risk of underperforming. Filter-based feature selection methods are commonly used in educational data mining due to their efficiency and ability to rank important features affecting academic success. However, because of their independence from classifiers and relying on a fixed threshold or predefined feature count, filter-based methods can sometimes negatively affect model performance. To address this, the present study introduces an optimized chi-square-based feature selection technique that dynamically selects the optimal features for each learning algorithm, ensuring that model performance is not compromised. The effectiveness of five classifiers—k-Nearest Neighbour (k-NN), Decision Tree (DT), Naïve Bayes (NB), Support Vector Machine (SVM), and Logistic Regression (LR)—has been evaluated using three configurations: no feature selection, traditional chi-square feature selection, and proposed optimized chi-square based feature selection. These evaluations were conducted on two distinct student datasets, one from secondary schools (DS1) and another from engineering institutions (DS2). The results demonstrated that the optimized chi-square method consistently improved prediction accuracy across all classifiers. Additionally, a bagging-based ensemble classifier, constructed using the best-performing individual classifier, further enhanced predictive performance. The highest accuracies achieved were 94.62% for DS1 and 96.36% for DS2, outperforming traditional feature selection and ensemble methods. This study presents a scalable, reliable, and stable approach to student performance prediction, integrating optimized feature selection with ensemble learning.