Academic Performance Prediction Models Based on Multi-Head Attention LSTM Mechanisms
Downloads
This study focuses on accurately predicting student academic performance to support personalized teaching and optimize educational resource allocation. Addressing the limitations of traditional prediction methods in handling long-term dependencies within time-series data and extracting key features, this paper proposes an innovative model that integrates multi-head attention mechanisms with Long Short-Term Memory (LSTM) networks. The method utilizes LSTMs to efficiently process sequential data while employing multi-head attention mechanisms to concentrate on critical information synergistically. Experiments demonstrate that this model achieves significantly higher prediction accuracy than traditional methods on specific datasets, with an MAE of 4.12 and an R² of 0.94, fully showcasing its outstanding performance. This model pioneers new pathways for academic performance prediction in education, supporting scientific educational decision-making and driving high-quality educational development.
Downloads
[1] Singh, A. K., & Kumar, A. (2025). Multi-objective: hybrid particle swarm optimization with firefly algorithm for feature selection with Leaky ReLU. Discover Artificial Intelligence, 5(1), 192. doi:10.1007/s44163-025-00428-0.
[2] Chen, M., Li, Y., Li, N., Zhong, Y., & Guo, G. (2024). Dangerous attack paths analysis for power networks based on adaptive limited depth search and improved Z-score pruning. Ain Shams Engineering Journal, 15(12), 103135. doi:10.1016/j.asej.2024.103135.
[3] Prasath, N. A. (2025). A Multi-Source Deep Learning Model Utilizing Campus Data to Enhance Early Academic Performance Prediction. Mathematical Modelling of Engineering Problems, 12(4), 1311–1320. doi:10.18280/mmep.120422.
[4] Jia, C., He, H., Zhou, J., Li, K., Li, J., & Wei, Z. (2024). A performance degradation prediction model for PEMFC based on bi-directional long short-term memory and multi-head self-attention mechanism. International Journal of Hydrogen Energy, 60, 133–146. doi:10.1016/j.ijhydene.2024.02.181.
[5] Li, F., Ma, G., Ju, C., Chen, S., & Huang, W. (2024). Data-driven forecasting framework for daily reservoir inflow time series considering the flood peaks based on multi-head attention mechanism. Journal of Hydrology, 645(Part B), 19. doi:10.1016/j.jhydrol.2024.132197.
[6] Ahmad, K., Iqbal, W., El-Hassan, A., Qadir, J., Benhaddou, D., Ayyash, M., & Al-Fuqaha, A. (2023). Data-driven artificial intelligence in education: A comprehensive review. IEEE Transactions on Learning Technologies, 17, 12-31. doi:10.1109/TLT.2023.3314610
[7] Liu, Q., Wang, J., Dai, H., Ning, L., & Nie, P. (2025). Bridge Structural Damage Identification Based on Parallel Multi-head Self-attention Mechanism and Bidirectional Long and Short-term Memory Network. Arabian Journal for Science and Engineering, 50(3), 1803–1815. doi:10.1007/s13369-024-09035-0.
[8] Wang, J., Zhang, D., Huang, Q., & Cui, Z. (2025). Multiple-step accurate prediction of wave energy: A hybrid model based on quadratic decomposition, SSA and LSTM. International Journal of Green Energy, 22(1), 100–123. doi:10.1080/15435075.2024.2406849.
[9] Abín, A., Núñez, J. C., Rodríguez, C., Cueli, M., García, T., & Rosário, P. (2020). Predicting Mathematics Achievement in Secondary Education: The Role of Cognitive, Motivational, and Emotional Variables. Frontiers in Psychology, 11. doi:10.3389/fpsyg.2020.00876.
[10] Xu, F., & Qu, S. (2021). Data mining of students’ consumption behaviour pattern based on self-attention graph neural network. Applied Sciences (Switzerland), 11(22), 10784. doi:10.3390/app112210784.
[11] Alturki, S., Alturki, N., & Stuckenschmidt, H. (2021). Using Educational Data Mining to Predict Students’ Academic Performance for Applying Early Interventions. Journal of Information Technology Education: Innovations in Practice, 20, 121–137. doi:10.28945/4835.
[12] Wang, Y., & Lewis, S. E. (2022). Towards a theoretically sound measure of chemistry students’ motivation; Investigating rank-sort survey methodology to reduce response style bias. Chemistry Education Research and Practice, 23(1), 240–256. doi:10.1039/d1rp00206f.
[13] Li, G., & Gao, W. (2022). Achievement Prediction of English Majors Based on Analytic Hierarchy Process and Genetic Algorithm. Mobile Information Systems, 2022, 6542300 1–6542300 10. doi:10.1155/2022/6542300.
[14] Gatzka, T. (2021). Aspects of openness as predictors of academic achievement. Personality and Individual Differences, 170. doi:10.1016/j.paid.2020.110422.
[15] Hamdani, M. (2021). Predicting Academic Achievement of Students: A Kolb’s Learning Styles Approach. Journal of Educational, Health and Community Psychology, 10(4), 623–641.
[16] Véronneau, M. H., Vitaro, F., Poulin, F., Ha, T., & Kornienko, O. (2025). Academic Achievement, Externalizing Problems, and Close Friends in Middle School: Testing a Developmental Cascade Model Leading to Educational Attainment in the Late Twenties. Journal of Youth and Adolescence, 54(6), 1489–1505. doi:10.1007/s10964-025-02143-6.
[17] Zhang, X., & Yue, J. (2024). Predictive Modeling of Student Performance through Classification with Gaussian Process Models. International Journal of Advanced Computer Science and Applications, 15(6), 1214–1227. doi:10.14569/IJACSA.2024.01506123.
[18] Meruelo, A. D., Castro, N., Nguyen-Louie, T., & Tapert, S. F. (2020). Substance use initiation and the prediction of subsequent academic achievement. Brain Imaging and Behavior, 14(6), 2679–2691. doi:10.1007/s11682-019-00219-z.
[19] Brzozka, B. (2025). Machine Learning Algorithms in Predicting College Students’ Grades: A Review. Journal of Applied Automation Technologies, 3, 1–12. doi:10.64972/jaat.2025v3.1.
[20] Nuankaew, P., & Nuankaew, W. S. (2022). Student Performance Prediction Model for Predicting Academic Achievement of High School Students. European Journal of Educational Research, 11(2), 949–963. doi:10.12973/EU-JER.11.2.949.
[21] Eltayar, A., Aref, S. R., Khalifa, H. M., & Hammad, A. S. (2023). Prediction of Graduate Learners’ Academic Achievement in an Online Learning Environment Using a Blended Trauma Course. Advances in Medical Education and Practice, 14(000), 137–144. doi:10.2147/AMEP.S401695.
[22] Yildiz, M., & Börekci, C. (2020). Predicting Academic Achievement with Machine Learning Algorithms. Journal of Educational Technology and Online Learning, 3(3), 372–392. doi:10.31681/jetol.773206.
[23] Willems, J., van Daal, T., van Petegem, P., Coertjens, L., & Donche, V. (2021). Predicting freshmen’s academic adjustment and subsequent achievement: Differences between academic and professional higher education contexts. Frontline Learning Research, 9(2), 28–49. doi:10.14786/flr.v9i2.647.
[24] Bird, K. A., Castleman, B. L., & Song, Y. (2025). Are algorithms biased in education? Exploring racial bias in predicting community college student success. Journal of Policy Analysis and Management, 44(2), 379–402. doi:10.1002/pam.22569.
[25] Zhang, Z., Deng, Q., He, W., & Yang, C. (2024). A New Method Based on Belief Rule Base with Balanced Accuracy and Interpretability for Student Achievement Prediction. Mathematics, 12(20), 3283. doi:10.3390/math12203283.
[26] Wallace, E. R., Ola, C., Leroux, B. G., Speltz, M. L., & Collett, B. R. (2021). Prediction of school age IQ, academic achievement, and motor skills in children with positional plagiocephaly. Paediatrics and Child Health (Canada), 26(3), E132–E137. doi:10.1093/pch/pxaa012.
[27] Pérez-González, J. C., Filella, G., Soldevila, A., Faiad, Y., & Sanchez-Ruiz, M. J. (2022). Integrating self-regulated learning and individual differences in the prediction of university academic achievement across a three-year-long degree. Metacognition and Learning, 17(3), 1141–1165. doi:10.1007/s11409-022-09315-w.
[28] Kumari, D. N. (2023). Contributory role of self-esteem, academic achievement and vocational aspirations in prediction of career attitude. International Journal of Arts, Humanities and Social Studies, 5(1), 107–110. doi:10.33545/26648652.2023.v5.i1b.54.
[29] Kawakami, K., Procopio, F., Rimfeld, K., Malanchini, M., von Stumm, S., Asbury, K., & Plomin, R. (2024). Exploring the genetic prediction of academic underachievement and overachievement. NPJ Science of Learning, 9(1), 39. doi:10.1038/s41539-024-00251-9.
[30] Corredor Valderrama, M. C., de Ruiz Sandoval, D. S., & Gómez, S. M. (2025). Prediction of Academic Achievement of University Students: Learning Styles and Strategies. Electronic Journal of Research in Educational Psychology, 23(1), 51–72. doi:10.25115/ZM2Q9N35.
[31] Iines R, P., Sami J, M., Vesa M, N., & Hannu K, S. (2023). ADHD symptoms and maladaptive achievement strategies: the reciprocal prediction of academic performance beyond the transition to middle school. Emotional and Behavioural Difficulties, 28(1), 3–17. doi:10.1080/13632752.2023.2189404.
- This work (including HTML and PDF Files) is licensed under a Creative Commons Attribution 4.0 International License.
