Integrating Large Language Models into Decision Support Systems for Objective Student Competition Participant Selection
DOI:
https://doi.org/10.62872/wfdsxn09Keywords:
Large Language Model , RAG, Sistem Rekomendasi, Kompetisi Mahasiswa, Kecerdasan Buatan, ScoutAbstract
The selection process for competition participants in the Informatics Engineering Department at Gorontalo State University is still carried out manually, potentially leading to subjectivity and suboptimal identification of deserving students. This study developed an artificial intelligence-based selection system using a Large Language Model (LLM) integrated with the Retrieval-Augmented Generation (RAG) approach. The developed system, named Scout, recommends competitions that match student profiles based on academic data, interests, experience, and achievements. The system evaluation used the Precision@K and Hit@K metrics to measure recommendation accuracy, and RAGAS to assess the quality of retrieval and chatbot responses. Test results showed that the Scout system obtained a Precision@3 score of 0.87 and a Faithfulness score of 0.91, indicating high recommendation relevance and factual consistency. Thus, the implementation of LLM and RAG has proven effective in increasing the objectivity and efficiency of the selection process and has the potential to become the basis for the development of an AI-based academic decision support system in higher education
Downloads
References
Arsyad, A. A., Mashud, M., & Sumardin, A., 2022. Implementasi Metode Agile Scrum Pada Sistem Informasi Akuntasi CV Tritama Inti Persada. Jurnal Ilmiah Sistem Informasi Akuntansi 2 (2), 82–87.
Bao, Y., Liu, Z., Wang, Y., & Zhang, M., 2025. Lost in sequence: Improving sequential recommendation with Large Language Models. arXiv preprint arXiv:2502.01234.
Chang, Y., et al., 2024. A Survey on Evaluation of Large Language Models. ACM Transactions on Intelligent Systems and Technology 15 (3), 1–45.
Deswanti, A. I., Novitasari, D., Asbari, M., & Purwanto, A., 2023. Pengaruh Tingkat Pendidikan dan Pengalaman Kerja terhadap Kinerja Karyawan: Narrative Literature Review. Journal of Information Systems and Management (JISMA) 2 (3), 34–40.
Efniasari, M., Wantoro, A., & Susanto, E. R., 2022. Pengembangan Sistem Informasi Pelayanan Kesehatan Berbasis Web Menggunakan Metode Scrum (Studi Kasus: Puskesmas Kisam Ilir). Jurnal Teknologi Dan Sistem Informasi 3 (3).
Es, S., James, J., Martin, L., Scelles, J., Rault, F., Blanquer, T., Raj, A., & de la Torre, D., 2023. RAGAS: Automated Evaluation of Retrieval Augmented Generation. arXiv preprint arXiv:2309.15217.
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., & Wang, H., 2023. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint arXiv:2312.10997.
Karyadi, B., 2023. Pemanfaatan Kecerdasan Buatan Dalam Mendukung Pembelajaran Mandiri. Educate: Jurnal Teknologi Pendidikan 8 (02), 253–258.
Lewis, P., et al., 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems 33, 9459–9474.
Nugroho, B. I., Santoso, N. A., & Murtopo, A. A., 2023. Prediksi Kemampuan Akademik Mahasiswa dengan Metode Support Vector Machine. Remik 7 (1), 177–188.
Rachmat, N., & Kesuma, D. P., 2024. Implementasi LLM Gemini Pada Pengembangan Aplikasi Chatbot Berbasis Android. Jurnal Ilmu Komputer (JUIK) 4 (1), 40–52.
Sutiyo, F. R. A., Harahap, N. S., Agustian, S., & Candra, R. M., 2024. Implementasi Question Answering Berbasis Chatbot Telegram Pada Tafsir Al-Jalalain Menggunakan Langchain dan LLM. KLIK: Kajian Ilmiah Informatika Dan Komputer 4 (5), 2464–2472.
Widodo, A., 2023. Strategi Seleksi Mahasiswa dalam Kompetisi Akademik: Studi Kasus pada Perguruan Tinggi di Indonesia. Jurnal Pendidikan dan Pengembangan Sumber Daya Manusia 12 (3), 145-160.
Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G., 2024. Expel: Llm agents are experiential learners. Proceedings of the AAAI Conference on Artificial Intelligence 38 (17), 19632–19642.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Eka V. Dangkua, Fazrul Anugrah Sahi, Nikmasari Pakaya, Indhitya R. Padiku, Muchlis Polin, Rahmat Taufik R. L Bau (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
