Sistem Chatbot Berbasis Large Language Model (LLM) untuk Materi Pembelajaran di Program Studi Teknologi Informasi dengan Metode Retrieval-Augmented Generation (RAG)

Abstract

This study aims to develop a chatbot system based on a Large Language Model (LLM) that implements the Retrieval-Augmented Generation (RAG) method as a learning support tool for courses in the Information Technology Study Program. The system is designed to provide accurate and contextual answers to student questions based on learning documents from ten courses, which have been converted into text format. The integration of The LLM model from Cohere with embedding techniques and vector storage using FAISS is applied to improve the accuracy of information retrieval. The system is implemented using the Flask framework with an HTML and CSS-based user interface. System evaluation includes testing the chatbot's responses to 20 questions using qualitative analysis and quantitative metrics such as ROUGE (ROUGE-1, ROUGE-2, and ROUGE-L). The evaluation results show that the chatbot is able to provide relevant and informative responses, with the highest average F1-score of 57.58% on ROUGE-1. In terms of response time, the chatbot responds within a range of 5.75 to 19.85 seconds. Overall, the system demonstrates that the chatbot can serve as a useful tool in supporting student learning. Further development may include expanding the scope of materials, improving model quality, and adding additional support features.