However, for illustrative purposes, we used example datasets available at ’https://startg2545.github.io/item_tutorial.csv’ for item data and ’https://startg2545.github.io/user_item_tutorial.csv’ for user data. During model evaluation, we emphasized the importance of splitting the dataset into training and testing sets. Commonly used ratios for this split include 60:40, 70:30, and 80:20. The training set is used to train the model to recognize patterns in course descriptions, while the testing set is used to evaluate the model’s performance without bias.
We evaluated the model’s performance using metrics such as hit rate and F1 score. The hit rates achieved were 14.69% ,48.24% , and 69.18% for the first approach, second approach, and hybrid approach, respectively. Similarly, the F1 scores were 2.58% , 7.92% , and 11.62% for the same approaches, indicating the accuracy of our recommendation system across different methodologies. By incorporating these evaluation metrics, we gained insights into the effectiveness of our recommendation system in providing personalized course recommendations, contributing to the overall success of our project.

6. EXPERIMENTATION AND RESULTS

In this project, I integrate insights from diverse studies to develop a recommendation system grounded in comprehensive research. The system is designed to recommend the most suitable courses for learners who have previously taken at least one course. It offers personalized recommendations, conducts thorough user data analysis to enhance accuracy, and contributes to improving the overall learning experience. However, it’s important to note that the system’s applicability is currently limited to online learning platforms that adhere to specific data formats required for implementation.
6.1 First Approach Analysis
The first approach has relatively low accuracy in both Hit Rate and F1 Score, indicating that the model’s performance in recommending relevant courses to users is not very effective. A Hit Rate of 14.69%means that only about 14.69% of the recommended courses match with the actual courses users take. Similarly, an F1 Score of2.58% suggests poor precision and recall, further highlighting the limitations of this approach in accurately predicting user preferences.
6.2 Second Approach Analysis
The second approach shows improved accuracy compared to the first approach, with a Hit Rate of 48.24% and an F1 Score of7.92% . This indicates that the model in the second approach performs better in recommending courses that users are likely to take. However, the accuracy is still relatively moderate, suggesting room for further improvement.
6.3 Hybrid Approach Analysis
The hybrid approach demonstrates the highest accuracy among the three approaches, with a Hit Rate of 69.18% and an F1 Score of11.62% . This indicates that combining multiple recommendation techniques, such as content based filtering and collaborative filtering (as implied by the hybrid approach), leads to more accurate and personalized recommendations for users. The higher Hit Rate and F1 Score suggest that the hybrid approach has a better understanding of user preferences and is more successful in recommending relevant courses.
6.4 Challenges
The challenges encountered in the project are diverse, with the most significant being the process of gathering and comprehending studies related to the recommender system. We conducted a deep investigation into the evidence provided by authors to ensure reliability. Additionally, our main obstacle lies in Machine Learning Operations (MLOps) concerning deployment, as we are unfamiliar with the process.
6.5 Suggestions and further improvements
A suggestion for improving the project is to expand its compatibility with a broader range of online learning platforms. This would reduce the concerns developers have about data consistency between their datasets and the recommender system, making it more accessible and user-friendly across various platforms.
6.6 Essential Knowledge Applied in This Project
For developers who wish to comprehend data analysis, machine learning, and generative ai for understanding user item configurations in large datasets. They should further acquaint knowledge such as data sparsity [23], the cold start problem [24], scalability issues, algorithm complexity, proposing solutions like data augmentation, cold start handling techniques, scalability improvements, and algorithmic enhancements. These perceptions encompass user-item configuration, behavior analysis, and item characteristics, crucial for addressing challenges like data sparsity and scalability [25]. It ensures the development of robust recommendation systems that enhance user experiences across various domains.

7. ACKNOWLEDGEMENT

I (Mr. Newin Yamaguchi) would like to express my deepest gratitude to all those who have contributed to the completion of this project. Without their support, this endeavor would not have been possible. First and foremost, I would like to thank my advisor (Mr. Kampol Woradit) for their guidance and invaluable insights throughout the duration of this project. Their expertise and encouragement have been instrumental in shaping the direction of my research. Furthermore, I would like to acknowledge the support of my colleagues (Ms. Patcharaporn Satantaipop) for their constructive feedback and encouragement during the writing process. I am deeply thankful to my friends and family for their unwavering support and understanding throughout this journey. Their encouragement has been a constant source of motivation. Finally, I would like to express my gratitude to the Department of Computer Engineering, Chiang Mai University for their financial support, which made this project possible.

8. REFERENCES

[1] Y. Liu, H. Yang, G. Sun, and S. Bin. Collaborative filtering recommendation algorithm based on multi-relationship social network.Ingenierie des Systemes d’Information , 25(3):359–364, 2020
[2] Dominic Wong. A critical literature review on e-learning limitations. Journal for the Advancement of Science and Arts , 2(1):55–62, 2007.
[3] Q. Zhang, Y. Liu, L. Liu, S. Lu, Y. Feng, and X. Yu. Location identification and personalized recommendation of tourist attractions based on image processing. Traitement du Signal , 38(1):197–205, 2021.
[4] China Internet Network Information Center (CNNIC). The 46th statistical report on the current situation of internet development in china. Technical report, China Internet Network Information Center (CNNIC), Beijing, China, 2020.
[5] Mustansar Ali Ghazanfar and Adam Prugel-Bennett. A scalable, accurate hybrid recommender system. In 2010 Third International Conference on Knowledge Discovery and Data Mining , pages 94–98, 2010.
[6] S. Bhaskaran and B. Santhi. An efficient personalized trust based hybrid recommendation (tbhr) strategy for e-learning system in cloud computing. Cluster Computing , 22(1):1137–1149, 2019.
[7] H. Tan, J. Guo, and Y. Li. E-learning recommendation system. InProceedings of the International Conference on Computer Science and Software Engineering , pages 430–433, Wuhan, China, December 2008.
[8] L. Sharma. A survey of recommendation system. Int. J. Eng. Trends Technol ., 4:1989– 1992, 2013.
[9] S. Shishenhchi, S.Y. Banihashem, and N.A.M. Zin. A proposed semantic recommendation system for e-learning: A rule and ontology based e-learning recommendation system. In Proceedings of the International Symposium on Information Technology , pages 1–5, Kuala Lumpur, Malaysia, June 2010.
[10] S.S. Khanal, P.W.C. Prasad, A. Alsadoon, and A. Maag. A systematic review: Machine learning based recommendation systems for e-learning. Educ. Inf. Technol. , 25:2635– 2664, 2020.
[11] Xu Chen, Zheng Qin, Yongfeng Zhang, and Tao Xu. Learning to rank features for recommendation over multiple categories. InProceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’16, page 305–314, New York, NY, USA, 2016. Association for Computing Machinery.
[12] Hualong Ma, Xiande Wang, Jianfeng Hou, and Yunjun Lu. Course recommendation based on semantic similarity analysis. In 2017 3rd IEEE International Conference on Control Science and Systems Engineering (ICCSSE) , pages 638–641, 2017.
[13] Rajae Zriaa and Said Amali. A comparative study between k-nearest neighbors and k-means clustering techniques of collaborative filtering in e-learning environment. In Mohamed Ben Ahmed, İsmail Rakıp Karaș, Domingos Santos, Olga Sergeyeva, and Anouar Abdelhakim Boudhir, editors, Innovations in Smart Cities Applications Volume 4 , pages 268–282, Cham, 2021. Springer International Publishing.
[14] J. Shen, T. Zhou, and L. Chen. Collaborative filtering-based recommendation system for big data. International Journal of Computational Science and Engineering , 21(2):219–225, 2020.
[15] P. Thakkar, K. Varma, V. Ukani, S. Mankad, and S. Tanwar. Combining user-based and item-based collaborative filtering using machine learning. In Information and Communication Technology for Intelligent Systems , pages 173-180. Springer, Singapore, 2019.
[16] J. Xiao, M. Wang, B. Jiang, and J. Li. A personalized recommendation system with combinational algorithm
[17] Freddy Lécué. Combining collaborative filtering and semantic content-based approaches to recommend web services. In 2010 IEEE Fourth International Conference on Semantic Computing , pages 200–205, 2010.
[18] J. Lin, H. Pu, Y. Li, and J. Lian. Intelligent recommendation system for course selection in smart education. Procedia Computer Science , 129:449-453, 2018.
[19] AvinashR.Sonule, Mukesh Kalla, Amit Jain, and Deepak Singh Chouhan. Unsw-nb15 dataset and machine learning based intrusion detection systems. International Journal of Engineering and Advanced Technology , 2020.
[20] Nitish K. Panigrahy, Jian Li, and Don Towsley. Hit rate vs. hit probability based cache utility maximization. ACM SIGMETRICS Performance Evaluation Review , 45(2):21–23, 2017.
[21] Reda Yacouby and Dustin Axman. Probabilistic Extension of Precision, Recall, and F1 Score for More Thorough Evaluation of Classification Models. In Steffen Eger, Yang Gao, Maxime Peyrard, Wei Zhao, and Eduard Hovy, editors, Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems , pages 79–91, Online, November 2020. Association for Computational Linguistics.
[22] G. Xu, A. Zhai, J. Wang, Z. Zhang, and X. Li. Cross-media semantic matching based on sparse representation. Technical Gazette , 26(6):170-1713, 2019.
[23] Y. Tian, B. Zheng, Y. Wang, Y. Zhang, and Q. Wu. College library personalized recommendation system based on hybrid recommendation algorithm. Procedia CIRP , 83:490–494, 2019.
[24] J. Gope and S. Jain. A survey on solving cold start problem in recommender systems. In Proceedings of the 2017 International Conference on Computing, Communication and Automation (ICCCA), pages 133-138, Greater Noida, India, 2017.
[25] J. Aguilar, P. Valdiviezo-Diaz, and G. Riofrio. A general framework for intelligent recommender systems. Applied Computing and Informatics , 13(2):147–160, 2017.