However, for illustrative purposes, we used example datasets available
at ’https://startg2545.github.io/item_tutorial.csv’ for item data and
’https://startg2545.github.io/user_item_tutorial.csv’ for user data.
During model evaluation, we emphasized the importance of splitting the
dataset into training and testing sets. Commonly used ratios for this
split include 60:40, 70:30, and 80:20. The training set is used to train
the model to recognize patterns in course descriptions, while the
testing set is used to evaluate the model’s performance without bias.
We evaluated the model’s performance using metrics such as hit rate and
F1 score. The hit rates achieved were 14.69% ,48.24% , and 69.18% for the first approach, second
approach, and hybrid approach, respectively. Similarly, the F1 scores
were 2.58% , 7.92% , and 11.62% for the same
approaches, indicating the accuracy of our recommendation system across
different methodologies. By incorporating these evaluation metrics, we
gained insights into the effectiveness of our recommendation system in
providing personalized course recommendations, contributing to the
overall success of our project.
6. EXPERIMENTATION AND RESULTS
In this project, I integrate insights from diverse studies to develop a
recommendation system grounded in comprehensive research. The system is
designed to recommend the most suitable courses for learners who have
previously taken at least one course. It offers personalized
recommendations, conducts thorough user data analysis to enhance
accuracy, and contributes to improving the overall learning experience.
However, it’s important to note that the system’s applicability is
currently limited to online learning platforms that adhere to specific
data formats required for implementation.
6.1 First Approach Analysis
The first approach has relatively low accuracy in both Hit Rate and F1
Score, indicating that the model’s performance in recommending relevant
courses to users is not very effective. A Hit Rate of 14.69%means that only about 14.69% of the recommended courses match
with the actual courses users take. Similarly, an F1 Score of2.58% suggests poor precision and recall, further highlighting
the limitations of this approach in accurately predicting user
preferences.
6.2 Second Approach Analysis
The second approach shows improved accuracy compared to the first
approach, with a Hit Rate of 48.24% and an F1 Score of7.92% . This indicates that the model in the second approach
performs better in recommending courses that users are likely to take.
However, the accuracy is still relatively moderate, suggesting room for
further improvement.
6.3 Hybrid Approach Analysis
The hybrid approach demonstrates the highest accuracy among the three
approaches, with a Hit Rate of 69.18% and an F1 Score of11.62% . This indicates that combining multiple recommendation
techniques, such as content based filtering and collaborative filtering
(as implied by the hybrid approach), leads to more accurate and
personalized recommendations for users. The higher Hit Rate and F1 Score
suggest that the hybrid approach has a better understanding of user
preferences and is more successful in recommending relevant courses.
6.4 Challenges
The challenges encountered in the project are diverse, with the most
significant being the process of gathering and comprehending studies
related to the recommender system. We conducted a deep investigation
into the evidence provided by authors to ensure reliability.
Additionally, our main obstacle lies in Machine Learning Operations
(MLOps) concerning deployment, as we are unfamiliar with the process.
6.5 Suggestions and further improvements
A suggestion for improving the project is to expand its compatibility
with a broader range of online learning platforms. This would reduce the
concerns developers have about data consistency between their datasets
and the recommender system, making it more accessible and user-friendly
across various platforms.
6.6 Essential Knowledge Applied in This Project
For developers who wish to comprehend data analysis, machine learning,
and generative ai for understanding user item configurations in large
datasets. They should further acquaint knowledge such as data sparsity
[23], the cold start problem [24], scalability issues, algorithm
complexity, proposing solutions like data augmentation, cold start
handling techniques, scalability improvements, and algorithmic
enhancements. These perceptions encompass user-item configuration,
behavior analysis, and item characteristics, crucial for addressing
challenges like data sparsity and scalability [25]. It ensures the
development of robust recommendation systems that enhance user
experiences across various domains.
7. ACKNOWLEDGEMENT
I (Mr. Newin Yamaguchi) would like to express my deepest gratitude to
all those who have contributed to the completion of this project.
Without their support, this endeavor would not have been possible. First
and foremost, I would like to thank my advisor (Mr. Kampol Woradit) for
their guidance and invaluable insights throughout the duration of this
project. Their expertise and encouragement have been instrumental in
shaping the direction of my research. Furthermore, I would like to
acknowledge the support of my colleagues (Ms. Patcharaporn Satantaipop)
for their constructive feedback and encouragement during the writing
process. I am deeply thankful to my friends and family for their
unwavering support and understanding throughout this journey. Their
encouragement has been a constant source of motivation. Finally, I would
like to express my gratitude to the Department of Computer Engineering,
Chiang Mai University for their financial support, which made this
project possible.
8. REFERENCES
[1] Y. Liu, H. Yang, G. Sun, and S. Bin. Collaborative filtering
recommendation algorithm based on multi-relationship social network.Ingenierie des Systemes d’Information , 25(3):359–364, 2020
[2] Dominic Wong. A critical literature review on e-learning
limitations. Journal for the Advancement of Science and Arts ,
2(1):55–62, 2007.
[3] Q. Zhang, Y. Liu, L. Liu, S. Lu, Y. Feng, and X. Yu. Location
identification and personalized recommendation of tourist attractions
based on image processing. Traitement du Signal , 38(1):197–205,
2021.
[4] China Internet Network Information Center (CNNIC). The
46th statistical report on the current situation of internet development
in china. Technical report, China Internet Network Information Center
(CNNIC), Beijing, China, 2020.
[5] Mustansar Ali Ghazanfar and Adam Prugel-Bennett. A scalable,
accurate hybrid recommender system. In 2010 Third International
Conference on Knowledge Discovery and Data Mining , pages 94–98, 2010.
[6] S. Bhaskaran and B. Santhi. An efficient personalized trust
based hybrid recommendation (tbhr) strategy for e-learning system in
cloud computing. Cluster Computing , 22(1):1137–1149, 2019.
[7] H. Tan, J. Guo, and Y. Li. E-learning recommendation system. InProceedings of the International Conference on Computer Science
and Software Engineering , pages 430–433, Wuhan, China, December 2008.
[8] L. Sharma. A survey of recommendation system. Int. J. Eng.
Trends Technol ., 4:1989– 1992, 2013.
[9] S. Shishenhchi, S.Y. Banihashem, and N.A.M. Zin. A proposed
semantic recommendation system for e-learning: A rule and ontology based
e-learning recommendation system. In Proceedings of the
International Symposium on Information Technology , pages 1–5, Kuala
Lumpur, Malaysia, June 2010.
[10] S.S. Khanal, P.W.C. Prasad, A. Alsadoon, and A. Maag. A
systematic review: Machine learning based recommendation systems for
e-learning. Educ. Inf. Technol. , 25:2635– 2664, 2020.
[11] Xu Chen, Zheng Qin, Yongfeng Zhang, and Tao Xu. Learning to
rank features for recommendation over multiple categories. InProceedings of the 39th International ACM SIGIR Conference on
Research and Development in Information Retrieval, SIGIR ’16, page
305–314, New York, NY, USA, 2016. Association for Computing Machinery.
[12] Hualong Ma, Xiande Wang, Jianfeng Hou, and Yunjun Lu. Course
recommendation based on semantic similarity analysis. In 2017 3rd
IEEE International Conference on Control Science and Systems Engineering
(ICCSSE) , pages 638–641, 2017.
[13] Rajae Zriaa and Said Amali. A comparative study between
k-nearest neighbors and k-means clustering techniques of collaborative
filtering in e-learning environment. In Mohamed Ben Ahmed, İsmail Rakıp
Karaș, Domingos Santos, Olga Sergeyeva, and Anouar Abdelhakim Boudhir,
editors, Innovations in Smart Cities Applications Volume 4 , pages
268–282, Cham, 2021. Springer International Publishing.
[14] J. Shen, T. Zhou, and L. Chen. Collaborative filtering-based
recommendation system for big data. International Journal of
Computational Science and Engineering , 21(2):219–225, 2020.
[15] P. Thakkar, K. Varma, V. Ukani, S. Mankad, and S. Tanwar.
Combining user-based and item-based collaborative filtering using
machine learning. In Information and Communication Technology for
Intelligent Systems , pages 173-180. Springer, Singapore, 2019.
[16] J. Xiao, M. Wang, B. Jiang, and J. Li. A personalized
recommendation system with combinational algorithm
[17] Freddy Lécué. Combining collaborative filtering and semantic
content-based approaches to recommend web services. In 2010 IEEE
Fourth International Conference on Semantic Computing , pages 200–205,
2010.
[18] J. Lin, H. Pu, Y. Li, and J. Lian. Intelligent recommendation
system for course selection in smart education. Procedia Computer
Science , 129:449-453, 2018.
[19] AvinashR.Sonule, Mukesh Kalla, Amit Jain, and Deepak Singh
Chouhan. Unsw-nb15 dataset and machine learning based intrusion
detection systems. International Journal of Engineering and
Advanced Technology , 2020.
[20] Nitish K. Panigrahy, Jian Li, and Don Towsley. Hit rate vs. hit
probability based cache utility maximization. ACM SIGMETRICS
Performance Evaluation Review , 45(2):21–23, 2017.
[21] Reda Yacouby and Dustin Axman. Probabilistic Extension of
Precision, Recall, and F1 Score for More Thorough Evaluation of
Classification Models. In Steffen Eger, Yang Gao, Maxime Peyrard, Wei
Zhao, and Eduard Hovy, editors, Proceedings of the First Workshop
on Evaluation and Comparison of NLP Systems , pages 79–91, Online,
November 2020. Association for Computational Linguistics.
[22] G. Xu, A. Zhai, J. Wang, Z. Zhang, and X. Li. Cross-media
semantic matching based on sparse representation. Technical
Gazette , 26(6):170-1713, 2019.
[23] Y. Tian, B. Zheng, Y. Wang, Y. Zhang, and Q. Wu. College
library personalized recommendation system based on hybrid
recommendation algorithm. Procedia CIRP , 83:490–494, 2019.
[24] J. Gope and S. Jain. A survey on solving cold start problem in
recommender systems. In Proceedings of the 2017 International
Conference on Computing, Communication and Automation (ICCCA), pages
133-138, Greater Noida, India, 2017.
[25] J. Aguilar, P. Valdiviezo-Diaz, and G. Riofrio. A general
framework for intelligent recommender systems. Applied Computing
and Informatics , 13(2):147–160, 2017.