1. INTRODUCTION

The obstacles of online learning become the main issue on e-Learning platform. The primary aim of this project is to address these challenges by employing various feedback models to recommend courses tailored to individual user interests [1]. Many e-Learning platforms lack sophisticated algorithms for suggesting courses based on learners’ preferences [2]. Consequently, users frequently resort to selecting courses prominently displayed on the homepage, leading to a detrimental impact on study intention. The objectives of the project are twofold: firstly, to offer personalized course recommendations tailored to individual learners, and secondly, to systematically gather and analyse user data with the aim of enhancing the quality and accuracy of course recommendations over time.
The project scope involves identifying and utilizing appropriate algorithms based on relevant methodologies to develop a recommendation system. Clear and verifiable success criteria will be defined, aligned with the chosen methodologies. Necessary libraries for implementation will be specified, and a suitable dataset will be selected for experimentation. Our recommendation system aims to enhance user experience by predicting ratings for specific items based on individual preferences. We will utilize content-based filtering, analysing user features and preferences to recommend items, and collaborative filtering, leveraging past user interactions to personalized recommendations [3].
Our research is fortified by a robust framework comprising essential Python libraries such as Scikit-learn, SciPy, NumPy, and Pandas. These libraries form the backbone of our methodology, empowering us to conduct predictive data analysis, solve complex mathematical equations, handle scientific computing tasks, and perform agile data analysis crucial for our research objectives.
With a focus on personalized course recommendations and user data analysis, our research endeavours to achieve two primary outcomes. Firstly, we anticipate a significant enhancement in user experience and satisfaction levels. By leveraging advanced recommendation algorithms, we aim to provide tailored course suggestions that align with individual preferences, ultimately fostering a more engaging and rewarding learning experience for users. Secondly, our research aims to enhance content discovery mechanisms, ensuring that users can easily explore and access relevant and valuable course content.
To achieve these objectives, we look forward to Kaggle as a trusted source of high-quality datasets essential for building and evaluating AI models. Kaggle not only provides access to diverse datasets but also facilitates data exploration, analysis, and collaboration, thereby enriching our research with valuable insights and fostering a collaborative environment for data-driven discoveries. Moreover, we also retrieve some datasets from an educational institution within Chiang Mai University. This research aims to improve decision-making, educational model, and technical way [4] to benefit society by enhancing health, safety, legal compliance, and cultural inclusivity.

2. RELATED STUDIES

To understand a whole picture of our recommendation, it is vital to thoroughly look for an overview of e-Learning course recommendations , focusing on the application of machine learning techniques [5]. It draws upon relevant research articles to establish a comprehensive understanding of the topic, aiming to provide insight into the complexities of these systems and their relevance in the field of machine learning.
Focusing on an e-Learning recommendation system that assists learners in accessing diverse topics without requiring personal information. The core functionality of the recommendation algorithm facilitates learners’ engagement with various online courses [7]. The implementation incorporates different techniques to address challenges, including the utilization of collaborative filtering [8]. The economic challenges faced by many individuals, especially during the COVID pandemic, highlight the importance of affordable education [9]. Developers should prioritize key components such as datasets, explicit and implicit ratings, learner behavior data, and results when implementing recommendation mechanisms [10].
2.1 Problems
The primary challenge in e-Learning course recommendation systems is the inefficient presentation of products to users, requiring them to make decisions to discern their needs or preferences. To address this challenge, it is essential to consider the pros and cons of different approaches and their practical behaviour [6].
2.2 Content-Based Filtering
The first step is the Feature Rating, where features describe users and items in a recommendation system [11]. This method works best when all items share the same set of features. However, it’s not suitable for items with different features. Compatibility across items’ features must be considered. In this case, content-based filtering assesses the relevance of courses based on their descriptions. Term Frequency and Inverse Document Frequency (TF-IDF) [12] are implemented to calculate the weights of courses’ descriptions.
Equation 1: Term Frequency
\begin{equation} \text{tf}\left(term,document\right)=\frac{f(term,document)}{\sum_{\text{term}^{\prime}\in document}{f(term^{{}^{\prime}},document)}}\nonumber \\ \end{equation}\begin{equation} \text{idf}\left(term,allDocument\right)=\log{\left(\frac{N}{1+df(t)}\right)+1}\nonumber \\ \end{equation}\begin{equation} \text{tfidf}\left(term,document,allDocument\right)=tf\times idf\nonumber \\ \end{equation}
Content-based filtering in recommendation systems offers advantages such as quick processing tailored to individual users and the ability to capture niche interests effectively. However, its reliance on feature definition and user knowledge can impact recommendation quality, and it is limited to suggesting items within a user’s current interests without the ability to expand those interests. Therefore, the collaborative approach is proposed.
2.3 Collaborative Filtering
Collaborative filtering recommends items based on the ratings of similar users. K-Nearest Neighbors (KNN) [13, 14] is employed for this purpose, utilizing a similarity metric to identify the most similar users. In addition to traditional collaborative filtering, the system also incorporates an item-based algorithm that analyses users’ behaviour records to calculate item similarities. This approach filters out item sets with high scores among those generated by the target user and utilizes the item similarity matrix to identify other items most like each item in the collection. By sorting and filtering items selected by the target user, the system then recommends the remaining items to enhance the recommendation accuracy and user satisfaction [15]. K-Nearest Neighbors (KNN) is a recommendation technique that uses a user-item matrix to calculate cosine distances between courses and then selects the top similar courses based on these distances. K-Nearest Neighbors (KNN) in recommendation systems has advantages such as not requiring explicit feature definitions and the ability to recommend diverse items without feature specification. However, it faces drawbacks like delayed recommendations for new items until users rate them, decreased accuracy in sparse matrices, and a tendency to prioritize popular courses over less-attended ones.
One common thread in recommender systems is the need to combine recommendation techniques to achieve peak performance [16]. All the known recommendation techniques in different ways. Hybrid recommendation systems [17] combine multiple techniques to enhance performance by customizing recommendations based on specific conditions and dataset requirements.

3. EVALUATION

Evaluation helps you assess how well your recommendation system is performing. It allows you to measure the accuracy of the recommendations made by the model and understand its effectiveness in suggesting relevant courses to users [18].
3.1 Training and Testing
Training and testing allow you to evaluate the performance of your recommendation system. By training the model on a subset of data and testing it on another subset that it hasn’t seen during training, you can assess how well the model generalizes to new data [19]. This helps in understanding if the recommendations made by the system are accurate and relevant.
3.2 Hit Rate
The performance of models is evaluated by predicting a certain number of courses for each user in the training dataset. Then, for each user in the test dataset, it will check if any of the courses they took are among the N predicted courses. if at least one of the predicted courses matches a course taken by the user in the test dataset, it is considered as a Hit [20].
3.3 F1 Score
This is a metric used to evaluate the performance of a classification model. The precision, recall, and F1 value are used as evaluation indexes to measure the performance of each algorithm [21]. The precision refers to the probability that the user is interested in the recommended course list, and recall refers to the probability that the course that the user is interested in appears in the recommendation list [22].

4. PROJECT STRUCTURE

Overall, the methodologies involved data preprocessing, implementing two different recommendation algorithms (TF-IDF and KNN), combining their results into a hybrid approach, and integrating the system to become a part of backend system behind the e-Learning platform for user interaction and visualization of results. Here’s a step by-step summary of the experimentation and results:
4.1 Data Cleaning
To prepare user and item datasets for analysis, several preprocessing steps are required. First, load the datasets in a format compatible with the specified structure. For the user dataset, reformat dates to a standardized format, standardize payment statuses and education levels for consistency, clean addresses to remove any inconsistencies or errors, and extract email domains from email addresses for further analysis if needed. Additionally, rename all columns in the user dataset for clarity and uniformity. In the item dataset, rename the course and description columns to enhance readability and understanding. Finally, after completing these preprocessing steps, save the modified datasets to new files for future use in analyses or recommendation system implementations.
4.2 Course Recommendation using TF-IDF and Linear Kernel
To perform course recommendation using TF-IDF (Term Frequency-Inverse Document Frequency) and a Linear Kernel approach, several steps are involved. First, the cleaned dataset containing user and item information is loaded. Then, the courses taken by a specific user are matched between the user dataset and the item dataset to retrieve their descriptions. Using TF-IDF vectorization, the weights of words in these course descriptions are calculated, giving higher importance to words that are rare across all courses but frequent within a specific course description. Next, a Linear Kernel is applied to compute the cosine similarities between all courses based on their TF-IDF vectors, measuring the degree of similarity between course descriptions. Finally, the top N recommended courses are returned based on their cosine similarity scores, providing personalized recommendations for the user based on their previous course choices. This approach leverages text analysis techniques and machine learning algorithms to generate relevant and tailored course suggestions.
In this example, we employ the Term Frequency-Inverse Document Frequency (TF-IDF) technique to assess word relevance in course descriptions across multiple documents. Initially, we calculate the Term Frequency (TF) for each word within a document, indicating how frequently a word appears in that specific document. For instance, in the course description ”The programming language is difficult, The local language is easy,” we compute TF values for words like ”The,” ”programming,” ”language,” ”is,” ”difficult,” ”local,” and ”easy.”
Next, we determine the Inverse Document Frequency (IDF) for each word, which evaluates the significance of a word by considering its frequency across all documents. Common terms like ”The” and ”professor” have lower IDF scores due to their prevalence across many documents, while specific terms like ”web” or ”network” have higher IDF scores due to their uniqueness. We combine the TF and IDF values using the TF-IDF formula (TF-IDF = TF * IDF) to gauge the relevance of a word within a specific document compared to its importance across all documents. For example, the TF-IDF score for the word ”language” in Document 1 is calculated by multiplying its TF (1.3) with its IDF (approximately 0.18), resulting in a TF-IDF score of around 0.23. This process helps identify words that are both frequently occurring in a document and distinctive across the entire dataset, aiding in the determination of key terms for course recommendation purposes.
Following the TF-IDF computation, we move to cosine similarity. Cosine similarity measures the similarity between two vectors by calculating the cosine of the angle between them. In the context of course recommendation, we construct a user-item matrix where each row represents a course and each column represents a word, with values indicating TF-IDF scores. We then compute the dot product between each pair of course vectors to obtain a course-course similarity matrix. This matrix, filled with cosine similarity values ranging from 0 to 1, reflects the similarity between courses based on their descriptions. A higher cosine similarity implies greater similarity between courses, enabling us to recommend relevant courses to users based on a given course’s similarity to others in the dataset. For instance, a cosine similarity matrix like the one shown (Table 3.6: Item-Item Matrix) allows us to identify courses with similar content or themes, enhancing the precision of course recommendations in educational platforms or e-learning systems.
4.2 Course Recommendation using Feature Ratings and KNN
To implement a course recommendation system, the first step is to load the cleaned dataset containing information about users and courses. Next, feature ratings for each course given by users are determined and calculated. These ratings reflect user preferences and help quantify the level of interest users have in different courses. Following this, the model is trained using the user-course data to learn the relationships between courses based on user preferences. This training process enables the system to understand patterns and correlations in user behaviour, enhancing its ability to make accurate recommendations. Once the model is trained, it can predict and rank the distances of courses based on the courses that the user has taken, using techniques such as collaborative filtering or content-based filtering. Finally, the system returns the top N recommended courses for the given user, leveraging the learned relationships and user preferences to provide personalized and relevant course suggestions.
4.2.1 Feature Ratings Calculation
The feature ratings are also considered to recommend courses to individual users. The principle underlying this approach is to calculate the rating scores for each course based on the user’s historical background. The measurement of impressive score is calculated by observing the behaviours of users in each column as follows. Firstly, for the Email Column, users receive points based on the type of email domain provided: 0 points for no email, 1 point for a common email domain, and 2 points for an educational email domain. The Age-Education Column assigns points based on the user’s status: 0 points for being in the educational system, 1 point for just graduating, and 2 points for being in the working age. The Time Column awards points based on the suitability of the registration time for learning: 0 points for an unsuitable time, and 1 point for a suitable time. The Payment Column assigns points based on payment status: 0 points for overdue payment, 1 point for unapproved payment, and 2 points for on-time payment. Finally, the Address Column gives 1 point if the address is filled and 0 points if it’s blank. This scoring system helps assess various aspects of user information to categorize and prioritize users based on their readiness and suitability for engaging in learning activities or accessing certain services.
Furthermore, find the impressive level individually by considering the historical background of the user and the course, remove the feature rating columns to remain only user, course, and score, pivot the table to get the user-course matrix.
4.2.2 Nearest Neighbors Model
The process described involves fitting a k-nearest neighbors (KNN) model to a dataset where yellow, red, and blue colours represent three different courses. In this context, the KNN model is used to calculate the cosine distance between the red course and all other courses in the dataset. The cosine distance is a measure of similarity between two vectors, in this case, representing the features or characteristics of each course. After calculating the cosine distances for the red course with respect to all other courses, the next step is to select the top-most similar courses based on these distances. The KNN model identifies the nearest neighbors (courses) to the red course by ranking the cosine distances, where smaller distances indicate higher similarity. The topmost similar courses are then determined based on a predefined number of neighbors (k) specified in the KNN model.
Overall, this process leverages the KNN algorithm to assess the similarity between courses using cosine distances, enabling the identification of courses that are most closely related or like the red course based on their features or characteristics.
4.3 Hybrid Course Recommendation
This process outlined involves creating a hybrid recommendation system by combining results from two distinct approaches: TF-IDF and KNN. Firstly, the cleaned dataset is loaded, containing information about users and courses. The TF-IDF approach is utilized to analyse the relevance of words in course descriptions and calculate the importance of each word in recommending courses to users. Simultaneously, the KNN approach is employed to determine the similarity between courses based on user preferences and past interactions.
In the hybrid recommendation system, the results from both TF-IDF and KNN approaches are combined to enhance the accuracy and effectiveness of course recommendations. This integration allows for a more comprehensive analysis of course similarities and user preferences, leading to more personalized and relevant recommendations. Finally, the system returns the top N recommended courses for the given user by leveraging insights from both TF-IDF and KNN approaches, providing a holistic and improved recommendation experience.
To combine TF-IDF and KNN in a hybrid recommendation system, several steps are crucial. Firstly, we need to set weights and normalize the matrices of TF-IDF and KNN to ensure their compatibility and meaningful integration. This step involves assigning appropriate weights to the TF-IDF and KNN results based on their relative importance in the recommendation process. Normalizing the matrices ensures that their values are scaled appropriately for accurate comparison and combination. Next, the normalized matrices from TF-IDF and KNN are stacked in sequence horizontally to create a unified dataset that captures both the semantic similarities derived from TF-IDF and the user-item interactions modelled by KNN. This stacked matrix represents a comprehensive view of course similarities and user preferences, incorporating insights from both approaches. Finally, we apply the Nearest Neighbors algorithm to the stacked matrix to generate the final recommendations. This process is like traditional KNN, where the algorithm identifies the nearest neighbors to a given course based on the stacked matrix, ranking them by their similarity scores. The top N recommended courses are then returned to the user, leveraging the combined strengths of TF-IDF and KNN for more accurate and personalized recommendations.
4.3 Evaluation
The outlined evaluation process provides a structured approach for developers and researchers to measure and enhance the effectiveness of recommendation systems in delivering personalized and relevant recommendations to users. Firstly, the evaluation considers users who have engaged with multiple courses, reflecting a realistic scenario where users have diverse interests and preferences. Next, the training and testing datasets are divided based on a specified ratio, ensuring a balanced representation of user interactions across courses in both datasets. This division allows the model to familiarize itself with the training dataset, minimizing bias and ensuring that the model learns from a diverse set of user behaviours. Once trained, the model predicts courses for all users in the dataset, leveraging the learned patterns and relationships to generate recommendations. The accuracy of the recommendations is then calculated by measuring the similarities between the predicted courses and the actual test courses, providing a quantitative assessment of the model’s performance. Finally, researchers and developers investigate the results of the evaluation and adjust parameters as needed to enhance the quality of the recommendation system. This iterative process of evaluation, adjustment, and refinement helps optimize the system for improved accuracy, relevance, and user satisfaction over time.
4.3 Training and Testing Splitting
When designing the training and testing split for evaluating a recommendation system, several considerations are crucial to ensure an effective assessment of system performance. Firstly, considering only users who have taken more than one course ensures that there is sufficient data available for each user to conduct meaningful analysis and evaluation. This approach helps in capturing diverse user preferences and behaviours, enhancing the reliability of the evaluation results. Secondly, dividing the courses into two parts individually according to specified proportions is essential. This step ensures that each user’s data, including their interactions with courses, is appropriately divided based on the specified ratio. This division helps maintain a balance between training and testing data, preventing biases and ensuring that the model learns from a representative sample of user interactions. Finally, splitting the dataset corresponding to the curriculum based on the proportions defined in the previous step further enhances the evaluation process. This division ensures that the dataset is divided into two distinct parts: one for training the recommendation system and the other for testing its performance. By separating the training and testing datasets based on curriculum proportions, researchers and developers can conduct a comprehensive evaluation that reflects real-world usage scenarios and provides reliable insights into the system’s effectiveness in recommending courses to users.
4.3 Accuracy Management
By incorporating both hit rate and F1 score as performance measurements, researchers and developers gain a comprehensive understanding of the recommendation system’s effectiveness. The hit rate focuses on user interaction and satisfaction, while the F1 score provides insights into the system’s accuracy and completeness in making recommendations. Together, these metrics offer valuable insights for optimizing the recommendation system and improving the quality of course recommendations for users.
4.3.1 Hit Rate Calculation
The hit rate is a metric that measures the percentage of users who receive at least one relevant recommendation from the system. It reflects the system’s ability to provide personalized and valuable suggestions to users, impacting user engagement and satisfaction. A higher hit rate indicates better performance in delivering relevant content to users, enhancing the overall effectiveness of the recommendation system.
\begin{equation} HitRate=\frac{\text{Number\ of\ Hits}}{\text{Number\ of\ Users}}\nonumber \\ \end{equation}
The hit rate of a recommendation system depends on the number of recommended courses provided. A larger number of recommendations can improve the hit rate by offering more opportunities to include relevant items. However, the quality and relevance of the recommendations are also essential factors that influence the hit rate. Simply increasing the number of recommendations may not guarantee a higher hit rate if the recommendations are not tailored to the user’s preferences and needs.
4.3.2 F1 Score Calculation
The F1 score is a crucial metric in evaluating the performance of a recommendation system, as it combines both precision and recall providing a balanced assessment. Precision measures the accuracy of the system’s positive predictions, specifically how many of the recommended courses are relevant to the user. Recall, on the other hand, evaluates the system’s ability to identify all relevant courses among those available.
\begin{equation} Recall=\frac{\text{Number\ of\ correctly\ predicted\ items}}{\text{Total\ number\ of\ relevant\ items}}\nonumber \\ \end{equation}\begin{equation} Precision=\frac{\text{Number\ of\ correctly\ predicted\ items}}{\text{Total\ number\ of\ recommended\ items}}\nonumber \\ \end{equation}\begin{equation} F1\ Score=\frac{2\times Precision\times Recall}{Precision+Recall}\nonumber \\ \end{equation}
The F1 score is calculated using the harmonic mean of precision and recall, ensuring that both metrics contribute equally to the final score. This harmonic mean nature prevents one metric from dominating the score, making the F1 score a reliable measure of the system’s overall effectiveness in providing accurate and comprehensive recommendations.
\begin{equation} F1\ Score_{\text{average}}=\frac{\sum_{i=1}^{n}{F1\ Score}_{i}}{\text{Number\ of\ Users}}\nonumber \\ \end{equation}
A high F1 score is achieved when both precision and recall values are close to 1, indicating that the recommendation system is successfully balancing accuracy and completeness in its recommendations.

5. EXPERIMENTATION AND RESULTS

In the experimentation phase of our project, we evaluated the performance of our recommendation system in providing personalized course recommendations to users. Our machine learning application, hosted on PyPI under the name ’isne-recommendation,’ comprises three key components: TfidfLinearKernel, FeatureRatingsKNN, and Hybrid classes. To maintain consistency and compatibility with our system, the dataset implemented in our project follows a specific pattern as an example table below.
Table 1: A user dataset pattern