No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A Social-aware Gaussian Pre-trained model for effective cold-start recommendation
... Therefore, several recommendation paradigms exist that leverage social connections among users to augment the modeling of user-item interactions with supplementary information sources. These methods(e.g., SFRec [4], SGP [5]) explicitly capture the inter-user influence or cross-user impact of social preferences in recommendations. ...
... In our LDGSR method, we adjusted the dimensions of the embeddings within the range of [5,10,15,20,25,30] to explore their impact on the model's performance. The learning rate was fixed to 0.01, while the batch size was turned within the range of 512 to 4096. ...
... In our test dataset, any user with fewer than 20 interaction instances is identified as a cold-start user. These users are then partitioned into four groups: (0, 5), [5,10), [10,15), and [15,20). Individually, we gauged the recommendation accuracy for each of these user groups. ...
Graph Neural Networks (GNNs) have been utilized in social recommendation, leveraging social relations to enhance the representation of learning for recommendation. Most social recommendation models unify the user-item interactions and social relations in the user representation. Although existing recommender systems have made great progress, most methods struggle to effectively capture the diverse behavioral patterns of users across the two domains, and this limitation hampers the ability to represent users and their preferences accurately. To overcome this limitation, we introduce a novel social recommendation disentangled learning framework (LDGSR). Our model not only highlights the significance of incorporating heterogeneous relationships and latent factor decomposition in social network recommendation models but also explores the rich relationships between items. We conducted comprehensive experiments on four widely used benchmark datasets to validate the effectiveness of the proposed method.
The purpose of this article is to conceive an advanced decision support environment that assists a company in selecting the best service requiring several constraints (expert or machine). The proposed contribution integrates the recommendation in deep learning technique deployed in Cloud. The first step involves preprocessing data using deep learning techniques, particularly NMT and BERT encoders, while the second step uses the neural network to generate recommendations. The neural network’s architecture includes two hidden layers consisting of 16 and 8 neurons, configured with the “ReLU” activation function, while the output layer uses the “Softmax” function. The experiments have been conducted on a dataset of 20, 000 services. Results demonstrate that migrating the DL-based approach to cloud computing significantly reduces response time and memory consumption by approximately 15% and 10% for classification tasks, and 50% and 30% for recommendation tasks, respectively. Compared to an earlier version of the proposed approach based on machine learning, the DL-approach improves recommendation accuracy by approximately 3%. However, the results in terms of response time and memory usage remain variable, suggesting that deep learning requires considerable computing resources. The proposed method was benchmarked using evaluation metrics such as accuracy, MAE, response time, and user satisfaction, demonstrating its practical superiority and applicability across various industries.
Sequential recommender systems often struggle with accurate personalized recommendations due to data sparsity issues. Existing works use variational autoencoders and generative adversarial network methods to enrich sparse data. However, they often overlook diversity in the latent data distribution, hindering the model’s generative capacity. This characteristic of generative methods can introduce additional noise in many cases. Moreover, retaining personalized user preferences through the generation process remains a challenge. This work introduces DGFedRS, a Federated Recommender System Based on Diffusion Augmentation and Guided Denoising, designed to capture the diversity in the latent data distribution while preserving user-specific information and suppressing noise. In particular, we pre-train the diffusion model using the recommender dataset and use a diffusion augmentation strategy to generate interaction sequences, expanding the sparse user-item interactions in the discrete space. To preserve user-specific preferences in the generated interactions, we employ a guided denoising strategy to guide the generation process during reverse diffusion. Subsequently, we design a noise control strategy to reduce the damage to personalized information during the diffusion process. Additionally, a stepwise scheduling strategy is devised to input generated data into the sequential recommender model based on their challenge levels. The success of the DGFedRS approach is demonstrated by thorough experiments conduct on three real-world datasets.
Contrastive learning (CL) recently has spurred a fruitful line of research in the field of recommendation, since its ability to extract self-supervised signals from the raw data is well-aligned with rec-ommender systems' needs for tackling the data sparsity issue. A typical pipeline of CL-based recommendation models is first augmenting the user-item bipartite graph with structure perturbations (e.g., stochastic edge/node dropout), and then maximizing the node representation consistency between different graph augmentations. Although this paradigm turns out to be effective, what underlies the performance gains is still a mystery. In this paper, we first experimentally disclose that, in CL-based recommendation models, CL functions by learning more uniform user/item representations that can implicitly mitigate the popularity bias. Meanwhile, the graph augmentations, which were considered necessary, just play a trivial role. Based on this finding, we propose a simple CL method which discards the graph augmentations and can freely adjust the uniformity of learned representations only by adding uniform noises to the embedding space for creating contrastive views. Although it appears strikingly simple, a comprehensive experimental study on three benchmark datasets demonstrates that the proposed method has distinct advantages over its graph augmentation-based counterparts in terms of recommendation accuracy and model training efficiency. The code is released at https://github.com/Coder-Yu/QRec.
Recommender Systems (RSs) aim to model and predict the user preference while interacting with items, such as Points of Interest (POIs). These systems face several challenges, such as data sparsity, limiting their effectiveness. In this paper, we address this problem by incorporating social, geographical, and temporal information into the Matrix Factorization (MF) technique. To this end, we model social influence based on two factors: similarities between users in terms of common check-ins and the friendships between them. We introduce two levels of friendship based on explicit friendship networks and high check-in overlap between users. We base our friendship algorithm on users’ geographical activity centers. The results show that our proposed model outperforms the state-of-the-art on two real-world datasets. More specifically, our ablation study shows that the social model improves the performance of our proposed POI recommendation system by 31% and 14% on the Gowalla and Yelp datasets in terms of [email protected], respectively.
Recommender Systems are decision support tools that adopt advanced algorithms in order to help users to find less-explored items that can be interesting for them. While recommender systems may offer a range of attractive benefits, they may also intensify undesired effects, such as the Popularity Bias, where a few popular users/items get more popular and many unpopular users/items get more unpopular.
In this paper, we study the impact of different recommender algorithms on the popularity bias in different application domains and recommendation scenarios. We have designed a comprehensive evaluation methodology by considering two different recommendation scenarios, i.e., the user-based scenario (e.g., recommending users to users to follow), and the item-based scenario (e.g., recommending items to users to consume). We have used two large datasets, Twitter and Movielens, and compared a wide range of classical and modern recommender algorithms by considering a diverse range of metrics, such as PR-AUC, RCE, Gini index, and Entropy Score.
The results have shown a substantial difference between different scenarios and different recommendation domains. According to our observations, while the recommendation of users to users may increase the popularity bias in the system, the recommendation of items to users may indeed decrease it. Moreover, while we have measured a different level of popularity bias in different languages (i.e., English, Spanish, Portuguese, and Japaneses), the above-noted phenomena has been consistently observed in all of these languages.
Self-supervised learning (SSL), which can automatically generate ground-truth samples from raw data, holds vast potential to improve recommender systems. Most existing SSL-based methods perturb the raw data graph with uniform node/edge dropout to generate new data views and then conduct the self-discrimination based contrastive learning over different views to learn generalizable representations. Under this scheme, only a bijective mapping is built between nodes in two different views, which means that the self-supervision signals from other nodes are being neglected. Due to the widely observed homophily in recommender systems, we argue that the supervisory signals from other nodes are also highly likely to benefit the representation learning for recommendation. To capture these signals, a general socially-aware SSL framework that integrates tri-training is proposed in this paper. Technically, our framework first augments the user data views with the user social information. And then under the regime of tri-training for multi-view encoding, the framework builds three graph encoders (one for recommendation) upon the augmented views and iteratively improves each encoder with self-supervision signals from other users, generated by the other two encoders. Since the tri-training operates on the augmented views of the same data sources for self-supervision signals, we name it self-supervised tri-training. Extensive experiments on multiple real-world datasets consistently validate the effectiveness of the self-supervised tri-training framework for improving recommendation. The code is released at https://github.com/Coder-Yu/QRec.
In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters. However, it is a challenge to deploy these cumbersome deep models on devices with limited resources, e.g., mobile phones and embedded devices, not only because of the high computational complexity but also the large storage requirements. To this end, a variety of model compression and acceleration techniques have been developed. As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. It has received rapid increasing attention from the community. This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, teacher–student architecture, distillation algorithms, performance comparison and applications. Furthermore, challenges in knowledge distillation are briefly reviewed and comments on future research are discussed and forwarded.
Deep learning techniques have become the method of choice for researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in general, it has, as a result, become difficult to keep track of what represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications point out problems in today's research practice in applied machine learning, e.g., in terms of the reproducibility of the results or the choice of the baselines when proposing new models. In this work, we report the results of a systematic analysis of algorithmic proposals for top-n recommendation tasks. Specifically, we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or graph-based techniques. The remaining one clearly outperformed the baselines but did not consistently outperform a well-tuned non-neural linear ranking method. Overall, our work sheds light on a number of potential problems in today's machine learning scholarship and calls for improved scientific practices in this area.
Semi-supervised learning is the branch of machine learning concerned with using labelled as well as unlabelled data to perform certain learning tasks. Conceptually situated between supervised and unsupervised learning, it permits harnessing the large amounts of unlabelled data available in many use cases in combination with typically smaller sets of labelled data. In recent years, research in this area has followed the general trends observed in machine learning, with much attention directed at neural network-based models and generative learning. The literature on the topic has also expanded in volume and scope, now encompassing a broad spectrum of theory, algorithms and applications. However, no recent surveys exist to collect and organize this knowledge, impeding the ability of researchers and engineers alike to utilize it. Filling this void, we present an up-to-date overview of semi-supervised learning methods, covering earlier work as well as more recent advances. We focus primarily on semi-supervised classification, where the large majority of semi-supervised learning research takes place. Our survey aims to provide researchers and practitioners new to the field as well as more advanced readers with a solid understanding of the main approaches and algorithms developed over the past two decades, with an emphasis on the most prominent and currently relevant work. Furthermore, we propose a new taxonomy of semi-supervised classification algorithms, which sheds light on the different conceptual and methodological approaches for incorporating unlabelled data into the training process. Lastly, we show how the fundamental assumptions underlying most semi-supervised learning algorithms are closely connected to each other, and how they relate to the well-known semi-supervised clustering assumption.
Word embeddings techniques have attracted a lot of attention recently due to their effectiveness in different tasks. Inspired by the continuous bag-of-words model, we present prefs2vec, a novel embedding representation of users and items for memory-based recommender systems that rely solely on user–item preferences such as ratings. To improve the performance and prevent overfitting, we use a variant of dropout as regularization, which can leverage existent word2vec implementations. Additionally, we propose a procedure for incremental learning of embeddings that boosts the applicability of our proposal to production scenarios. The experiments show that prefs2vec with a standard memory-based recommender system outperforms all the state-of-the-art baselines in terms of ranking accuracy, diversity, and novelty.
Many recommender systems suffer from popularity bias: popular items are recommended frequently while less popular , niche products, are recommended rarely or not at all. However, recommending the ignored products in the "long tail" is critical for businesses as they are less likely to be discovered. In this paper, we introduce a personalized diversification re-ranking approach to increase the representation of less popular items in recommendations while maintaining acceptable recommendation accuracy. Our approach is a post-processing step that can be applied to the output of any recommender system. We show that our approach is capable of managing popularity bias more effectively, compared with an existing method based on regularization. We also examine both new and existing metrics to measure the coverage of long-tail items in the recommendation.
Feature learning on point clouds has shown great promise, with the introduction of effective and generalizable deep learning frameworks such as pointnet++. Thus far, however, point features have been abstracted in an independent and isolated manner, ignoring the relative layout of neighboring points as well as their features. In the present article, we propose to overcome this limitation by using spectral graph convolution on a local graph, combined with a novel graph pooling strategy. In our approach, graph convolution is carried out on a nearest neighbor graph constructed from a point’s neighborhood, such that features are jointly learned. We replace the standard max pooling step with a recursive clustering and pooling strategy, devised to aggregate information from within clusters of nodes that are close to one another in their spectral coordinates, leading to richer overall feature descriptors. Through extensive experiments on diverse datasets, we show a consistent demonstrable advantage for the tasks of both point set classification and segmentation. Our implementations are available at https://github.com/fate3439/LocalSpecGCN.
Recommendation systems play a vital role to keep users engaged with personalized content in modern online platforms. Deep learning has revolutionized many research fields and there is a recent surge of interest in applying it to collaborative filtering (CF). However, existing methods compose deep learning architectures with the latent factor model ignoring a major class of CF models, neighborhood or memory-based approaches. We propose Collaborative Memory Networks (CMN), a deep architecture to unify the two classes of CF models capitalizing on the strengths of the global structure of latent factor model and local neighborhood-based structure in a nonlinear fashion. Motivated by the success of Memory Networks, we fuse a memory component and neural attention mechanism as the neighborhood component. The associative addressing scheme with the user and item memories in the memory module encodes complex user-item relations coupled with the neural attention mechanism to learn a user-item specific neighborhood. Finally, the output module jointly exploits the neighborhood with the user and item memories to produce the ranking score. Stacking multiple memory modules together yield deeper architectures capturing increasingly complex user-item relations. Furthermore, we show strong connections between CMN components, memory networks and the three classes of CF models. Comprehensive experimental results demonstrate the effectiveness of CMN on three public datasets outperforming competitive baselines. Qualitative visualization of the attention weights provide insight into the model's recommendation process and suggest the presence of higher order interactions.
Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.
Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.
Recommender systems commonly suffer from the long-standing data sparsity problem where insufficient user-item interaction data limits the systems’ ability to make accurate recommendations. This problem can be alleviated using cross-domain recommendation techniques. In particular, in a cross-domain setting, knowledge sharing between domains permits improved effectiveness on the target domain. While recent cross-domain recommendation techniques used a pre-training configuration, we argue that such techniques lead to a low fine-tuning efficiency, especially when using large neural models. In recent language models, prompts have been used for parameter-efficient and time-efficient tuning of the models on the downstream tasks - these prompts represent a tunable latent vector that permits to freeze the rest of the language model’s parameters. To address the cross-domain recommendation task in an efficient manner, we propose a novel Personalised Graph Prompt-based Recommendation (PGPRec) framework, which leverages the efficiency benefits from prompt-tuning. In such a framework, we develop personalised and item-wise graph prompts based on relevant items to those items the user has interacted with. In particular, we apply Contrastive Learning (CL) to generate the pre-trained embeddings, to allow an increased generalisability in the pre-training stage and to ensure an effective prompt-tuning stage. To evaluate the effectiveness of our PGPRec framework in a cross-domain setting, we conduct an extensive evaluation with the top- k recommendation task and perform a cold-start analysis. The obtained empirical results on four Amazon Review datasets show that our proposed PGPRec framework can reduce up to 74% of the tuned parameters with a competitive performance and achieves an 11.41% improved performance compared to the strongest baseline in a cold-start scenario.
Few-shot learning aims at heuristically resolving new tasks with limited labeled data; most of the existing approaches are affected by knowledge learned from similar experiences. However, interclass barriers and new samples insufficiency limit the transfer of knowledge. In this article, we propose a novel mixture distribution graph network, in which the interclass relation is explicitly modeled and propagated via graph generation. Owing to the weighted distribution features based on the Gaussian mixture model, we take class diversity into consideration, thereby utilizing information precisely and efficiently. Equipped with minimal gated units, the “memory” of similar tasks can be preserved and reused through episode training, which fills a gap in temporal characteristics and softens the impact of data insufficiency. Extensive trials are carried out based on the MiniImageNet and CIFAR-FS data sets. The results turn out that our method exceeds most state-of-the-art approaches, which shows the validity and universality of our method in few-shot learning.
Social recommender systems (SRS) aim to study how social relations influence users’ choices and how to use them for better learning users embeddings. However, the diversity of social relationships, which is instructive to the propagation of social influence, has been rarely explored. In this paper, we propose a graph convolutional network based representation learning method, namely multi-perspective social recommendation (MPSR), to construct hierarchical user preferences and assign friends’ influences with different levels of trust at varying perspectives. We further utilize the attributes of items to partition and excavate users’ explicit preferences and employ complementary perspective modeling to learn implicit preferences of users. To measure the trust degree of friends from different perspectives, the statistical information of users’ historical behavior is utilized to construct multi-perspective social networks. Experimental results on two public datasets of Yelp and Ciao demonstrate that the MPSR significantly outperforms the state-of-the-art methods. Further detailed analysis verifies the importance of mining explicit characteristics of users and the necessity for diverse social relationships, which show the rationality and effectiveness of the proposed model. The source Python code will be available upon request.
When ranking a list of documents relative to a given query, the vocabulary mismatches could compromise the performance, as a result of the different language used in the queries and the documents. Though the BERT-based re-ranker have significantly advanced the state-of-the-art, such mismatch still exist. Moreover, recent works demonstrated that it is non-trivial to use the established query expansion methods to boost the performance of BERT-based re-rankers. Henceforth, this paper proposes a novel query expansion model using unsupervised chunk selection, coined as BERT-QE. In particular, BERT-QE consists of three phases. After performing the first-round re-ranking in phase one, BERT-QE leverages the strength of the BERT model to select relevant text chunks from feedback documents in phase two and uses them for the final re-ranking in phase three. Furthermore, different variants of BERT-QE are thoroughly investigated for a better trade-off between effectiveness and efficiency, including the uses of smaller BERT variants and of recently proposed late interaction methods. On the standard TREC Robust04 and GOV2 test collections, the proposed BERT-QE model significantly outperforms BERT-Large models. Actually, the best variant of BERT-QE can outperform BERT-Large significantly on shallow metrics with less than 1% extra computations.
Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling physics systems, learning molecular fingerprints, predicting protein interface, and classifying diseases demand a model to learn from graph inputs. In other domains such as learning from non-structural data like texts and images, reasoning on extracted structures (like the dependency trees of sentences and the scene graphs of images) is an important research topic which also needs graph reasoning models. Graph neural networks (GNNs) are neural models that capture the dependence of graphs via message passing between the nodes of graphs. In recent years, variants of GNNs such as graph convolutional network (GCN), graph attention network (GAT), graph recurrent network (GRN) have demonstrated ground-breaking performances on many deep learning tasks. In this survey, we propose a general design pipeline for GNN models and discuss the variants of each component, systematically categorize the applications, and propose four open problems for future research.
Social recommendation has emerged to leverage social connections among users for predicting users’ unknown preferences, which could alleviate the data sparsity issue in collaborative filtering based recommendation. Early approaches relied on utilizing each user’s first-order social neighbors’ interests for better user modeling, and failed to model the social influence diffusion process from the global social network structure. Recently, we propose a preliminary work of a neural influence
Diff
usion
Net
work (i.e., DiffNet) for social recommendation L. Wu, P. Sun, Y. Fu, R. Hong, X. Wang, and M. Wang, “A neural influence diffusion model for social recommendation,” in
Proc. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval
, 2019, pp. 235–244.. DiffNet models the recursive social diffusion process for each user, such that the influence diffusion hidden in the higher-order social network is captured in the user embedding process. Despite the superior performance of DiffNet, we argue that, as users play a central role in both user-user social network and user-item interest network, only modeling the influence diffusion process in the social network would neglect the latent collaborative interests of users hidden in the user-item interest network. To this end, in this paper, we propose DiffNet++, an improved algorithm of DiffNet that models the neural influence diffusion and interest diffusion in a unified framework. By reformulating the social recommendation as a heterogeneous graph with social network and interest network as input, DiffNet++ advances DiffNet by injecting both the higher-order user latent interest reflected in the user-item graph and higher-order user influence reflected in the user-user graph for user embedding learning. This is achieved by iteratively aggregating each user’s embedding from three aspects: the user’s previous embedding, the influence aggregation of social neighbors from the social network, and the interest aggregation of item neighbors from the user-item interest network. Furthermore, we design a multi-level attention network that learns how to attentively aggregate user embeddings from these three aspects. Finally, extensive experimental results on four real-world datasets clearly show the effectiveness of our proposed model. We release the source code at
https://github.com/PeiJieSun/diffnet
.
Graph Convolutional Networks~(GCNs) are state-of-the-art graph based representation learning models by iteratively stacking multiple layers of convolution aggregation operations and non-linear activation operations. Recently, in Collaborative Filtering~(CF) based Recommender Systems~(RS), by treating the user-item interaction behavior as a bipartite graph, some researchers model higher-layer collaborative signals with GCNs. These GCN based recommender models show superior performance compared to traditional works. However, these models suffer from training difficulty with non-linear activations for large user-item graphs. Besides, most GCN based models could not model deeper layers due to the over smoothing effect with the graph convolution operation. In this paper, we revisit GCN based CF models from two aspects. First, we empirically show that removing non-linearities would enhance recommendation performance, which is consistent with the theories in simple graph convolutional networks. Second, we propose a residual network structure that is specifically designed for CF with user-item interaction modeling, which alleviates the over smoothing problem in graph convolution aggregation operation with sparse user-item interaction data. The proposed model is a linear model and it is easy to train, scale to large datasets, and yield better efficiency and effectiveness on two real datasets. We publish the source code at https://github.com/newlei/LR-GCCF.
Human communication using natural language, specially in social media, is influenced by the use of figurative language like irony. Recently, several workshops are intended to explore the task of irony detection in Twitter by using computational approaches.
This paper describes a model for irony detection based on the contextualization of pre-trained Twitter word embeddings by means of the Transformer architecture. This approach is based on the same powerful architecture as BERT but, differently to it, our approach allows us to use in-domain embeddings.
We performed an extensive evaluation on two corpora, one for the English language and another for the Spanish language. Our system was the first ranked system in the Spanish corpus and, to our knowledge, it has achieved the second-best result on the English corpus. These results support the correctness and adequacy of our proposal.
We also studied and interpreted how the multi-head self-attention mechanisms are specialized on detecting irony by means of considering the polarity and relevance of individual words and even the relationships among words. This analysis is a first step towards understanding how the multi-head self-attention mechanisms of the Transformer architecture address the irony detection problem.
Many collaborative recommender systems leverage social correlation theories to improve suggestion performance. However, they focus on explicit relations between users and they leave out other types of information that can contribute to determine users' global reputation; e.g., public recognition of reviewers' quality.
We are interested in understanding if and when these additional types of feedback improve Top-N recommendation. For this purpose, we propose a multi-faceted trust model to integrate local trust, represented by social links, with various types of global trust evidence provided by social networks. We aim at identifying general classes of data in order to make our model applicable to different case studies. Then, we test the model by applying it to a variant of User-to-User Collaborative filtering (U2UCF) which supports the fusion of rating similarity, local trust derived from social relations, and multi-faceted reputation for rating prediction.
We test our model on two datasets: the Yelp one publishes generic friend relations between users but provides different types of trust feedback, including user profile endorsements. The LibraryThing dataset offers fewer types of feedback but it provides more selective friend relations aimed at content sharing. The results of our experiments show that, on the Yelp dataset, our model outperforms both U2UCF and state-of-the-art trust-based recommenders that only use rating similarity and social relations. Differently, in the LibraryThing dataset, the combination of social relations and rating similarity achieves the best results. The lesson we learn is that multi-faceted trust can be a valuable type of information for recommendation. However, before using it in an application domain, an analysis of the type and amount of available trust evidence has to be done to assess its real impact on recommendation performance.
Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications, where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on the existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this article, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art GNNs into four categories, namely, recurrent GNNs, convolutional GNNs, graph autoencoders, and spatial-temporal GNNs. We further discuss the applications of GNNs across various domains and summarize the open-source codes, benchmark data sets, and model evaluation of GNNs. Finally, we propose potential research directions in this rapidly growing field.
Learning vector representations (aka. embeddings) of users and items lies at the core of modern recommender systems. Ranging from early matrix factorization to recently emerged deep learning based methods, existing efforts typically obtain a user's (or an item's) embedding by mapping from pre-existing features that describe the user (or the item), such as ID and attributes. We argue that an inherent drawback of such methods is that, the collaborative signal, which is latent in user-item interactions, is not encoded in the embedding process. As such, the resultant embeddings may not be sufficient to capture the collaborative filtering effect.
In this work, we propose to integrate the user-item interactions - more specifically the bipartite graph structure - into the embedding process. We develop a new recommendation framework Neural Graph Collaborative Filtering (NGCF), which exploits the user-item graph structure by propagating embeddings on it. This leads to the expressive modeling of high-order connectivity in user-item graph, effectively injecting the collaborative signal into the embedding process in an explicit manner. We conduct extensive experiments on three public benchmarks, demonstrating significant improvements over several state-of-the-art models like HOP-Rec [39] and Collaborative Memory Network [5]. Further analysis verifies the importance of embedding propagation for learning better user and item representations, justifying the rationality and effectiveness of NGCF. Codes are available at https://github.com/xiangwang1223/neural_graph_collaborative_filtering.
In recent years, Graph Neural Networks (GNNs), which can naturally integrate node information and topological structure, have been demonstrated to be powerful in learning on graph data. These advantages of GNNs provide great potential to advance social recommendation since data in social recommender systems can be represented as user-user social graph and user-item graph; and learning latent factors of users and items is the key. However, building social recommender systems based on GNNs faces challenges. For example, the user-item graph encodes both interactions and their associated opinions; social relations have heterogeneous strengths; users involve in two graphs (e.g., the user-user social graph and the user-item graph). To address the three aforementioned challenges simultaneously, in this paper, we present a novel graph neural network framework (GraphRec) for social recommendations. In particular, we provide a principled approach to jointly capture interactions and opinions in the user-item graph and propose the framework GraphRec, which coherently models two graphs and heterogeneous strengths. Extensive experiments on two real-world datasets demonstrate the effectiveness of the proposed framework GraphRec.
Translational models have proven to be accurate and efficient at learning entity and relation representations from knowledge graphs for machine learning tasks such as knowledge graph completion. In the past years, knowledge graphs have shown to be beneficial for recommender systems, efficiently addressing paramount issues such as new items and data sparsity. In this paper, we show that the item recommendation problem can be seen as a specific case of knowledge graph completion problem, where the “feedback” property, which connects users to items that they like, has to be predicted. We empirically compare a set of state-of-the-art knowledge graph embeddings algorithms on the task of item recommendation on the Movielens 1M and on the LibraryThing dataset. The results show that translational models outperform typical baseline approaches based on collaborative filtering and popularity and that the dimension of the embedding vector influences the accuracy of the recommendations.
Deep learning, as one of the most currently remarkable machine learning techniques, has achieved great success in many applications such as image analysis, speech recognition and text understanding. It uses supervised and unsupervised strategies to learn multi-level representations and features in hierarchical architectures for the tasks of classification and pattern recognition. Recent development in sensor networks and communication technologies has enabled the collection of big data. Although big data provides great opportunities for a broad of areas including e-commerce, industrial control and smart medical, it poses many challenging issues on data mining and information processing due to its characteristics of large volume, large variety, large velocity and large veracity. In the past few years, deep learning has played an important role in big data analytic solutions. In this paper, we review the emerging researches of deep learning models for big data feature learning. Furthermore, we point out the remaining challenges of big data deep learning and discuss the future topics.
With the advent of online social networks, the use of information hidden in social networks for recommendation has been extensively studied. Unlike previous work regarded social influence as regularization terms, we take advantage of network embedding techniques and propose an embedding based recommendation method. Specifically, we first pre-train a network embedding model on the users' social network to map each user into a low dimensional space, and then incorporate them into a matrix factorization model, which combines both latent and pre-learned features for recommendation. The experimental results on two real-world datasets indicate that our proposed model is more effective and can reach better performance than other related methods.
We extend variational autoencoders (VAEs) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still largely dominate collaborative filtering research.We introduce a generative model with multinomial likelihood and use Bayesian inference for parameter estimation. Despite widespread use in language modeling and economics, the multinomial likelihood receives less attention in the recommender systems literature. We introduce a different regularization parameter for the learning objective, which proves to be crucial for achieving competitive performance. Remarkably, there is an efficient way to tune the parameter using annealing. The resulting model and learning algorithm has information-theoretic connections to maximum entropy discrimination and the information bottleneck principle. Empirically, we show that the proposed approach significantly outperforms several state-of-the-art baselines, including two recently-proposed neural network approaches, on several real-world datasets. We also provide extended experiments comparing the multinomial likelihood with other commonly used likelihood functions in the latent factor collaborative filtering literature and show favorable results. Finally, we identify the pros and cons of employing a principled Bayesian inference approach and characterize settings where it provides the most significant improvements.
Recommender Systems are currently highly relevant for helping users deal with the information overload they suffer from the large volume of data on the web, and automatically suggest the most appropriate items that meet users needs. However, in cases in which a user is new to Recommender System, the system cannot recommend items that are relevant to her/him because of lack of previous information about the user and/or the user-item rating history that helps to determine the users preferences. This problem is known as cold-start, which remains open because it does not have a final solution. Social networks have been employed as a good source of information to determine users preferences to mitigate the cold-start problem. This paper presents the results of a Systematic Literature Review on Collaborative Filtering-based Recommender System that uses social network data to mitigate the cold-start problem. This Systematic Literature Review compiled the papers published between 2011–2017, to select the most recent studies in the area. Each selected paper was evaluated and classified according to the depth which social networks used to mitigate the cold-start problem. The final results show that there are several publications that use the information of the social networks within the Recommender System; however, few research papers currently use this data to mitigate the cold-start problem.
Recommender systems usually make personalized recommendation with user-item interaction ratings, implicit feedback and auxiliary information. Matrix factorization is the basic idea to predict a personalized ranking over a set of items for an individual user with the similarities among users and items. In this paper, we propose a novel matrix factorization model with neural network architecture. Firstly, we construct a user-item matrix with explicit ratings and non-preference implicit feedback. With this matrix as the input, we present a deep structure learning architecture to learn a common low dimensional space for the representations of users and items. Secondly, we design a new loss function based on binary cross entropy, in which we consider both explicit ratings and implicit feedback for a better optimization. The experimental results show the effectiveness of both our proposed model and the loss function. On several benchmark datasets, our model outperformed other state-of-the-art methods. We also conduct extensive experiments to evaluate the performance within different experimental settings.
In this paper we revisit matrix completion for recommender systems from the point of view of link prediction on graphs. Interaction data such as movie ratings can be represented by a bipartite user-item graph with labeled edges representing observed ratings. Building on recent progress in deep learning on graph-structured data, we propose a graph auto-encoder framework based on differentiable message passing on the bipartite interaction graph. This framework can be viewed as an important first step towards end-to-end learning in settings where the interaction data is integrated into larger graphs such as social networks or knowledge graphs, circumventing the need for multistage frameworks. Our model achieves competitive performance on standard collaborative filtering benchmarks, significantly outperforming related methods in a recommendation task with side information.
In recent years, deep neural networks have yielded immense success on speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks on recommender systems has received relatively less scrutiny. In this work, we strive to develop techniques based on neural networks to tackle the key problem in recommendation --- collaborative filtering --- on the basis of implicit feedback.
Although some recent work has employed deep learning for recommendation, they primarily used it to model auxiliary information, such as textual descriptions of items and acoustic features of musics. When it comes to model the key factor in collaborative filtering --- the interaction between user and item features, they still resorted to matrix factorization and applied an inner product on the latent features of users and items.
By replacing the inner product with a neural architecture that can learn an arbitrary function from data, we present a general framework named NCF, short for Neural network-based Collaborative Filtering. NCF is generic and can express and generalize matrix factorization under its framework. To supercharge NCF modelling with non-linearities, we propose to leverage a multi-layer perceptron to learn the user-item interaction function. Extensive experiments on two real-world datasets show significant improvements of our proposed NCF framework over the state-of-the-art methods. Empirical evidence shows that using deeper layers of neural networks offers better recommendation performance.